A driving force behind world-leading facial recognition performance
"I want to keep up the challenge, aiming for facial recognition with zero false matches."
I was full of mathematical expressions, from head to toe
--Tell us about your first steps as a researcher when you joined NEC.
Imaoka: My area of expertise is now learning algorithms for facial recognition, but in university I researched statistical mechanics that applied simulations and mathematics. While continuing with my theoretical research, I became interested in more tangible brain research. Right around that time, I learned there were openings for research staff needed to process brain-related information at NEC's Fundamental Research Laboratories, where well-known researchers in a variety of fields were assembled. That's when I decided to join NEC. For five years after I joined, I made use of the mathematical knowledge from my university days to conduct research into processing visual information of the brain. In simple terms, this research dealt with how people see things, and this later proved valuable for my facial recognition research.
In 2002, I moved to the Information and Media Processing Laboratories, and that's where my research into facial recognition started. Facial recognition concerns two areas: detection to find a face, and matching to distinguish who that face belongs to. I specialized in research into the matching side of this, but at first the false match rate was over 30%. At that time, we found ourselves in a quandary, as we didn't even know what we should improve to boost performance, in part because we didn't have a fixed methodology.
--What did you go through to improve the accuracy of facial recognition?
Imaoka: After a period of trial and error, we developed a learning algorithm. Match accuracy differs for each algorithm, and back then I spent each day dealing with them alone, making improvements to the current algorithm, as well as developing new ones. Specifically, I'd create a program with updated match criteria parameters during the day, and run the system at night to check the results. The next day I'd look over the results and make improvements to the program again, before once again performing checks at night. This cycle was repeated over and over. In those days I was full of mathematical equations, from head to toe. At least, that's what it felt like. My research progressed greatly through this algorithm development.
Another important factor was the collection of facial image data to facilitate machine learning. This is because the more image data I collect, the greater the learning ability of the machine will be, leading to more accurate matching. I made an effort to grow my collection of image data, taking photos of many people, and asking acquaintances to bring me passport photos and pictures of them as a child to aid matching of changes due to age. As a result of these endeavors, the false match rate dropped to between 5% and 10%, and in 2006 the Hong Kong government decided to implement our work in their border control system. After it was implemented, I traveled to Hong Kong for the purpose of evaluating actual operation and inspecting system issues to aid future research.
NEC's facial recognition system is cropping up all over the world
--In what areas are NEC's facial recognition technology being utilized right now?
Imaoka: Starting with the border control system for the Hong Kong government that I just mentioned, our technology is seeing use in police and judicial institutions, as well as border control mechanisms, in a number of countries including the United States and Chile. Its use in the area of security is also spreading at corporations and educational institutions, such as the entry control system at Tokyo Denki University, and facial recognition for NTT Docomo's mobile phones. Another unique example that uses NEC's facial recognition is the entry gate system at Universal Studios Japan®.
Visitors with an annual pass can register their face the first time they enter, enabling facial recognition to be performed smoothly the next time they visit.
This has apparently drastically reduced the amount of time and effort spent on issuing passes and verifying identity at Universal Studios Japan®.
The verification of visiting guests now takes just one second, making entry much easier. Guests also apparently appreciate the novelty of gaining entry through a "face pass."
--What areas do you think use of facial recognition technology will spread to in the future?
Imaoka: As I touched on in the beginning, facial recognition has the benefit of not requiring any action from the person involved, allowing authentication from afar. I think utilization of facial recognition will spread in other fields where benefits like this can be taken advantage of. For example, in hotels and department stores, it can be used to quickly detect VIP guests and customers, and provide high value-added services. It should also enable lost children to be found quickly and easily. On top of that, for entrance tests at places like universities where a flood of students descend upon one location, it is possible to confirm the identity of test-takers swiftly and accurately. Unlike a manual process there is no risk anyone being overlooked, so it would be impossible to have someone take an exam in your place. Implementing facial recognition at public institutions and public offices would also eliminate the need to bring a license or health insurance card each time to confirm your identity. I'd like to see facial recognition spread beyond crime and security-related fields, and into areas such as the creation of new service improvements or added value.
The ongoing challenge to achieve zero false matches
--Tell us about your current initiatives in facial recognition, and what your vision is for the future.
Imaoka: I'm still working on improving accuracy through the evolution of learning algorithms. Since attaining the top ranking in the facial recognition benchmark test results that NIST announced in June 2010, we continue to strive to remain the defending champion, and we are performing research to create algorithms with fewer false matches. We are also continuing to collect facial image data. Sometimes I'll even ask people on the street in New York City to let me take pictures of their faces, having them pose with a range of expressions and orientations.
In the future, I'd like to keep striving for zero false matches. Along with algorithm research, in the coming months I'll aim for more accurate matching using facial images taken at different angles, such as from the side or at a slant. Improving detection and matching accuracy for low resolution images, such as those taken from far away, is another issue we have yet to tackle fully. NEC has cutting-edge image analysis technology called super-resolution for sharpening low resolution images, so we will incorporate this technology, and continue to set our sights on highly-accurate facial recognition that only NEC can achieve.
--What do you take particular care with as a researcher? Also, what are your personal hobbies?
Imaoka: The NIST benchmark test caused me to cast my eyes out beyond the company's wall rather than staying insular. By taking notice of global trends such as research papers and patents and guessing what other companies are going to do, my own objectives become more clear. While taking on the NIST challenge, each Sunday I went to a family restaurant and concentrated on what I should do for the next week, what I could improve, and the priority for each of these. I then swrote this out as a mind map in my notebook. In 2010 I received the president's award within NEC, and this was a great honor, but seeing the technology I developed, which led to me receiving that award, used all around the world makes me even happier. That is the real appeal for me, as a researcher who chose to join a company like NEC over a research institute.
My hobby is running. Three times a week, I run approximately 4 km around my neighborhood at about 1:00 am in the morning. I do it more for stress relief than keeping in good shape. I like to run, and I've completed a full marathon about seven or eight times. If possible, I'd like to try running the Honolulu Marathon at the end of the year.
NEC IS OFFICIAL MARKETING PARTNER OF UNIVERSAL STUDIOS JAPAN®.
© & ® Universal Studios.All rights reserved.CR14-1139