A driving force behind world-leading facial recognition performance
"I want to continue to lead the world in fast, accurate facial recognition."
In 2010, NEC's facial recognition technology was ranked highest in the world for performance, leaving other leading vendors around the world in the dust. Hitoshi Imaoka made a significant contribution to this moment of glory, as a researcher of the learning algorithms that hold the key to matching accuracy. Here he discusses his determination to win, his daily battles with mathematical expressions, and the thought and hard work he has put into the challenge of creating the best facial recognition technology in the world.
Identifying individuals quickly and accurately by their facial features.
--What is facial recognition? Please explain in layman's terms.
Imaoka: Facial recognition is a form of what is called biometric authentication. Biometric authentication involves the identification of individuals using physical features that only that person can have, such as their face, fingerprints, finger veins, iris, palm, or ears. Biometric authentication has major benefits not found in systems such as passwords, authentication numbers, and IC cards, including the fact that the key cannot be forgotten, lost, stolen, or forged. It is well known that fingerprint and finger vein authentication are used by border control systems at airports, bank safety-deposit boxes, and PCs and smartphones to verify identity.
Let me explain how facial recognition is used to identify individuals. To perform facial recognition, we first need a facial image database containing universal characteristics suitable for matching. This information is extracted from the facial image data of individuals, and includes the distance between their pupils, the angle of their nose or mouth, and the position of their cheekbones. Then, facial image footage taken for matching purposes and registered images are detected and matched with high speed and accuracy, via computer-basedpattern recognition technology and machine learning techniques using advanced algorithms. The identification of individuals using facial recognition is technology for quickly "finding" a corresponding person from a database containing millions of facial images, and then "distinguishing" whether or not that person really is the individual in question. Although the basic techniques used by development vendors may appear similar, authentication accuracy varies wildly based on the criteria and algorithm used.
--Why has facial recognition come into the spotlight recently?
Imaoka: Facial recognition is receiving more attention than other various biometric authentication methods because it is significantly different from them in a number of ways. One of these is that no action needs to be taken to carry out authentication. Once an individual's facial image is registered, authentication can be carried out smoothly and speedily by shooting footage of that person as needed and matching it. This greatly reduces the time spent waiting for authentication. The second point is that individual authentication can be carried out from a distance. It is even possible to accurately identify an individual walking or in a crowd without them being aware. Facial recognition also enables people to quickly confirm matching data by sight, making tasks such as record management easy. On the other hand, unlike methods such as fingerprint authentication, facial recognition is commonly affected by variations, including lighting and shadow, facial orientation, and expression. That means it is very difficult to improve the accuracy of matching in a variety of environments. Facial recognition has been attracting attention for some time, and dealing with the range of variations that affect it has been a significant issue. NEC was quick to start tackling these facial recognition issues, and we now have 20 years of development history and accomplishments behind us.
Identifying someone even when they change their facial hair, glasses, or hairstyle.
--Tell us about the features and strengths of NEC's facial recognition technology.
Imaoka: "Accuracy" and "speed" are particularly important for facial recognition, and there is a trade-off between these two things. We could increase the amount of facial feature data to improve matching accuracy, but this would cause the detection and matching process to take longer. At NEC, we are independently developing a range of technology and knowledge to achieve both accuracy and speed. For example, the Multiple Face Comparison method uses a highly-accurate learning algorithm to quickly find the target facial image from registered image data of various sizes, both large faces and small, based on characteristics of the eyes. Another proprietary technique we have developed to distinguish between faces is called the Perturbation Space Method. This is a technique for instantly creating a variety of facial images by estimating changes such as the face's orientation, illumination and shadow, and environment from a single registered image. This is performed at an astonishing speed, taking just 0.1 seconds. Additionally, the Multi-Feature Identification method technique can cope with facial changes over the years, and is compatible with a diverse range of races. Drawing upon this range of techniques, we created "NeoFace®," NEC's face detection / face matching engine. NEC's "NeoFace®" enables accurate detection and matching under a variety of conditions, such as changes based on facial hair and eyebrows, hairstyles, and glasses, as well as the orientation or incline of the face. It also demonstrates the ability to cope with changes to faces based on aging. In our proof-of-concept tests, accurate authentication results were produced even when matching images that were taken over 20 years apart.
The best performance in the world by far
--Can you talk about the U.S. benchmark test that NEC was ranked number one in the world for?
Imaoka: That was the benchmark test carried out by the U.S. National Institute of Standards and Technology (NIST), which is a global authority in the field of biometric authentication. The evaluation test is carried out using facial image data of over a million individuals in the possession of NIST. Leading vendors from around the world send the algorithm programs they have developed to NIST, and their performance is evaluated in completely blind tests. The evaluation tests are performed using a range of facial image data, including images taken with a high performance digital camera, images reduced in size and compressed such as those stored in IC passports, images taken where there was not enough light such as an auditorium or hall, and images taken in direct sunlight. Of course, the facial photos used also depict people of various races.
NEC decided to take on the NIST benchmark test challenge in 2008, and for two years after that we focused our energy on performance improvements, such as fine-tuning our algorithm and developing proprietary technology. For example, we carried out repeated simulations by envisaging a variety of places in which facial recognition systems would actually be used, and creating mock tests using our own facial image data. After making these efforts, our matching accuracy was evaluated at 99.7% in results for the facial recognition benchmark test announced by NIST in June 2010. We were the clear winner, with an error rate about 1/10th that of other leading vendors from around the world. Our processing speed was also number one by a wide margin, with a search time of about 0.3 seconds when searching for an image among the 1.6 million registered. Furthermore, NEC's facial recognition showed almost no performance decline in trends for the error rate over the course of one to eight years of aging.
--Why do you think NEC was able to reach number one in the world?
Imaoka: One of the underlying reasons for tackling the NIST benchmark test was my desire as a researcher to win worldwide recognition for NEC's technology and strengths. But at first, I truly thought it would be difficult to prevail on a global stage. Later, as I continued to research, my mindset shift from considering why we would win, to thinking about how we could win. Along with this change, I started to see the objectives I should focus on more clearly, such as what I had to do, and which would be the fastest route to the goal. Before long, my feelings shifted to the belief that I wanted to win convincingly if I was going to take on this challenge. Rather than victory by a small margin, I wanted to win by a landslide to show NEC in a good light, and ensure that this technology would be used throughout the world. I wanted to provide the values of security and peace of mind to the entire world through facial recognition technology. I think these beliefs served as fuel for the advancement and development of the technology.
The tireless efforts of our team members. That's another reason for our success. I believe it is the result of professionals, with areas of expertise including detection, matching, characteristics and algorithms, working hard without compromise to improve our accuracy. When the results of the evaluation test were released, rather than rejoicing, I think we actually felt relieved that we were able to achieve our objective. We faced many struggles along the way, but taking on the NIST benchmark test really sparked a breakthrough in NEC's facial recognition technology.