The face of face recognition and the young researchers supporting NEC's technology: What strengths has NEC cultivated by harnessing the power of various "faces"?
NEC recently received The Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology "Awards for Science and Technology" (Reiwa 4; 2022) for its face recognition technology, which has been ranked number one in the world. The five recipients of this award are Hitoshi Imaoka (age 52), an NEC Fellow who has led the way for NEC in face recognition, Principal Researcher Akihiro Hayasaka (age 39), Principal Researcher Yusuke Morishita (age 38), Principal Researcher Koichi Takahashi (age 34), and Senior Research Architect Hiroshi Hashimoto (age 33) of NEC Biometrics Research Laboratories. The strength of NEC's face recognition technology has been driven by the growth of its young researchers.
From 70% accuracy to number one in the world
This award was presented to the team for its achievements in developing a highly accurate face recognition technology that contributes to the safety and security of the world. But what exactly is it that makes NEC's face recognition technology so remarkable?
The September 11 terrorist attacks on the United States in 2001 led to a dramatic increase in demand for facial recognition, most notably the introduction of biometric passports. Now, two decades later, face recognition technology is not only used at airports but also in various other situations, including smartphone personal authentication and entry/exit control at events.
NEC has made significant contributions to the development of face recognition for these and other uses.
According to Hitoshi Imaoka, "NEC's strength lies in the ability of its technology to adapt to both changes over time and face orientation."
NEC has made continuous efforts to improve both the accuracy and speed of its face recognition technology, resulting in it being ranked first in the world five times in benchmark tests conducted by the U.S. National Institute of Standards and Technology (NIST) for face recognition technology. Today, NEC's face recognition technology is used in 45 countries around the world, and it is Mr. Imaoka who has led the way.
Mr. Imaoka explains, saying, "I first became involved in face recognition research at NEC's Biometrics Research Laboratories in 2002. Accuracy was approximately 70% at the time, and at one point the Biometrics Research Laboratories was comprised of just two members. Fast forward to today—we now have an error rate of less than 1%, which is outstanding compared to our competitors around the world. This award is the culmination of those times and the hardships we have experienced along the way. I am delighted that the results of my many years of research have been recognized, and that I was able to receive this award together with the four other members of our team."
Mistakes are simply not an option when it comes to detection: A sense of mission unique to the business world
Young researchers are being nurtured under NEC's face of face recognition—Hitoshi Imaoka. Aside from Mr. Imaoka, the other award recipients are all in their thirties. NEC's strength in face recognition can be better understood by taking a look at the accomplishments of these four individuals.
The first step in face recognition is face detection, which is used to identify where the face is in the image. This is the area for which Mr. Morishita is in charge. Upon joining NEC in 2008, Mr. Morishita was immediately appointed as a member of the face recognition benchmark project team. He has remained involved in research for the more than ten years since then ever since.
Mr. Morishita reflects, saying, "Back in my university days, it was said that face recognition technology was not cutting-edge research. However, once I joined NEC, I realized that there were still issues to tackle before it could be utilized in the real world.
Even back then, NEC's face detection rate was 99%. Yet, if you test 10 million images with an error rate of 1%, you will make errors on 100,000 of the images. This is simply not acceptable in the business world. 'Mistakes are not an option when it comes to detection.' These are the words I clung to as I engaged in daily efforts aimed at improvement. Our error rate is currently at 0.00000……％, which is nearly zero. Receiving this award makes me feel as if I have contributed as a member of the team, and that makes me very happy."
I had been reluctant, but had no choice once the baton of leadership was passed on to me
Mr. Hayasaka has also been a member of Mr. Imaoka's team for more than ten years, and is in charge of facial feature extraction, which is the step following face detection. This is what makes it possible to determine whether the person in an image is the same person or someone else.
"I actually researched fingerprint recognition back when I was a student, and participated in NIST benchmark tests. I was initially reluctant to do so because the work involved was so demanding.
Participating in benchmark tests at NEC also proved to be tough. There was even a time when we collected face data for 3,000 people over a period of two months. I feel a great sense of accomplishment having contributed to NEC being ranked number one in the world through the improvement of face matching.
Now that I have been involved in this for so long, it is strange to recall how hesitant I was at first. After receiving this award, I would like to work towards a new goal with pride."
Starting in 2021, Mr. Imaoka took a step back from being the direct development leader for the benchmark test, entrusting Mr. Hayasaka with the position of leader and Mr. Morishita with the position of subleader.
While Mr. Hayasaka felt apprehensive at first and wondered if he was really the right person for the job, the team led by himself and Mr. Morishita once again claimed the top spot in the world in 2021. According to Mr. Morishita, "With strong members like Mr. Hashimoto and Mr. Takahashi, I believed that we would do just fine as long as we did what needed to be done."
Even a researcher like me who is not necessarily well balanced can demonstrate their abilities
One of the younger team members in which Mr. Hayasaka and Mr. Morishita placed their trust is Mr. Hashimoto, who joined NEC in 2016. In the area of facial feature extraction, deep learning was utilized to achieve the highest accuracy in the world.
"When I first joined the team, there were around 10 members. I was truly grateful that they gave a young member like myself a chance.
Deep learning began garnering attention in the 2010s. Experiencing the evolution of deep learning both in and outside the company in real time enabled us to incorporate a variety of technologies in a complex manner."
Mr. Imaoka gives his stamp of approval, telling us that Mr. Hashimoto became a force to be reckoned with starting with the 2018 benchmark test.
"While my specialties are deep learning, mathematics, and programming, I am not a well-balanced researcher. Nevertheless, the older members of our team have already built a strong foundation, and there are individuals I can rely on in each area. This has enabled me to demonstrate my abilities despite not being a well-balanced researcher. I believe this is something that is unique to NEC," says Mr. Hashimoto.
Not only do I find it interesting, but the fact that others can use it brings me joy
In face recognition, feature point detection, which analyzes the positioning of feature points (i.e., distinctive features) such as eyes, nose, and mouth, as well as the size of the facial area, is another important step in face recognition. Mr. Takahashi is active in this area.
Mr. Takahashi has been researching feature point detection since he was a student, and was actually scouted by Mr. Imaoka. After entering graduate school and completing an internship, he joined NEC in 2015.
Mr. Morishita, who is Mr. Takahashi's senior, was also in charge of feature point detection at one point. However, he decided to leave this area to Mr. Takahashi because he believed his approach was better.
Mr. Takahashi has demonstrated his abilities in face orientation, one of NEC's strengths, and in recognition from difficult angles, such as from the side. He has also greatly contributed to improving video recognition performance.
"Back when I was a student, it was important for me to be interested in what I was creating. Now, however, I find it more interesting to create things that can be used by various people. Part of what I enjoy about working at NEC is being able to contribute to the world."
Masked face recognition
NEC has rapidly demonstrated the capabilities of its face recognition technology even amidst the COVID-19 pandemic. Since masks hide nearly two-thirds of the face, they can interfere with recognition carried out using conventional technology. However, with the spread of infection that has occurred since 2020, improving the accuracy of face recognition when masks are worn has become an urgent task around the world.
We embarked on this endeavor in May of 2020, completed our research and development in about two months, and launched sales of the product in October. In our internal evaluations, NEC's masked face recognition rate has reached an accuracy of more than 99.9%.
While we would ordinarily make yearly plans for commercialization, we deployed this product at a furious speed to meet society's needs. The timing of this was incredibly fast even from a global perspective.
Mr. Imaoka reflects on the experience, saying, "It was thanks to team work and the strenuous efforts put forth by our members that we were able to accomplish this with such speed."
He then continued, emphatically stating, "When I first heard we had won the award, many faces came to mind. Not only fellow members and everyone else on the team but also the divisions, affiliated companies, and country affiliates that supported us. I would like to express my sincere appreciation to all of them."
"Our goal is to create a safe, secure, and convenient environment in countries around the globe, and to contribute to people's everyday lives."