Global Site
Breadcrumb navigation
Person Re-identification Technology That Matches People Even If They Are Facing Away or Their Bodies Are Occluded
Featured TechnologiesFebruary 8, 2019
Our new technology accurately matches people from their full-figure images recorded in videos and still pictures even when their faces are not visible. We asked our developer more about NEC’s person re-identification technology.
Identifying the same person from behind
Principal Researcher
Biometrics Research Laboratories
― What kind of technology is the person re-identification technology?
The person re-identification technology matches person from the full figure of the person captured in videos and images. It can match up not only people captured face-front into the camera, but also when they are captured from the sides when walking or from the back.
NEC is a world leader in biometric authentication, including face recognition. Face recognition, as you may know, needs the target person’s face to be in the camera picture in order to identify the individual. For that reason, it is impossible to match people with their faces turned away with the face recognition approach.
Now, I’d like you to think about how we humans identify people. Often we can tell who people are even when they are turned around, like someone you have passed by before. We identify and distinguish people by their body shapes and clothes, as well as other information that form the impression they make on us. If you think of this technology as something that simulates this human matching ability, that may make it easier to understand this technology.
Applying this technology, operators can easily find the same person from images captured on multiple cameras in a facility. The same person can be found in real time across multiple camera captures. Because there is no need to have particular body parts, such as face, to be captured on camera, there is little risk of losing sight of the person, which is the greatest feature of this technology. From the person re-identification technology, we can expect powerful solutions for facilities where security is a key concern.
Precision matching even with the body partially occluded
― What kinds of techniques are used in person matching?
The core of this technology is the image recognition based on machine learning. High-accuracy person matching is enabled by having the machine learn massive amounts of person image data. Image recognition is already in practical use as technology that identifies what is pictured in the image, such as identifying whether a subject is a cat or a dog, as in a familiar example of image search. The system gets exponentially complex once you start matching people. Matching people requires an enormous variety of data to be learned.
First, individuals need to be identified in order for them to be matched across different images, which means that the same number of categories to be distinguished as the number of individuals is needed. In addition, humans change into a broad variety of forms, such as when bending down or waving their hands―we need to prepare different variations of postures for each individual. This includes variations from different angles, from the front, side, back, and so on, which bloats the patterns to be learned.
NEC’s person re-identification technology overcame this problem by using the know-how of NEC’s World No.1 accuracy face recognition along with NEC’s original techniques based on deep learning. We also succeeded in matching 150 registered individuals with an accuracy close to 90% in an experiment using a public database.
The main feature of our technology is that you do not necessarily need to capture the full figure of the target person. Even when two-thirds of the lower body is occluded behind an object, we have confirmed that the person re-identification technology can still identify people. When considering applying security cameras in a facility, often in real life a part of a body is occluded by other people or objects such as chairs. In that sense, this technology is developed to a practically applicable level.
- ※MBGC 2009 (Multiple Biometric Grand Challenge)
MBE 2010 (Multiple Biometrics Evaluation)
FRVT 2013 (Face Recognition Vendor Test)
FIVE 2017 (Face In Video Evaluation)
Further evolution combining with face recognition and past data
―What solutions do you have in your plans?
For example, we think it can help look for children who got lost. Street cameras may not always capture the face, and lost children can often slip into crowds. Even in such cases, if we can find the body in the camera images, we can identify the lost child and the location. We also think that this technology can be used in tracking suspects in case of crimes or accidents. Such suspects very rarely allow their faces to be captured on camera, but since this technology can identify them with images of them simply running away―we can find where they headed to.
To pursue diverse applications, we need to respect privacy to the fullest extent. At the same time, the cameras installed spread out across an area and matching figures captured on such cameras can go a long way in diverse applications. We would like to cooperate with a broad range of customers to co-create new value.
- ※The information posted on this page is the information at the time of publication.