The new face of
Imagine arriving at an airport, dropping your bag on a conveyor belt, and walking to your gate without any lines or other check-in formalities. Your face is your ticket and your passport. It's all you need.
That day is approaching. Already, face recognition is being tested at dozens of airports around the world to conduct security checks faster and more accurately than humans can. In 15 U.S. airports, face recognition is being used to finally comply with laws passed as long as two decades ago to check credentials of people leaving the country. A 2006 law specifically requires biometric checks at entry and exit. It’s a big job—the U.S. has more than 3,000 international airport gates.
Technology in aviation hubs must also keep pace with growing air traffic. The number of airline passengers has grown 67% in the past decade to more than 4 billion travelers in 2017 and is expected to nearly double from that to 7.8 billion passengers in 2036, according to the International Air Transport Association. It's essential to move more passengers more quickly.
A slender face recognition scanner right at the gate not only verifies that the passenger is boarding the correct flight but also checks for security risks, including immigration violations, such as overstaying a visa.
The technology takes just 1.6 seconds to conduct a match for both steps, versus 10-15 seconds for a gate agent to verify a passenger's ticket against their passport and scan the ticket barcode. This means an airplane with more than 600 seats can board in about 15 minutes.
The device takes each passenger's picture and digitizes it--turns into 1s and 0s--to produce a template. It then sends the encrypted template to a secure database, which compares it against other faces until it gets a match. The system then tells the gate agent “we’ve got a match,” explains Benji Hutchinson, Vice President of Federal Operations for NEC Corporation of America’s Advanced Recognition Systems Division.
There are different databases, depending on their purpose. In the case of boarding a plane, the photo of the person at the gate is compared to their passport or visa photo in the flight manifest. "We expect this to match," he says. Someone who doesn't match is likely at the wrong gate, or has arrived at the right gate but at the wrong time.
The other, simultaneous, check is for security, comparing boarding passengers with databases of no-fly lists, security risks as well as immigration databases—like that photo you took on entry into the U.S.
If a person is flagged, they would be taken aside for a manual inspection.
While some privacy advocates worry about use of biometrics, the technology is mainstream. A growing number of mobile phones can be unlocked with fingerprints and face recognition, says Anil K. Jain, a face-recognition expert and professor at Michigan State University.
A study of the public’s attitudes toward biometrics shows that willingness to use such systems depends on the perceived benefits and the perceived security—for example, the extent to which legal and technological structures will protect people and their data, says Cristian Morosan, a professor at the University of Houston specializing in biometrics, security and privacy. “I don’t think that’s a problem anymore. If you go outside, there are quite a lot of cameras. People have cameras on their phones. Biometric passports have been issued for quite a while now. The fact that you’ll be photographed in a security setting isn’t likely to be a problem.”
The government has stored citizen’s biometric data from passports and driver’s licenses for many years without incident. With the biometric gate data, U.S. Customs and Border Protection (CBP) discards photos of U.S. citizens within 12 hours and those of non-citizens within 14 days.
While Department of Homeland Security (DHS) and CBP agents are very diligent, humans are prone to fatigue, resulting in recognition errors.
“If you give the same passport photo to the same human inspector at different times of the day, there is a small possibility that you may get different matching results,” Dr. Jain says. “There’s lots of variability in decision-making due to intra-inspector and inter-inspector variations. In other words, the same inspector could make different decisions, and different inspectors could make different decisions when presented with a pair of facial images. Automated face recognition algorithms, on the other hand, are objective; they will always make the same decision when presented with two facial images.”
Accuracy, he says, depends on “PIE” variations: changes in Pose, Illumination and Expression in face images. In an airport setting, depending on the application scenario for face recognition, we could have large or small PIE variations. For example, when subjects are moving about, their faces may be partly occluded, they may be talking or laughing, and the illumination can vary from bright sunlight to nighttime. However, in applications such as comparing passenger's faces to faces on the flight manifest as they are boarding, PIE variations can be controlled. In this controlled scenario, automated face recognition can achieve close to 99% accuracy, Dr. Jain says.
NEC’s face recognition system, called NeoFace™, doesn’t just use points on the face—that’s old hat, Mr. Hutchinson explains. Now that algorithms are faster and more complex, they use vector analysis. Computer vision detects patterns indiscernible to the eye.
“From roughly the time you’re 15 until you’re 60, the structure of your face that a computer can analyze is basically the same,” Mr. Hutchinson says.
The NeoFace system can unmask disguises, see through beards and even Hollywood-grade masks. It can adjust for aging, guess a person’s age and sex and tell identical twins apart.
“Ours is forensic grade, trained with artificial intelligence,” Mr. Hutchinson added, “NEC have been investing lots of time, technology and funds to overcome challenges such as poor ambient lighting or biases based on gender or color.”
NEC’s face recognition typically achieves 99% accuracy in tests by government agencies, and the U.S. National Institute of Standards and Technology (NIST) has ranked NEC’s technology first in nine of the past 10 years.
NeoFace scanners are about a foot tall and five inches deep. Airports and airlines are testing different devices and biometrics, including fingerprints, from an array of vendors, but regardless of the scanner, all the airports use NEC’s backend software.
Of course, if you undergo plastic surgery or change your weight considerably, you won’t look much like your passport photo, and you’ll need to get a new one—which would be the case even without biometrics. But somebody who had plastic surgery to look like a photo in a stolen passport is unlikely to pass.
CBP paid for the lion’s share of the technology because they bought NEC’s back-end software, the database that stores all the biometric data and contains NEC’s algorithms for comparison. CBP opens a channel for certified entities such as airlines or airports to connect their systems to the big database for face checks. The airlines can buy any front-end scanner device to connect to the CBP database.
The price of the technology has fallen dramatically over the past decade. In the mid-2000s, aviation experts estimated the federal government would have to spend $6 billion to upgrade all airports, gates and security checkpoints with a biometric entry and exit system. “That was a truly overinflated number,” Mr. Hutchinson says. “Today that number is approximately $500 million, and that is most likely overblown.”
Hiring CBP agents to manually check identities of departing international passengers at the top 30 U.S. airports, which handle 97% of international departures, would be far more expensive. In written testimony to the Senate, Homeland Security officials explained that “CBP would need seven to nine officers to handle a large aircraft…. CBP estimates that in order to inspect 95% of all ‘in-scope’ travelers departing by air, a manual solution at the top 30 airports would require approximately 3,400 more officers at an average annual cost of $790 million.”
By contrast, biometric technology not only costs less up front but has far lower operating costs. Not to mention that it’s far quicker than manual inspection. That’s likely to leave many passengers with a smile.