Due to the spread of the novel coronavirus infection (COVID-19), safe and secure management of airports is now more important than ever before, especially in cities where tourism is essential. To carry out effective mitigating countermeasures while maintaining existing airport operations, digital transformation (DX) is critical and solutions addressing urgent issues are quickly needed. As solutions for infection control at multiple airports to ensure safe and secure operations for both passengers and airport employees, NEC has begun to provide services that fully leverage our accumulated biometrics and video analytics technologies. This paper introduces DX offerings in airport facilities management that utilize biometrics and video analytics.

1. Introduction

The novel coronavirus infection (COVID-19) pandemic has had an enormous impact on the tourism industry as well as airlines and airports. However, in anticipation of eased travel restrictions, introducing certain measures to prevent the spread of infection is urgently needed for a safe and secure journey both at home and abroad. To address this need, NEC has decided to offer solutions that utilize biometric technology.

This paper will introduce the suite of DX offerings to mitigate the spread of infectious diseases in section 2, the features of the system and the on-site commitment to introduce the services in section 3, and the DX offerings to improve convenience by utilizing biometrics and video analytics in section 4.

2. Safe and Secure Airport Management

2.1 Overview of DX-UX in facilities management

The introduction of new technology while scaling up and reforming existing operations helps improve customer experience (CX) , employee experience (EX) and accelerates DX in facilities management overall (Fig. 1).

The overview of NEC’s DX in facilities management includes verification of employee health conditions and screening (core body temperature detection) and monitoring (video analysis and biometric technology) of passengers, as well as improvement of usability (ID management and biometrics).

2.2 First step to safety and security

NEC’s Infection Prevention DX Offering Suite is a service that provides solutions for various issues in airport facilities management. For the first step in the promotion of DX in airport management, this service helps implement border control measures that can help quickly manage urgent issues on a large-scale basis to ensure safety and security (Fig. 2).

2.3 Issues in airport management and measures against them

NEC’s Infection Prevention DX Offering Suite is designed to provide solutions for airport facilities management regarding infection control. Airports need to take measures against COVID-19 in their operations, however, it is not as simple as measuring core body temperature and health conditions. In addition to improving the accuracy of the technology and enhancing the safety of systems, solutions should consider the following: (1) keeping the passenger’s flow paths which is the same as before, (2) immediate availability for the on-site staff to use without complex training, (3) monitoring the conditions of the airport facilities by the management, and (4) no additional measures necessary for handling personal data and privacy.

Consequently, NEC has considered the following points to use and manage the airport safely and securely as well as pleasantly (Fig. 3).

3. Features of the Biometrics/Video Analytics Platform

3.1 Overview of the system

Achieving solutions for infection control at the airport, this system is comprised of the NEC I:Delight Services¹⁾ face recognition platform, which makes possible walkthrough usage, the integrated user interface of screening applications and monitoring, core body temperature detection thermal cameras, and RGB (color) cameras that sense broad bands of red, green, and blue to capture faces for matching the registered detected elevated body temperature images (Fig. 4).

Thermal cameras are installed at all disembarking gates at the airport. When the system detects that the core body temperature of an individual exceeds the threshold value, the face of that individual is acquired and sent to the face authentication platform, where a high-quality image is selected by the direction and blurriness of the face. The registered facial image is synchronized with the face matching system that monitors the hallways of the airport to get information on the routes registered individuals take. The sequence of this information is displayed on the integrated user interface to allow the airport staff to recognize the facial images of the registered individuals and their travel routes.

3.2 Technical issues

Preparing high-quality face images for the face recognition system is the key point in successful face matching. Preparing quality face images such as those used in passports and driver’s licenses is ideal. If the registered image has a low resolution or is blurry, the feature information cannot be extracted properly. For this reason, it is common to register an image taken from the front in a still position — without a face-obstructing object such as a hat, sunglasses, and a mask. However, the need to have the disembarking passengers stop at the gate prolongs the procedural time, which would be problematic. Due to measures against COVID-19, moreover, wearing a mask is mandatory or recommended at all airports, bringing the issue of the need to even register faces completely different from those in conventional face registration.

The system should be usable by staff with minimal training needed. Managers should also be able to monitor current conditions effectively — even in large airports.

3.3 Advance verification

We estimated the behaviors of disembarking passengers and conducted advance verification in how the walkthrough registration is affected by the factors such as a facial direction, walking speed, light source location, camera installation height, subject-to-camera distance, mask pattern difference, and camera setting difference between activation and deactivation of WDR^* mode. We set up a testing facility simulating an airport environment at NEC Tamagawa Plant (in Kawasaki, Kanagawa Prefecture). Obtaining cooperation from 20 employees, we conducted advance verification tests using the face recognition engine compatible with masks, which was just released at that time. In some cases, such as when a person looked down while looking at their smartphone or looked around quickly to find a place to move after disembarking, the face could not be recognized and therefore could not be registered. Also, when the subject-to-camera distance is short, the camera angle to the face became too sharp, making the image not suitable for face registration. This made it necessary to make adjustments to the viewing angle to maintain resolution even from a distance using optical zoom. We also found that patterned masks led to erroneous detection of eye positions and made it impossible to obtain the feature data. When shadows are cast on the face due to strong backlight or oblique light, the face image quality deteriorated. Detection of mask-covered faces — which we were not sure if it would work — could be performed relatively satisfactorily except for some interference from the patterns, allowing us to establish the effectiveness of the new engine. Considering these results comprehensively, we set forth the guidelines recommending modification of the camera positions and passenger flow paths to have the disembarking passengers direct their faces to the cameras as much as possible, elimination of blurriness in images by setting the shutter speed fast enough, and assurance of availability of sufficient amount of light.

3.4 Parameter adjustment in an actual environment

When we introduced the system in airports, there were various restrictions on camera installation positions and passenger flow paths due to the ceiling construction and the locations of the counters. Even if the cameras could be installed right in front, the passengers might look at the left and right as soon as they exited the disembarking area. So, it was difficult to get frontal images of faces in those areas. In order to register as many frontal faces as possible, a function was added to sort by the angle of the detected face. Face matching thresholds were also adjusted to eliminate false positives. As a result of these adjustments, we were able to conduct a demonstration in the presence of the client to get their approval to launch a formal operation. To sum up, it is important to have advance coordination with the client, especially in cases where walkthrough face registration is involved (non-cooperative authentication) to ensure that the cameras can be installed in optimal locations in consideration of passenger flow paths.

As for the operability and ease of use of the system, we performed start-up procedures in cooperation with the on-site staff when setting up the system at the airport. This allowed the staff to get accustomed to the operation methods.

3.5 Other points we considered

Since video is used for detection in walkthrough authentication, the same individuals end up being detected multiple times. This can result in an enormous number of alarms and non-essential information on the user interface display. To solve this problem, the integrated user interface utilizes face recognition to combine multiple alarms into one when there is a similarity between faces. Also provided is a function that shows the current location of an individual on the map if that person is detected in a different location. So, it is possible to track the whereabouts of the subject chronologically. From the viewpoint of personal information protection, the data is kept only for 30 minutes after the disembarkation, which is generally the length of time passengers stay in the airport, and all the personal information is deleted from the system database when 30 minutes have elapsed.

4. Commitment to Future Expansion

4.1 Linkage of IDs and services inside and outside the airport

Because ID management with biometric authentication improves the convenience of services concerning airport operations such as airport management companies, airlines, and tenants, we are pushing forward to expand the linkage between systems in addition to checking the health conditions of departing and arriving passengers (Fig. 5).

Linking biometric information with passports, boarding passes, vaccination certificates, and negative PCR test certificates and sharing that with checkpoints such as boarding, customs, and quarantine (PCR testing), can allow passengers to pass through checkpoints without showing physical credentials. Biometric authentication is also employed at facilities in the airports such as lounges, restaurants and hotels as well as various other services to simplify operation procedures and payment. As we have seen, we are leveraging the ID linkage service to push forward our commitment to providing DX offerings that achieve a seamless experience in the entire airport.

4.2 Cloud-based data linkage between airports

It is expected that linking data between arrival and destination airports can omit redundant operational tasks and handling passengers especially when those airports share routes. By doing so, they can also improve UX and make the airports and travel experience more pleasant (Fig. 6).

In case an organization manages multiple airport facilities, they can improve system management efficiency by linking and utilizing data in the cloud, making it possible to track the history of health conditions of passengers for smooth airport operations. Also, by using biometric information, as an authentication tool, the boarding and quarantine processes can be integrated into a system so that the burden on both the operators and passengers can be reduced.

5. Conclusion

We have introduced infection control solutions to multiple airports in just six months, and have successfully provided safe and secure operations with the use of our biometric technology and know-how. This could only happen with the cooperation of staff in Japan and abroad as well as our partners and customers. We are sincerely grateful to them for their efforts. NEC will continue to provide DX offerings with high convenience and increase opportunities in which biometric authentication and ID management services can be used safely and securely. In order to pursue our aim, we will optimize biometric and ID management technologies and also enhance usability so that enterprises and individual users will be able to adopt the services more easily and effectively.

Reference

Authors’ Profiles

Related URL