Sensing technologies utilising penetration characteristics of radio waves have been attracting attention as a measure to enhance security screening in public transportation and various venues. For these applications, it is important that the security check be performed without impeding the flow of people, with minimum human effort, and in a non-contact manner. In this paper, we introduce a walk-through security screening system called Invisible Sensing, which is the radio wave-based sensing technology and is capable of detecting weapons or dangerous objects concealed inside bags and/or worn on the body without requiring persons to stop during the scan.

1. Introduction

As the threat of terrorism has increased, detecting concealed weapons has become more important in public transportation and facilities. As an example, stationary body scanners are used in airports as illustrated in Fig. 1 (left), which employ radio waves to detect concealed weapons. However, these systems require persons to stop during the scan, resulting in low throughput at a security check. Moreover, these systems require people to remove items from their pockets and adopt a specified posture for screening that may be uncomfortable for some people. Therefore, such body scanners are not ideal for use at venues in urban areas where large crowds gather and higher throughput is required, such as railways, schools, commercial facilities, stadiums, and amusement parks. Implementing measures to control infectious disease is also crucial as we continue to manage the COVID-19 pandemic, and this highlights the need to eliminate congestion and promote social distancing to reduce person-to-person contact at the security check.

NEC has been promoting research and development of a screening system called Invisible Sensing (IVS) that is capable of detecting concealed weapons or dangerous objects while persons are walking through it, as shown in Fig. 1. This paper presents an overview of the IVS walk-through security screening solution developed by NEC.

2. Requirements and Needs for Expected Security Solutions

When considering security measures for facilities in urban areas, one of the key points to consider is seamless flow of people. Therefore, in areas where many people pass, it is important that the security check be performed without impeding the flow of people, with minimum human effort, and in a non-contact manner. Thus, the requirements for security screening systems in such areas include the following:

The following is a case of practical needs for enhanced security measures.

Traffic congestion in Metropolitan Manila in the Philippines is regarded as one of the worst in Asia, causing economic losses of 3.5 billion pesos (about 7.4 billion yen) per day as of 2017, according to estimates by the Japan International Cooperation Agency (JICA)¹⁾. To solve this problem, there are projects going on to extend and build urban railways and subways. Currently, Manila has three urban railway lines (one Light Rail Transit and two Metro Rail Transit lines) with a total of 44 stations, which will increase to 57 stations after the extension. In addition, the Metro Manila Subway project is in progress as the Philippines’ first mass underground transport system with 15 stations, which is scheduled to launch the service in 2025. It is expected that the number of passengers in Metropolitan Manila will increase to 1.67 million per day by that time.

Another point to consider is that citizens in the Philippines have a legal right to own guns and use them for shooting competitions and self-defence. For this reason, passengers in the Philippines are checked for guns and knives to ensure the safety of railway transportation, and hence, hand-luggage checks in front of ticket gates at stations are a common scene. At some major stations, passengers are subject to security screenings before entering the station.

The current security screening is carried out using walk-through metal detectors with one or two security guards per lane. However, since these detectors can also be set off by metal objects that are not dangerous, security guards still have to rely on visual checks, that takes a long time. During rush hours in the morning and evening, it can take half an hour or longer to buy a ticket, and even longer to enter the train due to long lines for hand-luggage screening. The Department of Transportation (DOTr) of the Philippines has shown strong interest in IVS as a solution to improve this situation and to realise safe and secure railway transportation for passengers.

3. Invisible Sensing System

3.1 System overview and features

NEC’s IVS walk-through security screening system reconstructs images of a target person by utilising microwave radar, and detects weapons concealed inside bags and/or worn on the body in real time. Since microwaves have a longer wavelength than millimetre-waves, our IVS system requires fewer antenna elements per scanning area, and so the radar activation period can be shorter. This makes the system more robust to movement of the target object, and hence walk-through operation is possible. The reduction in the number of antenna elements also contributes to lower the cost.

The IVS system has the following functional and technical features.

The developed demonstration prototype has obtained Technical Standard Conformity Certification in Japan (7.28 GHz to 10.23 GHz). This frequency specification meets FCC regulation in the United States as well. The operating frequency band can be changed to the European CE standard (6.0 GHz to 8.5 GHz), which is also referred to in Asian countries.

3.2 System configuration and processing overview

The IVS system scans depending on the position of the target person relative to the sensor panels that form a gate. As shown in Fig. 2, scans are conducted on the front of the target person when entering the gate, their right and left sides inside the gate, and their back when exiting the gate. This allows a full-body walk-through inspection while passing through the IVS gate.

Fig. 3 shows the general structure of the IVS system. The system consists of two sensor panels, a main panel and a subordinate panel, which are installed opposite each other to form a gate. Both sensor panels operate synchronously and cooperatively in the following procedure²⁾³⁾.

The main panel and the subordinate panel alternately transmit radio waves in time slots of 100 ms intervals to avoid interference each other. In each panel, each of the processing steps (1) to (3) is pipelined to operate at 10 fps. In step (4), the frame-basis scan results of both panels are united in the main sensor panel, and the final judgement of the screening for weapons is output.

4. Key Elemental Technologies for Invisible Sensing

4.1 Fast radar imaging with motion blur suppression

The IVS system analyses the reflected waves captured by the radar sensor and constructs an image in real time. The radar sensor is based on a multiple-input multiple-output (MIMO) antenna array, which consists of multiple transmitters (Tx) and multiple receivers (Rx). In the prototype developed by NEC, 198 Tx elements and 198 Rx elements are arranged in a 1 m x 1 m MIMO array per sensor panel. The system controls each Tx to activate transmission at a different time to avoid interference, while all the Rx can receive reflected waves simultaneously. The waveform is a stepped frequency continuous wave (SFCW).

Every Rx acquires radar signal Ae^-jφ from the reflected wave that is transmitted from each Tx and frequency step, where A denotes amplitude and φ denotes the phase difference between Tx and Rx. Regarding the imaging technique, we applied the MIMO range migration algorithm (MIMO-RMA)⁴⁾ as a base, which is known as a computationally efficient imaging method by utilising frequency-domain processing. We further optimised the imaging algorithm to be suitably implemented on a GPU.

In walk-through screening, it is important to suppress blurring caused by the movements of a walking person during radar scanning. Therefore, to suppress motion blur⁵⁾, we developed an imaging method that estimates the amount of movement of the person from the radar signal, and applied it to MIMO-RMA to compensate for the movement.

Fig. 4 shows an example of fast radar imaging with the developed MBS technique. The upper radar images show the side of a person carrying a model gun concealed in a bag when passing through the IVS gate in briskly walking speed at 1.5 m/s. It is seen that MBS effectively suppresses motion blur that occurred in radar images from previous systems (left) and generates images of improved quality (right). The bottom half of Fig. 4 shows the radar images of a gun and the front of the person with a gun concealed near the abdomen when entering the IVS gate, confirming the effectiveness of our MBS technique.

4.2 AI-driven weapon detection based on radar images

Fast, accurate weapon detection is performed with the use of AI by inputting radar images of both the right and left sides as well as those of the front and back constructed from the radar signals. The radar images of the IVS system are of three-dimensional (3D) complex data in the size 119×119×69 for the side images and 90×100×50 for the front and back images. To process this enormous amount of data in real time, NEC developed a method to create a compressed representation of the original radar image⁶⁾.

From the 3D complex radar image, the data on the right and left sides is compressed by selecting only the complex values with the maximum reflection intensity (amplitude) from the data in the direction orthogonal to the radar sensor placement plane, and a two-dimensional (2D) complex radar image is obtained. Similarly, for the front and back, the information is compressed in the direction parallel to the radar sensor plane. The data is compressed to obtain a 2D radar image. The complex value of each data pixel are divided into amplitude and phase and converted into colour space to produce an RGB image. Using this as an input, a standard Convolutional Neural Network (CNN) can be applied. Thus, by this method, the compressed representation of the radar image maintains the the complex value information across the channels which achieved the real time detection with high accuracy.

We evaluated the performance of this compressed representation method for 3D object detection on the developed IVS system prototype. The experimentation was carried out with several different model guns concealed inside bags and/or worn on the body as target objects for the scanning of a number of people⁷⁾. We applied the aforementioned compressed representation method to the measured 3D radar images and evaluated the detection performance of the sides and the front/back images independently using a model trained by applying the latest 2D CNN framework. We obtained a detection rate (recall) of 90.95% and a false alarm rate of 1.5%. The processing time was less than 100 ms on GeForce GTX1080Ti when using compressed representation. This is about six times faster than that with the original 3D radar images, while maintaining the comparable detection performance.

4.3 IVS system user interface and demonstrations

Fig. 5 shows an example of the actual operation of the IVS system when scanning from the side. A series of continuous images are taken every 100 ms when a person passes through the IVS gate carrying a gun concealed inside a bag. In each image, the points indicating a weapon was detected were automatically determined and then displayed in a square.

Fig. 6 also shows the graphical interface for the IVS system demonstrations. At the top of the screen, radar images of a person passing through the IVS gate (front, right, left, and back) can be selected and displayed as a video, and the image of the person taken from the camera embedded in the sensor panel can be displayed to show the conditions for the time when the scan was conducted. The person’s privacy is taken into consideration when displaying the radar images, and any areas other than the location of the detected weapon is scrambled. The results of the security screening can be viewed in the band at the top of the screen for each person. “Detected” is displayed when a weapon is found, and “Pass” is displayed when a weapon is not found. In addition, the built-in camera at the entrance of the IVS system takes the image when the person enters the screening area, which can be linked with the screening result.

5. Service Deployment

NEC plans to offer the IVS walk-through security screening system for the detection of dangerous objects not only as a stand-alone product but also as a security solution and service. As shown in Fig. 7, each IVS equipment can be controlled via a monitoring centre in the cloud. The target objects to be screened at each installation point can be selected and set from the monitoring centre’s menu, and the detection engine can be updated as needed. It is also possible to add value by integrating multiple IVS screening results as well as data from face recognition scans and other sensors.

The IVS system can be configured as a separate device dedicated to hand-luggage screening. It can also be integrated with a security system already installed in a client’s cite. We will offer a comprehensive security solution together with IVS system according to the needs of clients.

6. Conclusions

The IVS walk-through security screening system detects weapons concealed inside bags and/or worn on the body without stopping the flow of people. This system achieves high throughput and user convenience and has the ability to distinguish dangerous objects from acceptable daily items. We believe this system will be effective as a measure to enhance security in urban facilities.

In the future, we will proceed with demonstration experiments in real environments, aiming for early social implementation.

Reference

Authors’ Profiles