High-precision detection of suspicious persons from massive videos data: Quantification of appearance patterns and automated classification
Featured Technologies
June 12, 2018
Background
Today, there are many security cameras installed virtually everywhere in public places such as shopping malls, buildings, stations, and public facilities. The video data collected from these cameras accumulate every second to such an amount that it is nearly impossible for humans to analyze by hand. Particularly, analyzing video data to identify such individuals as those who show up on the same spot many times or various random places, i.e., seemingly suspicious behaviors, from the lengthy video recordings from numerous security cameras is considerably time-consuming and extremely difficult to handle by manpower.
Technological Challenges
To solve these issues, NEC announced back in 2015 the "Profiling Across Spatio-Temporal Data (Note 1)" technology that rapidly extracts individuals that appear frequently. While this technology can quickly find frequently-appearing individuals, it is also important to distinguish between the patterns of how they show up in the video (appearance patterns), such as wandering, passing-through, and stopping, and to classify them for further use. Since conventional technologies had relied solely on human eyes to check and categorize individual appearance patterns, this was the bottleneck for the further operations on the analysis results from massive video data.
Overview of New Technology
NEC has developed an AI-based new technology to prevent overlooking suspicious individuals when analyzing vast amounts of video data. This technology focuses on the differences in appearance patterns to extract suspicious persons (Fig. 1).
Fig. 1: Suspicious person detection system based on differences in appearance patterns
This technology statistically processes the appearance frequency, movement, stay duration, and other attributes of persons appearing in videos, and then draws curves to show their changes. The statistical items are weighted to enable the extraction of only the individuals showing specific appearance patterns. The evaluation results of experiments conducted on a public video data confirmed that people's appearance patterns, such as wandering, longtime standing still, and passing-through, are accurately classified. The experiment thus indicates that this technology can practically discover suspicious individuals without non-detection (Fig. 2).
Fig. 2: Wandering detection rate
Features of New Technology
Quantification and visualization of people's appearance patterns Camera images are divided into a grid to statistically process image information, including appearance frequency, movement (moving range, amount of activity), and stay duration, in detail to quantify the state per frame. This is organized into time-series data to capture the degree of changes in change curves (Fig. 3). Fig. 3: Change curves showing the differences in appearance patterns Based on these change curves, appearance patterns can be classified into different categories for a frequently-appearing person: "stopping" when the degree of change is small during an extended appearance on camera video, "get lost" or "wandering" if the degree of change is greater. For example, a steep gradient in the curve indicates fast movement and a moderate gradient indicates a slower movement. Looking at the behavior of the person represented by a blue curve in Fig. 3, notice the curve rapidly rises up and becomes moderate as it changes over time. This indicates that this specific person has been "wandering," going back and forth over a wide range after appearing in that place.
On the other hand, there are appearance patterns that need to be excluded in order to definitely discover suspicious appearance patterns. The two typical examples are explained below. Detection of suspicious persons can be facilitated by eliminating these appearance patterns prior to analysis.
<Example 1: Movement pattern curve surges at a fixed level while a person is passing through> <Example 2: Movement pattern curve drops when a person stops>
Setting priorities based on target appearance patterns using purpose-oriented weights The priority order of target individuals for observation can be changed by weighting and adjusting the appearance frequency, movement, and stay duration, with respect to the appearance pattern derived from the degree of changes shown in above figures.
By changing the weights according to your purpose, you can set the priority so that the individuals with the target appearance pattern come up in higher ranks. For example, if you want to find people passing through, you can increase the weight on "movement." To find people stopping, you can increase the weight on "stay duration." To find people wandering, you can increase the weight on both "movement" and "stay duration." In this way, target individuals can be extracted by appearance patterns.
The GUI that allows easy operation of the above enables the operator to extract frequently-appearing persons and understand how long those people appeared on which camera, how they walked among the cameras, and other appearance pattern information at a glance.
NEC has positioned the safety business as our global growth engine in the 2020 Medium-term Management Plan and is promoting the switchover to a service-oriented business leveraging platforms as well as the expansion of business into areas beyond public safety to cover areas such as digital government. This technology is one of our core technologies that substantiates the development of NEC Safer Cities. NEC aims to roll out this technology during fiscal 2018 for practical application in security and hospitality to tourists.
※
The information posted on this page is the information at the time of publication.
