Since 2017 NEC has been providing “NeoFace Cloud”, a face recognition cloud service using the high accuracy NeoFace engine. NeoFace Cloud features the benefits of cloud services at a reasonable monthly fee. Such benefits increase the applications of face recognition services, even in locations that are unsuitable for installing and managing the servers. These include outdoors or in facilities not attended by system managers. “NeoFace Cloud” is increasingly being adopted by customers for linking face recognition to their business systems or services. This paper introduces the technical points of the said service.

1. Introduction

As a pioneer corporation in the field of biometric authentication, NEC has achieved a world top level technology through long years of R&D and by also extending the range of R&D to new authentication technologies and multimodal matching functions. At the same time, NEC has also been leading the expansion of the biometrics market by promoting utilization in various scenarios from national to private enterprises projects, with the introduction of more than 700 systems in more than 70 countries worldwide over the past 40 years.

One of the advantages of biometric authentication methods is the low risk of loss, theft and falsification of authentication media because it is based on the physical and behavioral features of each person. Among the various biometric authentication methods, face recognition has particular high convenience. NEC has been focusing on improving the impersonation prevention function for personal identification and also on the enhancement of crime prevention measures. Based on these improvements, NEC has proposed and introduced a large number of face recognition systems that meet the needs of each customer’s operational demands. NEC also provides a wide range of platform products supporting system building such as entrance/exit gates, computer security software and face recognition appliance servers.

2. NeoFace Cloud was Developed Based on Customer Requests

Requests from customers such as: “we want to use the face recognition service in a location where installing servers is not easy, such as outdoors or at event sites”, “even though we are introducing the system, we’d like to avoid burdening the server or system administration because it presents difficulties” and “we would like to use NEC’s face recognition system with high accuracy” have recently been increasing. To respond to these requests, NEC began the provision of face recognition cloud service NeoFace Cloud in October 2017.

3. Mechanism of Service Provision, Advantages of Cloud Services

With the face recognition function of NeoFace Cloud, the image of the subject’s face (face data) is registered in the cloud in advance. The user app in the edge terminal and the software in the cloud then exchange communications such as the transmission of the face data from the camera-equipped face recognition device (edge terminal) to the cloud. Additionally, it provides the API for user application development so that the user may develop an app according to the precise purpose of use of the face recognition service.

As NeoFace Cloud runs the latest face recognition engine in the cloud, its advantages for customers include the possibility of the latest face recognition technologies following the daily advancement and improvement of NEC’s technologies. Unlike the previous face recognition products, it does not store face data for use in matching in the edge terminal but saves it in the cloud. Consequently, once the face data is registered, its use can be shared by multiple locations and on the multiple cloud-based face recognition systems of the user, thereby significantly improving the convenience of face data management, including registration and updating. In addition, the safe management by NEC of the face data, which is secure personal information, is one of the advantages proper to the cloud service.

4. Rapid Expansion of the Scenarios of Use of Face Recognition

The use of NeoFace Cloud is disseminated among the users who need highly accurate face recognition cloud services such as “personal identification for financial account opening via networks” or “face-recognized payments at store”. The advantages of the usability of various devices, including smartphone, tablet terminal and digital signage contribute to expanding the range of use of face recognition even in outdoors which are unsuitable for installing servers or other equipment. In consequence, new usages scenarios of face recognition are thereby introduced such as “recording labor conditions of outdoor workers by using a location information system”, the “hospitality” and “games”. The following introduces some specific examples of the use of face recognition in innovative areas.

(1) Usage with an interactive robot

The face recognition function and user app are incorporated in an interactive robot, which is then installed in a facility (hotel, membership fitness club, golf course, etc.). A pre-registered VIP member is detected quickly and by interlocking with the CRM system, customer information is displayed on terminals attended by the staff allowing several stores to provide high-quality customer hospitality services in common. Uniform treatment that is not affected by transfer of or retreatment by highly skilled staff is also offered.

(2) “Stamp Rally” using face recognition

“Stamp Rally” is a popular event in Japan which is similar to orienteering. People journey different stations or tourist spots to collect stamps on the provided stamp card. NEC allows people to participate in the “face recognition Stamp Rally” by means of face recognition using equipment installed at check points. When it is combined with a smartphone app provided for end users, it is also possible to send push-type information delivery to the target customers. As it can for example be used in the promotion of visits to locations that enhance visits to a touristic site or theme park, it is expected to bring about effects that can be a part of the regional activation measures of local governmental bodies.

5. Mode of NeoFace Cloud provision

This section details the considerations made in the provision of NeoFace to customers as a cloud service while maintaining a satisfactory usability level, such as in cases as described in section 4.

5.1 Image transfer and feature data transfer

Face recognition follows the process of capturing face images on camera, detecting the face from the shot image, extracting the features data from the detected face, and matching it with the database. As NeoFace Cloud performs face recognition as a cloud service, the face recognition database target is placed in the cloud. However, as the camera for capturing the face image is on the edge terminal, it is necessary to consider how to allocate each series of the above operations on either the edge terminal side or cloud side. A pattern of the process allocation is shown in Fig. 1.

As seen in Fig. 1, allocating more operations on the edge terminal side has the advantage of reducing the size of data transferred to the cloud. However, this allocation is also accompanied with disadvantages, such as 1) the long processing time to be taken on the edge terminal side, and 2) narrowing the choice of an edge terminal because it should incorporate a platform and OS that is compatible with the system used at the service providing side.

To provide NeoFace Cloud with the universality of a cloud service, NeoFace Cloud is provided as pattern (1) independent to the edge terminal as NeoFace Cloud Type A (hereinafter Type A) and as pattern (3) for Android as NeoFace Cloud Type B (hereinafter Type B) for use in cases with limited network bandwidth.

5.2 Functions provided as cloud services

Fig. 2 shows the functions provided by NeoFace Cloud.

5.2.1 Face recognition function

The face recognition function has various usage cases, as described in section 4, and the number of usage cases is equal to the number of apps. As it is hard to cover all of them in a service, the face recognition function is provided as API assuming that it is incorporated in the customer’s application. It is provided as REST API in Type A and as a library for Android in Type B.

5.2.2 Management functions

Face recognition requires its targeted individuals to be registered in a database in advance. In order to facilitate the management work, such as registration of face recognition targets, NeoFace Cloud provides the “NeoFace Cloud Portal” as the web user interface. The functions provided by NeoFace Cloud Portal include the face recognition history check, API (key for using the REST API) management and Type B edge terminal management in addition to the registration of face registration targets.

Functions equivalent to those provided by NeoFace Cloud Portal are also provided by the REST API. This procedure not only enables registration manually via a web user interface but also with linkages such as by registration of face recognition targets from a customer app to NeoFace Cloud, via a personnel system or customer management system.

5.3 Access control

Because NeoFace Cloud is published as a cloud service on the Internet, optimum access control is required for each of the functions described in Section 5.2.

5.3.1 Access control of face recognition function

Since face recognition is assumed to be used from a standing face recognition gate or robot as well as from specific user device, the authority should not be tied to a specific user but should be tied to each device.

With Type A, the face recognition can access the REST API by using the face recognition API key registered on NeoFace Cloud Portal.

With Type B, the library provided for the edge terminal side is given an activation function that activates the connection authority of the edge terminal. The procedure until the completion of activation consists of interim registration of the edge terminal on NeoFace Cloud, issuance of the one-time authentication code and the input of the code on the edge terminal for actual registration of the edge terminal. After completion of activation, the edge terminal can use the face recognition function of NeoFace Cloud through the library.

5.3.2 Access control of management functions

As NeoFace Cloud Portal handles the management functions, the user is required to log in. Assuming that the authority on each management function should be divided according to a usage case, it is possible to grant authority individually for each of the following: face recognition target management, face recognition history viewing, API key management and edge terminal management. In a usage case this procedure enables for example, for entrance/exit management to grant the minimum required authority for each of the staffers: responsible for entering/exiting persons, in charge of entrance/exist history check, in charge of linked app registration and in charge of entrance/exit gate equipment management.

On the other hand, the REST API handling the management functions assumes the case of usage by login of a user having the management authority and the case of background use by an app regardless of the user’s authority. It provides two means of authentication; the user-based authentication and the API key-based authentication equivalent to NeoFace Cloud Portal.

The user-based authentication conforms to the OpenID Connect¹⁾ specifications. The user app works as the Relying Party (RP) of OpenID Connect and the REST API is used by obtaining the access token based on the user authentication by OpenID Provider (OP) provided by NEC.

The API key-based authentication allows the management function to access the REST API by using the API key for management functions registered on NeoFace Cloud Portal.

5.4 Example of app configuration

This section introduces an example of an application that uses one of the provided functions described above to unlock a door by face recognition interlocked with the personnel data. Fig. 3 shows an example of app building and Fig. 4 an example of actual operation usage.

When corporate staff are registered in the personnel system, the same data is automatically registered in NeoFace Cloud so that door unlocking based on face recognition becomes possible. As the data is registered in the cloud, it can also be applied to other branches or offices at the construction sites.

6. Conclusion

As described in section 3, one of the features of NeoFace Cloud is the possibility of “shared usage of face data for use in matching by a number of cloud-based face recognition systems”. Benefiting from this advantage, NEC has begun to provide the “Face Information Management Service.” This is a cloud service that enables the shared usage and unified management of face data and face recognition log data by face recognition platform products and face recognition-based business solutions, as well as by NeoFace Cloud. Some face recognition products (entrance/exit gate. PC security software, settlement package, NeoFace Cloud, etc.) are already capable of sharing data using this service. The service will be made progressively compatible with NEC’s various face recognition solutions.

Reference

Authors' Profiles