The railway industry has recently begun to accelerate efforts to upgrade operations with AI and IoT which is frequently referred to as railway digital transformation or railway DX. This operational upgrade is an ideal candidate for the technology and other resources that NEC can bring to bear. Leveraging our extensive experience in adding new values to railway transportation and passenger services, together with our commitment to supporting maximum railway safety and lives of passengers, we are now systematizing our accumulated know-how to provide a foundation for further expansion in this field.

1. NEC’s Vision for Railway DX

The issues currently facing railway operators can be roughly divided into four areas:

NEC is fully committed to resolving these issues and supporting the continuity of railway services as a keystone of social infrastructure with focus on the four key concepts below. (Fig. 1):

The actions we are taking to achieve these goals are described in the following sections (Fig. 2).

2. Smart Maintenance Solution

2.1 Failure symptom detection

As any sudden failure in in-vehicle equipment or signaling systems could have a negative impact on services, preventive maintenance—that is, addressing symptoms before they become problematic—is critical to ensuring reliable operation. In the event that a failure does occur, it is necessary to identify the specific component in a complex device that precipitated the failure and to immediately perform partial repairs to minimize downtime.

To accomplish this, NEC is applying its System Invariant Analysis Technology (SIAT), which has been tested at various plants, to railway facilities. Verification of the technology is now underway with a view to practical implementation in railway applications. As one of the world’s most safety-oriented industries in which operations are, in principal, accident free, the rail industry poses a challenge to conventional machine learning systems that rely on anomaly data to train their error detection systems. NEC’s SIAT, on the other hand, learns those conditions considered normal and detects any variances as abnormal, allowing it to be applied to equipment where failure events do not follow any same pattern.

2.2 Deterioration prediction

Repair of railway equipment and facilities is performed in compliance with the inspection standards prescribed by the Ministry of Land, Infrastructure, Transport and Tourism, irrespective of the usage environment and state of deterioration. This can actually lead to excessive repair in some cases, making reduction of maintenance costs an important aspect of railway management.

To address this issue, NEC is applying the Heterogeneous Mixture Learning Technology to help optimize maintenance frequency and visualize factors contributing to deterioration. The system is currently being verified in power equipment, as well as signaling and communications equipment. When calculating prediction results, it is necessary to clarify the assumptions on which the results are based and to specify the factors contributing to deterioration. To make sure we know the basis for the results, we use a white-box type AI represented by the Heterogeneous Mixture Learning Technology.

2.3 Facility defect detection

Railway facilities can be located in mountainous areas, on bridges, and inside tunnels. Thus, on-site visual inspection requires a significant expenditure of time and labor. Often, it may also be dangerous.

An increasingly popular supplemental measure is to use camera-installed trains to perform inspection. Further laborsaving and automation are needed here, as in most cases, humans are still required to visually confirm the recorded results.

To address this issue, NEC has been applying image analysis technology that utilizes cutting-edge deep learning technology—RAPID Machine Learning—to advance our efforts to achieve automatic detection of defects in railway facilities and equipment. Since much of the equipment and facilities used by railways are located outdoors, they are likely to be affected by the weather and by the difference in available light between day and night. Development efforts are underway to solve this problem and apply automatic detection function in practice.

3. Smart Operations

This section highlights the various efforts we have undertaken to optimize rail transportation by utilizing our AI and IoT technologies including a suite of cutting-edge technologies called NEC the WISE in order to support next-generation railway service operations while advancing the sophistication in decision making, improving the efficiency of tasks, and facilitating succession of skills.

3.1 Supporting command operations

Trains are operated by command controls in railway industry. When a failure occurs in a transportation service, the service need to be restored as quickly as possible. Critical to this is the transportation command center which is central to communicating with control center staff, station staff, and crew members. If the schedule is disrupted as a result of the failure, the timetable will be revised by weighing various factors such as stagnation of passengers, the time of the day, operation conditions, crew status, and equipment conditions. Ultimately, the final decision is based less on the data than on the knowledge and know-how of experienced technicians. Because no two failure events are the same, it is not possible to unearth a similar failure in the past, making it difficult to optimize recovery plans with machine learning.

This led us to wonder if it would be possible to propose optimal timetables by generating various scenarios that combined historical data with artificially generated data based on various simulations and using pseudo-data on transportation failure accumulated in machine learning. In collaboration with the NEC Central Research Laboratories, we are now developing a high-speed simulator for machine learning that reproduces operation control of trains and passenger flow and applying that simulator in machine learning with a view to discovering a reliable way to create optimal timetables and minimize the stagnation of passengers. The core of this system is NEC’s Logical Thinking AI, which we expect will facilitate efficient machine learning of simulation data on a massive scale.

Clearly, this approach has implications across many fields and we intend to expand our efforts into a broad range of applications such as airline, logistic industry , and ship schedules. At the same time, we will focus on achieving flexible operation management that transcends the borders of various mobility modalities in the age of the “new normal.”

3.2 Supporting station operations

As the primary contact point with passengers, a train station handles various duties. From the viewpoint of universal design, we are also investigating how we can support various station operations such as providing appropriate assistance for passengers with disabilities. Our number one goal is to improve operational efficiency by seamlessly linking guidance information within a railway system and connecting it to the expressed needs of passengers, assuring maximum convenience and a comfortable journey for all passengers.

4. Smart Station

In this section, we show how we are helping railway companies reduce operational loads in the daily duties of station staff (ticket window, turnstile, station attendant, and so on) and improve their station-based passenger services.

4.1 Utilization of AI at a station
(JR East Mobility Innovation Consortium, validation conducted at Ueno Station)

An AI guidance system is among the many projects in the JR East Group Management Vision “Move UP” 2027. As one of the companies involved in this project, NEC conducted validation tests of an AI guidance system at JR Ueno Station.

In validation phase 1, we applied an AI chatbot, NEC auto response solution, to learn frequently asked questions and check the responses’ accuracy. In validation phase 2, we added multiple languages to improve services for foreign visitors. Also looking into future deployment at all stations nationwide, we linked the system to external application programing interfaces in transit search apps to maximize usability (Fig. 3).

Using the knowledge gained on this project, we intend to expand services while further improving response accuracy. We also plan to make those services available via passengers’ own smart devices (multi-device compatibility) and to tailor them to meet the needs of each passenger.

4.2 Utilization of biometrics at stations

NEC I:Delight is a new concept that uses common biometric IDs such as faces and fingerprints to offer customers a seamless experience in multiple places and with multiple services. Our goal is to create a world where everyone is able to enjoy an experience customized to their own needs in all aspects of public and social life. As public transportation is one of the most important and widely used parts of the social infrastructure, we believe it’s an ideal field in which to work on co-creation of values.

To recapitulate, use of biometric authentication (face recognition) enables passengers to pass freely through turnstiles at train stations, on buses, and many other places. Meanwhile it is now expanding into new services such as payment for purchases.

5. Smart Mobility

5.1 Smart mobility and MaaS

The concept of smart mobility involves creating a digital experience (DX) that solves the issues of transport operators and customer experience (CX) that enhances the convenience of users.

Smart mobility embraces the achievement of MaaS as well.

In the MaaS concept, railway systems are seen as the primary means of transportation in large cities, while buses and light rail transit (LRT) systems play a central role in smaller cities. In accordance with this concept, we are now sharing our ICT expertise with public transportation companies and the government (Fig. 4).

5.2 Smart mobility for operators

We have already discussed the issues faced by railways and other public transportation companies in section 1 above.

To address these issues, we have set the following parameters for the desired mobility services:

5.3 What NEC is doing

As part of the validation testing of self-driving mid-size buses, which is a joint project between the Ministry of Economy, Trade and Industry and the Ministry of Land, Infrastructure, Transport and Tourism, we are collaborating with Shinki Bus to conduct face recognition trials that will enable advanced bus operations and diversification of in-vehicle payment systems (as of July 2020).

To achieve advanced bus operations, trials to monitor the bus by video and audio from a remote office is underway. We are gathering data to ensure the safety of buses using the adaptive video distribution technology and abnormal sound detection technology. This combination of video and audio enables stable in-vehicle monitoring even when the bus is moving, ensuring on-board safety with a minimum crew.

As for face recognition, validation tests for practical usage are now underway at the immigration and customs checkpoints of airports, as well as at railway turnstiles. Face recognition in train, cars, and buses is challenging, however, because passengers change at each station and bus stop. There is also the issue of privacy to be considered. We are using our face recognition cloud service to verify whether or not recognition can be conducted smoothly and is acceptable to the local community (Fig. 5).

5.4 Future of smart mobility

As discussed above, we have started working on solutions aimed at both public transportation companies and users in big and small cities. We will continue our efforts to address the issues of public transportation companies and help people enjoy more abundant lives by building digital platforms that connect big cities with small cities, while continuing to improve those solutions and collaborating with distribution and various service providers.

6. Conclusion: The Future of Railway DX

Utilizing our AI and IoT technology, NEC will continue to expand and enhance railway DX and create various values in public transportation and passenger services. We are confident that the know-how and experience we have gained in the railway field can be effectively deployed in other transport modalities, enhancing people’s lifestyles and leading to safer and more convenient public transportation.

Authors’ Profiles