In May 2023, we launched the NEC Generative AI Service (NGS) with the aim of internal business use. For the large language model (LLM), we used not only GPT-3.5 from Microsoft’s Azure OpenAI Service but also GPT-4 and NEC’s LLM. We not only made mechanisms available but also established rules so employees could appropriately use generative AI and prepared policies for its use. In addition, to make full use of generative AI within the NEC Group and lead the way in achieving overwhelming improvements in productivity, we launched the NEC Generative AI Transformation Office and started to provide the MyData service from December 2023, aiming to improve the value of the NGS. At the end of this paper, we present a variety of utilization cases.

1. Introduction

Generative AI has rapidly gained attention with OpenAI’s release of ChatGPT in November 2022, but until then it had only been a topic of conversation among experts. Under these circumstances, NEC declared that it would proactively use generative AI for its internal operations, research and development, and business domestically and internationally in the press release “NEC Group policy on the use of ChatGPT” (April 2023) ¹⁾. To ensure that generative AI can be used for internal operations in line with this declaration, NEC’s internal IT department needed to provide ways that employees of the NEC Group can use generative AI for business with a sense of security. This paper presents how we prepared and implemented them and how they are being used.

2. Launch of the NEC Generative AI Service (NGS)

Consideration for establishing the NEC Generative AI Service (NGS) began in earnest in early April 2023. We began by creating a use policy and then built a system in accordance with it, preparing means for effective use. Based on these three pillars, we began to provide NGS services on May 8, 2023.

2.1 Establishment of a use policy

The Management Information Systems Department and Smart Work DX Center played a central role in considering the use policy, which was later checked and evaluated by many relevant departments including the Human Resources and General Affairs Department, Legal Department, Risk Management & Compliance Department, Digital Trust Business Strategy Department, and Software & System Engineering Department. Despite being a relatively new field, we understood the nature of the risks and worked to swiftly establish rules that do not impose excessive restrictions on the use of generative AI.

Major points of consideration in establishing the use policy were:

The following is an overview of the rules that were created after discussing and considering these points:

We make sure that only employees who agreed upon these conditions and submitted an application for use are allowed to use the NGS.

In addition to these rules, the NGS logs all the input to and the output from the generative AI in a format that can be audited at any time to avoid information leaks, copyright infringement, and other risks. In the unlikely an employee is involved in a lawsuit due to the use of generative AI, this makes it possible to trace how the generative AI was used and to ensure a smooth response.

2.2 System creation

The NGS is achieved by a sparse structure that has few non-zero components, which is commonly called composable architecture. The internal DX Development Center (NDX) used the services and platforms in which it is proficient, enabling it to achieve speedy development. Specifically, the NGS uses Mendix for the front end of the chat screen, MuleSoft for the API, Snowflake for logging, and ServiceNow for the application system.

At the time of the service launch, we adopted the Azure OpenAI Service, provided by Microsoft in partnership with OpenAI, as the Large Language Model (LLM) that forms the core of the generative AI. This is because OpenAI’s technology was already the de-facto standard, and it does not use input information to train and fine-tune the model. At the time of the service launch, only GPT-3.5-Turbo was available.

We confirmed that the environment we created met the appropriate security requirements. After performing basic checks when each part of the infrastructure was introduced, we also checked the network and access control in cooperation with the Cyber Security Strategy Department to fully confirm that there is no risk of information leaks due to the infrastructure before providing services.

2.3 Measures for use

Generative AI is a fairly new service so we focused on the promotion of its utilization and visualization. The following are measures to promote its use we prepared after the service launch:

The usage status dashboard initially only covered the number of service registrants, number of chats, and some other elements; but it later became usable for aggregation by model or organization. This is how the visualization of the usage state is continuing.

VoE is one of the measures that have been promoted by the internal IT team since 2022 as an initiative to improve services by listening to employee feedback. Digital Workplace (DWP), a service that brings together Microsoft 365, Zoom, Box, and NEC’s proprietary add-on functions, is an example of VoE. We applied this to the NGS to ensure that we can achieve the functionality improvements needed by users. By preferentially addressing requirements that receive many likes, we became able to improve usability in a relatively short period.

We are also focusing on collecting examples of how generative AI is used. First, we launched the NEC Prompt Pad, a site for sharing generative AI prompt templates, in August 2023 and achieved constant results by aggregating prompts. Subsequently, we redesigned it as the AX Acceleration Hub in November 2023 with a view of handling a wide range of use cases and ideas. We also organized a monthly award competition to select the person who best used generative AI—in the same way the most valuable player (MVP) is selected after a game— and have been accelerating the accumulation of use cases as a result.

In addition to these, we tried to make sure that generative AI is used by as many people as possible through presentations in internal e-mail newsletters and the portal website, video manuals on how to use generative AI, and internal contests.

2.4 NEC Generative AI Transformation Office

In addition, to make full use of generative AI within the NEC Group and lead the way in overwhelming improving productivity, we launched a virtual organization called the NEC Generative AI Transformation Office, directly under the CIO/CISO in May 2023. Its roles and mission are as follows:

3. Expansion of Service Contents

3.1 Provision of GPT-4

When NGS was launched, only GPT-3.5-Turbo was available, but it was a time when the world was only beginning to be astonished by the overwhelming capabilities of GPT-4.

Because NEC had applied to Microsoft Corporation for use before other companies, NEC could acquire the GPT-4 environment relatively quickly compared to the competition and began offering it on the NGS in late May 2023. In particular, GPT-4-32k can handle a large number of tokens and has high accuracy, so many users use and benefit from it on a daily basis.

3.2 Provision of NEC’s LLM

On July 6, 2023, NEC announced the development of a lightweight LLM with world-class Japanese language proficiency and performance. In conjunction with the announcement, we made NEC’s unique LLM model available to the NGS so that employees can use it as Client Zero.

3.3 MyData service

One of the most common comments received after the launch of the NGS was about the need to make inferences based on internal content and knowledge. Several methods for generative AI are available, and NEC decided to use a method called Retrieval Augmented Generation (RAG). For implementation, we adopted LlamaIndex. This paper does not explain the technical aspects in depth, but by registering Office documents or PDF files you have in advance, you can make use of that knowledge. As of December 2023, the accuracy of generative AI is not yet high, but we have started providing the NGS in response to feedback from employees who are willing to try it out. Through measures such as replacing LlamaIndex with another means, we aim to improve the accuracy of generative AI in the fourth quarter of FY2023.

4. Presentation of Use Cases within the NEC Group

A variety of initiatives have been launched, including the following:

We will present some examples from among these initiatives.

4.1 Support tool for setting operational goals

AI provides support for setting smart operational goals and for setting and agreeing on goals (Fig. 1).

4.2 Outlook add-in to check for suspicious e-mail

We are currently developing an Outlook add-in to check for suspicious e-mail (Fig. 2). It uses the NGS API services to check if the email in question is suspicious or not. This is expected to make it possible to handle cases that are difficult to detect with conventional pattern matching.

4.3 Operational and reputational risk assessment and countermeasures

The reputational risk assessment and creation of countermeasure plans based on news articles that are thought to be related to NEC are automated thanks to generative AI (Fig. 3). Previously, the contents of related articles were checked manually. Automation using generative AI has reduced the work hours for the task by 80%.

References

Author’s Profiles

Related URL: