Hearable devices open the way to new user experiences

Professor, Department of Advanced Media,
Faculty of Advanced Engineering,
Chiba Institute of Technology Masaya Ando
NECSatoshi Furutani

In recent years, wearable devices have been gathering attention. NEC has developed a “hearable” device that is worn in the ear. What are the hidden possibilities of hearables that allow for a new computing style? We spoke with Professor Masaya Ando who has taught and conducted research in user experience (UX) for many years and Satoshi Furutani who is in charge of the hearable business at NEC.

Masaya Ando
Masaya AndoMasaya Ando
Professor, Department of Advanced Media, Faculty of Advanced Engineering, Chiba Institute of Technology
He graduated from the Culture of Media Society Department in the School of Cultural and Social Studies at the Graduate University for Advanced Studies with a PhD (Doctor of Philosophy). He teaches and conducts research in the areas of user experience, human centered design, and ethnographic design approaches. Member of the Japan ISO/TC159 (Ergonomics) Committee for International Standards for human centered design and accessibility. Chair of the draft committee for JIS standards relating to human centered design.
Satoshi Furutani
Satoshi FurutaniSatoshi Furutani
After joining NEC, he was involved in the planning and social implementation of products and services to connect people and cyber technology, and he is currently in charge of expanding business in Japan and overseas as the manager of the hearable business. His hobby is traveling solo on his motorcycle.

Hearable devices open the way to new user experiences

First off, please tell us about hearable devices.

The concept of these devices is a hands-free and eyes-free computing style. Since they can be worn in the ear like a wireless earphone, both hands are free, and since there is no operation screen, nothing occupies your line of sight and prevents you from living your life. Additionally, it is possible to conduct personal authentication using the acoustic characteristics of the ear and maintain a continuous internet connection, so you can constantly sense the conditions of a particular user. The inside of the device is full of sensors. The microphone and speaker are directed inwards, and when the device is worn, it analyzes the acoustic characteristics of the user’s ear canal to execute personal authentication. The motion sensor can even tell the posture of the person wearing the device and differentiate when the user is sitting and standing. The device can also measure the position of the user with high precision when they are in areas without GPS signals, such as inside a building or underground, and by equipping it with an optical sensor, it can take steady measurements of vital data such as blood flow and pulse waves.

These days, devices and services using a voice interface are popular. With the spread of Amazon Echo, the fire was ignited in the US, and just the other day, Line announced its new Clova service, so we can expect the age of voice UX to come to Japan soon. When you think of an interface as speaking and listening, we must think about how accustomed to the technology the individual is and the culture and how it might affect the experience. I believe we need to create a culture for the use of this type of device to spread. NEC’s flip phones were a great hit. I think the shape that brings the microphone to the user’s mouth fit the cultural etiquette in which the user could show the people around them that they were currently speaking on the phone. Nowadays, we see more people using earphones and a mic when talking instead of holding the phone up to the ear. Perhaps the foundation for the spread of hearable devices is already in place. I personally like voice UX, so I write emails using voice typing and listen to the news while walking by using the text-to-speech function. Hearable devices are potentially life-changing, and I have high expectations for this technology.


Hearable devices connect things and contexts

What kind of UX do hearable devices make possible, and how will this change the services?

The devices will make it possible to monitor the situation of the user by determining who they are and where they are at any point in time. You can say that it senses the context of the user so it can provide information at the appropriate time, expanding the scope in which users can use a service to a variety of situations regardless of whether they are at work or not. In these cases, the important thing is that the device senses and determines the situation and can choose the right timing to provide information. The information provided can be useful or an unwelcome favor depending on the person and the context. To make this part clearer, we have AI technology called active learning. This is the technology to learn about a person through dialog. Hearable devices may also need technology for understanding the preferences of the user through voice dialogs with a chat bot.

I just gave a presentation the other day at a conference regarding information on unwelcome favors where we conducted an experiment to see the differences in the way users receive information when it can be provided at the right times based on context depending on the individual characteristics of the users. What we discovered was that it was necessary to not just read the context and push out information but also to understand the attitudes toward everyday actions. In this sense, I see very high potential for services using hearable devices because they are continuously connected and can understand the user’s behavioral characteristics.


The potential of hearable devices

Please tell us what kind of people will use hearable devices, how its use will spread, and your vision for the future.

I think it will start spreading from the B2B market. The labor population is currently declining, so if we can get them to use this device as part of a wearable worker support system, it can meet the needs of employees and the information system. Additionally, in the automotive world, we are seeing advancements not only in electrification and automation but also computerization, and as things develop from connected cars to connected drivers, I can see hearable devices fitting in.

I think they can be used in many places. In the service industry, digital wireless headsets are already in general use. The information that is currently flowing out of the headsets can be personalized and provided at the necessary points if hearable devices are used, and this will improve the quality of services. I am also interested in the indoor positioning function. I conduct research on “altruistic UX,” which is UX for helping people in need. When you consider the system for mutual aid as an emergency response in the event of cardiac arrest of a resident in a high-rise condo, GPS information cannot tell you which floor they live on. Wearable devices may be able to resolve issues like this.

Eventually, we would like to establish an optimized computing style to connect the real world with the cyber world. The scene we see now of people staring down at their smartphones on the train is a behavior we only started seeing in the past decade. I believe this is not the final shape of how things will be. In the future, by creating a style and environment where people can use both their hands and both eyes freely, have continued connection to the network, and receive timely and personalized assistance, we can bring about innovation in services and workstyles.

When we are doing something on a laptop computer, and we need to stop to go somewhere, we sometimes carry the computer with it half open. This comes from our desire to keep the context or in this case, to maintain the internet connection. With hearable devices, I believe we can bridge the subtle lag between the context of real behaviors and the online context, which cannot be achieved with other devices.

Related article


The future of corporate wellness - a healthtech perspective

FiNC Founder and CEOYuji Mizoguchi
NEC Hiroaki Nakano
Learn more

Accurate demand forecasting through AI technology

Deputy General Manager, Digital Strategy Department, Corporate Planning Division, Asahi Breweries, Ltd.Kaoru Yamamoto
NEC Shizuka Kanie
Learn more