New solutions that utilize video streaming are now being proposed in the area of public safety such as security services and emergency medical care. In the past, reliable video streaming required dedicated, high-cost communications facilities such as satellite communications and mobile base station vehicles, restricting utilization of this service. This paper discusses new Adaptive Video Streaming Control technology developed by NEC that can be applied to low-cost public communication networks that do not guarantee communication quality - such as mobile networks and the Internet - enabling them to reliably streaming high-quality video in real time. Also introduced in this paper is an application example involving special police units known as the “running police.” We believe that this technology will help raise the level of video streaming in the public safety field to one of practical usage and promote total utilization in the future.

1.Introduction

Rapid development in mobile networking technology and the proliferation of smart devices in recent years are leading to a rush of proposals for new solutions and services in the field of public safety, including security services and emergency medical services.

Security services typically rely on visual observation using CCTV and security guards. Now, NEC is proposing a solution which will capture and stream on-site video shot with wearable cameras worn by security guards. This will help minimize lost coverage caused by camera blind spots, while making it easy for on-site personnel to share images with security headquarters.

In emergency medical care, NEC is also proposing a support solution based on streaming of video captured with wearable cameras. For example, video images of incoming patients captured by a wearable camera worn by paramedics can be transmitted to the receiving hospital to help staff better prepare appropriate treatment even before the arrival of the patient, speeding up the intake process and potentially saving more lives.

The effectiveness of these solutions depends on the ability to stream high-quality video images in real time. Until now, that has meant video streaming could only be handled by high-cost, dedicated communications facilities such as satellite communications and mobile base stations. Naturally, this has limited the ability of companies and organizations to take advantage of this technology. While low-cost public communications networks such as mobile networks and the Internet have been expected to eventually be able to offer similar levels of stability and reliability, these networks can still only provide what is known as “best-effort” streaming (that is, they cannot guarantee communication throughput). Consequently, reliable, affordable streaming of high-quality video remains difficult. In this paper, we introduce NEC’s Adaptive Video Streaming Control technology which makes it possible to reliably streaming high-quality video in real time - even via such best-effort networks.

2.Video Streaming Difficulties

In best-effort networks such as mobile networks and the Internet, communication throughput (data size that can be communicated per unit time) fluctuates substantially moment by moment depending on the radio wave environment and congestion conditions (other traffics). Since video uses a lot of data, video data may stall or get lost in the network in an environment where communication throughput is unstable. As a result, the streamed video images may suffer interruptions or be marred by noise (Fig. 1).

The typical solution to the video streaming problems that occur on a network such as stalling or data loss has been to reduce the image quality (bit rate and frame rate) to minimize the data size of video images and thereby minimize the adverse impact of network throughput issues. In practice, however, it is impossible to know how low to actually set the bit rate and frame rate. Settings that are too low reduce the image quality to a level that it not useful for operations.

3.Adaptive Video Streaming Control Technology

NEC has developed Adaptive Video Streaming Control technology that will solve these issues in video streaming. This technology uses advanced algorithms capable of predicting in real time the moment-by-moment fluctuations in communication throughput. These predictions can be then used to dynamically control the video bit rate and frame rate as required to assure smooth real-time streaming of high-quality video even in an unstable communication environment.

Communication throughput prediction is critical to the performance of NEC’s Adaptive Video Streaming Control technology. We have succeeded in the development of a methodology able to predict fluctuations of communication throughput anywhere from 1 to 3 minutes beforehand with 80% or higher accuracy. It does this by analyzing the throughput’s time sequence from the preceding period which can range from a few tens of seconds to 1 minute1). By analyzing a massive amount of measured communication data, we have been able to determine that communication throughput has a complex mixture of conditions where it fluctuates stably (stationary state) and conditions where it fluctuates violently (non-stationary state). This makes it possible to identify current throughput conditions (stationary or non-stationary state). We have also achieved high-precision real-time prediction according to momentarily fluctuating throughput conditions by mixing the stationary and non-stationary models to generate a prediction model (mixture model) for future throughput.

Fig. 2 shows prediction result examples of communication throughput in the future. The stationary state prediction result is shown in (a), and the non-stationary state prediction result is shown in (b). These graphs represent past throughput fluctuations and future expansion of probability distribution (stochastic diffusion). The graph in the stationary state indicates that the stochastic diffusion is insignificant and there is a high probability of stable communication continuing in the future. On the other hand, the graph in the non-stationary state shows that there is a probability of significant stochastic diffusion occurring, which may cause a substantial decrease in communication throughput.

Predicting the stochastic diffusion of communication throughput allows us to know the approximate volume of video data to be streamed. In the examples shown in the graphs in Fig. 2, the lower half of the parabola indicates the likelihood that communication throughput will decrease. When we can predict the decrease in communication throughput in this manner, we will also be able to predict the maximum volume of video data that can be passed through without it stalling or getting lost in the network. This also will allow us to calculate the appropriate video bit rate and frame rate of the video. Dynamic, real-time control of the video bit rate and frame rate makes it possible to deliver high-quality video without interruption or noise even in an unstable communication environment.

Photo 1 shows a comparison between video images streamed by a conventional video streaming system and by the new system incorporating the Adaptive Video Streaming Control technology in a typical mobile network environment. Conventional systems, which stream video with VGA (640 x 480) resolution at fixed rates of 3 Mbps and 10 fps, cannot reliably transmit video data if communication throughput decreases, generating noise as shown in (a) that makes the image incomprehensible. The new system incorporates an HD (1280 x 720) resolution operating with adaptive rates of 0.3 to 5 Mbps and 2 to 30 fps to control the rates at which video data is streamed. This assure stable streaming of high-quality video images as shown in (b), ensuring that images are clear and comprehensible even in the same low-quality communication environment.

4.Security Application

As mentioned previously, our Adaptive Video Streaming Control technology is expected to play a particularly active role in the field of public safety. An example of this is the “running police” validation experiment we conducted in cooperation with the Tokyo Metropolitan Police Department (TMPD).

“Running police” is one of the “overt” policing methods used at major marathon events, in which police officers sporting wearable cameras participate in the race as runners and keep watch over the course (Photo 2). The Tokyo Metropolitan Police Department wanted to stream and utilize video images captured from the runner’s perspective to police headquarters, but found themselves hamstrung by the problems with video streaming via mobile networks that we discussed earlier. By applying NEC’s Adaptive Video Streaming Control technology to the video streaming system, the TMPD was able to stream high-quality video images in real time with continuously assured stability. The success of this experiment has led to rapid growth in security solutions that utilize the video distribution from wearable cameras.

5.Conclusion

This paper has introduced NEC’s Adaptive Video Streaming Control technology, which is capable of ensuring consistent, reliable distribution of high-quality video images in real time even in an unstable communication environment, and discussed its application to public safety. Today, as utilization of video images expands in the fields of security, paramedic services, and disaster prevention, we anticipate that the importance of this technology will continue to grow in the future.

Reference

1) H. Yoshida et al., “Constructing stochastic model of TCP throughput on basis of stationarity analysis,” Proc. of 2013 IEEE Global Communication Conference, pp.1544-1550, 2013.12

Authors’ Profiles

YOSHIDA Hiroshi
Doctor of Engineering
Principal Researcher
System Platform Research Laboratories

KAI Natsuki
System Platform Research Laboratories

KANETOMO Dai
Principal Researcher
System Platform Research Laboratories

SATODA Kozo
Senior Manager
System Platform Research Laboratories

Reference