NEC strengthens Heterogeneous Mixture Learning Technologies that automatically discover massive patterns hidden in big data - Accommodates super-large-scale demand forecasts on millions of analysis targets -

*** For immediate use June 19, 2014



NEC has achieved fast and high-precision analysis of super-large-scale data by utilizing its Heterogeneous Mixture Learning Technologies that discover massive patterns hidden in big data to automate processes that data analysts with advanced expertise have until now performed manually.

NEC has developed new analysis techniques that automate trial-and-error processes that analysts have traditionally had to perform manually in order to discover patterns in analyzed data, namely "partitioning data" based on conditions such as day of the week or weather and "combining factors" that are important in making forecasts. This enables super-large-scale demand forecasting (for example, sales forecasts by store and product, energy demand forecasts, etc.) on millions of analysis targets, something which has been limited by conventional manual techniques.

Today, there are heightened expectations for technologies that rapidly analyze big data and help make future predictions. In 2012, NEC independently developed Heterogeneous Mixture Learning Technologies that enable high-precision big data analysis (Note 1).By detecting the patterns hidden in big data, breaking them down into multiple simple expressions and combining factors based on the analysis one wishes to perform, Heterogeneous Mixture Learning Technologies achieve advanced prediction and differentiation.

Previously, data analysts possessing advanced expertise had to manually perform the processes of partitioning data and combining factors. For instance, when forecasting sales in a retail business, an analyst would have to repeatedly perform the data partitioning process based on conditions such as day of the week and weather, in addition to carrying out statistical analysis of sales trends in different store locations. Moreover, when investigating how an important factor combined with a certain product can influence the sales of other types of products, an analyst would have to form and evaluate hypotheses for each product in advance.

1. High-speed search of data partitioning conditions
   
   
   
   
   NEC has developed techniques to efficiently and automatically perform partitioning of data subject to analysis that was previously performed manually. By simultaneously searching for multiple patterns hidden in large volumes of data (formulas represented by combinations of multiple factors) and the conditions that establish those patterns, the optimal conditions for partitioning data can be quickly identified from among vast quantities of conditions.



2. Automated optimization of the factor combinations needed for prediction and forecasting
   
   
   
   
   NEC has developed techniques to automatically identify the optimum factor combinations needed for prediction and forecasting from among a large volume of candidate factors extracted from data subject to analysis as an interim step in the process described in Feature 1. This enables the investigation of factor combinations that were not possible when using conventional manual techniques and the development of highly precise predictions.

(Note 1) June 22, 2012 "NEC Technology Automatically Detects Patterns from Big Data"
http://www.nec.com/en/press/201206/global_20120622_02.html

(Note 2) Conference Presentation
International Conference on Machine Learning (ICML) 2014：http://icml.cc/2014/
Research Paper URL : http://jmlr.org/proceedings/papers/v32/liub14.pdf