As the cost of infrared cameras comes down and their performance goes up, demand is surging and user requirements are becoming ever more diversified. As the market for infrared cameras continues to expand, these cameras are now being widely deployed as non-destructive inspection devices to ensure public safety and security. This paper discusses the image processing technology used to obtain the higher resolution required for non-destructive inspection and looks at different examples of infrared camera applications.

1. Introduction

Thanks to their ability to visualize by detecting infrared rays irradiated from the subject, infrared cameras offer capabilities not available with visible-light cameras, such as shooting in the dark or measuring the subject’s temperature. Over the past few years, fast-improving performance and falling prices have spurred rapid expansion in the use of infrared cameras in a wide variety of fields. Accompanying the expansion of the market has been a growing demand for a wider and more diverse range of capabilities. For example, in the Japanese market, infrared cameras, which have conventionally been popular in research and development fields such as temperature measurement, are now being deployed as non-destructive inspection devices in areas such as maintenance and management of roads and structures such as factories. Today, infrared cameras are a critical component that helps to protect the safety and security of individuals and of society as a whole, and also accelerates to achieve NEC’s vision, “To realize an information society friendly to humans and the earth.”

This paper discusses image processing technology designed to increase the resolution of infrared cameras - something that is essential for non-destructive inspection and other new applications.

2. Improved Spatial Resolution Using Multi-frame Super-resolution Processing

One of the applications where infrared cameras have proven invaluable is in detecting interior cracks in concrete. Cracks inside the concrete affect the temperature distribution on the surface of the concrete, making it possible to detect abnormalities inside concrete using an infrared camera. However, measurement objects that are the target of structure diagnosis are often located high above and far away, so high spatial resolution and temperature resolution capabilities that enable detection of minute temperature distribution from a distance are required. Infrared cameras with high resolution are not only suitable for shooting subjects that are high and distant, they also enhance operation efficiency because they can shoot a wide area at once without decreasing spatial resolution. Yet despite the obvious benefits to be obtained by increasing resolution, development of infrared imaging sensors has focused on narrowing pitches and reducing prices rather than increasing pixels. As a result, the number of pixels in commercially available cameras has not increased significantly over the past few years. Moreover, those infrared cameras that are equipped with high-resolution sensors are expensive, making it difficult to expand the market.

To meet the market’s needs for higher-resolution infrared cameras at lower prices, Nippon Avionics has equipped its infrared thermal imaging camera with a multi-frame super-resolution processing function that uses software processing to increase the number of pixels and improves spatial resolution.

An overview of how multi-frame super-resolution processing works is shown in Fig. 1. This technology restores the original subject detail by improving resolution and increasing spatial frequency while reducing blur and image noise. This effect is achieved by synthesizing several consecutive low-resolution images using a sophisticated algorithm to generate a single super-resolution image. By precisely overlapping multiple frames and utilizing deviation of less than one pixel caused by camera shake to interpolate the information between the pixels, this technology actually makes it possible to quadruple the number of pixels and improve spatial resolution by 150%. Fig. 2 shows an example of the application of super-resolution technology in an infrared camera with resolution of 320 × 240 pixels. It is evident that minute temperature changes are now rendered more clearly. Super-resolution image processing makes possible the acquisition of higher-resolution data using existing sensors with conventional resolution. Because it updates rather than replaces existing system, it keeps the initial costs of introduction to a minimum. Moreover, it can also meet the requirements for higher temperature resolution because random noise can be reduced by overlapping multiple frames.

A camera in which this function has been implemented, Nippon Avionics’s InfReC R500 Series (Photo 1) is equipped with a 640 × 480 pixel image sensor, yet is able to record still images with 4 times as many pixels (1280 × 960) in super-resolution mode. In other words, it means that high-quality images with 1.2 million pixels can now be recorded with a camera at a price range of 640 × 480 pixels. Fig. 3 and Fig. 4 show samples of the images shot with the InfReC R500. In these images, quarter sections of the frames have been cut out and expanded to make it easier to see the actual effect of the improved resolution that multi-frame super-resolution processing makes possible. These images make it clear that sharper images with higher spatial resolution can be obtained using super-resolution processing.

However, because multi-frame super-resolution processing requires very advanced computation, it is necessary to incorporate a high-specification CPU and memory comparable to those used in computers. Germany’s Testo took the lead in applying this technology to an infrared camera, but they did it using PC software. Nippon Avionics is the first to implement this function directly in an infrared thermal imaging camera. The technology developed by Nippon Avionics optimizes the computation process to reduce processing time, finally making it possible to incorporate this function in a camera and enables accurate restoration of original subject detail at high speed and with high precision on location. It is also important to note that the temperature accuracy of the restored image data fits within the specifications of the product.

Applicable across the full spectrum of infrared camera, from low-cost, low-resolution models to top-of-the-line high-resolution models, super-resolution technology will make it possible to develop new high-resolution models that offer the highest levels of resolution imaginable, while also pairing with low-erresolution sensors to produce low-cost models able to deliver remarkably high image quality. In addition, it is expected that further increases in the processing speed will make it possible to achieve real-time super-resolution processing with moving pictures, not just still pictures.

3. Application Examples of Infrared Cameras

Inspections of road bridges are introduced here as examples of how infrared cameras can be applied to structure diagnosis. Hammering tests are mainly performed in bridge inspections, but special vehicles are required as shown in Photo 2. Therefore, efficient detection of targets requiring inspection is an effective way to reduce the time and cost of hammering tests.

Fig. 5 shows the results of a bridge inspection. While no abnormalities can be detected in the visible-light image (a), the infrared camera image (b) clearly shows uneven temperature distribution. In this inspection, it was confirmed by a hammering test that an abnormality was definitely present (see (c)) in the area with unusual temperature distribution. When bridges are difficult to access with inspection equipment because of distance or height, the improvement of the resolution using multi-frame super-resolution processing is particularly helpful in detecting abnormal sections (see Section 2). The detection rate of abnormal sections can be expected to increase still further with improvements in the resolution of infrared sensors, progress in image processing technology, and increased measurement expertise.

4. Conclusion

In this paper, we have reviewed the advanced new technology used to improve the resolution of infrared cameras and looked at examples of how it can be applied in structure diagnosis. As the prices of infrared cameras are expected to drop further, while their performance continues to improve, existing markets will expand and new markets will develop. Going beyond the provision of infrared cameras, Nippon Avionics is also committed to helping increase the value of NEC Group’s contributions to public safety and security by working closely with our customers to expand our measurement capabilities.

The bridge inspection data used in the preparation of this paper was provided by West Nippon Expressway Engineering Kansai Company Limited. We are very grateful for their assistance.

Authors' Profiles