In recent years, deep learning has become the driving force of AI technology, propelling many of the innovative breakthroughs that are transforming the world we live in. The field of biometrics is no exception. Deployment of deep learning is well underway ― especially in face recognition. More recently, deep learning has been applied to fingerprint matching, taking advantage of capabilities like image enhancement and feature extraction. In this paper, we will examine the current status of the application of deep learning to fingerprint matching and consider the future prospects of this technology.

1. Introduction

Biometrics is now essential to our daily life and is widely established as a reliable form of authentication that can be used for identification and access control. Biometrics is already widely used by public agencies including crime investigation¹⁾, national ID²⁾, immigration control, and election administration. One of the best known and longest used forms of biometrics is fingerprint matching, which can now be fully automated for rapid, accurate identification. For more than forty years, NEC has been a leader in the development of automated systems for matching of latent fingerprints in criminal investigations and has been ranked number one in fingerprint matching accuracy in benchmark tests conducted by one of the world’s top technology standards organizations³⁾.

One of the most commonly used methods for fingerprint matching is to compare the positions and directions of a ridge bifurcation or a ridge ending on a fingerprint ― plot points called minutiae. Using the minutiae information to calculate the similarity between fingerprints makes matching possible. While this method has changed very little since its inception, NEC has been conducting research to incrementally improve performance at a more granular level for almost half a century. This involves discovering ways to extract minutiae data from fingerprint images with higher precision and more accurately assessing whether that data belongs to the person in question or some other person. To achieve this, we have been working on noise elimination technology⁴⁾ and on improving feature extraction and matching algorithms. Over the years, these incremental improvements in accuracy and performance have made possible today’s high-precision matching in large-scale databases.

However, the rule-based system that has evolved over many years through the accumulation of algorithms is now undergoing a radical change thanks to deep learning. For the first time in decades, the fingerprint matching paradigm is about to shift. In this paper, we will introduce a number of use cases and application methods that show how deep learning is transforming fingerprint matching and discuss what that means for the future of this technology.

2. Use Cases of Deep Learning for Fingerprint Matching

2.1 Pattern classification

Fingerprint patterns are roughly classified into four types according to their ridge structures, as shown in Fig. 1. This classification is used to identify matching targets. In conventional automatic pattern classification, ridge structures are analyzed heuristically. NEC has succeeded in achieving the same level of classification accuracy by using deep learning. We are now conducting research and development (R&D) into pattern classification ― which offers higher usage value ― at higher accuracy.

2.2 Specification of digits

When a digit ― the thumb of the right hand, for example ― is specified on a fingerprint image, in addition to identification of the matching target, errors, if any, in fingerprint collection can be detected. Experienced fingerprints examiners can easily identify digits from fingerprint images. However, because that knowledge cannot easily be turned into algorithms, the technology for automatic digit specification was never developed. Deep learning has changed that particular reasoning as it has the potential to identify digits with the same degree of accuracy as experienced fingerprints examiners. We are currently analyzing a methodology to visualize deep learning judgment criteria to achieve a system that can identify digits with greater accuracy than human experts.

2.3 Classification of abnormal data

In biometric systems, it is extremely important to maintain the quality of data for higher matching accuracy. Achieving this would, for example, require coverage on a national scale using a national identification system. However, because this would also require many system operators, the improvements in accuracy achieved by using a national system may be offset by the difficulty in maintaining a high level of training when the number of staff increases, and that could result in the collection of poor-quality specimens. For example, samples from the left and right hands might be inverted. To counter problems of this nature, a key element of any large-scale fingerprint matching system would be a function that can detect erroneous input.

Deep learning is especially useful in this regard. It can be used to analyze incoming images and issue warnings when the possibility of erroneous input is high.

2.4 Detection of central axes

In fingerprints, a specific point called a central axis can be uniquely defined in most patterns. As shown in Fig. 2, the position of the central axis and its direction are used as points of comparison to start a search for a match.

If an error occurs in setting the central axis, the matching accuracy will be compromised. To avoid this, fingerprints examiners check the central axis set by conventional algorithms and make corrections if necessary. Deep learning can use the corrected central axis and fingerprint images to derive a central axis that is more correct than when performed by a conventional system.

2.5 Detection of fingerprint position

A fingerprint scanner usually scans four fingers simultaneously and puts them in the same image. To effectively match four-finger images, a method to detect each finger is widely used for management of fingerprint information. This differentiation of fingers is essential for high-precision matching. If it fails, matching itself will be impossible.

Currently, a rule-based method is commonly used. However, detection of fingerprint position using deep learning as shown in Fig. 3 is now possible, achieving a much higher level of accuracy than the conventional method.

2.6 Extraction of feature points using ridge recognition

Ridge recognition of fingerprint images is the core technology of the fingerprint matching systems currently in use. Ridge recognition can also be handled by deep learning by which a system can learn fingerprint images and skeletons by fingerprints examiners, thereby achieving high-precision ridge recognition.

Utilization of deep learning makes it possible to set fingerprint regions more correctly than the conventional method as shown in Fig. 4, effectively eliminating parts below the distal interphalangeal crease or any text in the background of the image, which are outside the fingerprint region, and thereby reducing feature points with erroneous factors.

2.7 Extraction of features

Utilization of deep learning also makes it possible to extract features that can be used for matching. By learning that the distance is small when the collected fingerprint is the same as the fingerprint image and that the distance is large when the collected fingerprint is different from the fingerprint image as shown in Fig. 5, features that can be used for matching can be extracted. Unlike a conventional rule-based matching method, simple distance calculation makes it possible to match fingerprints, substantially reducing matching costs.

3. Current Utilization of Deep Learning

3.1 Deep learning in combination with conventional methods

While an increasing number of deep learning–based methods are already suitable for practical use, we have not yet reached the point where deep learning–based methods can replace conventional rule-based methods. Efforts are now underway to improve the accuracy of fingerprint matching systems by combining the conventional method with the deep learning–based method. For example, the feature extraction method using ridge enhancement described in section 2.6 is now being combined with its conventional counterpart for fusion matching.

Fusion matching combines multiple feature extraction methods to improve accuracy. Conventional methods perform fusion matching by using various rule-based methods. Higher accuracy can be expected by combining a conventional method with the deep learning–based method. In fusion matching, the lower the correlation between the component methods, the greater the complementarity there is ― that is, each method compensates for the weakness of the other method. For this reason, fusion matching promises to be much more reliable than any of these methods would be on their own. The key to improving accuracy is how to combine methods in each stage of feature extraction from fingerprint detection to ridge enhancement.

3.2 Utilization of deep learning for multi-stage matching by using macro data

Whereas micro data like minutiae data refers to very specific aspects of a single feature, macro data refers to features that aggregate a wide range of data including the entire fingerprint. As described in section 2.7, deep learning can extract features from the entire image and achieve very high-speed matching. In terms of accuracy, however, deep learning–based matching is not yet as good as minutiae matching. Currently, deep learning is increasingly used as part of multi-stage matching.

To increase the speed of tenprint inquiry, rule-based macro information is conventionally used at an early stage of multi-stage matching. Efforts have been made to replace this stage with the deep learning–based method and use it as a filter on an early stage of minutiae matching. The fruition of these efforts is that the deep learning–based method has achieved accuracy superior to the conventional rule-based macro information matching method by using higher-speed comparison processing to revolutionize the speed of fingerprint matching.

4. Prospects for Deep Learning–Based Fingerprint Matching

4.1 Accelerating the transition from the rule-based method

The power of deep learning is becoming manifest as new studies produce results in which the deep learning–based method surpasses the conventional rule-based method. Improvements in accuracy are accelerating and the conventional rule-based method is increasingly likely to cease to be necessary at some point in the future. The deep learning–based method has already successfully performed detection and judgement at relatively high speeds with a reasonable degree of accuracy. As accuracy continues to improve, deep learning’s speed advantage should encourage more and more users to make the change.

Deep learning’s ability to accurately extract and match features is also improving rapidly. So much so that minutiae matching ― which is now essential ― may be deemed unnecessary in the near future for fingerprint matching when using fingerprint databases, where the quality is relatively stable, and will be replaced by the deep learning–based method. On the one hand, minutiae matching is extremely accurate, but on the other hand, the speed at which matches are made is slow and the number of features is large. As a result, this is likely to further advance the transition to the deep learning–based method when higher-speed matching on a larger scale is necessary.

4.2 Self-evolution by automatic learning

It is a well known fact that the accuracy of deep learning can be improved through learning from data that comes as close to actual real-world conditions as possible. Due to the characteristics of a fingerprint matching system, operators often check judgments deemed to be correct. In other words, correct data accumulates over time as the actual system operates. For example, features for macro information can become usable as a pair of correct judgments only when operators verify the identity of the person to whom the fingerprints belong. By learning features that can distinguish between the person in question or some other person more clearly, accuracy can be improved. As the process of deep learning–based matching continues ― that is, as the AI evolves ― accuracy will continue to improve on its own.

5. Conclusion

In the past few years, deep learning has become one of the most dynamic forces driving technological change, shattering traditional concepts and methods in almost every aspect of modern life. This is even more so for such heavily data-dependent fields as fingerprint matching. The incorporation of deep learning into fingerprint matching technology is already underway, and its importance is expected to grow rapidly in the years to come. More than forty years have passed since NEC first began R&D into fingerprint matching technology. Now that technology is poised for a paradigm shift as deep learning challenges minutiae analysis, which has for decades been the core of fingerprint matching technology. By continuing to make the most of deep learning, NEC will continue to develop higher-speed, higher-precision fingerprint matching technology that we hope will contribute to the creation of a safer and more secure society.

Reference

Authors’ Profiles

Related URL