System and method for transforming fingerprints to improve recognition

[0002] 1. Field of the Invention

[0003] The present invention generally relates to image processing and, more specifically, to a system and method for transforming fingerprint images to improve recognition by distortion removal.

[0004] 2. Background Description

[0005] For many years, fingerprints have been used to identify individuals. There exist systems for accomplishing automatic authentication or identification of a person using images of the fingerprint. A fingerprint of a person comprises a distinctive and unique ridge pattern structure which allows for such identification of a person among millions. The image of a fingerprint can be recorded in one of a number of ways. Traditionally, fingerprints were recorded by inking the finger and rolling it in a controlled way across a piece of paper. Fingerprints can also be captured by dabbing the inked finger onto paper.

[0006] Non-ink processes have also been developed to leave a permanent record, and fingerprints left in sweat and natural oils on a variety of substances can also be enhanced and photographed.

[0007] More recently, automatic fingerprint identification (AFI) systems have been developed, and for quick capture of a fingerprint “live scan” devices have been used. These sensors, typically ultrasound, electronic or optical, can capture a fingerprint image directly from the finger without the intermediate paper or ink stage.

[0008] Regardless of how the fingerprint image is obtained, the AFI system is used to compare images and determine if a query print matches a stored print in a database. However, the process of imaging by any method inherently introduces a distortion into the representation. Since the distortion for the query print and database print cannot be exactly equal the distortion makes it hard to compare the fingerprints.

[0009] Distortion is introduced in a number of ways, but for most sensors it comes about by trying to capture an image of a surface in three dimensions with a two dimensional imaging device. As the finger is pressed against the paper or against the imaging device (typically an integrated circuit, prism or other optical element), the curved, elastic finger is compressed and its surface is deformed into a flat plane (in some cases a surface curved in one direction, but never a surface of the exact shape of the finger). This distortion can be minimized by trained personnel controlling the way the finger is dabbed or rolled, but because of the transformation from a three-dimensional surface to a two-dimensional surface, some distortion is inevitable, and the image will not be consistent when a subsequent print is taken of the same finger.

[0010] It is possible to scan the finger shape in free space, without pressing it against a flat surface (e.g., with a laser scanner, a binocular optical system, etc.), but there is still inherent variability in the image shape because the shape of the finger changes (depending on the current water content of the skin for instance).

[0011] Thus, it can be seen that fingerprint images are inherently variable because of distortions. These variations make the matching of two fingerprints difficult since two images of the same fingerprint will never be the same. It is thus hard to tell whether differences between two prints are due to noise processes like the distortion mentioned above, or due to a difference in identity between the fingers represented.

[0012] Previous AFI systems have not tackled the problem of distortion. While a trained professional fingerprint officer can make allowance for the presence of distortion, computer systems are misled by the distortion which changes the ridge spacings, the distances between features arid the relationships between features (e.g., points collinear on one image may not be on the next because of distortion).

[0013] Since AFI systems rely on this kind of representation of fingerprints (inter-feature relations, distances and ridge spacing), the distortion means that prints from the same finger do not look similar and the match between the prints may be overlooked. Historically, such systems have either relied on trained operators to reduce the distortion to a minimum, allowed multiple presentations of a finger to the device, with the hope that one with minimal distortion will be obtained, or the systems have been used in situations where errors are tolerated.

[0014] Each of these “solutions” bypasses the real problem and has inherent disadvantages. Trained operators are expensive and defeat much of the object of automating the rest of the system. Multiple presentations are time consuming and annoying to the users, increase false acceptance error rates (allowing an impostor multiple attempts a being recognized) and increase wear on the sensor. Doing nothing about the problem lends to high error rates, either by rejecting legitimate matches, or if the system is made lax to avoid that problem, by falsely accepting more impostors.

[0015] It is therefore an object of the present invention to provide a system and method to reduce the amount of variation in a fingerprint image due to distortion from elastic deformation of the finger surface and projection onto a planar coordinate system.

[0016] The invention works by estimating the amount and extent of distortion, and inverting the process in a systematic way, so that:

[0017] 1. the amount of distortion is minimized; and

[0018] 2. the distortion present is consistent, so that different images of the same print will have very similar distortion and thus matching of the fingerprints will be reliable and unaffected by the distortion.

[0019] According to a preferred embodiment of the invention, there is provided a system for estimating the distortion present in a fingerprint which comprises a computer with one or more central processing units, one or more memories, and one or more input devices. An input device captures one or more images, and the captured image is converted into a representation of the locations of the fingerprint ridges or valleys. An estimating function estimates the distance between two ridges (valleys) on the image, and the estimates of the ridge distances are combined. Based on the combined estimates, the fingerprint image is transformed to remove estimated distortion.

[0020] The method according to the invention reduces the amount of distortion present in a fingerprint representation by first capturing a fingerprint image. The captured fingerprint image is distorted by the capture process. The captured fingerprint image is preprocessed, and information of the locations of ridges or valleys in the captured fingerprint image are extracted. The ridge or valley spacing throughout the captured fingerprint image is estimated, and based on this estimate, an average ridge or valley spacing in the captured fingerprint image is estimated. Then, localized warps in the captured fingerprint image are estimated to normalize ridge or valley spacing. Minutiae information from the captured fingerprint image is extracted, and local warps are applied to the estimated minutiae locations without distortion. Finally, existing minutiae matching algorithms are applied to establish fingerprint identity.

[0021] The foregoing and other objects, aspects and advantages will be better understood from the following detailed description of a preferred embodiment of the invention with reference to the drawings, in which:

[0022] FIG. 1 is a block diagram of one preferred embodiment of the fingerprint distortion removal system according to a preferred embodiment of the invention;

[0023] FIG. 2 shows a fingerprint image taken under controlled conditions, minimizing the distortion;

[0024] FIG. 3 shows the fingerprint image of FIG. 2 with distortion due to elastic deformation;

[0025] FIG. 4 shows the fingerprint of FIG. 3 with the distortions removed by the method according to the invention;

[0026] FIG. 5 is a flow diagram of the procedure used to estimate the fingerprint distortion as implemented on the system of FIG. 1; and

[0027] FIG. 6 shows the ridge-spacing measurements used to estimate the distortion.

[0028] Referring now to the drawings, and more particularly to FIG. 1, there is shown a block diagram representation of a general computer hardware environment that is used as the image processing system 100. This system includes a computer 110 which may be a personal computer (PC), such as one of International Business Machines Corporation (IBM) Personal System/2 (PS/2) family of Personal Computers, a work station, such as IBM's RISC System/6000 workstation or workstations in a network. The invention may be implemented on a variety of other and equivalent hardware platforms such as, for example, an n-WAY IBM SP2 machine with AFS for storage. The SP2 is a collection IBM RS/6000 engines running under a common umbrella and the AIX operating system (IBM's version of the UNIX operating system) and connected with a high speed switch. It could also be implemented on other hardware platforms including, but not limited to, mini computers, stand alone UNIX or Windows NT workstations, or, or mainframes, including IBM AS400 and ES9000 computers.

[0029] The computer 110 includes one or more central processing units (CPUs) 115, which may conform to any general computer architecture (e.g., Intel X86 architecture or a reduced instruction set computer (RISC) architecture such as the IBM PowerPC® microprocessor). The CPU 115 is attached to a system bus (not shown) to which are attached a read/write and/or random access memory (RAM) 120 that can include one or more cache memories, a read only memory (ROM) 140, and an input/output (I/O) adapter 125. The RAM 120 provides temporary storage for one or more application program programs 126 containing code and/or data while the ROM typically includes the basic input/output system (BIOS) code of the computer. One or more Direct Access Storage Devices (DASDs), here represented by a hard disk drive 130, are also connected to the CPU by an appropriate adapter (not shown). The hard disk drive 130 typically stores the computer's operating system (OS), such as IBM's OS/2 operating system for PCS or AIX for workstations, and various application programs, data, and/or databases. These databases include intermediate results and fingerprint image data 135. Typically, the I/O adapter 125 has attached to it a keyboard 127, a mouse 128, and/or other user interface devices (not shown).

[0030] The computer 110 also can include a display 138, here represented as a cathode ray tube (CRT) display but which may be a liquid crystal display (LCD) or other suitable display which supports a graphic user interface (GUI). The display 138 is connected to the system bus via a display adapter (not shown).

[0031] The computer 110 is also interfaced with a frame grabber 150 and an image acquisition device, e.g., a camera 160 and imaging subsystem 180, to capture a live scan fingerprint image onto the computer memory/disk. Alternatively, the computer might be communicating with a document scanning device 165 that scans the fingerprint image from a document like an inked fingerprint card 170. Any other known means can be used to enter a fingerprint image to the database 135, e.g., transmitting an image over a network 166 from other equivalent computers 110A.

[0032] The hardware for system 100 and equivalents of these systems are well known to those skilled in the art. Personal System/2, PS/2, OS/2, RISC System/6000, Power Parallel System, SP/x, PowerPC, and IBM are trademarks of the International Business Machines Corporation.

[0033] FIGS. 2, 3 and 4 serve to illustrate the problem solved by the present invention. FIG. 2 shows a fingerprint which is largely undistorted. FIG. 3 shows the same fingerprint but badly distorted due to elastic deformation.

[0034] It will be observed that the fingerprint in FIG. 3 is in part contracted with respect to the fingerprint in FIG. 2 and, as a result, the distances between ridges have become closer in some parts of the fingerprint and farther apart in other parts of the fingerprint. Therefore, even though the two fingerprints are in fact the same, the distortion of the fingerprint in FIG. 3 makes it difficult in an AFI system to make a positive match with the fingerprint in FIG. 2.

[0035] The restored version of the distorted print is shown in FIG. 4. This fingerprint is remapped by the process according to the invention to be a more accurate representation of the fingerprint in FIG. 2.

[0036] FIG. 5 is a flow diagram of the method of processing fingerprints implemented on the system shown in FIG. 1. The process begins in function block 510 by acquisition of an image of the fingerprint to be processed. This may be accomplished in any of the several ways described with reference to FIG. 1. The image is processed in function block 520 to perform ridge finding. In this implementation, steps shown in function blocks 510 and 520 correspond to prior art. In particular the processes in function block 520 are described by N. K. Ratha, K. Karu, S. Chen, and A. K. Jam in “A real-time matching system for large fingerprint databases”, IEEE Transactions on Pattern Analysis and Machine Intelligence18(8): 799-813 (1996). These processes involve algorithms to identify the locations in the image which correspond to ridges in the fingerprint. The step shown in function block 520 results in a new image whose pixels are in one of two states, indicating whether the pixel is part of a ridge or not. The ridges are one-pixel-wide chains of pixels. U.S. patent application Ser. No. 08/837,069 filed Apr. 11, 1997, by Rodulf Maarten Bolle, Scott Eric Colville, and Sharathchandra U. Pankanti for “SYSTEM AND METHOD FOR DETERMINING RIDGE COUNTS IN FINGERPRINT IMAGE PROCESSING” gives a detailed description of the generation of ridge images. The disclosure of application Ser. No. 08/837,069 is hereby incorporated by reference.

[0037] After the image processing of function block 520, the process branches to perform parallel operations. In the main branch, ridge spacing estimation is done in function block 530. Once the ridge spacing estimation occurs as shown in the step in function block 530, a count accumulation is performed in function block 540. Then, as shown in function block 550, dilation estimation and smoothing take place. Finally, as shown in function block 560, a distortion removal map f(x) is calculated. In the parallel branch, from function block 520, a feature extraction function takes place in function block 570. The output of function block 560 is used in function block 580 to apply the calculated distortion removal map f(x) to the output of the feature extraction of function block 570.

[0038] In the preferred embodiment, the distortion map is estimated by making an assumption about the nature of the pattern on the fingerprint. The ridge pattern on the fingerprint is assumed to have a uniform spacing, p, between ridges. Thus, where two adjacent ridges run side-by-side, uninterrupted by breaks, bifurcations, crossings or high curvature, the shortest distance from any point on one ridge to the other ridge is assumed to remain approximately constant. It is also assumed that large distortions will only be seen in restricted areas of the print.

[0039] Given these assumptions, and a fingerprint image, any place where the distance between adjacent ridges deviates from that observed across most of the print must be an area of distortion. By estimating the local deviations from the mode ridge spacing, the deformation can be estimated.

[0040] To estimate both the typical ridge spacing for the whole print, and the local ridge-spacing in each area, the ridge separation is measured at a large number of points in the image in function block 530.

[0041] One method for estimating ridge spacing is best described with reference to FIG. 6. As shown in FIG. 6, a given ridge 610 in the fingerprint can be followed by standard algorithms, whether the ridges are represented as chains of pixels such as those resulting from a thinning algorithm, or as geometric lines or parametric curves. At regular intervals (the “search points” 620) along each ridge, the separation from the nearest ridge 680 is measured as follows.

[0042] To measure separation from a given search point 630, the system searches for another ridge in a direction 640 perpendicular to the current ridge at the search point 630. The search can be carried out either to both sides sequentially, or to just one side. At the point of intersection 650 with another ridge 680, the distance 670 from the given search point 630 to the intersection point 650 is measured. If the distance is too large to be that between two adjacent ridges (either by an absolute threshold, from knowing the physical limits on ridge spacings under deformation, or from a preliminary estimate of the ridge-spacing for this fingerprint), the intersection is discarded. If the ridge spacing is plausible, then the distance 670 becomes the ridge spacing estimate and is recorded, together with the location of midpoint 660 between the search point 630 and intersection 650. If no intersection is found, or the intersection is discarded, no ridge-spacing estimate is made for this search line.

[0043] By following all the ridges in this manner, and sampling sufficiently frequently, a large number of ridge spacing estimates, r′(x), are accumulated over the whole area, A, of the fingerprint. Averaging these estimates (the preferred embodiment chooses the mode length) gives an estimate, ρ′, of the midpoint between the ridges on the original fingerprint.

[0044] In addition to the overall estimate, there is now a large number of local ridge-spacing estimates. Differences between the local estimate and the overall ridge spacing indicate distortions. To reduce the effect of noise in the estimates, and to allow an estimate of the local ridge spacing at any point in the image, the local estimates may be smoothed with a technique such as that of Parzen windows (See chapter 4 of Duda & Hart, Pattern Classification and Scene Analysis, Wiley Interscience, 1973). In addition, in the preferred embodiment a vector estimate of the distortion is made by resolving the ridge spacing estimates into horizontal and vertical components.

[0045] The distortion at each point is modeled as an anisotropic scaling or dilation, which varies as a function of the position x. The separation between two ridges at any point is assumed to have a true length ρ, but to have been scaled by the distortion process At each point, we wish to know the magnitude of this scaling and its direction. Thus, we find the ratio of ρ′ to the local estimates, r′(x), resolving into orthogonal components. $\frac{{\rho }^{\prime }}{r^{\prime }\left(x\right)}=\frac{{\rho }^{\prime }}{r_x^{\prime }\left(x\right)}\hat{i}+\frac{{\rho }^{\prime }}{r_y^{\prime }\left(x\right)}\hat{j}$

[0046] Where î and ĵ are the unit vectors in the x and y directions respectively. The unit vector in the direction separation is also found. ${\hat{r}}^{\prime }\left(x\right)=\frac{r^{\prime }\left(x\right)}{\Vert r^{\prime }\left(x\right)\Vert }$

[0047] and resolved {circumflex over (r)}′_x={circumflex over (r)}′_x(x)î+{circumflex over (r)}′_y(x)ĵ.

[0048] A set of bins is set up across the fingerprint area, typically one per pixel, or at regular intervals. For each bin, B(x), four sums are accumulated for any local estimates at points x falling into that bin (function block 540). The values accumulated are the sums of $\frac{{\rho }^{\prime }}{r_x^{\prime }\left(x\right)}\mathrm{and}\frac{{\rho }^{\prime }}{r_y^{\prime }\left(x\right)}$

[0049] and the counts {circumflex over (r)}′_x(x) and {circumflex over (r)}′_y(x).

[0050] In addition, since a near-vertical ridge spacing estimate provides little information about the horizontal component of the local dilation, it may be decided that the near-vertical or near-horizontal ridge-spacings be discarded or discounted when estimating the horizontal and vertical additions, respectively.

[0051] After the sums and counts have been accumulated for all the local estimates, each is smoothed by convolution with a two-dimensional Gaussian kernel, G( ), whose variance is determined by the number of samples available and the amount of error in the estimation process. The smoothed counts are then divided (in function block 550) to give estimates of the components of the dilation vector, d′(x), for each bin: $d^{\prime }\left(x\right)=\frac{{\int }_A\frac{G\left(x-y\right){\rho }^{\prime }}{r_x^{\prime }\left(y\right)}dy}{{\int }_{A}G\left(x-y\right){\hat{r}}_x^{\prime }\left(y\right)dy}\hat{i}+\frac{{\int }_A\frac{G\left(x-y\right){\rho }^{\prime }}{r_y^{\prime }\left(y\right)}dy}{{\int }_{A}G\left(x-y\right){\hat{r}}_y^{\prime }\left(y\right)dy}\hat{j}$

[0052] These dilations are estimates of the reciprocals of the stretching magnitudes in the î and ĵ directions for every point in the fingerprint. In the preferred embodiment, to reduce noise, the standard error of the mean is also estimated and the estimated distributions are combined with a prior probability distribution for the expected distortion (which is one), to give a maximum a posteriori estimate for the displacement.

[0053] When it is desired to invert the distortion process, any point in the original fingerprint image can be mapped to a new point by inverting the estimated distortion. Points x=(x,y) in the original image are mapped to points x′=f(x). If d′(x) is estimated for every pixel, its value is an estimate for the dimensions of that pixel in the undistorted image, since every image pixel is a unit square to be anisotropically scaled by d′(x), so the actual displacement of point x, is the cumulative sum of these dilations (function block 460). The dilations are accumulated outward from the center. Without loss of generality, assume that the origin is at the center of the image. Then, $f\left(x\right)=\sum _0^x{d}_x^{\prime }\left(x\right)\hat{i}+\sum _0^y{d}_y^{\prime }\left(x\right)\hat{j}$

[0054] The distortion removal can be applied to any representation of the fingerprint. The original bitmap image can be scaled by the dilation factors, as can a ridge image or any other derived representation with the same domain (such as an orientation map). A parametric curve representation of the ridges has its coordinates (e.g. the coordinates of the control points of a spline) remapped by f(x). FIGS. 2 and 3 show such a spline representation for the undistorted and distorted fingerprints, respectively. Similarly, feature points located in the image, such as core, delta or minutiae have their locations remapped by f(x), and relative features such as minutia angles or curvatures can be worked out in the new coordinates x′. The result of this remapping is illustrated in FIG. 6 which shows the restored version of the distorted print in FIG. 4. This remapped fingerprint is a more accurate representation of the fingerprint in FIG. 2, making a reliable identification more certain.

[0055] The assumption of uniform ridge spacing underlying these procedures is only an approximation. However, for most fingerprints, it seems to be a good approximation for the majority of the area considered. In practice, failure of the assumption does not matter because the “distortion removal” procedure will be applied consistently to all images of a fingerprint. Areas where the ridge spacing is lower than the average will always be enlarged. While the resultant image will then actually be further from the true picture, it will be applied consistently and thus two images of the finger will have the same enlargement and be comparable by a fingerprint matcher, with the advantage that any extra distortion on the prints will have been reduced, making the match easier than if no distortion removal had been carried out.

[0056] While the invention has been described in terms of a single preferred embodiment, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims. In particular, the procedures for estimating the distortions and inverting them may use a wide variety of methods and not necessarily the particular method described here.