US 5,577,181 AGrant
Method for Autonomous Determination of Tie Points in Imagery
Issue Date:1996-11-19
•17 Claims
•6 Drawing Sheets
Abstract
A method for selecting tie point locations within a series of overlapping images. Initially the series of images are ordered to maximize the potential for successful phase correlations between adjacent images. Adjacent pairs in the ordered images are then phase correlated to determine the translational offsets between adjacent pairs. The overlapping regions in adjacent pairs are assessed for normalized cross correlation and initial candidate tie points are selected within the regions at a reduced resolution. The initially selected tie points are then phase correlated at full resolution and any surviving candidate tie point locations are further refined to sub-pixel accuracy.
Metadata
Assignee
- E-Systems, Inc.
Inventors
- Fenton L. Givens
- Ronald L. Bowden
Application Information
Application Number:US 4789725
Filing Date:1995-06-07
Priority Date:1995-06-07
Art Unit:242
Classifications
IPC:
G06F 1500
Field of Search:
395345135;133113;114
Patent Drawings (6 sheets)
Description
Technical Field
The present invention relates to imaging systems, and more particularly to a method for selecting tie points within a sequence of geographically overlapping digital images.
Background of the Invention
Image registration techniques play an important role in terrain assessment and mapping. From a sequence of geographically overlapping digital images, image registration techniques identify and automatically register points of correspondence ("tie points") among the plurality of images. These images are digital images with different orientations and/or scales. These images may also have been obtained at differing times of year and from different geographical perspectives.
Once tie points are identified, coordinates of each corresponding point in each image are calculated so as to be invariant with respect to the scale, orientation and position of the image of which the points are a part. From this information, the areas in common of the overlapping images can be analyzed. It may be desirable to assemble a single large image based on a collection of all of the overlapping points. Thus, a need has arisen to provide a method and apparatus for automatically identifying the same geographical points in different digital images having common coverage in a more versatile and useful fashion.
Summary of the Invention
The present invention comprises a method and apparatus for selecting tie point locations within overlapping images which overcomes the foregoing disadvantages associated with the prior art.
Brief Description of the Drawings
For a more complete understanding of the present invention and the advantages thereof, reference is now made to the following Detailed Description taken in conjunction with the accompanying Drawings in which:
FIG. 1 is a flow diagram illustrating the process for achieving autonomous tie point registration;
FIG. 2 is a flow diagram illustrating the method for ordering geographically overlapping images;
FIG. 3 illustrates the data array resulting from calculating the inverse Fourier transform of an adjacent data array pair;
FIG. 4 is a flow diagram illustrating the method for phase correlating image data arrays at a first resolution;
FIG. 5 is a flow diagram illustrating the method for phase correlating image data arrays at a second resolution;
FIG. 6 is a flow diagram illustrating the method for assessing potential tie point locations;
FIG. 7 is a flow diagram illustrating the normalized cross correlation method of the present invention;
FIG. 8 is a flow diagram illustrating the method for ranking candidate tie point locations;
FIG. 9 is a flow diagram illustrating the method for performing the consensus check function of the present invention; and
FIG. 10 is a flow diagram illustrating the method for fitting a quadratic surface to a 3.times.3 pixel window extracted from a normalized cross correlation solution array.
Detailed Description
Referring now to the Drawings, and more particularly to FIG. 1, there is shown a flow diagram of the process for achieving autonomous tie point registration. The process is broken down into the selection process 10 and mensuration process 12. The selection process 10 identifies coarse tie point locations from reduced resolution imagery (generally 1/64.times.). The mensuration process 12 refines the locations of the rough tie points estimations provided by the selection process 10 to sub pixel array.
The selection process 10 includes optimize stereo pairing 14. The optimize stereo pairing 14 places a plurality of digital images provided by an image gathering apparatus (not shown) in a circular order such that all pairings, including the pairing of the first and last images, possess a high degree of likelihood for successful phase correlation. This approach minimizes the differences between the seasons of adjacent images and in the angles defining the taking geometry of adjacent images. The smaller the degree of difference between these two parameters, the greater the possibility of a successful phase correlation between the adjacent images.
Referring now to FIG. 2, there is more fully illustrated the method for ordering the geographically overlapping images according to the present invention. Each image is assigned at step 16 a numerical tag from 1 to N, where N=the total number of images. The ordered images are stored in a two column array M, wherein the total number of rows equals the number of same flight image pairs plus the remaining number of single unpaired images. The first location of array M contains the image closest to midyear. Midyear is defined to be midnight July 1 in the northern hemisphere (including the equator) and as midnight, January 1 in the southern hemisphere. The numerical tags of the same flight image pair occurring closest to the midyear date is stored at step 18 in the first row of array M with the tag of the fore perspective image in Column 1, and the tag of the aft perspective image in Column 2.
For each of the remaining same flight, stereo image pairs, the stereo geometry angles of the image pair most similar to those of the previously stored stereo image pair are calculated, and the tags for the image pair are stored at step 20 in the next row of array M. The stereo image pair containing the most closely associated stereo geometry angles are determined by defining the vector of the last stereo geometry angles placed in the array M as (a.sub.1, a.sub.2, . . . a.sub.6) and the ith unplaced same flight stereo image pair as (b.sub.i1, b.sub.i2, . . . b.sub.i6). The following equation: ##EQU1## is used to determine the index of the same flight stereo image pair minimizing equation (1). Inquiry step 22 continues to the next image pair until all same flight stereo image pairs have been stored within array M.
After inquiry step 22 determines all same flight image pairs have been stored in array M, the single image nearest in time to the image most recently stored in array M is determined at step 24, and the numerical tag of this image is stored in Column 2 of the next row of array M. This procedure continues via inquiry step 26 until the numerical tags of all single images have been stored within array M.
Referring back to FIG. 1, once all images have been ordered by the optimized stereo pairing 14, the phase correlator 21 determines the translational offsets between consecutively ordered images. In order to determine the translational offset between each of the adjacent images, phase correlations are performed between the data arrays describing the images. Adjacent arrays of image data are phase correlated by taking the inverse Fourier Transform of the arrays to compute the phase of the cross power spectrum of the arrays.
This procedure generates a third array, as shown in FIG. 3, that peaks at the desired offset location. The degree of correlation between the two data arrays is measured by correlator Figure of Merit (FOM), defined as the difference between the maximum correlation value in the signal region and the maximum correlation value in the noise region. The peak value of the phase correlation surface is denoted by .O slashed..sub.max and its location is given by (.DELTA.1, .DELTA.s). The noise peak (N.sub.max) is located outside a 5.times.5 pixel signal region centered at (.DELTA.1, .DELTA.s).
Referring now to FIG. 4, there is shown a flow diagram illustrating the process for phase correlating adjacent image data arrays to determine the translational offset between the images. The process superimposes a pair of images and shifting them against each other without rotation until image features in the overlapping region coincide. Phase correlation is performed twice for each consecutive pairing in the ordering. Each of these phase correlations is performed at a different resolution size. The first resolution size is chosen depending upon the amount of data available in the vicinity of the proposed tie point locations, and the second resolution size is normally 1/8S.times..
A 64 square pixel window region (W) in the first image and a 128 square pixel search region (S) in the second image are selected at step 30. Within the search region(s), twenty-five (25) overlapping 64 square pixel subregions, identified as candidate match areas, are selected at step 32. The candidate match areas lag each other by 16 pixels in each dimension. Phase correlation is performed at step 34 between the first image window and each of the candidate match areas. For each correlation, a feature vector is stored. The feature vector consist of (.O slashed..sub.max, N.sub.max, FOM, (.DELTA.1, .DELTA.s)). Also recorded are the index of the candidate match area having the maximum FOM, as well as the index of the candidate match area having the maximum value of .O slashed..sub.max.
max
Once each candidate match area within the search region (S) has been phase correlated with the window (W), an additional phase correlation, the Valid Match Verification (VMV), is performed at step 36. The VMV adds the offsets (.DELTA.1, .DELTA.s) obtained from correlating (W) with each of the candidate match areas to the location of each candidate match area and phase correlates (W) with the translated candidate match areas. The feature vector (.O slashed..sub.max, N.sub.max, FOM, (.DELTA.1, .DELTA.s)) for these additional correlations is also stored.
If the maximum FOM from the VMV exceeds at inquiry step 38 the threshold FOM value, phase correlation is considered successful. Thus, at step 40, the center of window (w) is mapped to the center of the 64 square pixel array of the first image and the center of the best matching candidate match area is mapped to the center of a 96 square pixel array in the second image at a point (S*) defined by the following equation:
wherein:
(.DELTA.1, .DELTA.s).sub.i =initial candidate match offset
(.DELTA.1, .DELTA.s).sub.v =VMV offset
Phase correlation is performed at step 41 at the second resolution according to the procedure illustrated in FIG. 5.
If the maximum Figure of Merit value does not exceed the threshold Figure of Merit value at inquiry step 38, the translational offsets of all twenty-five (25) candidate match areas are added at step 42 to their respective locations, and the translated candidate match areas are phase correlated at step 44. If any of the candidate match areas predict the same match location (.+-.2 pixels in each dimension separately) as the previous phase correlation predicted at inquiry step 46, this candidate is deemed to successfully match the window in the first image. The 64 square pixel first image window is centered at the initial window location at step 40, and the 96 square pixel search region is centered at the point (S*) corresponding to:
Phase correlation proceeds at step 41 according to the process outlined in FIG. 5. If none of the candidate match areas predict the same location at step 46, correlation is deemed to have failed and new tie point locations must be proposed by the redensification function at step 47.
Referring now to FIG. 5, there is illustrated the method for processing candidate tie points at the second resolution. A 64 square pixel window region (W) in the first image and a 96 square pixel search region (S) in the second image are selected and centered at step 48 at respective proposed tie point locations from the process described in FIG. 4. Within the search regions, nine (9) overlapping candidate match areas (subregions) are selected at step 49. These candidate match areas lag each other by 16 pixels in each dimension. The window is phase correlated at step 50 with each of the candidate match areas. For each correlation, a feature vector is stored as described previously with respect to FIG. 4. Also recorded are the index of the candidate with the largest FOM, the index of the candidate with the largest .O slashed..sub.max, and the value of .O slashed..sub.max for the central candidate match area (S.sub.5).
Inquiry step 51 determines if .O slashed..sub.max of the central candidate match area exceeds a maximum threshold value. If so, a valid match verification is performed at step 52 on S.sub.5, and the tie points for the current image pair are selected at step 54 at the center of W for the left image, and at S.sub.5 +(.DELTA.1, .DELTA.s).sub.5 +(.DELTA.1, .DELTA.s).sub.v for the right image. If .O slashed..sub.max of the central candidate match area does not exceed the predetermined maximum threshold value for .O slashed..sub.max, a valid match verification procedure, as described previously with respect to FIG. 4, is executed at step 56.
If the maximum Figure of Merit from the VMV exceeds the threshold FOM value at inquiry step 58, phase correlation is considered successful. Thus, at step 60, the center of window (W) is mapped to the center of the 64.times.64 pixel array of the first image and the center of the best matching candidate area is mapped to the center of a 96.times.96 pixel array in the second image. If the maximum Figure of Merit value from the VMV process at step 56 does not exceed the threshold Figure of Merit value at inquiry step 58, the translation offsets of all twenty-five (25) candidate match areas are added at step 64 to their respective locations, and the translated candidate match areas are phase correlated at step 66.
If any of the candidate match areas predict the same match location (.+-.2 pixels in each dimension separately) as the previous phase correlation predicted at inquiry step 68, this candidate is deemed to successfully match the window in the first image and the 64 square pixel first image window is centered at the initial window location at step 60, while the 96 square pixel search region is centered at the point (S*) corresponding to:
If none of the candidate match areas predict the same location at step 68, correlation is deemed to have failed and new tie point locations must be proposed by redensification 71.
Once an image pairing has been successfully phase correlated, a proposed tie point location exists in the right hand image of the pairing. Coordinates for this tie point location are given in a plane. To move to the next pairing, this tie point location is mapped to rectified (x, t) space using the appropriate coefficients for image rectification, and the point is then mapped to (r, s) space of the next pairing using a different set of coefficients at step 62. This yields a new tie point location in the left hand image of the next pairing that forms the center of the new Window W.
Referring back to FIG. 1, the candidate area assessment 53 provides an assessment of signal level and scene content correlation for candidate areas within the first image. The signal level and scene content correlation are calculated at sixteen (16) test point locations within each candidate area. By analyzing the signal level and scene content correlation, test point locations to be used in finding tie point locations within the remaining images are generated.
Referring now to FIG. 6, there is more fully illustrated the method for assessing candidate areas. The number of candidate areas within the first image are determined at step 72 and 16 test point locations are selected within each candidate area of the first image. A signal level test is performed at step 74 on the test points within each candidate area. The signal level test determines the standard deviation of signal strength within a 7.times.15 pixel array centered about each test point location. It is to be understood that the size of the pixel array surrounding the test point locations in the signal level tests may be of a variety of dimensions selected by a user. The calculated standard deviation is compared to a predetermined threshold level. If eleven (11) of the standard deviations computed for the 16 test point locations are less than the threshold level as determine by inquiry step 76, the candidate area is considered non-suitable due to a weak signal-to-noise ratio and processing proceeds to the next candidate area at step 78.
If inquiry step 76 determines a candidate area contains sufficient signal to noise ratio, the scene content correlation is calculated at step 60 for the candidate area. For each test point location within a candidate area, a 7.times.15 pixel window centered about a given test point is phase correlated with four (4) different 7.times.15 windows, whose window centers are displaced some by distance (.tau.) with respect to the test point center. Locations of the four (4) window midpoints are at (l, s-.tau.), (l, s+.tau.), (l-.tau., s) and (l+.tau., s). The correlation procedure provides four (4) autocorrelation estimates p.sub.1, p.sub.2, p.sub.3 and p.sub.4. Estimates of correlation content for a lag .tau. are made for the sample (s) and line (l) directions according to the following equations: ##EQU2##
If inquiry step 82 determines the correlation content for the line and sample directions does not lie within a predetermined threshold range, the candidate area is considered unsuitable and processing proceeds to the next candidate area via step 78. Otherwise, the candidate areas are identified at step 84 as suitable candidates for the normalized cross correlation function.
Referring again to FIG. 1, the candidate tie points determined from the candidate area assessment 53 are forwarded to normalized cross correlation (NCC) 90. Candidate tie point locations in the left hand image of the first stereo pair are assigned as designated by the CAA function. The NCC 90 attempts to find conjugate pairs in the right hand image. The maximum correlator values contained in the NCC solution array is chosen as a conjugate tie point location.
A FOM test is applied to the selected conjugate tie point locations as an additional check on validity. Points that fail the FOM test are dropped from further processing. Points that pass the FOM test are mapped to the left hand image of the next stereo pair which has been specified by the optimized image pairing 14. The entire NCC process is repeated on each stereo pair. Candidate tie points that pass all of the FOM tests (one (1) per stereo pair) are mapped back to the original image, and a closure test network 110 locates blunders and false matches.
Referring now to FIG. 7, there is illustrated in more detail the method for accomplishing the normalized cross correlation of the normalized cross correlation function. Candidate tie point locations in the first image received from the CAA network 53 are correlated at step 92 with the second image, and the maximum correlation values are selected as conjugate points within the second image at step 94. Once the maximum correlation value is determined, the Figure of Merit (FOM) for the image correlation is determined according to the following equation: ##EQU3## where p.sub.max and .eta..sub.max are defined in FIG. 3 and PGF is computed by the following: ##EQU4## and N is the total number of P.sub.ij values generated in the correlation process. The FOM value is compared to a threshold FOM value and any candidate locations not exceeding a threshold FOM value are dropped as candidate locations at step 96.
A stereo image pair comprised of the second image of the previous image pair and the following image are dewarped as a stereo pair at step 98. The candidate points surviving the FOM test from the previous normalized cross correlation process are again correlated for the next image pair at step 100. Points passing the FOM test for this correlation are determined at step 102, and the points are forwarded for dewarping and correlation if inquiry step 104 verifies the existence of additional image pairs. Once all image pairs have been correlated, the last and first images are dewarped as a stereo pair at step 106, and the images are correlated at step 108. Candidate matches passing the FOM threshold requirements throughout the process are forwarded for further analysis.
The preceding discussion of the normalized cross correlation procedure assumes there will be successful correlations in each image. If, for any reason, this does not occur, normalized cross correlation processing will terminate for those images, and control passes to the redensification 71. If all alternate tolerance regions are exhausted, and no candidate tie points are found, the procedure reverts to interactive selection of the tie point candidates.
Referring again to FIG. 1, the closure test 110 performs a deterministic test arising from the concept that a single point, when correlated and mapped through all images of interest with zero (0) errors, will have the same computed coordinate value that was assigned to it at the beginning of NCC processing. Therefore, the difference between its initial assigned coordinate and its final mapped value at the end of closure test processing will be zero (0). There is some residual error after closure test processing for valid candidate tie points. A tolerance threshold of .+-.2 pixels has been established. Points having closure errors greater than this magnitude are considered blunders and are not retained for further processing.
Once normalized cross correlation 90 has generated the plurality of candidate tie point locations, the closure test 110 maps the location of the candidate tie points in the last image from (r, s) coordinates to (x, t) coordinates and then back to (r, s) coordinates in the left hand image of the first stereo pair. The mapped candidate tie points are compared to the original position of the tie points and any pixels differing by more than two (pixels) are discarded. The two pixel variations are incorporated to account for correlating and mapping errors caused by the normalization cross correlation procedure.
The candidate tie point ranking 112 receives the information vector of candidate tie points having passed the closure test 110. The candidate tie point ranking 112 selects three (3) candidate tie points having the highest probability of successfully passing normalized cross correlation and closure test procedures during the mensuration process 12 based upon data contained within the information vectors of the candidate points.
Referring now to FIG. 8, there is illustrated the method for ranking candidate tie point locations according to the candidate tie point ranking 112. All candidate tie points are read at step 114 from the correlation file generated by the normalized cross correlation and closure tests 90, 110. For each candidate tie point, the averages of RATIO and PGF are computed at step 116. The averages for RATIO and PGF form a mean information vector (V.sub.m) for each candidate tie point location, as illustrated by the following equation: ##EQU5##
The RATIO and PGF averages are sorted at step 118 in descending order. The candidate tie point ranking 112 selects at step 120 the first three (3) candidate tie points according to the sorted RATIO average list. While the present description describes sorting the candidate tie points according to the RATIO average of the candidate tie points, the candidate tie points may also be sorted according to the PGF parameter as selected by a user. Finally, the top three (3) candidate tie points from the sorted lists are output at step 122 for further processing.
Referring back to FIG. 1, once all candidate tie points are ranked by the candidate tie point ranking 112, the points are passed to the consensus check 130. The consensus check 130 selects the top two (2) or three (3) tie point locations from the ranked candidates and calculates the distance between the points in three dimensional object space for each stereo pairing.
The triangle formed in three dimensional object space by three (3) candidate tie point locations should be congruent in all pairings. Candidate tie points passing these criteria minimizes the possibility of false match locations with any of the three (3) points. To calculate the distances between the candidate tie points forming the triangles, the consensus check 130 processes the ranked candidate tie points and direction cosines in all stereo pairings of interest. In the event that only one tie point candidate location exists, the consensus check 130 is bypassed, and the point is passed to the mensuration process 12.
Referring now to FIG. 9, there is more fully illustrated the method by which the consensus check 130 performs the analysis of the ranked candidate tie points. At inquiry step 132, a determination is made of the existence of more than one (1) tie point candidate location. If only one (1) candidate tie point exists, processing passes to step 146 to pass the candidate tie point to the mensuration process 12. If more than one (1) tie point candidate exists, the top candidate locations, normally two (2) or three (3), are read at step 134, and the distances between these points in all stereo pairings are calculated at step 136. Utilizing the distances computed at step 136, the standard deviation of distances between particular tie point candidates in the stereo pairs are determined at step 138. These standard deviations are compared at step 140 to a predetermined tolerance range, and if all deviations exists within the specified tolerance range, the top two (2) or three (3) candidates are passed to mensuration at step 146.
If one or more of the standard deviations are not within the specified tolerance range, the candidate point to be rejected is determined at step 142, and the next ranked candidate tie point is selected at step 144. The determination of candidate tie points for rejection at step 142 will be more fully discussed. Once a new tie point location is selected at step 144, control returns to step 136 and the distances and standard deviations between the new candidate tie point locations are again determined.
If any of the standard deviations do not fall within one of the predetermined tolerance ranges one of the following three (3) procedure will be followed at step 142.
CASE 1: One standard deviation is greater than a tolerance level; and
a) the other two standard deviations are not equal; or
Procedure: The common tie point of the two distances with the largest standard deviation is rejected.
b) the other two standard deviations are equal.
Procedure: The lower ranked tie point of the tie points comprising the distances in question is rejected.
CASE 2: Two standard deviations are greater than a tolerance level.
Procedure: The common tie point of the two distances is rejected.
CASE 3: All three standard deviations are greater than tolerance; and
a) all three standard deviations are unequal; or
Procedure: The common tie point of the two largest standard deviations is rejected.
b) two standard deviations are equal and the third standard deviation is greater; or
Procedure: The lower ranked tie point of the segment comprising the largest standard deviation is rejected.
c) two standard deviations are equal and the third standard deviation is less; or
Procedure: The common tie point of the two largest standard deviations is rejected.
d) all three standard deviations are equal.
Procedure: The common tie point of the two lowest ranked tie points is rejected.
Referring back to FIG. 1, the redensification 71 provides the logic required to redensify an alternate location for a tolerance region when a candidate tie point location cannot be found in the primary tolerance region. The first time redensification 71 is utilized for determining an initial or primary tolerance region, alternate locations of tolerance region on either side of the primary region are computed. These alternate locations of tolerance regions are utilized when there is a failure to find candidate tie point locations at any point within the selection procedure 10.
After completion of the selection procedure 10, the surviving candidate tie points (normally 3) are forwarded to the mensuration process 12. The mensuration process 12 provides many of the same functional processes as the selection process 10 but performs the procedures at full resolution. Normalized cross correlation 150 utilizes the normalized crossed correlation procedure discussed previously with respect to FIG. 7. However, there are a few notable differences between the two procedures. The FOM test of step 96 applied to the correlated peak values during the selection process 10 is not performed for the normalized cross correlation 150 of the mensuration process 12. All correlator peak locations are accepted as valid and the closure test 152 is used to determine which locations are kept as candidate points. Also, image data used to fill 7.times.15 Windows are sampled four (4) or more pixels apart in both the left and right images. Finally, the normalized cross correlation procedure is carried out at full resolution as opposed to one quarter resolution.
After the normalized cross correlation 150 analyses, the surviving candidate tie points are forwarded at full resolution to a closure test 152. The closure test 152 analyzes and forwards tie point candidate locations passing the closure test in the same manner as that discussed with respect to the closure test 100 of the selection process 10.
Once normalized cross correlation 150 and closure test 152 have processed the candidate tie points, any surviving tie point candidates are processed by tie point accuracy 154 to estimate the accuracy of the tie point candidates. Tie point accuracy 154 refines the candidate tie point locations to decimal line and sample (sub-pixel) accuracy and determines the error associated with tie point candidate location estimate. Initial integer tie point locations are refined by fitting a quadratic surface to a 3.times.3 window extracted from the normalized cross correlation solution array and determining the surface fit maximum needed to refine the tie point integer location to sub-pixel accuracy.
Referring now to FIG. 10, there is illustrated the method for fitting the quadratic surface to a 3.times.3 pixel window extracted from the normalized cross correlation solution array. The 3.times.3 pixel window centered about the maximum value of the cross correlation array is extracted at step 160 from the array. The coefficients of a quadratic surface satisfying the least squares constraint on the nine 9) points included in the extracted 3.times.3 array are determined at step 162 for the following quadratic equation:
The surface coefficients for the quadratic surface are determined according to the following equations:
Where ##EQU6## and the P.sub.ij value indicates the maximum value produced by the correlation process. The remaining P.sub.lk values (l.noteq.i, k.noteq.j) denote a 3.times.3 window of correlation values having P.sub.ij as the midpoint and maximum value.
Finally, the coordinates (x,y) of the peak of the quadratic surface are found at step 154 according to the equation: ##EQU7## where: H=4a.sub.3 a.sub.4 -a.sub.5 Under any other conditions for a.sub.3 and H, the location of the peak value for the quadratic surface fit defaults to the center of the window such that (x,y)=(0,0).
A second estimate of the integer tie point location is calculated at step 166 and the two (2) estimates are combined at step 168 to provide a final sub-pixel tie point location estimate. Generation of the second estimate is based upon the assumption that autocorrelation values of the 7.times.15 pixel window and the direction of a line (L) containing the integer tie point and first sub-pixel estimate diminish according to an exponential function given by:
The fall off rate a is computed by locating the integer lag (N1 for N1.ltoreq.5) along line (L) whose autocorrelation value p is the smallest of the positive autocorrelation values. Upon this bases, .alpha. is computed to be -ln (p)/N1. This autocorrelation model is designed to fit exactly the autocorrelation values along line p computed at lags 0 and N1. The second subpixel estimate is then given simply by the lag value along (L) for which the model assumes the value of .O slashed..sub.max (the peak value of the normalized cross correlation function) given as: ##EQU8##
Previous locations of tie points to sub-pixel accuracy using simulated stereo imagery has indicated accuracies to generally be less than 0.2 pixel (line and sample) for either the surface fit estimation on the expeditial correlation model described in equation 10. It appears that general scene content may be adequately modeled by a process generally providing results somewhere between the two models discussed. For this reason, the final tie point estimation P.sub.TP is determined by combining results from the two estimates at step 64.
Although a preferred of the present invention has been illustrated in the accompanying Drawings and described in the foregoing Detailed Description, it will be understood that the invention is not limited to the embodiment disclosed, but is capable of numerous rearrangements, modifications and substitutions of parts and elements without departing from the spirit of the invention.
Claims
We claim:
1. A method for selecting tie point locations within a series of overlapping images, comprising the steps of: (a) ordering the series of overlapping images to maximize successful phase correlations between adjacent image pairs; (b) phase correlating the adjacent image pairs according to the ordering of the series of overlapping images to determine an offset between overlapping areas in the adjacent image pairs; (c) selecting initial candidate tie point locations within the overlapping areas of the adjacent image pairs valid for each at a reduced resolution; (d) determining the candidate tie point location likely valid for phase correlation between each of the adjacent image pairs in the plurality of overlapping areas at full resolution; (e) phase correlating at full resolution the candidate tie point locations determined in step (d) through all adjacent images in the plurality of images to determine surviving tie point locations; and (g) refining the surviving tie point locations to sub-pixel accuracy.
2. The method of claim 1, wherein the step of ordering further includes the steps of: (a) placing a first image pair from the plurality of overlapping images into a first location in an array, the first image pair being generated closest to a predetermined point in time; (b) placing a next image pair closest in geometry to a previously placed image pair in a next storage location in the array; and (c) repeating step (b) unit all image pairs are placed in the array.
3. The method of claim 2, wherein the step of ordering further includes the steps of: (a) selecting an unpaired single image closest in time to an image previously stored in the array; (b) placing the selected unpaired single image in the array; and (c) repeating step (a) and (b) until all unpaired single images from the plurality of overlapping images have been placed in the array.
4. The method of claim 1, wherein the step of phase correlating the adjacent image pairs further includes the steps of: phase correlating a search region in a first image to a plurality of candidate regions in a second image at a first reduced resolution to determine a first translational offset; translating a center of the first and second images to a location indicated by the first translational offset; phase correlating the translated images to determine the centers of the overlapping regions at a second reduced resolution.
5. The method of claim 1, further including the step of assessing suitability of overlapping areas in adjacent images for normalized cross correlation, comprising the steps of: selecting a plurality of candidate areas within the overlapping areas of adjacent images; determining whether the signal to noise ratio for a plurality of candidate tie points within a first candidate area pass predetermined threshold requirements; rejecting the candidate tie points failing the predetermined threshold requirements; determining the candidate tie points located within a predetermined scene correlation range for the candidate tie points passing the predetermined threshold requirements; and classifying candidate tie points within the predetermined scene correlation threshold as suitable candidate tie points.
6. The method of claim 1, wherein the step of selecting initial candidate tie point locations further includes the steps of: (a) phase correlating a first adjacent image pair from the plurality of overlapping images to select initial candidate tie points; (b) rejecting initial candidate tie points not satisfying a threshold figure of merit value; (c) dewarping unrejected initial candidate tie points to a next adjacent image pair; (d) phase correlating the next adjacent image pair to determine initial candidate tie points surviving for the next adjacent image pair; (e) rejecting surviving candidate tie points not satisfying a threshold figure of merit value; and (f) repeating steps (c) through (e) to determine candidate tie points surviving phase correlation and threshold figure of merit value requirements for each adjacent image pair in the plurality of overlapping images.
7. The method of claim 6, wherein the step of selecting initial candidate tie point locations further includes the steps of: dewarping the candidate tie points surviving phase correlation and threshold figure of merit value requirements from the final image of the plurality of overlapping image to the first image of the plurality of overlapping images; comparing the surviving dewarped candidate tie points to corresponding initial candidate tie points; and rejecting the surviving dewarped candidate tie points not falling within a predetermined number of pixels of the corresponding initial candidate tie points.
8. The method of claim 1, wherein the step of determining the candidate tie point location further includes the steps of: computing a RATIO average for each candidate tie point; sorting the candidate tie points according to the RATIO average into a list; and selecting the top candidate tie points from the sorted list.
9. The method of claim 1, wherein the step of determining the candidate tie point location further includes the steps of: computing a PGF average for each candidate tie point; sorting the candidate tie points according to the PGF averages into a list; and selecting the top candidate tie points from the sorted list.
10. The method of claim 1, wherein the step of selecting initial candidate tie point locations further includes the step of testing the top ranked candidate tie points.
11. The method of claim 10, wherein the step of testing the top ranked candidate includes the steps of: (a) determining distances between the top candidate tie points in each of the adjacent images within the plurality of overlapping images; (b) determining standard deviations of the distances between the top candidate tie points from the distances determined in step (a); (c) comparing the standard deviations for each of the distances between the top candidate tie points to predetermined threshold standard deviations; (d) passing the candidate tie points if all standard deviations do not exceed the predetermined threshold standard deviations; and (e) rejecting candidate tie points if any of the standard deviations exceed the predetermined threshold standard deviations.
12. The method of claim 1, wherein the step of phase correlating at full resolution candidate tie points further includes the steps of: (a) phase correlating a first adjacent image pair from the plurality of overlapping images to select initial candidate tie points; (b) rejecting initial candidate tie points not satisfying a threshold figure of merit value; (c) dewarping unrejected initial candidate tie points to a next adjacent image pair; (d) phase correlating the next adjacent image pair to determine initial candidate tie points surviving for the next adjacent image pair; (e) rejecting surviving candidate tie points not satisfying a threshold figure of merit value; and (f) repeating steps (c) through (e) to determine candidate tie points surviving phase correlation and threshold figure of merit value requirements for each adjacent image pair in the plurality of overlapping images.
13. The method of claim 12 wherein the step of phase correlating at full resolution the candidate tie points further includes the steps of: dewarping the candidate tie points surviving phase correlation and threshold figure of merit value requirements from the final image of the plurality of overlapping images to the first image of the plurality of overlapping images; comparing the surviving dewarped candidate tie points to corresponding initial candidate tie points; rejecting surviving dewarped candidate tie points not falling within a predetermined number of pixels of the corresponding initial candidate tie points.
14. The method of claim 1, wherein the step of refining the surviving tie point locations further includes the steps of: fitting a quadratic surface to an array generated by the phase correlation of adjacent image arrays; determining a first peak location of the quadratic surface; determining a second peak location according to a predetermined function; and combining the first and second peak locations to determine a final tie point location.
15. A method for image correlation to determine candidate tie point locations comprising the steps of: (a) phase correlating a first adjacent image pair from the plurality of overlapping images to select initial candidate tie points; (b) rejecting the initial candidate tie points not satisfying a threshold figure of merit value; (c) dewarping unrejected initial candidate tie points to a next adjacent image pair; (d) phase correlating the next adjacent image pair to determine initial candidate tie points surviving for the next adjacent image pair; (e) rejecting surviving candidate tie points not satisfying a threshold figure of merit value; and (f) repeating steps (c) through (e) until candidate tie points surviving phase correlation and threshold figure of merit value requirements for each adjacent image pair in the plurality of overlapping images.
16. The method of claim 15, further including the steps of: dewarping the candidate tie points surviving phase correlation and threshold figure of merit value requirements from the final image of the plurality of overlapping image to the first image of the plurality of overlapping images; comparing the surviving dewarped candidate tie points to corresponding initial candidate tie points; rejecting surviving dewarped candidate tie points not falling within a predetermined number of pixels of the corresponding initial candidate tie points.
17. A method for selecting tie point locations within a series of overlapping images, comprising the steps of: selecting a plurality of candidate tie point locations from the overlapping images at a reduced resolution, the selecting further comprising: ordering the overlapping images to maximize successful phase correlations between adjacent images; phase correlating adjacent image pairs in an order provided by the step of ordering to determine an offset between overlapping areas; assessing suitability of overlapping areas for normalized cross correlation; selecting initial candidate tie point locations using normalized cross correlation between adjacent images; selecting top candidate tie point locations from the initial candidate tie point locations; refining the top candidate tie point locations to sub pixel accuracy at full resolution, the step of refining further comprising: phase correlating the candidate tie point locations between all adjacent images at full resolution to determine the surviving candidate tie points; and refining the surviving candidate tie points to a sub pixel location.