Passive/ranging/tracking processing method for collision avoidance guidance and control

[0002] This is a regular application of a provisional application, application No. 60/199,052, filed on Apr. 22, 2000.

[0004] 1. Field of the Present Invention

[0005] The present invention relates to a method for ranging/tracking a target, and more particularly to a method for passively ranging/tracking a target, wherein the direction measurements from two or more laterally displaced passive sensors and associated tracking control devices in a carrier and data from a GPS (Global Positioning System)/IMU (Inertial Measurement Unit) integrated navigation system are processed to provide the three dimensional position and velocity information of a moving target.

[0006] 2. Description of Related Arts

[0007] Nowadays, there is an increasing demand for real-time autonomous vehicles in wide application areas, such as industry, agriculture, health care, military, space, and underwater. Major performance characterization of the real-time autonomous vehicles depends on guidance and navigation processing and the control architecture.

[0008] Navigation concerns account for the optimal integration of information derived from multiple navigation sensors, such as GPS (Global Positioning System) receiver, IMU (Inertial Measurement Unit), etc. The guidance laws are optimally matched to generate the desired trajectory shape, based on the position information of the autonomous vehicle from the navigation subsystem and the position information of targets from the target tracking subsystem, so as to satisfy optimality criteria along different phases of the traversed path. Target acquisition and tracking accounts for the increasing desire for autonomy in system design. Vehicles are often required to sense their environment and tracking targets that are crucial to their navigation profile. Target state estimation is required to provide or predict an accurate target state from partial information and noisy sensor data.

[0009] Generally, conventional methods for producing range measurements of a target with respect to a carrier are to employ an active sensor, such as radio or acoustic radar or laser rangefinder sensor. The concept of operation of an active ranging/tracking sensor depends on measurement of the traveling time between the active sensor transmitted signals and target reflected signals.

[0010] Passive tracking methods offer significant advantages over active tracking methods. Unlike radar, laser, and other active tracking control devices, passive sensors do not emit out any kind of energy. They only receive target emitted energy and transform it to measurement data. This characteristic makes the passive tracking method an ideal technique in reconnaissance and surveillance applications, for it can detect the target but keep itself hidden from any external target, as it emits no signals.

[0011] However, in general, a passive tracking sensor can not measure the distance between the target and the sensor, as it is not based on the echoed-signal principle. The passive sensor devices offer only the measurement of the target direction with respect to the carrier in space. Therefore, it is very challenging to estimate the highly accurate three-dimensional position and velocity of a target from standalone passive sensors, under target uncertainties, poorly modeled dynamics, perturbations, nonlinear, and time-varying parameters.

[0012] The main objective of the invention is to provide a passive ranging/tracking method, which can provide the three dimensional position and velocity information of a moving target through the information from passive sensors and associated tracking control devices and GPS/IMU integrated navigation system.

[0013] Another objective of the invention is to provide a passive ranging/tracking method, wherein the passive pointing direction measurements of a moving target from two or more laterally displaced synchronized passive sensors are used to triangulate to determinate range measurements. The range measurements are further filtered to provide the three dimensional position and velocity information of the moving target.

[0014] Another objective of the invention is to provide a passive ranging/tracking method, wherein the pair of passive direction measurements of a moving target derived from two or more laterally displaced synchronized passive image sensors is used to triangulate to determinate range measurements. The range measurements are further filtered to provide the position and velocity information of the moving target.

[0015] Another objective of the invention is to provide a passive ranging/tracking method, wherein the position, velocity, and attitude information from an onboard integrated GPS/IMU navigation system are incorporated to provide high accuracy position and attitude information of the carrier to facilitate the passive ranging/tracking computation.

[0016] Another objective of the invention is to provide a passive ranging/tracking method, wherein a least squares method is employed in the construction of the algorithm of the passive ranging/tracking to obtain a form of optimal estimation.

[0017] Another objective of the invention is to provide a passive ranging/tracking method, wherein an optimization method is employed in the construction of the algorithm of the passive ranging/tracking to obtain a form of optimal estimation.

[0018] Another objective of the invention is to provide a passive ranging/tracking method, wherein a nonlinear Kalman filter is employed to achieve an effective, numerically convergent, and highly accurate passive ranging/tracking computation.

[0019] In order to accomplish the above objectives, the present invention provides a passive ranging/tracking method performed on a carrier, which comprises the steps of:

[0020] (a) producing two or more sets of direction measurements of said target with respect to a carrier, such as sets of elevation and azimuth angle of said target, from two or more synchronized sets of passive sensors through associated tracking control device, wherein said passive sensors are installed on different locations of said carrier;

[0021] (b) producing navigation data of said carrier, including position, velocity, and attitude data, using an onboard navigation system;

[0022] (c) computing said target range vector measurement of said target with respect to said carrier using said two or more sets of direction measurements; and

[0023] (d) extracting said three-dimensional position and velocity information of said target at said current epoch using said target range vector measurement.

[0024] FIG. 1 is a block diagram illustrating a passive ranging/tracking method according to a preferred embodiment of the present invention.

[0025] FIG. 2 is a block diagram illustrating the steps of the passive ranging/tracking method according to the above preferred embodiment of the present invention.

[0026] FIG. 3 shows the passive ranging geometry according to the above preferred embodiment of the present invention.

[0027] FIG. 4 shows the passive tracking coordinate system according to the above preferred embodiment of the present invention.

[0028] FIG. 5 is a block diagram illustrating the passive ranging/tracking processing steps according to the two passive image sensors of the present invention.

[0029] FIG. 6 shows the attributed graph representation according to the above preferred embodiment of the present invention.

[0030] FIG. 7 shows the feature matching algorithm according to the above preferred embodiment of the present invention.

[0031] The passive ranging/tracking method of the present invention provides a solution to obtain the three dimensional position and velocity of a target using onboard passive sensors only, wherein information from passive sensors that are controlled by tracking control devices, respectively, and navigation data of a onboard GPS/IMU integrated navigation system are processed to achieve the passive ranging/tracking solution.

[0032] The most common measurements of target tracking systems consist of the range, range rate, azimuth, or elevation angles. In the present invention, the range vector measurement is first obtained from output of a passive sensor array and the associated tracking control device. Then, the three coordinates to specify the position of the target are extracted.

[0033] The range vector measurement contains the direction and distance information of the target with respect to a carrier.

[0034] Referring to FIG. 1, each passive sensor 11, 12, 13 . . . of a passive sensor array 1 receives energy or signals emitted by the target and generates the target direction measurement. Under the control of each tracking control device 21, 22, 23 . . . of a tracking control device array 2, the passive sensors 11, 12, 13 . . . can keep pointing to the target space, so they are capable of giving out accurate direction information of the target. In order to isolate the maneuvering motion of the carrier to keep tracking the target, each of the passive sensors 11, 12, 13 . . . is generally mounted on a two degree-of-freedom observation platform of the respective tracking control device 21, 22, 23 . . . , which is further stabilized by a gyroscope-equipped control system.

[0035] For example, each of the tracking control devices 21, 22, 23 . . . may comprise an encoder installed in two gimbal axes to output the precise angular position of the passive sensor direction relative to the body coordinate system of the carrier. But in the computation of the target geographic position, the passive sensor direction relative to the navigation coordinate system is needed, so it is necessary to know the carrier attitude and combine it with the gimbal angles to obtain the angular position of the passive sensor relative to the navigation coordinate system through a target range vector computation 4. The navigation coordinate system can be an East, North and Up coordinate system, or North, East, and Down coordinate system, or North, West and Up coordinate system.

[0036] A GPS/IMU integrated navigation system 7 is required to provide accurate position and attitude information of the carrier through a target position and velocity estimation 6.

[0037] Referring to FIGS. 1 and 2, the passive ranging/tracking method performed on a carrier for tracking a target comprises the steps of:

[0038] (1) producing at least a first and a second sets of direction measurements of the target with respect to the carrier, such as sets of elevation and azimuth angle of the target, from two or more synchronized sets of passive sensors 11, 12, 13 . . . through associated tracking control device 21, 22, 23 . . . , wherein the passive sensors 11, 12, 13 . . . are installed on different locations of the carrier;

[0039] (2) producing navigation data of the carrier, including position, velocity, and attitude data, using an onboard navigation system;

[0040] (3) computing the target range vector measurement of the target with respect to the carrier using the two or more sets of direction measurements (through the target range vector computation 4 as shown in FIG. 1); and

[0041] (4) extracting three-dimensional position and velocity information of the target at a current epoch using the target range vector measurement (through the target position and velocity estimation 6 as shown in FIG. 1).

[0042] The preferred onboard navigation system is the GPS/IMU integrated navigation system 7, which can provide long-term, highly accurate navigation data of the carrier.

[0043] In most applications, two passive sensors 11, 12 are sufficient for the present invention to solve a passive ranging/tracking problem. A preferred embodiment of the present invention wherein two passive sensors 11, 12 are used is disclosed as follows. Note that the present invention is not limited to the scenario wherein only two passive sensors 11, 12 are used.

[0044] As the preferred embodiments, the step (1) further comprises steps of:

[0045] (1.1) producing the first set of direction measurements of the target with respect to the carrier, such as elevation and azimuth angle, using the first passive sensor 11 through first tracking control device 21; and

[0046] (1.2) producing the second set of direction measurements of the target with respect to the carrier, such as elevation and azimuth angle, using the second passive sensor 12 through the second tracking control device 22.

[0047] Referring to FIG. 3, the step (3) further comprises the steps of:

[0048] (3.1) forming the first presumed target-sensor vector d1, representing a direction measurement between the first passive sensor 11 and the target and expressed in the first passive sensor coordinate system (x1,y1,z1), using the elevation angle and azimuth angle measurement of the target from output of the first passive sensor 11 and the first unknown distance between the first passive sensor 11 and the target;

[0049] (3.2) forming the second presumed target-sensor vector d2, representing a direction measurement between the second passive sensor 12 and the target and expressed in the second passive sensor coordinate system (x2,y2,z2), using the second elevation angle and azimuth angle measurement of the target from the output from the second passive sensor 12 and the second unknown distance between the second passive sensor 12 and the target;

[0050] (3.3) converting the first presumed target-sensor vector d1 from the first passive sensor coordinate system (x1,y1,z1) to a navigation coordinate system of the carrier, using navigation data from the GPS/IMU integrated navigation system 7;

[0051] (3.4) converting the second presumed target-sensor vector d2 from the second passive sensor coordinate system (x2,y2,z2) to the navigation coordinate system of the carrier, using navigation data from the GPS/IMU integrated navigation system 7;

[0052] (3.5) computing the first passive sensor location vector r1, expressed in the local navigation coordinate system, using the knowledge of the first passive sensor location L1 in the carrier body coordinate system (Xb,Yb,Zb) and navigation data from the GPS/IMU integrated navigation system 7;

[0053] (3.6) computing the second passive sensor location vector r2, expressed in the local navigation coordinate system, using the knowledge of the second passive sensor location L2 in the carrier body coordinate system (Xb,Yb,Zb) and navigation data from the GPS/IMU integrated navigation system 7;

[0054] (3.7) forming the first presumed target vector (d1+L1), expressed in the navigation coordinates, by adding the first target-sensor vector d1 and the first passive sensor location vector L1;

[0055] (3.8) forming the second presumed target vector (d2+L2) expressed in the navigation coordinates, by adding the second target-sensor vector d2 and the second passive sensor location vector L2;

[0056] (3.9) finding the first unknown distance and the second known distance, using the first presumed target vector and the second presumed target vector;

[0057] (3.10) forming the first target vector, by inserting the first known distance into the first presumed target vector;

[0058] (3.11) forming the second target vector, by inserting the second known distance into the second presumed target vector; and

[0059] (3.12) forming a range vector measurement r, using the first target vector and the second target vector.

[0060] As a preferred embodiment, the step (3.9) further comprises the steps of:

[0061] (3.9.A1) forming a vector equation by differencing the first target vector and the second target vector; and

[0062] (3.9.A2) finding the first unknown distance and the second unknown distance, by resolving the vector equation.

[0063] The position of each passive sensor 11 and 12 in the carrier body coordinate system (Xb, Yb, Zb) and the target can determine a straight line in the 3-dimensional space. Under ideal conditions, the intersecting point of the two straight lines is the position of the ranged target.

[0064] Referring to FIG. 3, as described above, the positions of the two passive sensors 11, 12 are represented by the coordinates (x₁,y₁,z₁) and (x₂,y₂,z₂) in the navigation coordinate system, respectively. The first and second passive sensor location vectors are denoted by the vectors L₁and L₂. The two measured directions of the target are denoted by the unit vectors l₁, and l₂.

[0065] The direction of the target derived from the output of the passive sensor device can be expressed with two angles: the elevation angle β and the azimuth angle α, with respect to the body coordinates. So the direction of the target from each sensor device 11, 12 can be represented by a unit vector l₁, i=1,2. $\begin{array}{cc}{l}_i=\left[\begin{array}{c}-\cos {\beta }_i\cos a_i\\ \cos {\beta }_i\sin a_i\\ -\sin \beta ,\end{array}\right],i=1,2& \left(1\right)\end{array}$

[0066] Assume the position of the target is represented by the vector r in the carrier body coordinate system. $\begin{array}{cc}r=\left[\begin{array}{c}x\\ y\\ z\end{array}\right]& \left(2\right)\end{array}$

[0067] The presumed target vectors that are determined by the two passive sensor respectively can be written as follows

[0068] r=L₁+d₁l₁  (3)

[0069] r=L₂+d₂l₂  (4)

[0070] where, the variables d₁or d₂are the unknown distances between the passive sensors and the target. The intersecting point of the two lines caqn be determined by the following equation

[0071] d₁l₁−d₂l₂=L₂−L₁=Δr₀  (5)

[0072] which written in matrix form becomes $\begin{array}{cc}\left(\begin{array}{cc}-\cos {\beta }_1\cos {\alpha }_1& \cos {\beta }_2\cos {\alpha }_2\\ \cos {\beta }_1\sin {\alpha }_1& -\cos {\beta }_2\sin {\alpha }_2\\ -\sin {\beta }_1& \sin {\beta }_2\end{array}\right)·\left[\begin{array}{c}{d}_1\\ d_2\end{array}\right]=\left[\begin{array}{c}{x}_2-x_1\\ y_2-y_1\\ z_2-z_1\end{array}\right]& \left(6\right)\end{array}$

[0073] Notice that there are three equations but only two unknowns, d₁and d₂. So it is an over-determined equation set and it is possible that there may be no definite solutions for this set of equations. Corresponding to the ranging problem, there may be no definite position for the target according to the ranging measurements. That is, the two lines which are determined by the passive sensors 11, 12 may not intersect each other.

[0074] Under ideal conditions, when there is no error in the passive ranging/tracking system, the above equations should have a unique solution. But in a practical system, because of installation errors, processing errors, and other error sources, it is a coincidence for the tracking problem to have the two lines intersect each other. So in practical applications, it is necessary to estimate the most probable target position according to some kind of optimization principle.

[0075] To find a set of d₁and d₂, we use the criterion that the two points at each line correspond to the shortest distance between the two lines.

[0076] The distance between the two lines depends on d₁and d₂. If denoted by D, the distance can be expressed as $\begin{array}{cc}S==\sqrt{D^2}=\sqrt{\begin{array}{c}{\left(\Delta x-d_2\cos {\beta }_2\sin {\alpha }_2+d_1\cos {\beta }_1\sin {\alpha }_1\right)}^2+\\ {\left(\Delta y+d_2\cos {\beta }_2\cos {\alpha }_2-d_1\cos {\beta }_1\cos {\alpha }_1\right)}^2+\\ {\left(\Delta z+d_2\sin {\beta }_2-d_1\sin {\beta }_1\right)}^2\end{array}}& \left(7\right)\end{array}$

[0077] We can find d₁, d₂to minimize S using an optimization method. The minimum of S is the shortest distance between the two tracking lines.

[0078] We can also solve Equation (7) with a least squares method, and the result is the same as the minimization method above. The solution can be expressed in the form of vector arithmetic as follows $\begin{array}{cc}{d}_1=\frac{1}{d_h}\left[l_1\Delta r_0-\left(l_1{l}_2\right)l_2\Delta r_0\right]d_2=\frac{1}{d_h}\left[\left(l_1{l}_2\right)l_1\Delta r_0-l_2\Delta r_0\right]& \left(8\right)\end{array}$

[0079] where d_h=1−[cos β₁cos β₂cos(α₂−α₂)+sin β₁sin β₂]², and l₁Δr₀,l₁l₂,l₂Δr₀in the above equations means the dot product of two vectors.

[0080] Notice that except for the case where the two lines are parallel (β₁=β₂and α₁=α₂) we can always obtain definite d₁and d₂solutions to optimize the position estimate.

[0081] Inserting Equation (8) into Equation (3) and (4), we can get the two target range vector measurements corresponding to d₁and d₂. Averaging the two estimates, we arrive at

[0082] r=½(r₁₀+r₂₀+d₁l₁+d₂l₂)  (9)

[0083] This is the solution to the target ranging problem. The range measurement is expressed as

[0084] R=|r|={square root over (x

[0085] Additionally, we can obtain estimates of the orientations of the target with respect to center of the carrier: $\begin{array}{cc}{\theta }_v=a\tan \frac{z}{\sqrt{x^2+y^2}}& \left(10\right)\\ {\varphi }_v=a\tan \frac{y}{x}& \left(11\right)\end{array}$

[0086] In the ideal situation, if there is no error in the passive ranging/tracking system, the minimized S will be zero. We can use the S value as a real-time verification of the ranging error with a probabilistic interpretation.

[0087] Therefore, the step (3.9) further comprises the steps of:

[0088] (3.9.B1) forming a formula for a distance parameter, which represents the distance of the two points between the first target vector and the second target vector, using the first target vector and the second target vector; and

[0089] (3.9.B2) finding a set of the first unknown distance and the second unknown distance, which makes the value of the distance parameter be minimal.

[0090] Alternatively, the step (3.9) further comprises the steps of:

[0091] (3.9.C1) forming a vector equation by differencing the first target vector and the second target vector; and

[0092] (3.9.C2) finding the first unknown distance and the second unknown distance, by resolving the vector equation using a least squares method.

[0093] As a preferred embodiment, the step (3.12) further comprises the step of:

[0094] (3.12.A) forming a range vector measurement, averaging the first target vector and the second target vector.

[0095] At each epoch, the range vector measurement obtained from the step (3) is noisy. Therefore, the position and velocity that are extracted directly from the range vector measurement may be very noisy.

[0096] To obtain improved performance of the passive ranging/tracking method of the present invention, referring to FIG. 3 and 4, the step (4) further comprises the step of:

[0097] (4A) filtering the range vector measurement at each epoch to estimate the current position of the target by a filter at the current epoch.

[0098] The filter of the present invention is preferably chosen as a Kalman filter. Kalman filtering is a method for estimating the state variables of a dynamic system recursively from noise-contaminated measurements.

[0099] The Kalman filter of the present invention uses range vector measurements and orientation measurements to estimate the current position of the target with respect to the carrier and to further predict the future position of the target with respect to the carrier. The Kalman filtering algorithm includes estimation algorithms and prediction algorithms.

[0100] The tracking Kalman filter models the dynamics of the target as follows

[0101] X_k+1=F X_k+G W_k  (12)

[0102] where, X_Kis the state vector at time K. F is the system matrix, G is the input matrix, and W_kis a disturbance input vector. These quantities are defined below, where Q_kis the disturbance matrix. $X_k=\left[\begin{array}{c}{x}_{1k}\\ x_{2k}\\ x_{3k}\\ x_{4k}\\ x_{5k}\\ x_{6k}\end{array}\right]=\left[\begin{array}{c}x\\ \stackrel{.}{x}\\ y\\ \stackrel{.}{y}\\ z\\ \stackrel{.}{z}\end{array}\right]$ $F=\left[\begin{array}{cccccc}1& T& 0& 0& 0& 0\\ 0& 1& 0& 0& 0& 0\\ 0& 0& 1& T& 0& 0\\ 0& 0& 0& 1& 0& 0\\ 0& 0& 0& 0& 1& T\\ 0& 0& 0& 0& 0& 1\end{array}\right]$ $G=\left[\begin{array}{ccc}\frac{T^2}{2}& 0& \\ T& 0& \\ & \frac{T^2}{2}& \\ & T& \\ & & \frac{T^2}{2}\\ & & T\end{array}\right]W_k=\left[\begin{array}{c}w\\ w_y\\ w_z\end{array}\right]$ $W_k\sim N\left(0,Q_k\right),Q_k=E\left[w_k{w}_k^T\right]=\left[\begin{array}{ccc}{q}_x& 0& 0\\ 0& q_y& 0\\ 0& 0& q_z\end{array}\right]$

[0103] Where T is the sample time. Referring to FIG. 4, the measurements of the Kalman filter are the relative range, azimuth, and elevation

[0104] $\begin{array}{cc}r=\sqrt{x^2+y^2+z^2}& \left(13\right)\\ {\theta }_v=a\tan \frac{z}{\sqrt{x^2+y^2}}& \left(14\right)\\ {\varphi }_v=a\tan \frac{y}{x}& \left(15\right)\end{array}$

[0105] X_kitself is not measurable directly. Measurement equations (13), (14), and (15) can be further expressed as follows

[0106] z(k)=h(x(k))+v(k)  (16)

[0107] where z(k) represents the discrete measurement at the k-th sampling time l_k, h(x) is a vector nonlinear function which describes the relationship of the measurement with the state vector, and v(k) is the measurement noise with a variance of R(k).

[0108] Because the measurement equation (16) is nonlinear, an extended Kalman filter is needed to estimate the current position of the target and to predict the projected trajectory of the target.

[0109] Denote {circumflex over (x)}(k) and P(k) the estimate and its associated covariance error matrix obtained by the extended Kalman filter, respectively. The optimal estimation of the current position of the target is obtained in two steps as follows:

[0110] Time Propagation

[0111] {circumflex over (X)}(k/k−1)=F{circumflex over (X)}(k/k−1)  (17)

[0112] P(k/k−1)=FP(k/k−1)F^T+Q_k−1  (18)

[0113] Measurement Update

[0114] When the measurement z(k) is available at time t_k, the measurement update is performed to correct the estimate and its covariance error matrix, using the standard discrete extended Kalman filter. Define the Jacobi matrix $\begin{array}{cc}{\left(H\left(k\right)=\frac{\partial h\left(x\right)}{\partial x}|\right)}_{x=\hat{x}\left(k/k-1\right)}& \left(19\right)\end{array}$

[0115] The measurement update equations are:

[0116] η(k)=H(k)P(k/k−1)H^T(k)+R(k)

[0117] K(k)=P(k)H^T(k)η⁻¹(k)

[0118] {circumflex over (x)}(k)={circumflex over (x)}(k/k−1)+K(k)[z(k)−h({circumflex over (x)}(k/k−1))]

[0119] P(k)=[I−K(k)H(k)]P(k/k−1)  (20)

[0120] where {circumflex over (x)}(k/k−1) and P(k/k−1) in (19) and (20) are the solutions of (17) and (18) evaluated at time t_k, just before z(k) is processed. Furthermore, the outputs of the measurement update, {circumflex over (x)}(k) and P(k), provide the initial conditions for (17) and (18) starting at time t_k, just after z(k+1) is processed.

[0121] It can be easily seen from (17)˜(20) that the cases of irregular and intermittent measurements are easily handled. When no measurement is available, only a time update is performed to produce an optimal prediction.

[0122] When {circumflex over (x)}(k) at time t_kis available, the projected trajectory of the target can be predicted, based on the given prediction interval T_P,

[0123] {circumflex over (X)}(k+T_p/k)=F{circumflex over (X)}(k)  (21)

[0124] Generally, the predication T_Pis larger than the sample time T. If T_p=T, the prediction of the projected trajectory of the target can be included in the computation circle of the optimal estimation of the current position of the target (17)˜(20).

[0125] In a preferred application of the present invention, the passive sensor 11, 12, 13 . . . of the present invention can be a passive image sensor. Passive image sensors can output image sequences. Referring to FIG. 5, the passive ranging/tracking method comprises the steps of:

[0126] (1) determining a pair of feature areas in the two synchronized image data from the two passive image sensors, wherein the pair of feature represents the same target;

[0127] (2) computing the two centroids of the two feature areas in the two synchronized images;

[0128] (3) computing the two direction measurements, using the two centroids;

[0129] (4) producing navigation data of the carrier, including position, velocity, and attitude data, using an onboard navigation system;

[0130] (5) computing the range vector measurement of the target with respect to the carrier using the two sets of direction measurements; and

[0131] (6) extracting the three-dimensional position and velocity information of the target at the current epoch using the range vector measurement;

[0132] After the centroid of the target is determined in both images, the direction measurements of the target with respect to both passive image sensors can be derived using knowledge about the relative position/orientation of the two passive image sensors and their internal parameters.

[0133] As a preferred embodiment, the step (1) further comprises the steps of:

[0134] (1A) determining a pair of feature areas in the two synchronized image data from the two passive image sensors, using a feature matching method wherein the pair of features represents the same target.

[0135] There exists a great deal of literature on feature extraction and matching. One example of a sophisticated feature matching and scene registration technique that has been successfully applied to related applications is attributed graph matching. Attributed graphs are represented by nodes and arcs where each node corresponds to a derived image feature and arcs represent the relationships between nodes.

[0136] For example, in FIG. 6, a sensed image consists of three objects (1,2,3) containing certain spatial relationships and attributes (size, thickness, texture). An attributed graph can be formed from these three detected features as shown in FIG. 6.

[0137] The nodes of the graph correspond to the three individual detected features and the relations between nodes correspond to their angles of intersection or spatial relationships (above, to the right of, etc.). Nodes also contain attributes such as the size, thickness or texture associated with the detected image features. Attributed graph representation derived from imagery obtained from the passive image sensors; the three detected objects or image features 1, 2 and 3 are used to create or synthesize an attributed graph representation of the sensed scene; this sensed graph is then matched with store reference graphs.

[0138] The basic matching process requires the creation of a reference attributed graph from available sources (example, information regarding the manned vehicle) and the synthesis of a sensed attributed graph from the live sensor imagery. These comparable representations are then matched using a specialized search algorithm.

[0139] The matching or graph registration procedure is shown in FIG. 7. The output of the attributed graph matching algorithm is the largest common subgraph which represents the degree of matching between the reference and sensed data. The number of nodes and arcs in the output attributed graph can be used to assess the match quality, based on the number of nodes and arcs matched, and the uniqueness of their corresponding attributes.

[0140] The target models are used to form reference graphs that are input to the matching process (off line); the live imagery derived from the stereo sensors is synthesized into sensed attributed graphs; finally, the two graph representations (reference and sensed) are matched using a specialized matching algorithm. The output is the largest common subgraph and represents the registration of the reference and sensed data.

[0141] The attributed graph matching algorithm as shown in FIG. 7 uses a branch and bound technique to rapidly match the sensed and reference graphs. The process is mathematically a polynomial time algorithm. However, the attributes of the nodes and arcs in the graph collapse the search space and the registration can be easily performed in real time. The time consuming portion of the process is the feature extraction step where pixel data is processed to detect the objects in the sensed scene.

[0142] The features of using this type of approach include:

[0143] (i) Sensor independent feature representation,

[0144] (ii) Tolerant to perspective variations,

[0145] (iii) Attributed nodes and relationships

[0146] (iv) Incorporate camera parameters when available,

[0147] (v) Represent spacecraft features, and

[0148] (vi) Prior experience on related programs.

[0149] The benefits of using this type of approach include:

[0150] (i) Same technique can be applied to Visible/IR data,

[0151] (ii) Register wider range of data,

[0152] (iii) Constrains search for improved matching and low false alarms,

[0153] (iv) Improves search and matching accuracy,

[0154] (v) Flexible integration of diverse reference data sources, and

[0155] (vi) Lower risk approach.

[0156] As another preferred embodiment, the step (1) alternatively comprises the steps of:

[0157] (1B.1) detecting a moving target in both images from at least two passive image sensors; and

[0158] (1B.2) determining a pair of feature areas in the two synchronized image data from the two passive image sensors, using a feature matching method wherein the pair of features represents the same target.

[0159] Alternatively, the passive image sensor can be a visible/infrared (IR) camera. Therefore, the two passive image sensors can be externally synchronized with GPS's precise timer signal, so that they capture images simultaneously and the data in both images are directly related.

[0160] As a preferred embodiment of the present invention, the two passive sensors can also be installed on two individual carriers to range and track a target passively. Then, the passive ranging/tracking method performed in two individual carriers comprises the steps of:

[0161] (1) producing two or more sets of direction measurements of the target with respect to at least a first carrier and a second carrier, such as sets of elevation and azimuth angle of the target, from two or more synchronized sets of passive sensors through associated tracking control device, wherein the two passive sensors are installed on the first carrier and second carrier respectively;

[0162] (2) producing navigation data of the first carrier and the second carrier, including position, velocity, and attitude data, using the first onboard navigation system on the first carrier and the second onboard navigation system on the second carrier,

[0163] (3) computing the target range vector measurement of the target with respect to each of the first carrier and the second carrier, using the two or more sets of direction measurements, wherein the first carrier and the second carrier are data-linked; and

[0164] (4) extracting the three-dimensional position and velocity information of the target, expressed in the navigation coordinate system, at the current epoch using the target range vector measurement.

[0165] Alternatively, it is apparent that the carrier(s) can be in static condition or the two or more passive sensors can be directly located on the ground, i.e. the carrier can be the earth, according to the present invention. These passive sensors on the different locations on the ground are data-linked to implement the passive ranging/tracking of a target.