Method for tracking moving object by means of specific characteristics

[0002] The present invention relates to a method for detecting and tracking objects within digitized images on the basis of properties which clearly distinguish these objects from other structures appearing in the images.

[0003] The automation of processes, as well as autonomously operating systems and devices used in processing technology and systems employed by the military, require the dependable detection of objects to be manipulated, as well as of obstacles and targets, in particular highly developed partial systems contained in military systems for air defense for detecting the structure, position and direction of movement of aircraft, whose relevant data must be available extremely rapidly in order to dependably counter any threat. Because such systems are customarily equipped with active sensors, in particular surveillance and tracking radars, weapons systems have been developed which can effectively negate such sensors by using their radiation for their own target detection. Passive detection of targets is performed in the traditional way by appropriately trained personnel.

[0004] Besides the considerable outlay for training such personnel, the limited reaction speed of humans as well as the endangerment of the personnel employed at the targets which are attractive to the opponent should be mentioned as additional disadvantages.

[0005] Therefore several methods for detecting and tracking targets by extracting them from image data obtained from video and infrared cameras are already known. For example, a target can be detected within a section to be passed through an image, by comparing the statistical frequency of defined gray scale values in the section of a video image with that of the entire image. Besides the possibility of errors in the case of the presence of gray edges with corresponding gray scale value statistics extending through the entire image, there is the disadvantage that a certain minimum size of the object to be detected is required. This therefore cannot consist of only a few pixels of the entire image, so that the detection range of a video camera is even further removed from that of a surveillance radar.

[0006] The increase of the number of pixels by using special cameras also approaches a limit set by the speed with which calculations can be performed in the processors used. The first such methods are mainly reflected in the hardware, which was adapted to the specific use for achieving a high calculation speed.

[0007] Possible improvements of the method therefore resulted in an inordinate outlay because of the required adaptation of the hardware. Since implementations of such methods by providing the corresponding software for very fast and universally usable hardware have only very recently become possible at all, the special requirements for such implementable methods lie in their economical structure in respect to the volume of the required calculating operations.

[0008] It is therefore the object of the invention hereinafter described to assure the detection and tracking of targets in digitized images with a minimal outlay for calculating operations, while avoiding the described disadvantages of the prior art.

[0009] Methods in accordance with the prior art use a relatively large area for detecting gray scale values and similar characteristics in order to extract a target from an image. But the method in accordance with the invention uses only certain elements from the total amount of image pixels containing the target. These are moreover obtained by transformations of the actual amount of target pixels and their additional reduction to essential pixels. A characteristic pattern of the target obtained by this is employed in subsequent images for tracking by correlating it with all the pixels of an image section to be searched, wherein the correlation maximum indicates the new position of the target. The characteristic pattern of the target is furthermore continuously updated by means of an integration process in order to take into account changes in the target profile, for example if an aircraft rotates around its axis in the course of making turns.

[0010] A considerable advantage of the method in accordance with the invention lies in minimizing the calculation efforts required for detecting a target, establishing its pattern and tracking it, which consists of minimizing the number of pixels to be processed.

[0011] A further advantage is the result of the high degree of dynamics of the system in regard to the extension of the objects to be detected. Therefore it is possible by means of the described method steps to reduce an extended object to a few significant image pixels. However, since only a few pixels are required for identifying a target, it is also possible by means of the method of the invention to detect and track targets which only include a few unprocessed pixels. Therefore the possibility also exists of detecting and tracking guided missiles and drones without the employment of active sensors.

[0012] Further details, characteristics and advantages of the invention ensue not only from the claims and the characteristics to be taken from them, either by themselves and/or in combination, but also from the following description of preferred exemplary embodiments.

[0013] FIG. 1 is a representation of a target 2 within a search window 4, as well as various processing steps 6 within this search window,

[0014] FIG. 2 is a block diagram describing a method for tracking a target.

[0015] The image section 8 shown in FIG. 1 represents the contents of a gradient integrator, in which image pixels, which are the result of previous gradient images, have been stored. In gradient images, the gray scale values of the individual pixels are replaced by the differences of the gray scale values between the respective image pixel and the immediately adjacent image pixel both in the horizontal and the vertical directions. Because respective noise components in the gray scale values of the original images can even be emphasized by gradient formation, first the original images are subjected to a smoothing Gaussian filtering. The integration process takes place in that initially the gradient image, which follows the gradient image which was first loaded into the gradient integrator, is added pixel by pixel with a weighting factor k, wherein the image pixels of the existing image are entered with the weighting factor 1−k. The image section 10 represents a pattern of the target to be tracked, made from the data of the image section 8. The derivation of the pattern from the contents of the gradient integrator represented in the image section 8 takes place by determining the pixels of the image section 8 which represent local extremes. All pixels having neither a local maximum or minimum value are written over with the value zero. Thereafter, this image is correlated by pixel by pixel multiplication with the data of an actual smoothed gradient image, wherein local as well as an absolute maximum of the correlation process results from the mutual displacement of the pixels of both images.

[0016] The gradient image of the actual image 18 is represented with the overlaid pattern in the image section 12. The local relationship between the gradient image and the pattern is represented by the correlation maximum. The image section 14 shows the geometric course of the correlation progression which is valid for the actual image and pattern and represents a result which, though it is more definite, has been achieved with an incomparably greater calculation effort by means of a Hough transformation. After correcting the positional error of the target to be tracked, which can be recognized in the image section 12, the actual gradient image, on which the image section 12 is based, is entered with the weighting factor k into the gradient integrator for updating the data represented in the image section 8, from which subsequently the updated data of the pattern represented in the image section 10 are calculated, which thereafter are correlated with the data of a subsequent gradient image.

[0017] By means of a block diagram, FIG. 2 explains the method which was described by means of the respective image sections. A video camera 20, which can also be replaced by an infrared camera or other image-producing sensors suitable for this, generates a chronologically sequential series of video images 22, from which an image section 26 is extracted by assignment of a superordinated system, or by means of data determined by the described method by an appropriate process.

[0018] The image section 26 is a gray scale value image, whose curvature flow is evaluated and from which too fine structures are removed by a mean curvature flow process 28. By means of a convolution with an appropriately weighted matrix, the appropriately processed image section 30 is thereafter subjected to a Gauss-like low bandpass filtering 32 in order to eliminate very small structures created in the image section by the mean curvature flow processing, as well as noise components. For reasons of its expense, this process is preferably performed only once at the start and for initializing the method.

[0019] An image section 34 resulting from this is then used as the input value in a method step designated as gradient formation 35. Here, the differences of the gray scale values of the adjoining image pixels are used for a comparison with the respective image pixel, in which the change in the gray scale value at the transition to adjoining image pixels is stored, instead of the absolute gray scale value. In a specially adapted method of the invention, all image pixels located outside of a defined close vicinity of the target object are set to the value zero.

[0020] The gradient image 37 resulting from this is subjected to a shift by means of the target course data 62 obtained in the previous correlation process and is multiplied pixel by pixel by a factor k in a method step identified as weighting 36 in order to be thereafter added up pixel by pixel with an integrator image 46 derived from an integrator 42 and multiplied pixel by pixel in a weighting 44 by a factor 1−k, wherein a gradient image 40 being generated from this is loaded as an updated gradient image into the gradient integrator 42.

[0021] In actual use, a further task actually performed is the horizon suppression by means of the so-called horizontal or vertical edge suppression.

[0022] In a method step defined as a non-maximum suppression 50, all pixels which do not represent a local minimum or maximum in their vicinity are removed from the actual integrator image 46.

[0023] In methods of the invention which are adapted to special purposes, all pixels which fall below the maximum of the value of a pixel contained in the image by a defined, useful factor, are overwritten with the value zero.

[0024] Furthermore, in special embodiments only adjoining line- forming pixels of a value not equal to zero are considered, which do not fall below the maximum length of the lines in the image section by a defined, useful factor. In an image section which should be identified as a pattern 52 of a target, relatively few image pixels, which cover the edges of a gradient image of the target, remain because of this, all other pixels are assigned the value zero. Tracking of the target takes place in a method step identified as correlation 54. In this case a pattern 52 is multiplied pixel by pixel with a gradient image 37, taking into consideration the signs of the individual pixels, wherein the gradient image 37 used has not had an effect on the contents of the gradient integrator 42 and is laterally displaced in several method steps 56 in respect to the pattern 52. The pixel values of a two-dimensional correlation signal 58 obtained from this consist of the sum of the products between the pixel values of the correspondingly displaced gradient image 37 and the pattern 52, and the signal 58 is searched for an absolute maximum in the method step of a search 60 for a maximum, whose coordinates result in target course data 62 inside of the image section. These are passed on to downstream connected systems and are used for a correcting displacement 55 of subsequent gradient images, for the mechanical tracking of the video camera 20 as well as for tracking the extraction of the tracking window 24.

[0025] The connected changes of the gray scale values of the image pixels of a target are processed in the method of the invention, accordingly, there is a potential interference of such edges in the background of an image.

[0026] However, generally a moving target is detected and tracked in front of an immovable background. But borderline cases result in connection with strong horizontal edges, which correspond to the global direction of movement of a flying object, as well as in case of objects moving toward the video camera. To reduce these interferences, values in gradient images 37, which exceed the maximum value in the actual integrator image 46 stored in the gradient integrator 42 by a factor to be determined, are written over with the value zero.