Kalman Filter Preprocessor

The field of the invention relates to an image processing system in general, and in particular to an apparatus for providing optical differentiation between two images to be used as inputs for a Kalman filter algorithm.

An extended Kalman filter algorithm is a computerized technique developed in the general area of estimation theory for nonlinear systems. The algorithm is a sophisticated technique of performing an error analysis on real time measurements and making corresponding corrections to a system state. Typically, the Kalman filter algorithm is used with space navigation systems and usually involves the use of a covariance matrix and weighting coefficients. For more information about Kalman filter algorithms, see The Kalman Filter: Its Recognition and Development for Aerospace Applications American Institute of Aeronautics and Astronautics, Inc., Journal of Guidance and Control, Jan.-Feb. 1981, pg. 4. Also see Least-squares estimation: from Gauss to Kalman, IEEE Spectrum, Jul. 1970, pg 63. Another good reference is An Adaptive Extended Kalman Filter for Target Image Tracking, IEEE Transactions on Aerospace and Electronic Systems, Vol. AES-17, No. 2, Mar. 1981, pg 173.

At the present time, computer usage of the extended Kalman filter algorithm requires the calculation of three preliminary functions in order to be able to utilize the algorithm with data that is collected optically. Presently, these three functions are calculated digitally by the same computer used to execute the extended Kalman filter algorithm. Unfortunately, the computation of these functions by digital computers at video rates requires prohibitively high data rates for images larger than 8.times.8 pixels. Consequently, the digital computer is unable to efficiently perform lower data rate tasks required by the Kalman filter algorithm such as matrix manipulations and multiplications.

U.S. Pat. No. 3 888,591 discloses a system for subtracting two images utilizing encoding and filtering means. U.S. Pat. No. 4,075,604 teaches an apparatus having sampling means for real time recognition of test images by comparison with sample images. While both patents are useful devices for their intended purposes, they are unsuitable for providing the three parameters needed in executing an extended Kalman filter algorithm.

It is an object of the invention to provide a device that computes optically the input functions required by the extended Kalman filter thereby increasing the digital computer's efficiency for images larger than 8.times.8 pixels.

According to the invention, an incoming stream of time varying images, G, is integrated to form a reference image, h, which is then input to two light valve image subtraction systems to produce difference image approximations to the partial derivatives .differential.h/.differential.x and .differential.h/.differential.y; additionally the reference image h is then subtracted from each subsequent sampled image G. The result, G-h, is digitized and used as an input to an extended Kalman filter algorithm. The two partial derivatives are likewise digitized and inputted into the algorithm.

FIG. 1 is a block diagram of the invention according to the preferred embodiment; and

FIG. 2 is a schematic diagram of the optical elements comprising the preferred embodiment of the invention.

Referring to FIG. 1, a source of input images G, typically non-coherent telescope or television camera images, is fed into an integrator 12 for computing a time averaged signal h over several input frames. Source image G and averaged signal h are fed into an image subtractor 14 for subtracting the averaged image h point by point from the input image G to produce the desired two-dimensional difference function G-h. The averaged image h is additionally fed through a spatial filter 16 for noise reduction and then split into two separate paths. Each path passes the image into separate but parallel differentiators 18 and 19. One differentiator performs spatial differentiation on the image with respect to one dimension (i.e. the x direction), while the second differentiator likewise performs spatial differentiation on the image but with respect to the opposite dimension (i.e. the y direction). The resultant functions are known as .differential.h/.differential.x and .differential.h/.differential.y respectively. All three functions, G-h, .differential.h/.differential.x, and .differential.h/.differential.y, are coupled through an analog-to-digital converter to a computer 44 where they are utilized by the Kalman filter algorithm.

Referring to FIG. 2, a source of input images G is transitioned to a beam-splitter 20 for dividing G into two beams. One beam passes through a lens 22 for focusing the beam on a light valve 24. The other beam is transitioned via mirrors 26, 28 and lens 30 and is focused on a light valve 32 simultaneously with the beam striking light valve 24.

A light valve is a device in which the light transmission can be varied in accordance with an externally applied electrical signal. Such a light valve typically has a fiber optic face plate at the input surface, followed by a photoconductive layer, followed by a layer of an electro-optic material (typically a liquid crystal), and ending with a second transparent electrode on the output surface. Operationally, an incident intensity image is presented at the input side of the light valve and is transitioned via the fiber optic face plate and transparent electrode to the photoconductive layer. A voltage is placed across the input and output transparent electrodes which gives rise to an electric field across the entire device. The layer of electro-optic material, whose birefringence varies as the electric field across it varies, responds to produce an output image which is transitioned to the output electrode. Additional layers of index matching dielectric material are introduced by some manufacturers between the photoconductive and electro-optic layers to isolate the input and output light beams and to allow reflective readout through the output transparent electrode. The spatially varying irradiance of the input image gives raise to spatially varying conductivity across the photoconductive layer which in turn gives rise to a spatially varying electric field across the output electro-optic material. The birefringence of the output electro-optic layer thus varies spatially with the spatially varying electric field, and the polarization of a linearly polarized readout beam (coherent or incoherent) will exhibit a spatially varying polarization as it leaves the output surface of the light valve. If the output beam passes through a linear polarized oriented orthogonally to the original polarization of the readout beam, a positive output intensity image will be produced. If the output beam passes through a parallel linear polarizer, a negative output intensity image will be produced.

An input image G is presented simultaneously at light valves 32 and 24. Light valve 32 continuously integrates the input image producing a continuous reference image h, which consists of the time integral of the previous 100-200 ms of input data. This integration time is somewhat longer than the integration time of typical liquid crystal light valves and is controlled by the input image irradiance and the bias voltage. An incoherent light source S1 is situated such that its light beam is pointed directly at the output side of the light valve 32. Light valve 24 is positioned such that a line drawn perpendicular from the output surface of the valve is perpendicular to the light path from source S1 to valve 32. A polarized beamsplitter 34 is placed at the intersection of the light path from source S1 and the perpendicular line from valve 24 such that it will reflect the light from source S1 onto the output surface of light valve 24. This beamsplitter causes the reflected light to become polarized and also polarizes the light which passes through it. The light is then reflected from the output side of the light valve back through the polarizing beamsplitter 34 and onto a TV camera 36 lens. The character of the light reflected from the output side of the light valve is controlled by the image focused on the input side of the light valve.

Similarly, another polarizing beamsplitter 38 is placed in the light path from source S1, between light valve 32 and beamsplitter 34, such that it will reflect the light from the output side of light valve 32 by 90.degree. thereby sending it to a third light valve 40. A .lambda./4 retarder 42 is introduced between light valves 32 and beamsplitter 38 to provide an additional 90.degree. of rotation of the polarization of the reflected readout beam at valve 32 (The retarder 42 is traversed twice by the readout beam.), so that the image h is effectively read out between parallel polarizers, thus producing a negative image h at the output of the polarizing beam-splitter 34. The images G and -h are combined at a 30-60 Hz rate to form G-h at the input to the TV camera. Similarly, a positive image h is produced at the output of beamsplitter 38 and is sent to light valve 40.

The combination of valves 24 and 32, light source S1, polarizing beamsplitter 34, and the .lambda./4 retarder comprise the image subtraction system 14 of FIG. 1.

The output of TV camera 36 is a video signal which is coupled through an analog-to-digital converter and sent to digital computer 44 to be used in the extended Kalman filter algorithm.

Light valve 40 serves as an input to a coherent optical spatial filtering system. The reference image h is the input to the spatial filter. An analyzer 46 is installed just prior to light valve 40 for changing the polarization pattern back to an image amplitude pattern. The output side of light valve 40 faces a series of lenses 48, 50, and 52 oriented in a straight line to accept and transition an image appearing on the output side of the light valve. A laser 51 with its beam passing through a pinhole 56 and lens 58 is positioned with its beam oriented parallel with the output surface of light valve 40 and perpendicular to the light path passing between valve 40 and lens 48. A beamsplitter 60 is inserted into this light path such that the beam from the laser is reflected onto the output surface of valve 40 to readout the image h with the reflected light which then traverses beamsplitter 60 and is transitioned through lenses 48, 50 and 52 to appear on the input side of another light valve 54. An analyzer 47 is installed at the focal point of lens 50 for converting the polarization pattern to an image amplitude pattern while a spatial filter 16 is installed at the focal point of lens 48 for filtering and noise reduction purposes.

The output filtered version of reference image h is presented simultaneously at four light valves: 54, 55, 57 and 60. These four light valves comprise two optical subtraction systems equivalent to the system described above which use light valves 24 and 32. Light valves 54 and 55 comprise one subtraction system and valves 57 and 60 comprise the second.

The simultaneous presentation of reference image h at the two light valves 54 and 55, which are oriented at right angles to each other, is accomplished by splitting the light beam between lenses 50 and 52 using a beamsplitter 62. Another beamsplitter 64 is introduced within the reflected beam produced by beamsplitter 62, thus splitting the beam again and sending a new reflected beam into a mirror M1 which in turn reflects the beam through a lens 66 for focusing the reference image h onto the input side of light valve 55. A .lambda./4 retarder 70 is inserted just in front of the output side of valve 54 to provide a 90.degree. rotation of the polarization of the beam used for reflective readout of the image h as it appears on the output side of valve 54. An incoherent light source S2 is positioned directly in line with the output side of valve 55, and a TV camera 68 is positioned directly in line with the output side of valve 54 such that the light path from the camera to valve 54 intersects with and is orthogonal to the light path from source S2 to valve 55. A polarizing beamsplitter 72 is inserted at the intersection of the light paths such that light from source S2 passes through to valve 55 and is also reflected to valve 54 via retarder 70. Thus, image h appearing on the output side of valve 55 is reflected via polarizing beamsplitter 72 into camera 68 while output image h from the output side of valve 54 passes through retarder 70 and polarizing beamsplitter 72 and appears at camera 68 as a negative image. Operationally, mirror M1 is tilted slightly about the y-axis to introduce a slight offset in the x direction. Thus the image h is subtracted from an x-shifted version of h to produce a good approximation of the desired partial derivative, .differential.h,/.differential.x. TV camera 68 converts the resultant image into a video analog signal which is coupled through an analog-to-digital converter and made available for use in computer 44.

The simultaneous presentation of reference image h at light valves 56 and 60, which are oriented at right angles to each other, is accomplished by dividing the light beam that passes through beamsplitter 64 into two separate beams using a beamsplitter 74. Each of these two separate beams is reflected by a mirror; mirror M2 reflects one of the beams through lens 76 onto the input side of light valve 56 while mirror M3 reflects the other beam through lens 78 onto the input side of light valve 60. A .lambda./4 retarder 80 is inserted just in front of the output side of valve 68 to provide a 90.degree. rotation of the polarization of the beam used for reflective readout of the image h as it appears on the output side of valve 68. An incoherent light source S3 is positioned directly in line with the output side of valve 56, and a TV camera 82 is positioned directly in line with the output side of valve 60 such that the light path from the camera to valve 60 intersects with and is orthogonal to the light path from source S3 to valve 57. A polarizing beamsplitter 84 is inserted at the intersection of the light paths such that light from source S3 passes through to valve 57 and is also reflected to valve 60 via retarder 80. Thus, image h appearing on the output side of valve 57 will be reflected via polarizing beamsplitter 84 into camera 82 while output image h from the output side of 60 passes through retarder 80 and polarizing beamsplitter 84 and appears at camera 82 as a negative image. Operationally, mirror M2 is tilted slightly about the x-axis to introduce a slight offset in the y direction. Thus, the image h is subtracted from a y-shifted version of h to produce a good approximation of the desired partial derivative, .differential.h/.differential. y. TV camera 82 converts the resultant image into a video analog signal which is coupled through an analog-to-digital converter and made available for use in computer 44.

In summary, the functions G-h, .differential.h/.differential.x, and .differential.h/.differential.y are produced by the optical system presented schematically in FIG. 2. These functions are in the form of two-dimensional images which are input to TV cameras and then converted to digital signals and fed to a computer as inputs to the Kalman filter algorithm.

Thus, while preferred constructional features of the invention are embodied in the structure illustrated herein, it is to be understood that changes and variations may be made by the skilled in the art without departing from the spirit and scope of the invention.