Optical Image Processor

This invention relates to image processing, and more particularly relates to a novel optical image processor having particular application to the processing of X-ray images.

Image processors for medical X-ray images and the like are well known. Such image processors employ digital techniques to manipulate images.

Digital techniques are used in systems which employ digital computers for processing the received data. Typically, a video camera scans the output phosphor screen of an X-ray image intensifier and the output is digitized, and applied to a computer which performs varous manipulations on the data. Transparencies and other forms of input information can also be analyzed by the video camera.

Digital radiography equipment of this type is slow, is bandwidth-limited and has low spatial resolution and poor contrast detectability. Consequently, patients have to be exposed to fairly large doses of X-rays to employ this type of image processing.

Improved digital systems are available which have been developed principally for military reconnaissance applications and are characterized by moderate data through-put which might be equal to or greater than about 100 megabits per second and have the ability to implement advanced processing techniques including subtraction, convolution, correlation, etc. in real time or near real time. Even these digital signal processors have limitations, however, in that they are generally constrained to spatial resolutions of less than about 3 line pairs per millimeter. This is due to a combination of the limiting spatial resolution of the X-ray image intensifier and of the matrix size which is generally about 512.times.512 pixels. Further limitations exist on the quality of the end product image due to the television cameras used to image the output phosphor of the X-ray screen or image.

As the signal processing quality of digital signal processors is increased (that is, as the size of the matrix used is increased), the number of steps of gray to be manipulated is increased, the through-put is increased and the processing electronics becomes more complex. The equipment reaches a point where cost, maintenance, downtime, repair and environmental considerations become a serious problem in addition to the high initial cost of the equipment.

Optical image processing avoids many of the problems of digital signal processing at only a small fraction of the cost of the digital system. Conventional optical data processing equipment employs the properties and interactions which occur when light travels through a refracting medium. Various signal processing techniques can be used including convolution, correlation, image transformation, edge enhancement, deblurring and noise reduction among others.

Optical data processing systems generally employ operations on either coherent light or noncoherent light. The processing of coherent optical data streams offers a number of capabilities, principally the ability to perform a fourier transform in real time. This occurs when the coherent beam passes through a lens. Various types of spatial filters can be placed at the fourier plane of this lens. Such filters are capable of implementing various signal processing functions. The data stream is then subjected to an inverse fourier transform by a second lens. The resulting output image plane contains the enhanced data set.

Coherent optical data processing has certain limitations. These include a relatively small dynamic range and a noise problem known as "laser speckle" which is a snow-like noise image overlaid on the main image under certain circumstances. A second problem in coherent optical data processing is that the X-ray data which is initially input into the processor is incoherent so that appropriate transformation must be made prior to processing.

Noncoherent optical data processing is also known and is capable of performing noise reduction, feature enhancement, correlation, subtraction, level splicing among other data processing operations. Such processors offer a wide dynamic range and have a high signal-to-noise ratio with full two-dimensional parallel processing in real time of the data stream.

Noncoherent optical data processing techniques are based on two principles: geometric optical systems and diffraction optical systems. Each has its own advantages and disadvantages which are well known. Geometric optical systems employ shadow techniques which modify the optical data stream. Diffraction optics rely only on the interaction and diffraction of light waves to perform enhancement techniques.

The present invention provides a novel assemblage of optical components which integrates geometric optical techniques with diffraction techniques for a noncoherent optical data processor.

A novel assemblage of optical components is provided in accordance with the present invention to perform improved and relatively inexpensive image processing for an image which can be derived from any source, for example, an X-ray image intensifier. The novel assembly of the invention can perform most operations currently implemented by existing optical data processors and by existing digital radiography systems including:

(a) Fourier domain processing;

(b) Two-dimensional spatial filtering;

(c) Edge enhancement;

(d) Noise reduction;

(e) Real time subtraction;

(f) Convolution;

(g) Level splicing;

(h) Intensity transformation.

All of the above operations are accomplished in real time on high resolution medical X-rays or other imagery. The device of the invention is capable of operating with the following specifications:

(a) Spatial resolution: 100 line pairs per millimeter;

(b) Dynamic range: in excess of 60 db;

(c) Contrast range: 5000:1;

(d) Bandwidth: 1 gigahertz;

(e) Frame rate: 30 frames per second in all formats with higher speeds possible.

Among other features of the invention, there is provided a novel subtraction technique wherein the same image is divided into two paths, both of which have processing devices, and the images in the two paths are subtracted. This subtraction technique is to be contrasted, for example, to the subtraction of the image of the first transparency from the image of the second transparency as is common. By subtracting the image from itself, at least edge enhancement processing is obtained in the processed image. In the subtraction system, there is also a novel arrangement for obtaining convolution by alteration of the length of the subtraction paths.

A novel switching arrangement is provided in which optical paths can be selectively turned on and off to perform various image processing operations on the image applied to the system, such as only subtraction or only spatial filtering or both.

A novel system is provided for the storage of optical information in a buffer, such as a prom or the like. The purpose of the buffers is to permit storage of information from a weak source over a given period of time. Thereafter, the image stored in the buffers is read out through the flash beam. Note that the flash sends an intense beam through the entire system in order to preserve the signal-to-noise ratio of a noncoherent system while still obtaining the desired intense output normally available in a coherent system.

As a further feature of the invention, a novel spatial filter interferometer structure is provided in which spatial light modulators are contained in conventional apertures at each side of a conventional interferometer. In addition, a novel third geometric filter is provided in the spatial filtering system output path which follows the diffraction type spatial light modulators which are contained in the interferometer apertures. The spatial filter system enhances contrast. Note in prior art systems a pinhole or single, fixed aperture was provided in each leg of the system to limit spatial frequency in each leg and thus create bandpass filtration.

As a further novel feature of the present invention, means are provided for combining unprocessed and processed data in selected proportions so that the observer can mix processed and unprocessed data as desired.

A still further feature of the present invention involves the novel arrangement for packaging the overall assembly. Thus, optical processing systems are sensitive to dimensional change due to temperature variation. In accordance with one feature of the present invention, temperature drifts are avoided by intentionally heating the assembly to a given temperature above the highest expected ambient temperature. The input power to the heater is then regulated to maintain a constant assembly temperature regardless of generation of more or less heat within the assembly or fluctuation of ambient temperature.

All optical components are mounted within a hermetically sealed enclosure filled, for example, with dry nitrogen gas to eliminate the need to clean the optical system. Moreover, the entire housing may be mounted in a suitable shock-mounting to prevent inadvertent relative movement of the optical components due to shock.

All adjustments to equipment within the housing can be made from regions external of the housing, with manual adjustments, for example, employing a flexible control cable or the like which enters the housing, or employing electrical drive motors within the enclosure, controlled by electrical apparatus disposed externally of the housing.

FIG. 1 schematically illustrates the optical components which are at the optical input system and which are involved in the subtraction function of the equipment.

FIG. 2 shows the spatial filtering portion of the system and is a continuation of FIG. 1.

FIG. 3 schematically illustrates the novel housing of the invention along with its shock-mounting and heater control.

FIG. 1 shows the subtraction optics used for the system of the invention, while FIG. 2 shows the spatial filtering optics for the system. A bypass line for unprocessed data from the subtraction system to the spatial filtering system and the processed signal output of the subtraction system of FIG. 1 lead directly into the left-hand side of the spatial filtering arrangement of FIG. 2.

The system shows in FIGS. 1 and 2 employs non-coherent light for processing as contrasted to a coherent light processor. A typical source for a noncoherent light input which is to be processed can be the screen of the X-ray image intensifier 10 of FIG. 1. A significant feature of the invention is that the input light is a modulated source, rather than a transparency, as in the prior art. However, the present invention can also operate on input data from an appropriately illuminated transparency or the like.

The two-dimensional image of the X-ray image intensifier face is applied toward an appropriate collimating lens 11 to beam splitter 12 which divides the collimated light from collimating lens 11 into paths 13 and 14, respectively. If desired, a fiber optic cable can be used to couple the modulated source image to the collimating lens 11 and splitter 12. Paths 13 and 14 contain respective light switching devices illustrated as Pockels cells 15 and 16, respectively. Pockels cells 15 and 16, and the others described hereinafter, can be replaced by any other type of electro-optical shutter or mechanical shutter.

The image of X-ray intensifier 10 which passes through Pockels cell 15 is applied through a spectral filter 17 to a beam splitter 18. The input light to the beam splitter 18 then divides into two paths, the first including reflecting mirror 19 and focusing lens 20 and the second including a suitable optical delay line 21 and a focusing lens 22. The image focused by lens 20 is applied to buffer 23 and the image focused by lens 22 is focused onto the buffer 24. Buffers 23 and 24 as well as other buffers to be described hereinafter may be of any desired image storing device and can consist of conventional proms or liquid crystal light valves (LCLVs)of known structure. Buffers of this type are mainly used in coherent systems but are used in a novel manner in the present system as will be described.

The output light of buffers 23 and 24 follows paths 25 and 26, respectively. The light in path 26 is reflected by an appropriate mirror 28 and is applied through a 1/4 wave path 29 to one side of a conventional beam splitter 30. The light from path 25 is also applied to the beam splitter 30 and the light of paths 25 and 26 is combined by the beam splitter 30 to produce an image subtraction operation. The 1/4 wave plate 29 is used to adjust the polarized nature of the light in the two paths.

Buffers 23 and 24 can be replaced by a single buffer at the output of beam splitter 30. This is desirable to assist in keeping the optical properties of the two paths in the subtraction system as close to one another as psosible.

The output subtracted light 31 is then passed through an iris diaphragm 32 which acts as a light level control device and acts to sharpen the edges in the processed image. The processed and subtracted output signal 33 is then applied to the input side of the spatial filtering arrangement of FIG. 2 as will be later described.

In the subtraction path of FIG. 1, one of the focusing lenses, for example, lens 20, is adjusted to intentionally slightly blur the image focused on the buffer 23. By blurring this part of the image, there is produced a convolution effect similar in nature to the technique known as "blurred or unsharp masking" which is used in photographic processing. Note, however, that the implementation of this technique in the optical arrangement of FIG. 1 is unique.

An important and unique feature in the arrangement of FIG. 1 is that the images in buffers 23 and 24 are the real time images of exactly the same scene on the X-ray image intensifier 10 and these identical images are subtracted from one another. This is distinct from subtraction techniques which have been used in the past in which the images of two different transparencies or two different time scenes are subtracted from one another.

Also shown in FIG. 1 is a high intensity flash illuminator which consists of a strong flashlamp 40 which directs its illumination onto the beam splitter 18 when it is desired to read out buffers 23 and 24. At the time the illuminator lamp 40 is flashed (which flash is focused by the mirror 41 onto the beam splitter 18), the switches or Pockels cells 15 and 16 are closed. Similarly, other shutters and Pockels cells which are to be later described are closed during this readout time.

The output or processed signal of the subtract system of FIG. 1, as pointed out above, is applied to the input of the system portion shown in FIG. 2. Thus, the processed signal 33 is applied to beam splitter 50 in FIG. 2 which is operable to combine the processed signal 33 with the focused, unprocessed signal which is taken directly from the X-ray image intensifier. Thus, in FIG. 1 the unprocessed and collimated light from intensifier 10 has been applied by the beam splitter 12 through the Pockels cell 16 to mirror 51. The unprocessed light, shown by the ray 52, is then applied to the beam splitter 53 in FIG. 2 and is applied through the Pockels cell 54 to the beam splitter 50 so that processed and unprocessed images can be combined in the beam splitter 50 when the Pockels cells 54 is opened.

Processed and unprocessed light or the processed light combined with the unprocessed light is then applied from the beam splitter 50 to the collimator lens 55. The output light of collimator 55 is then applied to the interferometer consisting of beam slitter 56, reflecting mirrors 57 and 58 and aperture plates 59 and 60 associated with mirrors 57 and 58, respectively.

The interferometer, consisting of components 56 through 60, is known for spatial filtering purposes and optical processing systems. In accordance with the invention, however, the apertures in plates 59 and 60 contain spatial light modulators 61 and 62, respectively, to enable spatial filtering or modulating of the output signal of the interferometer which is ultimately applied through the focal lens 63 to the buffer 64. Modulators 61 and 62 may be any desired optical device such as a filter, grating, diaphragm, SLM or other device which can impose an appropriate function on the light beam passing therethrough.

The information in buffer 64 can be read when desired through the use of illuminator 70, with the Pockels cell 54 opened and other Pockels cells in the system appropriately closed as is necessary.

The output of the buffer 64 which is filtered or otherwise processed by the spatial light modulators 61 and 62 is then further processed in an appropriate manner after reflection from mirror 71 by the filter 72. The filter 72 is normally used in geometric systems but its use with the diffraction type filter including the spatial light modulators 61 and 62 is a novel combination of both geometric and physical optics in an optical processor.

The light which is processed by the filter 72 is then applied through the beam splitter 75 to appropriate zoom type optics 76, if desired, and then through a beam splitter 77 to either a hard copy-making apparatus 78 or by reflection from beam splitter 79 and through focal system 80 to a viewing screen 81.

The bypass channel shown in FIG. 2 for the unprocessed signal includes the Pockels cell 80 and beam splitter 81a. Light from beam splitter 81a takes the path through the Pockels cell 82 and the path to the reflecting mirror 83. When passing through the Pockels cell 82, the unprocessed light is combined with the processed light at the beam splitter 79 and can be presented to the diagnostician with any mix of processed and unprocessed light which he may select. The unprocessed light which passes directly through beam splitter 81a is reflected by the mirror 90 and is directed through an appropriate attenuator 91 to reflecting mirror 92. From the reflecting mirror 92, the unprocessed light is applied through the beam splitter 79 and the lens 80 to the viewing screen 81 to permit viewing of the unprocessed signal alone or in combination with the processed signal. The attenuator permits the application of whatever amount of unprocessed light is desired by the diagnostician.

In the novel arrangement of the invention, as set forth in FIGS. 1 and 2, it will be apparent that by appropriately turning on and off the various Pockels cells and by appropriate adjustment of attenuator 91, it becomes possible to excite the illuminators 40 and 70 and to turn on and off appropriate Pockels cells in order to read out the stored and processed image of the X-ray image intensifier 10 either on a viewing screen 81 or in the hard copy apparatus 78.

The observed signal will have either or both subtraction modification or spatial filtering modification or the like through the novel combination of geometric and physical optical systems and either the processed signal or unprocessed signal can be displayed or the signal displayed can include any desired proportion of processed and unprocessed signals.

The optical equipment shown in FIGS. 1 and 2 is also provided with appropriate mechanical adjustments which are indicated, for example, by the double-ended arrows. Thus, components 20, 22, 57 and 58 which are capable of mechanical adjustment can be adjusted by the turning of a mechanical screw or through a motorized system.

Stages of image intensification can be added to the system where needed. For example, an image intensifier 95 can be connected in front of splitter 12 (FIG. 1). Other image intensifiers (not shown) can be connected between components 30 and 32 and between components 76 and 77 (FIG. 2).

Note that two basic signal paths are provided in FIGS. 1 and 2. A third signal path can be added, if desired, to perform further processing on the applied input signal, if desired.

FIG. 3 schematically illustrates a hermetically sealed housing 100 for housing the optical components of FIGS. 1 and 2 and for fixing the components relative to one another as needed. The housing is preferably filled with a gas such as dry nitrogen to eliminate the need for cleaning the optical elements. Appropriate input and output hoods (not shown) are also provided.

The entire housing is preferably mounted on shock supports, schematically illustrated as spring mounts 101 and 102, and has external controls 103 at which manually and electrically available controls are provided to enable control of the position of lenses and other components within the housing 100. The external controls 103 can be coupled to the components within the housing 100 through conventional hermetically sealed cable arrangements. Adjustment motors within housing 100 can also be operated by electrical signals produced by appropriate circuitry in the control housing 103.

Since the various optical components may vary in spacing due to temperature change to affect the adjustment of the system, the housing is provided with an internal heater element 104 which is energized from an appropriate a.c. source through a heater power control system 105. Heater 104 is operated to intentionally heat the housing and its contents to a temperature higher than that which the ambient can be expected to reach. An appropriate temperature measurement probe, which may consist of one or more probes distributed about the interior of housing 100, is then connected to a temperature measuring circuit 106. Circuit 106 has a second input 107 from a temperature set reference source which can be manually set to a value related to some desired constant temperature for the interior of the housing 100 which is above the highest ambient temperature which is expected to be reached.

The output of the temperature measurement device 106 is then connected to the heater power control system 105 and consists of an error signal produced when the output of the temperature measurement within housing 100 varies from the set temperature input from the input reference source 107. Consequently, it becomes possible to increase or decrease the power applied to the heater 104 in order to maintain the temperature within housing 100 constant regardless of changes in ambient temperature or in changes of the heat produced within the housing by the system components.

Although the present invention has been described in connection with a preferred embodiment thereof, many variations and modifications will now become apparent to those skilled in the art. It is preferred, therefore, that the present invention be limited not by the specific disclosure herein, but only by the appended claims.