Enhancement of Fluroscopically Generated Images

This invention relates to the obtainment of radiographic images and, more particularly, to an apparatus and method for obtaining improved images of opacified anatomy using a fluoroscopic type of equipment in conjunction with a video processor.

A typical x-ray fluoroscopy apparatus includes an x-ray source and an image intensifier which is used to detect the x-radiation. The output of the image intensifier is viewed by a television camera, and the resultant television signal can be presented on a monitor and/or recorded. When a body, such as that of a patient, is interposed between the x-ray source and the detector, x-rays are absorbed in varying degrees depending upon the thickness and composition of different regions of the body. This results in the presentation of a two-dimensional image that can be used, for example, for diagnosing structural abnormalities within the body.

The ability to "see" structure in the body using the described technique depends on the x-ray absorption properties of the structure of interest in relation to the x-ray absorption properties of the material(s) adjacent to the structure. The greater the difference, the greater the "contrast" the structure of interest will have in the resulting television image. The greater the contrast, the greater the clarity of the structure in the image. Consequently, achieving high contrast is a desirable quality with this imaging procedure.

Radiographic contrast agents are used to create a large difference in x-ray absorption behavior where little or none previously existed. For example, blood vessels are virtually invisible on fluoroscopic images (except in the chest) because blood, muscle, fat and soft tissue all possess similar x-ray absorption behavior. Radiographic contrast agents contain material (e.g. air, barium, iodine) which has x-ray absorption properties dissimilar to blood, muscle, fat and the soft tissue. For example, when a bolus of iodinated liquid contrast material is injected into an artery or vein, the vascular structure is given artificially higher contrast on an x-ray image while the contrast material is present within a certain vascular segment.

Digital video processing has been previously employed to improve radiographic imaging. Successful prior art digital processing techniques for image contrast enhancement have taken advantage of a priori knowledge of the time course behavior of radio-opaque contrast agents. By isolating and imaging the flow of such contrast agents through preselected arteries and organs within the body, both anatomical and physiological information related to organ function have been obtained without the need for selective catherization and its attendant risks.

So-called mask-mode imaging is a straightforward form of time-dependent subtraction imaging implemented by digital processing. A patient is placed on an x-ray table and a region of interest is chosen for study, e.g., the carotid arteries, or a heart chamber. A small needle is placed in an arm vein, through which 30-50 ml of iodinated contrast agent later is typically injected. Prior to injection, a single digital image is formed over several video frames (typically one to four) and stored in a digital memory. The contrast agent then is injected rapidly (e.g. in three to five seconds) and flows to the right heart, then to the lungs and to the left heart chambers from where it is pumped throughout the body's arterial system. As the contrast material passes through the region of interest, a sequence of additional images is accumulated within a second digital memory. Each of these post-opacification images is subtracted sequentially from the preinjection image. Subtraction images formed in this way have been synthesized at about one image per second for relatively stationary arterial structures. For rapidly moving cardiac structures the rate has been increased to about fifteen to sixty images per second.

The time-dependent subtraction images thus formed ideally would display only opacified cardiovascular anatomy. In the absence of patient motion, image contrast due to unopacified anatomy, e.g. bones, is removed. The removal of extraneous image information permits contrast enhancement of the opacified structures. Once enhanced, the subtraction images are reconverted to video format and stored, e.g. on a video tape or a video disc. The entire processing and external storage can proceed in real-time with the patient on the table.

In the copending U.S. patent application Ser. No. 342,376, assigned to the same assignee as the present application, there is disclosed a technique whereby recursive filtering is employed to filter radiographically-generated video signals with a filter function having a temporal frequency substantially corresponding to the expected temporal frequency of the movement of the bolus of contrast material through a region of a body being imaged. The video image processed in this manner tends to enhance portions of the image through which contrast material has moved while attenuating contributions from stationary tissue and from tissue moving at a substantially higher temporal frequency than the bandpass peak.

While both subtraction and recursive filtering techniques, as described, are used to advantage in the art in improving radiographically-generated images, it is an object of the present invention to set forth a video enhancement technique that exhibits even further improvement.

The subtraction technique referred to above can be thought of as retrospective filtering which discards most of the data (i.e. video frames other than the frames which are the subject of the subtraction). The above-referenced recursive filtering technique, on the other hand, chooses a filter characteristic prospectively, the choice of filter being based on a priori assemptions as to the shape of the contrast dilution curve. In these respects, the prior art techniques represent compromises.

The present invention utilizes substantially all the data collected and also incorporates retrospective knowledge of the actual imaging characteristics of the body region of interest during the passage of the contrast bolus.

In accordance with an embodiment of the invention, there is provided an apparatus and method for generating a processed video signal representative of an image of a body. A source of radiation is directable at the body, and means are provided for detecting radiation received from the body and for converting the detected radiation into an electronic video signal. Means are provided for storing a series of frames of the video signal as digital pixel values. The digital pixel values are generally representative of the radiation absorption characteristic of the body portion corresponding to the elemental pixel position of the frame, although it will be understood that other characteristics, such as transmissivity, could be processed in accordance with the principles hereof, if desired. Means are provided for determining, for each frame i of the series, an average pixel value, S.sub.i, for the frame. Means are also provided for determining, for each frame, i, of the series, an enhancement value (or filter coefficient), k.sub.i, as a function of S.sub.i. Preferably, the enhancement value k.sub.i is determined as the difference between S.sub.i and S, where S is the average of S.sub.i over the sequence of frames. The enhancement values k.sub.i can be of either positive or negative value. Further means are provided for combining the pixel values of the pixels of each of the stored frames, i, with the enhancement value, k.sub.i, for the frame, to obtain enhanced pixel values. Means are then provided for combining the enhanced pixel values of corresponding pixels of the frames of the sequence to obtain a processed pixel value at each pixel position. The processed pixel values can be converted to analog form for display.

In the present embodiment, the absorption densities used in determining k.sub.i for each frame i are obtained within a preselected portion of the frame defined by a window area. If the window area is selected to include most or all of the body region of interest through which the bolus of contrast material will pass during the sequence of frames being processed, then limiting the determination of k.sub.i to depend on the pixel values within the window area will result in the effective filter characteristic providing a better "match" to the bolus' temporal response than if the full frame area was considered for this determination.

Applicants have determined that the technique of the present invention provides improvement of signal to noise ratio that allows very substantial reductions, as compared to prior art techniques, in the dose of contrast material necessary to obtain an image of given quality.

Further features and advantages of the invention will become more readily apparent from the following detailed description when taken in conjunction with the accompanying drawings.

FIG. 1 is a block diagram of an apparatus in accordance with the invention and which can be used to practice the method of the invention.

FIG. 2 is a basic flow diagram for controlling the processing circuitry of the FIG. 1 embodiment.

FIG. 3 is a flow diagram illustrating the pass I routine of FIG. 2.

FIG. 4 is a flow diagram of the routine of FIG. 2 for determining S.

FIG. 5 is a flow diagram of the pass II routine of FIG. 2.

Referring to FIG. 1 there is shown a block diagram of an apparatus 100 for obtaining a displayed image of the internal structure of a body 20. The apparatus 100 conventionally includes a radiation source 110, typically an x-ray source, a detector 120, and a video generator 130. The combination of detector and video generator may include, for example, an x-ray image intensifier in conjunction with a television camera. The output of video generator 130 is coupled to video recorder 150, and also to an analog-to-digital converter 151 which converts the television signal into digital form. The output of recorder 150 is also coupled to the converter 151. Equipment for obtaining the digitized television signal is well known in the art and commercially available, an example being the model AD-964310 manufactured by Thomson-CSF Broadcast, Inc. At each pixel of the video frame, the television signal digitizer generates an eight bit digital signal representative of one of 256 gradations of luminance level (for a monochrome signal--as considered in the present illustrated embodiment). The video recorder 150 may be any suitable recording device such as a video tape recorder or disc recorder. The video generator, the video recorder, and the analog-to-digital converter conventionally receive synchronizing signals, and the analog-to-digital converter also receives clock signals at the pixel rate.

The output of the analog-to-digital converter 151 is coupled to processing circuitry 200 which may comprise a general purpose digital computer or microprocessor or special purpose digital electronics. The output of the analog-to-digital converter 151 is also coupled to an arithmetic accumulator 160 which, in turn, is under control of and communicates with the processor 200. The video generator 130 and the video recorder 150 are also under control of the processor 200. The processor 200, may comprise for example a model 68000 microprocessor manufactured by Motorola Corp., or a so-called "bit slice" implementation thereof.

The output of the digital-to-analog converter 151 is also coupled to multiplier circuit 170 that is under control of the processor 200. The output of multiplier circuit is coupled to another accumulator 300 which includes frame store 305 and summing circuit 306. The output of accumulator 300 is coupled to a digital-to-analog converter 195 whose output is coupled to display 198 and to a video recorder, which may be recorder 150. Again, it is assumed that vertical and horizontal sync signals, as well as clock signals, are conventionally available from sync and clock generation circuitry (not shown).

Before describing the operation of the system and the control thereof, some theoretical considerations will be set forth.

Any temporal filtering scheme can be represented in the frequency domain by its associated temporal frequency response (amplitude and phase) or in the time domain by its impulse response. Consider the time domain. If a filter input signal is denoted by s(t) and the impulse response of that filter is denoted by f(t), then the filter output, s'(t), is given by ##EQU1## where * denotes convolution. For the case where the signal is a set of N discrete samples of s(t) spanning T seconds the filter output is a discrete sum given by ##EQU2## where s[i] is the i-th sample of s(t) and f[i] is the i-th sample of f(t). For a causal input signal and a causal filter, eq. (3) becomes ##EQU3## The n-th output is ##EQU4## Note that eq. (4)' is a weighted sum of all the input samples ending with the N-th sample.

Any filtering approach operating on a discrete set of data samples can be thought of as a linear combination of the data samples. Visualized in this way, mask mode subtraction imaging, noted in the Background hereof, is a special case of retrospective filtering where the k.sub.j of eq. (4)' are chosen according to ##EQU5## The l-th sample is chosen near the time of maximum opacification and the m-th sample is chosen at a time corresponding to low opacification. The other N-2 samples are discarded.

For the above-described recursive filtering, as in copending U.S. application Ser. No. 342,376, most of the k.sub.j are non-zero. Therefore, each data sample is used, but the k.sub.j, which are related to the filter impulse response, are chosen prospectively (e.g. to match the bolus' temporal response) and are uninfluenced by specific knowledge of the set of samples {s[j]}.

Subtraction techniques, therefore, can be thought of as a retrospective filtering process which discards most of the data. Recursive filtering, while using most of the data, employs a fixed prospective filter whose choice is based on a priori assumptions as to the shape of the contrast dilution curve. Each method represents a compromise.

The present filtering method, which utilizes substantially all the data collected and which incorporates retrospective knowledge of {s[j]} has advantages of both imaging approaches. Given the sample set {s[j]}, one can choose the k.sub.j in a way which will optimize ##EQU6## according to a selected criterion. If the k.sub.j are chosen such that the signal to noise ratio of the N-th output is maximized while concurrently eliminating the stationary background anatomy, the signal (S) and noise (n) associated with a single pixel for the filtering procedure are given as ##EQU7## where .sigma. is the standard deviation of the noise per pixel for each sample. Equation (7) assumes that the sample noise, .sigma., is uncorrelated from sample to sample, and that .sigma. is independent of j. In order to insure that the stationary background anatomy on the N-th output is cancelled we impose the condition that ##EQU8## With this condition we can rewrite Eq. (6) as ##EQU9## where c is an arbitrary constant. S/n is maximized by setting ##EQU10## Substituting Eqs. (7) and (8) into Eq. (9) and solving for k.sub.i gives the following expression ##EQU11## Eq. 10 can be rewritten as ##EQU12## where a({k.sub.j }) is a proportionally constant which depends on {k.sub.j }:

To determine how the quantity a({k.sub.j }) affects S/n, we substitute Eq. (12) into eqs. (7) and (8) and evaluate S/n to obtain ##EQU13## which is independent of a({k.sub.j }). For simplicity, choose a({k.sub.j })=1. A choice of k.sub.i which maximizes S/n, therefore is

Remembering that ##EQU14## we can evaluate c as follows: ##EQU15## where s is the mean of {s(i)}. Finally, a choice of k.sub.i which maximizes S/n is ##EQU16## Recalling Eq. (5), the impulse response of the filter is

Using Eq. (17) one can evaluate S/n from Eqs. (7) and (8) as ##EQU17## and (s[i]-s).sub.rms is evaluated only for the samples which are used for filtering. The quantity N+1 is the number of input samples which is retained for processing.

FIG. 2 is a basic flow diagram which, when taken in conjunction with the flow diagrams referred to therein, shows how processing circuitry 200 is programmed to control operation of the circuit of FIG. 1 in accordance with the present embodiment. The block 201 represents the determination of the average pixel value, S.sub.i for each frame i. The routine for this operation, designated as "pass I", is shown in FIG. 3. Having obtained value S.sub.i for each frame of the sequence, the S.sub.i values are averaged as represented by block 202, to obtain an overall average value, designated S. The routine for this operation is shown in FIG. 4. An enhancement value, k.sub.i, for each frame is then obtained, consistent with relationship (17) above, as represented by block 203. Pass II is then implemented, whereby, for each frame i of the sequence, the pixel value of each pixel is multiplied by k.sub.i. The routine for this operation is shown in FIG. 5. The corresponding enhanced pixel values from each frame of the sequence are accumulated in accumulator 300 (FIG. 1), consistent with relationship (4)' above. As each enhanced pixel value is output from multiplier 170, it is added, by summer 306, to the accumulated value for the particular pixel that is fed back from frame store 305. The new accumulated value is then restored to the particular pixel position of frame store 305. When all frames have been processed, the frame store 305 will hold the desired accumulated pixel value at each pixel position.

FIG. 3 illustrates the pass I routine for obtaining the value S.sub.i for each frame i of the sequence. The block 211 represents the operator selection of the window area. As described above, the window defines the area in which overall average pixel value S of the present embodiment is obtained. The operator may select the window area by, for example, moving a conventional cursor dot to the diagonally opposite corners of the window area and inputting the coordinates of these corners. Alternatively, the coordinates of the area corners could be conventionally input via a keyboard. The operator can also input the number of frames in the sequence of frames to be processed (block 212). The accumulator 160 (FIG. 1) is cleared, and the frame count i is initialized at one (block 213). The block 214 represents the inputting of the next frame of video from video generator 150 via analog-to-digital converter 151. The loop 220 is then entered, this loop representing the implementation of the procedure whereby the average S.sub.i is obtained for the frame i. In particular, individual pixel values are read in from analog-to-digital converter 151 (block 215), and the pixel element or line count, as the case may be, is incremented (block 216). A determination is made (diamond 217) as to whether or not the current pixel is within the predetermined window area. If it is, block 218 is entered, this block representing the incrementing of the accumulator count by the current pixel value. If the pixel is not within the window, block 218 is skipped. Determination is then made (diamond 221) if the last pixel of the window has been processed. If not, the loop 220 is continued until the frame i has been completed. (Since only pixels within the window enter into determination of S.sub.i in this embodiment, it is not necessary to continue considering pixels once the lower rightmost pixel of the window has been reached.) The value of S.sub.i is then stored in memory in conjunction with the frame index i (block 222). The accumulator is then cleared (block 223) and determination is made (diamond 224) as to whether or not the last frame of the sequence has been processed. If not, loop 230 is continued by incrementing the frame count (block 235) and reentering block 214. The loop 230 continues until all S.sub.i for the sequence of frames have been determined and stored.

The flow diagram of FIG. 4 is representative of the routine of the block 202 of FIG. 2 whereby the individual S.sub.i values are averaged to obtain S. The frame count is initialized (block 411), and a variable SBAR, used to accumulate the S.sub.i values, is initialized at zero (block 412). The stored value S.sub.i associated with i is fetched (block 413). SBAR is then increased by the fetched S.sub.i, as represented by the block 414. Inquiry is then made (decision diamond 415) as to whether or not the last frame of the series has been reached. If not, the frame count is incremented (block 416), and the loop 410 is continued until all values S.sub.i have been added to SBAR. The block 421 is then entered, this block representing the division of the sum SBAR by the number of frames in the series, to obtain S, the value which is then stored (block 422).

The flow diagram of FIG. 5 is representative of the pass II routine of block 204 of FIG. 2 whereby, for each frame i of the sequence, the pixel value of each pixel is multiplied by k.sub.i and the enhanced pixel value is added to the pixel value of corresponding pixels of the other frames of the sequence by accumulator 300 (FIG. 1). The frame index is initialized (block 511) and the enhancement value k.sub.i is output to multiplier 170 (block 512). The pixels of frame i of the sequence are input to multiplier 170 (block 513) one pixel at a time (block 514) from analog-to-digital converter 151. The loop 520 is continued until the last pixel of the frame is processed (diamond 515). Determination is then made (diamond 525) as to whether or not the last frame of the sequence has been processed. If not, the frame index is incremented (block 526) and the loop 530 is continued until all frames of the sequence have been processed.

The invention has been described with reference to a particular embodiment, but variations within the spirit and scope of the invention will occur to those skilled in the art. For example, it will be understood that, if desired, implementation of certain individual steps of the disclosed technique can be achieved under manual control or by using special purpose control circuitry.