Imaging

[0001] This invention relates to imaging and in particular to methods and systems for image enhancement.

[0002] Imaging involves transfer from the object domain into the image domain, but owing to limiting factors such as the finite size of energy source, detector size, sampling frequency, display density, software filter function, and possibly partial-volume effects experienced with some imagers, an infinitely fine delta function in the object domain cannot be faithfully reproduced in the image domain. Instead, a smeared-out image, or point-spread function (PSF), is observed. Similarly, an infinitely sharp edge-response function (ERF) in the object domain becomes a smeared-out ERF in the image domain. The smearing effect becomes more intense as the adjacent ERFs of discontinuities or contrast profiles get closer to each other.

[0003] It is an object of the present invention to provide a method and system by which the above problem can be at least partly overcome.

[0004] According to one aspect of the present invention there is provided a method wherein a de-convolution process is applied to the image-domain results of an object-scan to derive therefrom the respective point- or line-spread function effective in the object- to image-domain transfer of one or more object-discontinuities, and to derive from said function the location in the image domain of the respective discontinuity.

[0005] According to another aspect of the invention there is provided an imaging system comprising means for performing a de-convolution process on the image-domain results of an object-scan to derive therefrom the respective point- or line-spread function effective in the object- to image-domain transfer of one or more object-discontinuities, and means to derive from said function the location in the image domain of the respective discontinuity.

[0006] The method and system of the invention enable the location of the respective discontinuity in the image domain, to be established with a high degree of accuracy. This is critical to image definition free of any substantial smearing, and to this end the location of the respective discontinuity may be derived to sub-pixel accuracy simply from the mid-point of the full-width half-maximum of said function.

[0007] The said function may be correlated with the image-domain results of said transfer for enhancement of spatial resolution of the imaging of the one or more discontinuities. This enhancement may involve transfer of sub-pixels within the image-domain results of the respective one or more discontinuities, the sub-pixels being transferred within their respective image-domain results from one side to the other of said location.

[0008] The de-convolution process may be carried out using least-squares running filtering.

[0016] The method and system to be described with reference to FIG. 1 utilise MR scanning for medical diagnostic and treatment-planning purposes. In principle and in the general techniques described, the method and system of the invention can be used in other applications of MR scanning and also in circumstances where other scanning techniques are utilised. Furthermore, although both structure and function are represented by discrete ‘boxes’ 1 to 19 in FIG. 1, the method and system are to a substantial extent manifest in programmed digital data-processing operations.

[0017] Referring to FIG. 1, data derived in accordance with conventional operation of an MR scanner 1 is processed for imaging purposes within a processor 2. The output of the processor 2 is used to provide a display 3, and from this is subject to post-imaging processing 4. The post-imaging processing 4 includes the facility for selecting a region of the display 3 for more-detailed and closer inspection.

[0018] To the extent the imaging method and system of FIG. 1 have so far been described, they are conventional, and it is in further processing 5 of the image data of the selected region of interest obtained by the post-imaging processing 4 that a step forward from what is already known is achieved. More particularly, the further processing 5 is operative to define more clearly the true edges or boundaries of image contour(s) in the selected region of interest, and to enhance the accuracy of the imaging of those contours.

[0019] The definition and accuracy of transfer of features from the object domain scanned by the scanner 1, to the image domain manifest in the post-imaging processing 4, is limited by many factors. The limitations arise from within the scanner 1 itself (in particular the finite size of the energy source), within the processing performed by the processor 2, and within the display 3; limitations arise inherently from, for example, the data sampling frequency and display density used, and also from the filter-function of the software involved. More particularly, and referring to FIG. 2, an infinitely fine delta function 20 in the object domain is not faithfully reproduced in the image domain. Instead, the transfer as represented by the arrow 21 results in a point-spread function (PSF) or smeared-out image 22 in the image domain. Similarly, an infinitely sharp edge-response function (ERF) or step 23 in the object domain becomes through the transfer represented by arrow 24, a smeared-out transition 25 of spread represented by dimension arrows 26, in the image domain. When two image ERFs are close to one another, the smeared-out effects run into each other. The consequent deterioration of the spatial resolution is often monitored by the percentage modulation transfer which is given by the ratio, expressed as a percentage, of the amplitude of the modulation in the image domain to that in the object domain.

[0020] The smearing effect becomes more intense as adjacent ERFs of discontinuities or contrast profiles get closer to each other (or as the spatial frequency of the modulation becomes higher); this also causes loss of profile height. The inherent loss of the spatial resolution (that is, the part that is indicated by the smeared-out effect on the corner edge of the ERF) cannot be restored or partially restored even by re-scanning the image with an ultra high resolution digital scanner system.

[0021] The further processing 5 is operative in accordance with the invention to provide accurate image-edge definition and location, and to improve spatial resolution in the imaging. More especially, in the context of FIG. 2, the edge position corresponding to the discontinuity or step 23 of the object ERF is pin-pointed in the image domain from the mid-point of the full-width half-maximum (FWHM) of the image PSF; the pin-pointing is to sub-pixel accuracy for the image ERF. Low-contrast and ‘area’ filtering are used to remove ‘spurious’ edges, and sub-pixel sampling to detect detail to the resolution of the single-pixel modulation. The discontinuity or step 23 of the ERF is then restored within the image domain by removing the sub-pixel values from outside the optimum edge position to compensate for those within. It is to be noted that the sub-pixels then become pixels in display, and that the enhancement is equivalent to the performance of an extra high resolution image transfer system.

[0022] As represented in FIG. 2 by the arrows 27 and 28, the image ERF 25 of an infinitely-sharp step 23 can be produced by convolution of the image PSF 22 with the object ERF 23. In accordance with the present invention, de-convolution of the image ERF 25 using sub-pixel sampling, represented by the arrow 29, reproduces the image PSF 22 in a de-convolution space as image PSF 22′. The image ERF 25 is superimposed on the PSF 22′ within this space as image ERF 25′, and the optimum edge-position 30 is derived from the mid-point of the FWHM of the image PSF 22′, and is pin-pointed to sub-pixel accuracy.

[0023] For one-dimensional cases, the operation in accordance with the invention is relatively simple, as only either the x- or the y-profile, that is to say a line spread function LSF is involved. But for two-dimensional operations, both the x- and y-profiles, and if necessary, the xy-diagonal profiles to eliminate any possible streakings in the image, may be used; in this case, a proper weighting scheme will be required to re-construct the image.

[0024] Once the original sharp-edge feature represented by the object ERF 23 is pin-pointed at the position 30 within further processing 5, that feature may be restored by additional re-processing 6 (FIG. 1). In re-processing 6, the sub-pixel values occurring ‘outside’ the optimum edge-position 30 are transferred to compensate those ‘within’. This is illustrated in FIG. 2 by arrow 31 transferring sub-pixel blocks 32 from after point 30 in the image ERF 25′, to before it. The re-construction of image ERF 25′ into image ERF 33 conforming closely in configuration to object ERF 23 is represented by arrow 34. Image ERF 33 is displayed in enlarged form in display 7 (FIG. 1).

[0025] These techniques enable substantial recovery of the loss of spatial resolution in the imaging, without the trade-off loss of other properties such as image noise. Furthermore, the enhancement of spatial resolution in display 7 reproduces the region of interest selected from display 3, without blurring (or step) effects at the profile edge.

[0026] FIG. 3 is illustrative of some of the low-contrast results provided in practice from MR scanning of a pig's brain in fluid. Curve A is the image ERF produced, whereas curve B is the line spread function (LSF) resulting from de-convolution of curve A carried out in processing 5. The optimum edge-position is established from the mid-point C of the FWHM of curve B, and the additional re-processing 6 is operative by means of sub-pixel transfer, to re-construct curve A to conform substantially to the edge-feature from which it originated in the display 7.

[0027] It is to be noted that whereas curve A is stepped, curve B is nonetheless smooth and that mid-point C is located to sub-pixel accuracy. Furthermore curve B indicates a sensitivity of more than 8:1 between the profile-height and background noise.

[0028] Referring again to FIG. 2, the complete profile 35 of an image within the selected area of interest of display 3, is built up as indicated by arrow 36, from the edge-position data derived within processing 5. This data identifies the location of the point 30, together with the locations of all corresponding points derived from sampling the multiple x- or y-profiles of the selected area of interest. The build up and display of these points from the data takes place in display 8 so that a substantially true contour 37 for the profile 35 is defined. The sharpness of the true contour 37 is in contrast to the smeared contour that without de-convolution would have been obtained by virtue of the spread (represented by the arrows 26) of the relevant image ERFs 25.

[0029] A small portion of the contour 37 is shown enlarged in FIG. 4 and is defined as a best-fit line between optimum edge-positions derived respectively from x- and y-profiles; the x-profile positions are indicated by black dots and the y-profile positions by white dots. The closeness to one another of corresponding positions in the x- and y-profiles is indicative of the accuracy to sub-pixel level achieved. The smear-out that would have been manifest in the image-profile contour if the de-convolution technique were not used, would have extended throughout the space bounded by the dashed lines 38 and 39; these boundaries are indicated in dashed line in the representation of profile 35 in FIG. 2.

[0030] The accurate definition of the image contour 37 derived in the display 8 allows correspondingly accurate determination in calculation 9 of the area within that contour; the volume involved can also be derived from successive slices scanned. The determination of area and volume is especially useful for diagnostic and accurate assessment 10 of the size of a tumour or lesion before and after treatment. It is similarly useful for assessment of arterial dimensions in angiography.

[0031] Moreover, the accurate definition of the image contour 37 derived in the display 8, is particularly useful for segmenting anatomical structures for diagnostic and treatment planning 11. Furthermore, the ratio of intensities of two scans are derived by processing 12 to derive values of relaxation times T₁, and T₂, and of proton density. The values are then represented in display 13 within the boundary of the image contour, utilising standardisation data derived from a couch-top 14 used within the scanner 1. The couch-top 14, which also provides landmarks for determining position coordinates, has the form shown in FIGS. 5 and 6, and will now be described.

[0032] Referring to FIGS. 5 and 6, the couch-top 40, which is of polystyrene bubble foam, has the form of a flat slab, and is supported on a curved foam base 41 that fits to the scanner table of the MR installation. Two zig-zag runs of tubing 42 are embedded within the top 40 to extend along the length of the couch either side of centrally-embedded straight tubing 43.

[0033] The tubing 42 of each zig-zag run is of double-bore rectangular cross-section, whereas the tubing 43 is of single-bore cross-section. The five bores defined by the array of tubing 42 and 43 may be filled respectively with the five MR solutions S₀ to S₄ of Table I, for standardization and calibration purposes. The four MnCl₂.4H₂O solutions, S₁ to S₄, cover the full range of values of T₁, and T₂ for anatomical tissues, and the fifth solution, S₀, of CuSO₄.5H₂O, is nominally equivalent to “loosely bound water”.

[0034]

TABLE I
		T1	T2
Solution		at 0.5T	at 0.5T
S0	1.25 g/l CuSO4.5H2O	200 ms	200 ms
S1	3.41 × 1016 Mn+2 ions/ml	840 ms	300 ms
S2	1.15 × 1017 Mn+2 ions/ml	440 ms	120 ms
S3	2.30 × 1017 Mn+2 ions/ml	250 ms	60 ms
S4	4.37 × 1017 Mn+2 ions/ml	150 ms	30 ms

[0035] The tissue types revealed by the T₁, T₂ and proton density values in display 13 are determined by processing 15 from look-up tables, and tissue densities are assigned within the image-contour boundaries in display 16. The images of display 16 are furthermore corrected empirically for geometry distortion in accordance with data supplied from memory 17. The data stored in memory 17 is derived using a set of drum phantoms of the spider-web form, and correction for geometry distortion is realistically effective principally because of the accuracy and spatial resolution with which image-contours are defined.

[0036] The tissue types assigned to the corrected images are utilised in display 16 through the Bulk Heterogeneity Correction method described by Richard A, Geise et al, Radiology, 124:133-141, July, 1977, to establish for each image a normalised tissue density value; the up-datable look-up table for this is stored in memory 18. Accordingly, the display 16 when used at step 19 in conjunction with the positional datums derived from the couch 14, has all the tissue contours accurately mapped out with their respective tissue densities and locations. This establishes an accurate and readily-usable, stand-alone alone basis for diagnosis and treatment planning, and enables a true three-dimensional assessment and plan to be made when both orthogonal and oblique MR images are involved.

[0037] Although the method and system of the invention have been described above in the medical context they are applicable more widely than this, for example, in engineering, in physical science and in the field of instrumentation generally. Moreover, the method and system is not limited to MR imaging, but may be utilised where other forms of imaging are involved. The steps and structure represented in FIG. 1 by ‘boxes’ 1 to 11 are just as applicable to computer assisted tomography (CT), as they are to MR imaging. Other forms of imaging to which the invention is applicable include X-ray radiography, film- or print-image transformation to digital form, digital X-ray fluorography, ultra-sound imaging, nuclear medicine, positron emission tomography (PET) and other camera or imaging. The technique is particularly suitable for use in X-ray digital fluorography, in which small structures under study are highlighted by injection of contrast liquids; the small structures may also be isolated from surrounding interfering effects by using an image subtraction technique.

[0038] The inherent resolutions of X-ray radiography, ultra-sound imaging, nuclear medicine, and PET scanning are relatively low, and some are used for real-time study. Only the individual still frame or hard-copy images may be re-processed.

[0039] In the context of engineering, physical science and the field of instrumentation, the invention is applicable to one-dimensional imaging as used, for example, in regard to bar-code patterns, the spectrum of DNA analysis, iris patterns of eyes (for example, for identification purposes in commercial banking), finger-print identification, and emission spectroscopy. The invention is also applicable to two-dimensional imaging, for example, in relation to images obtained by satellite or pattern recognition, or from a surveillance camera or during laboratory experimentation. As a general matter, the invention is applicable where there needs to be accurate determination of the edge position in an image versus the true object-edge position, for the purpose, for example, of measurement of the positional displacement between object and image, distortion correction and manufacturing control.

[0040] As a further example of application of the present invention, a method and system that uses CT and MR imaging in conjunction with one another, will now be described.

[0041] The major contribution to the magnetic-resonance (MR) signal comes from the abundant protons content of water molecules and protein. It is a quantum process at the Larmor frequency according to the magnetic field in use. The ‘T₁-weighted’ and ‘T₂-weighted’ MR signals from protons provide contrast numbers that are relative in scale, whereas in CT, the X-ray absorption is a polychromatic attenuation process affected by the electron densities of all the atoms presented within the X-ray beam. There is no equation to correlate the CT number (or the linear attenuation coefficient, electron density, or tissue density) with the MR-contrast numbers; no direct calibration between the two types of signal is possible. This lack of correlation is confirmed by consideration of bone and air which are at opposite ends of the CT contrast (absolute) scale using water as the base-line reference, but which are at the same end of the MR-image contrast (relative) scale owing to their common low proton-population.

[0042] The lack of correlation between the CT and MR signals acts against their use in combination for imaging purposes, but the present invention provides a method and system by which the advantages of each may be utilised to improve image resolution and contrast information.

[0043] In the latter regard, CT provides a high spatial resolution but only in regard to view normal to the transverse slice. Resolution for all re-constructed non-transverse planes is poor owing to the need to use elongate voxels to improve signal-to-noise ratio. Also, the partial-volume effect of using elongate voxels may give rise to detection errors at the thin edge of a contrast profile of a lesion. MR, on the other hand, can give the same high degree of spatial resolution viewed in the normal direction to any image-slice plane, and can also provide isotropic resolution with cubic-voxel volume imaging.

[0044] To this end, multiple-slice transverse CT scans are collected across a section of the volume of interest in a patient or other subject. Corresponding multiple-slice transverse MR scans of the same volume are also collected. The slice thickness of the latter scans may be one half, or smaller, of the thickness of the CT slices, and may be collected two-dimensionally or three-dimensionally. The patient or other object scanned is constrained throughout on a couch that provides landmarks with respect to a coordinate reference arrangement on the couch-top. This is to ensure the reproducibility of, for example, anatomical positions and features to the first order accuracy for the corresponding CT and MR scans, and possibly for radiation treatment to be made.

[0045] The respective transverse planes of the CT and MR images are processed individually in the method and are matched with one another in a de-convoluted space for the CT and MR images. The two sets of de-convoluted maps are then merged together to a second order of accuracy in order that the CT numbers may be transferred over to replace the corresponding MR contrast numbers. Once this has been achieved, non-transverse (or oblique) planes can be obtained from the two-dimensional MR images, or from the re-arrangement of the corresponding voxels of the three-dimensional volume images; where two-dimensional MR is used, a further step of contrast transformation may be required.

[0046] The respective transverse planes of CT and MR images are processed individually by using ‘boundary’ or ‘finger-print’ matching techniques in a de-convolution space for the CT & MR images. In these transverse image planes, in particular when used medically, the skin-contour features along the sides of the patient may be best used for second-order alignment and matching purposes, as they are less affected by patient-movements. Transitional error may be readily corrected with respect to the coordinate positions of a rectangular tubing system embedded in the couch-top (for example that described above with reference to FIGS. 5 and 6). The processed data may then be used for a ‘diagnostic and statistics software package’ of CT image versus MR image for their exactly corresponding transverse slice(s), and an associated ‘statistical package’ for accurate computation of the ‘true’ area, and then the ‘true’ volume, of a lesion or tissue profile or contour.

[0047] The two sets of de-convoluted maps may also be merged together to a second order of accuracy in order that the CT numbers may be transferred over to replace the corresponding MR contrast numbers. Once this has been done, non-transverse (or oblique) planes are obtained from the two-dimensional image or from the re-arrangement of the corresponding voxels of the three-dimensional volume; in the two-dimensional MR case, a further step of contrast transformation is required. The transferred contrast data may then be used in a three-dimensional radiotherapy treatment planning software package for an in-plane, oblique-image pseudo-three-dimensional approach using MR images.

[0048] Software required for de-convolution processing of image data according to the invention may be implemented in conjunction with a least-squares curve-fitting method. A mathematical model of the method, from which the required software can be readily developed, will now be given in relation to FIG. 7 which indicates the convolution of an LSF L(x) with a step function SF having a step-down edge SDE, to produce an image ERF E(x). Considering the values of E(x) resulting from the convolution at points x=1 to x=n of SF: $\mathrm{at}x=1\text{:}\sum _0^{a}L\left(x\right)\Delta x={E\left(x\right)}_1$ $\mathrm{at}x=2\text{:}\sum _0^{a}L\left(x\right)\Delta x-\sum _{a-1}^{a}L\left(x\right)\Delta x={E\left(x\right)}_2$ $\mathrm{at}x=3\text{:}\sum _0^{a}L\left(x\right)\Delta x-\sum _{a-2}^{a}L\left(x\right)\Delta x={E\left(x\right)}_3$ $\mathrm{until}\mathrm{at}x=n\text{:}\sum _0^{a}L\left(x\right)\Delta x-\sum _1^{a}L\left(x\right)\Delta x={E\left(x\right)}_n$
For de-convolution, that is to say, for the ERF E(x) to manifest the LSF L(x): ${E\left(x\right)}_1-{E\left(x\right)}_2=\sum _{a-1}^{a}L\left(x\right)\Delta x={L\left(x\right)}_{\left(a-1\right)-a}\Delta x$  {E(x)₁−E(x)₂}/Δx=L(x)_(a-1)−a
thus:
{E(x)₂−E(x)₃}/Δx=L(x)_{(a-2)−(a-1)}
leading to
{E(x)_(n-1)−E(x)_n}/Δx=L(x)_(n-1)−n

[0049] Thus, the same shape of LSF is recovered by the de-convolution process independently of the sense of-the ERF; it does not matter whether a roll-up or a roll-down ERF of a positive or a negative contrast profile, is involved. This is an important property as a positive contrast contour will have roll-up (from low to high) ERFs at both ends whereas a negative contrast contour will have roll-down (from high to low) ERFs at both ends.

[0050] In the practical implementation, the LSF is derived from the ERF by de-convolution using a running filter. This enhances the accuracy of the method in overcoming the problem of noise that affects the digitised pixel values of the image. Use is made of a least-squares fitting of a set of data points that advances along the whole length of the function from one point to another.

[0051] Assuming that y=a₀+a₁x+a₂x² represents the ERF curve a five-point or seven-point fit is used, and the normal equation becomes: $\begin{array}{ccc}{\Sigma }_1& \Sigma x_i& \Sigma x_i^2\\ \Sigma x_i& \Sigma x_i^2& \Sigma x_i^3\\ \Sigma x_i^2& \Sigma x_i^3& \Sigma x_i^4\end{array}\begin{array}{c}\mathrm{a0}\\ \mathrm{a1}\\ \mathrm{a2}\end{array}=\begin{array}{c}\Sigma y_i\\ \Sigma x_i{y}_i\\ \Sigma x_i^2{y}_i\end{array}$

• [0052] where: i is 1+n, . . . 5+n; . . . until m−5, . . . m or
  • [0053] i is 1+n, . . . 7+n; . . . until m−7, . . . m n is 0,1, 2, . . . ; and
  • [0054] 1, . . . m is the span of the ERF profile.

[0055] The solution may be derived from either: $\begin{array}{c}\mathrm{a0}\\ \mathrm{a1}\\ \mathrm{a2}\end{array}={\begin{array}{ccc}{\Sigma }_1& \Sigma x& \Sigma x^2\\ \Sigma x& \Sigma x^2& \Sigma x^3\\ \Sigma x^2& \Sigma x^3& \Sigma x^4\end{array}}^{-1}\begin{array}{c}\Sigma y\\ \Sigma \mathrm{xy}\\ \Sigma x^{2}y\end{array}\mathrm{or}\text{:}$ $\frac{\mathrm{a0}}{\begin{array}{ccc}\Sigma y& \Sigma x& \Sigma x^2\\ \Sigma \mathrm{xy}& \Sigma x^2& \Sigma x^3\\ \Sigma x^{2}y& \Sigma x^3& \Sigma x^4\end{array}}=\frac{\mathrm{a1}}{\begin{array}{ccc}{\Sigma }_1& \Sigma x& \Sigma x^2\\ \Sigma x& \Sigma \mathrm{xy}& \Sigma x^3\\ \Sigma x^2& \Sigma x^{2}y& \Sigma x^4\end{array}}=\frac{\mathrm{a2}}{\begin{array}{ccc}{\Sigma }_1& \Sigma x& \Sigma y\\ \Sigma x& \Sigma x^2& \Sigma \mathrm{xy}\\ \Sigma x^2& \Sigma x^3& \Sigma x^{2}y\end{array}}=\frac{1}{\begin{array}{ccc}{\Sigma }_1& \Sigma x& \Sigma x^2\\ \Sigma x& \Sigma x^2& \Sigma x^3\\ \Sigma x^2& \Sigma x^3& \Sigma x^4\end{array}}$

[0056] For both of these equations to be valid: $\begin{array}{ccc}{\Sigma }_1& \Sigma x& \Sigma x^2\\ \Sigma x& \Sigma x^2& \Sigma x^3\\ \Sigma x^2& \Sigma x^3& \Sigma x^4\end{array}\ne 0$

[0057] The gradient at dy/dx_(3+n) or dy/dx_(4+n) can then be derived and plotted against x for the LSF profile.

[0058] The graph of dy/dx against x gives the LSF profile. The peak of this profile is located centrally of the mid-points of the ascending and descending limbs of the graph. These points define the extremes of the full-width-half-maximum (FWHM) of the profile and the mid-point of this is determined with an accuracy of sub-pixel level owing to the ‘average’ effect.

[0059] The point spread function (PSF) is the two dimensional profile which may be derived, in practice, from the two corresponding LSFs orthogonal to one another within an image plane. The peak position of the PSF profile is, therefore, from the ‘mean’ or ‘cross-over’ of the two peaks or the two LSF profiles. The PSF is obtained in practice from two orthogonal axes in a two-dimensional plane.

[0060] The generation of the LSF (or PSF) is, after phase-reversal correction, independent of the roll-up or roll-down nature of ERFs at the edges of the contrast contour. In other words, it is independent of the sense and the absolute value of the contrast numbers within the contour. The peak position of the LSF (or PSF) is the central half-way (50%) point of the roll-up or roll-down ERF which is the optimum position for true-edge definition