Radiographic scoring method

[0001] The present invention relates to computer-assisted radiological scoring of radiographic images.

[0002] In radiological practice, radiographic images are used for a variety of diagnostic purposes. Radiographs can be captured to detect and identify lesions, to diagnose an underlying pathology from radiological manifestations, to assess the existence and extent of a fracture etc.

[0003] There exist also a domain of specialized radiological examination procedures, which cannot be classified into any of the enumerated tasks, which is the field of radiological scoring.

[0004] In radiographic scoring methods,the radiographer or clinician already knows the pathology but wishes to assess the degree of severity of the disease or the developmental stage of the patient.

[0005] Several scoring methods exist in the state of the art. All scoring methods have the following specific characteristics in common:

• • [0006] (1) a number of pre-determined anatomical sites on a radiograph is examined; the number of anatomical sites being constant for a given method;
  • [0007] (2) the specific appearance of the anatomical site is rated against a number of reference pictures, each reference picture corresponding to a developmental or pathological stage, and the rating results in a stage being assigned; the number of reference stages being constant for the given anatomical site;
  • [0008] (3) each stage has an associated numerical score, the value of which in general differs according to the stage and the skeletal site;
  • [0009] (4) all scores are totaled and translated into a clinically useful index.

[0010] Although rating refers to the process of assigning a stage to an anatomical site and scoring refers to the attribution of a value to the stage, we will call both processes ‘scoring’ in the sequel.

[0011] Radiological scoring methods, which adhere to the aforementioned principles, are e.g. bone age determination (also known as skeletal maturity assessment), rheumatoid arthritis scoring, osteo-arthritis scoring, ankylosing spondylitis scoring, and osteoporosis assessment.

[0012] In the following scoring methods applied to different applications will be explained.

[0013] A first example of an application wherein radiographic scoring is used, is skeletal maturity assessment, a procedure frequently performed in paediatric radiology.

[0014] Biologically, it is desirable to assess the maturity of the whole skeleton, but because of several practical difficulties such as the time needed for the assessment of so many bones, the expense of radiographic film, the risk of excessive radiation exposure etc. a specific area is chosen for the assessment of skeletal maturity.

[0015] An area of the human skeleton, which has received considerable attention as a source of clinically relevant maturity indicator, is the hand-wrist. Based on a radiological examination of skeletal development of areas of the non-dominant hand and wrist, the bone age is assessed and compared to the chronological age. A discrepancy between these two values indicates abnormalities in skeletal development. This examination is performed on children with growth abnormalities to affirm clinical suspect, to predict height at adult age, or to monitor the effect of treatment of metabolic diseases.

[0016] A common method for assessing the skeletal maturity is the atlas method, and the hand-wrist was the first area of the body for which atlases became available. Currently, there are two frequently used atlas methods operating on a hand-wrist radiograph.

[0017] The most frequently used method (78%) is the Greulich and Pyle (GP) method. In this method, the radiograph of non-dominant hand wrist (mostly the left hand) is compared to a reference series of hand-wrist radiographs displayed in this atlas. Each hand-wrist radiograph corresponds to a certain year of bone age. The age interval between successive hand-wrist plates varies from 3 to 6 months. The reference radiograph, which globally compares best with the clinical image, is selected as the best match and its associated age is called the bone age. This atlas-derived bone age is then compared with the chronological age of the patient, and the age difference is used for diagnostic purposes.

[0018] The manual effectuation of this method is prone to error or susceptible to ambiguity.

[0019] Firstly, different radiologist performing this procedure may have different training experience; hence substantial inter-observer and intra-observer variance may result from the subjective nature of this comparison.

[0020] Second, the comparison is a global one. Depending on the weight the radiologist attaches to the similarity or dissimilarity of specific skeletal sites on the hand-wrist radiograph, ambiguous results may be obtained.

[0021] A method, which aims at minimizing the subjective observer errors, is the Tanner and White house method (TW2 method). Scores are assigned to grades of skeletal maturity indicators, and the sum of scores is later transformed to a skeletal age (also called bone age). Slight modifications to the reference values and charts have recently been made to reflect changes in the population, resulting in the TW3 method. Descriptions and manual rating of the stages of the bones, however, have not been altered.

[0022] The TW2 method is currently effectuated in paediatric radiology as follows.

[0023] A radiograph from the non-dominant hand and wrist is made either by (a) conventional screen/film recording system, or (b) by digital means such as film digitisation, a computed radiography system or a direct radiography system (based on direct or indirect flat panel, or CCD).

[0024] In case a film is used, it is viewed on a light box. A digital image on the other hand may be either printed on film and viewed conventionally on a light box, or it may be displayed on a viewing station.

[0025] Knowledge of gender and chronological age of the patient are the only two clinical data needed to start the analysis.

[0026] For each skeletal site indicated in the atlas for which a score needs to be established, the corresponding skeletal site in the actual hand-wrist radiograph is searched for visually. Specific salient anatomic details of the skeletal site are retained mentally.

[0027] Then the attention is directed to the different stages depicted in the atlas to identify the most similar one.

[0028] Use is made of (1) the reference pictures (two radiographic reproductions per stage in the TW2 atlas, to reflect the range of variation within the stage), (2) a sketch or drawing of the skeletal site depicting the outline of the bones, the radio-opaque lines on or within the margin of the epiphyses and bones and pointers to salient features of the particular stage, and (3) one or more criterions per stage, which are textual explanations hinting at the most prominent image features of the stage.

[0029] In the event of doubt, whether a particular feature of a certain stage is present or not in the actual image, visual attention is redirected back to the hand-wrist image on the light box or computer display.

[0030] As a matter of course, the correct skeletal site to be assessed needs again be relocated in the hand-wrist radiograph before such confirming visual analysis can be effectuated.

[0031] In many cases, if not all, several switches back and forth between hand-wrist radiograph and the reference stages in the atlas are needed before a conclusive assignment of a reference stage can be made.

[0032] The score, corresponding to the matching stage, is written down on paper or typed in a computer spreadsheet or database.

[0033] This iterative process of hypothesis generation and hypothesis verification as to the actual stage of a skeletal site is repeated s for every skeletal site.

[0034] In the full TW2 method, 20 skeletal sites in the hand-wrist need to be assessed this way; therefore this scoring method can be time-consuming, demanding and error-prone.

[0035] The total score (on a scale between 0 . . . 1000) is translated into a bone age by a table look up operation. A different table is applied depending on the gender of the patient.

[0036] The bone age derived is compared with the chronological age of the patient and the difference between them is used as an element in the clinical diagnosis process.

[0037] Analogous to the Greulich and Pyle method, care must be exercised when interpreting the bone age to chronological age difference for patients belonging to a race other than the Caucasian type, because the atlas shows reference pictures and associated scores for this ethnic type only.

[0038] Unfortunately, although reliable, the above method is complex to operate for several reasons:

• • [0039] It requires a well-trained radiologist or radiology operator.
  • [0040] Because it requires the assessment of a great number of different skeletal sites, a manual method is time consuming.
  • [0041] The application of the method by means of analogue screen-film radiography is particularly cumbersome. It requires 3 distinct media to effectuate this procedure:
    • [0042] An X-ray film of the hand or other skeletal sites, displayed on a light box. With the emergence of digital radiography modalities (film digitisation, computed radiography, digital radiography sensors), the digital image may be displayed on a computer display instead. However, such electronic medium still is physically distinct from the remaining components.
    • [0043] The TW2 atlas illustrating the different reference skeletal stages to compare with, and depicting the conversion table to manually convert the total score into a bone age.
    • [0044] Pencil/Paper to note and add the scores of each selected stage in the patient dossier. Alternatively, electronic spreadsheets may be used in conjunction with a database to store and compute the total score and the conversion to bone age. This way of operation is particularly cumbersome when scores determined at regular time intervals need to be retrieved and compared for clinical evaluation over time.
  • [0045] Both TW2 and GP atlases have been established for the Caucasian racial type. Therefore, the bone age derived from these atlases is in principle valid for this type only.

[0046] Another example of an application wherein radiographic scoring is used, is rheumatoid arthritis.

[0047] Conventional rheumatoid arthritis scoring (RA scoring) is based on comparison of the actual film radiography of hands and feet with reference pictures printed on film.

[0048] To establish the database of reference pictures, all stages of a joint or a group of joints are printed on one sheet of film in life size.

[0049] Scoring proceeds by displaying standard reference films and the actual radiography on a light box, and writing down scores associated with the matching stage of each joint.

[0050] Among other indicators, the most specific deformations in RA are erosions and joint space narrowing. The radiological study of these deformations is complementary; therefore, both are being assessed in a scoring method.

[0051] Radiological assessment serves as a recognized standard for the evaluation of rheumatoid arthritis. For standardized, epidemiological and therapeutic evaluation, scoring or grading systems have been proposed on X-ray images of specific skeletal sites. X-ray images have the advantages to be able to record the history of the damage of a joint, because the damage is mostly irreversible. Previous images can be recalled in a later stage, to assess the evolution and to subject them to newly developed scoring systems. The effectivity of drugs can be traced.

[0052] These advantages however, can only be obtained when the images are of sufficient diagnostic quality: (a) for serial assessment, the positioning of the joints during radiation exposure are of substantial importance, (b) correct exposure is important because overexposure or underexposure affects e.g. the correct assessment of erosions, and (c) the images recorded with sufficient resolution are essential for a reliable evaluation of previous erosions.

[0053] Still another example of an application wherein radiographic scoring is used, is osteoporosis scoring.

[0054] Spine or hip fracture due to minor or no trauma is an essential feature of symptomatic osteoporosis. Conventional radiographs are widely used to confirm or disprove suspected osteoporotic fractures, and may also demonstrate progress of existing fractures or development of new ones.

[0055] The scoring systems using conventional radiographs suffer the same drawback of other film-based diagnostic scoring systems. As indicated, the problem of repeated re-location of the anatomical site is even more compound because the spine is a highly repetitive structure of vertebrae having almost identical shape.

[0056] It is an object of the present invention to provide a user-friendly radiographic scoring method that overcomes the drawbacks of the prior art scoring methods, such as the need for different media and the need for repeated focusing of the attention between images on different media.

[0057] Further objects will become apparent from the detailed description given hereinafter.

[0058] The above-mentioned objects are realised by a method of radiographic scoring of a radiographic image comprising the steps of

• • [0059] acquiring (1) a digital image representation of an image to be subjected to radiographic scoring,
  • [0060] applying (2) the acquired image representation to a display device and generating a displayed image,
  • [0061] determining (3) at least one region of interest on the displayed image,
  • [0062] retrieving (4) for each of the determined regions of interest a number of reference stage images from a database (5), each reference stage image having an associated score number and
  • [0063] displaying (6) the reference stage images on said display device,
  • [0064] selecting (7) a displayed reference stage image which most optimally matches with a displayed region of interest,
  • [0065] associating (8) a score number corresponding with the selected reference stage image with the displayed region of interest,
  • [0066] combining (9) score numbers associated with said regions of interest into an overall score number pertaining to said radiographic image.

[0067] A displayed reference stage image which most optimally matches with a displayed region of interest can be selected by

• • [0068] displaying each of said region(s) of interest and the reference stage images pertaining to the displayed region of interest in close proximity to each other, and
  • [0069] visually comparing a displayed region of interest and displayed reference stage images pertaining to the displayed region of interest,
  • [0070] selecting a displayed reference image which visually most optimally matches with a displayed region of interest.

[0071] A reference stage image is an image which is representative for a developmental stage of a specific skeletal site. These reference stage images are commonly selected by an expert. In most cases these reference images are accepted as world wide standards.

[0072] In one embodiment the step of determining a region of interest is guided by means of a stencil consisting of a sketch of a body part of the patient that is to be evaluated. The sketch depicts at least one of the outline of the following three items: the body part, bone contours of bones within the body part, anatomical contours, as well as region of interest delineations superimposed on said sketch.

[0073] In one embodiment a region of interest is determined and extracted on a display device by manual operation.

[0074] The following steps are performed:

• • [0075] displaying a stencil comprising a sketch of a body part to be evaluated and region of interest delineations superimposed on said sketch,
  • [0076] mapping or dragging the region of interest delineations onto the actual position of the body part to be scored in the displayed image,
  • [0077] adjusting the shape of the region of interest by rotation and scaling so that it substantially matches the actual geometry of the local anatomy in the displayed image,
  • [0078] selecting image data within the dragged delineations as region of interest.

[0079] In one embodiment a digital image representation of a region of interest is extracted from a digital image representation of said image by re-sampling and interpolation.

[0080] Preferably a stencil is selected according to identification data of said radiographic image such as gender, age and examination type (hand, foot etc.).

[0081] After the regions of interest superimposed on the skeletal outlines in the stencil, are mapped to the correct position in the image, the image data in the regions of interest in the displayed image are extracted. Preferably the extracted regions of interest are subjected to rotation or to an affine transformation so as to have them in upright position.

[0082] Regions of interest may also be subjected to image enhancing processing (see below).

[0083] Next the ensemble of ROI's is subjected to the remainder of the scoring steps: display in proximity to the reference stage images, selection of the best matching reference stage image, associating of a score number and combining of score numbers of all regions of interest into an overall score number of the entire image.

[0084] This procedure is referred to as manual scoring procedure.

[0085] Alternatively a semi-automated method can be applied wherein all steps of the method described higher are also applied but wherein additional decision support is generated to assist the selection of the optimally matching reference stage image. In this semi-automated method additional support is generated to help to assess whether a specific feature is radiologically manifested in the region of interest. More specifically for each of the regions of interest (ROI) a sequence of specific image processing operators is applied to compute ROI specific feature values and to subject the computed feature values to a judgment device in order to determine presence or absence of a feature. Presence or absence of a feature will be used for assessing which reference stage image is most optimally matching. This system is called semi-automated scoring since the ultimate decision whether or not a computer-suggested decision is accepted, is still made by a radiologist. It may be possible in the current method that not for all regions of interest automatic feature based support is available. Furthermore it may also be possible that the features are not computed exhaustively for all stages of a specific region of interest.

[0086] In an enhanced semi-automatic operation mode all features applicable for a specific region of interest are computed and input to a judgment device to derive the stage of the region of interest. The difference with the above semi-automated method is that, in order to derive a suggestion as to the stage of the region of interest, all relevant features covering the whole developmental stage of the spectrum of the region of interest must be computed and used in the classification process. The only task left for the radiologist is then to accept or reject the computer suggestions for individual features. This form of semi-automated scoring will therefore be much faster, but because of the plurality of input features, the judgment device is faced with a more complex task. Bayesian classifiers and artificial neural networks are used for this purpose.

[0087] Data within a region of interest and/or data of a reference stage image may be subjected to image processing. Image processing is e.g. spatial reformatting and/or processing for the purpose of enhancing the intrinsic image quality. For example multi scale contrast enhancement can be applied, as will be explained further on.

[0088] Another aspect of this invention relates to a computer program product as set out in the claims.

[0089] Further details of the different steps performed by the computer program product are set out in the dependent claims.

[0090] A computer program product is meant to encompass a software product in the form of an electric signal as well as a computer program on a computer readable medium such as a CD-ROM.

[0091] Still another aspect relates to a computer readable medium such as a CD-ROM comprising program code adapted to carry out the above method.

[0092] Specific features for preferred embodiments of the invention are set out in the dependent claims.

[0093] It will be clear that the scoring method according to the present invention copes with the problems of the prior art.

[0094] The ROI extraction and display in the neighborhood of the reference stage images alleviates the problem of repeated focus of attention towards the correct skeletal site.

[0095] Furthermore, by applying image processing such as multi-scale enhancement, the available dynamic range of the output medium can be used optimally to render all relevant image details.

[0096] The current invention is particularly suited to implement different variations of the Sharp score in rheumatoid arthritis, since the ROI stencil can be customized so as to include only an abbreviated set of ROI's.

[0097] Alternatively, a score, based on a reduced number of ROI's, can also be derived from a superset including more ROI's having been evaluated previously.

[0098] The risk to evaluate a wrong ROI is substantially reduced, because the ROI stencil depicting fewer skeletal sites serves as a reliable hinting mechanism.

[0099] In the prior art the task of focusing on a ROI and scoring is alternated for all skeletal sites. In one embodiment of the present invention the order has been changed, first all ROI's are determined and next the scoring is performed.

[0100] In the field of osteoporosis the invention is particularly useful since by first extracting regions of interest around each vertebra, the visual search process is greatly simplified. No confusion with other vertebrae is possible when presenting each ROI in juxtaposition with the reference radiographs and graphic pictures.

[0101] For bone age determination, errors that might occur in the prior art due to inadvertently exchanging score numbers for boys and girls, are avoided.

[0110] Image Acquisition and Display

[0111] A digital image representation of an image to be subjected to radiographic scoring is acquired (FIG. 1, numeral 1).

[0112] A large variety of image acquisition systems are applicable. The radiation image can for example be recorded on a photostimulable phosphor screen. The screen carrying the radiation image is then read out by scanning it with stimulating radiation and detecting the image-modulated light which is emitted upon stimulation and by converting the image-wise modulated light into a signal representation of the radiation image.

[0113] In an alternative embodiment the radiation image can be stored in a direct radiation sensor rendering a signal representation of a radiation image.

[0114] Still further alternatives for acquiring a digital signal representation of a radiation image, such as scanning of radiographic film carrying a radiation image, are possible.

[0115] The acquired image representation is next applied to a display device and the image is displayed (FIG. 1, numeral 1).

[0116] ROI Positioning and Selection

[0117] Next, in the ROI (region of interest) positioning stage at least one ROI is determined on the displayed image (FIG. 1, step 2).

[0118] In the following explanation will be referred to the hand as a body part that is subjected to evaluation. However, the present invention is not limited to scoring methods using the hand as a body part to be evaluated.

[0119] In this context, the region of interest is the image region covering the joint or bone relevant for the examination. The ROI positioning stage aims at positioning all regions of interest (ROI's) on the body part that is examined. Although each of the ROI's may be extracted separately and analyzed immediately, a more convenient work flow is obtained by first extracting all ROI's simultaneously before processing them.

[0120] ROI positioning can be performed in an operator-guided way or alternatively in an automatic way.

[0121] In the case of operator-guided ROI positioning a method using a digital stencil can be used to facilitate the positioning.

[0122] In this context, a stencil is a digital sketch of a hand wrist image depicting the outline of the hand and all hand bone contours. The delineations of all possible ROI's are superimposed on these outlines.

[0123] As has already been explained higher, a hand-wrist image is taken as an example. Other body parts may be examined depending on the type of application and the radiographic scoring method.

[0124] For example for the TW2 analysis, these ROI outlines will typically be rectangular regions around joints of the tubular (radius, ulna, metacarpal and phalange) bones and around the carpal bones.

[0125] FIG. 2 shows a stencil used for bone age determination. The stencil shows all rectangles of regions of interest (dashed lines) superimposed on an outline of the diagnostic entities.

[0126] Each of the ROI outlines is mapped or dragged by the operator (translated e.g. by mouse and arrow keys) to its actual location in the displayed image, and the stencil serves as a guideline to indicate the intended image region the ROI should encompass, with sufficient precision for further analysis.

[0127] Basic operations such as translation, rotation and scaling may be applied to the stencil ROI so that the resulting ROI fully comprises its associated hand bone without irrelevant bones being present which would make the identification of the target bone ambiguous.

[0128] The manual placement may be speeded up by first translating, rotating and scaling the stencil in its entirety so that it roughly coincides with the actual hand and bony outlines, and by further adjusting the position of the individual ROI's so that each ROI accurately covers its intended bone/joint location.

[0129] An alternative for speeding up is the semi-automated positioning relative to a few user-defined points.

[0130] Because the superposition of all ROI's in the stencil may be visually confusing when they overlap substantially (this is especially the case for the carpal region of interest), the ROI's may be sequentially displayed in the stencil and positioned in the actual region. Size and orientation of the rectangles may be modified e.g. by corner point dragging to match the actual size and the orientation of the displayed image.

[0131] Because the hand-wrist anatomy changes its aspect in accordance with age, the stencil must preferably reflect this in order to more faithfully represent the actual skeletal sites. Therefore, preferably a number of hand-wrist stencils are stored as a function of chronological age, gender, race etc. In accordance with the patient's identification data such as age, race, gender, etc. which may be entered into the computer, the corresponding sketch can be selected.

[0132] Although the chronological age may not be representative for the skeletal maturity status of the patient (the bone age scoring is precisely meant to detect significant differences from it), it serves as a rough estimate for the initial selection of the stencil.

[0133] In an alternative embodiment, the ROI positioning can be automated by image processing algorithms, so that no user interaction is required in the placement of the ROI regions.

[0134] Automatic placement can be based on first automatically segmenting the bones based on Point Distribution Models (PDM) and Active Shape Models (ASM) such as disclosed in the prior art (Cootes TF and Taylor CJ, Active Shape Models—“smart snakes”, Proc. British Machine Vision Conference, Springer Verlag 1992, p. 256–275). An example for segmenting the individual vertebrae in the spine is given in (Smyth et al, Automatic measurement of vertebral shape using active shape models, Image and Vision Computing, Vol. 15, p. 575–581, 1997), for the hand and hip in (Behiels et al, Active shape model based segmentation of digital X-ray images, proceedings 2and international conference on medical image computing and computer-assisted intervention—MICCAI′99, lecture notes in computer science, Vol. 1979, p. 128–137, 1999).

[0135] Secondly, depending on the scoring method, the ROI's searched for may be located in the following ways with respect to the segmentation of the bones:

• [0136] at outer borders of the bony segmented regions (which is the case for RUS bone age scoring because the proximal and distal interface regions of neighboring bones contain the epiphyses and corresponding metaphyses; and for rheumatoid arthritis scoring, because these regions are affected by joint space narrowing and is erosion defects), around the bony region (which is the case for individual vertebra of a spine, such that each ROI contains the segmented vertebra and part of both neighboring vertebrae), inside the bony region at pre-determined locations given by human anatomy (which is the case for carpal bone age scoring when the region of carpal bones is segmented as a whole, because the each carpal bone has a fixed position with respect to all other carpal bones contained in the ROI).

[0137] FIG. 3 is an example of automatically segmented RUS bones. FIG. 3A shows the initial bounding box placement for automatic segmentation of the associated bones and FIG. 3B shows the segmentation of the phalanxes and carpal bones. The interface regions at the extremal ends of these bones determine the ROI's for bone age scoring.

[0138] Scoring Procedure—General Overview

[0139] The methodology used to achieve radiographic scoring according to the present invention will be described using bone age assessment as an example.

[0140] Whenever necessary, details specific for other clinical scoring methods( rheumatoid arthritis, osteo-arthritis, osteoporosis and ankylosing spondylitis) will be indicated.

[0141] A general overview of a scoring system is shown in FIG. 5.

[0142] The system comprises an engine and a user interface. The engine controls the navigation through different screens (e.g. one screen per region of interest), retrieves and supplies the previous scores and regions of interest from the database, controls juxtaposing of the regions of interest and the reference stage images on the display device, initiates appropriate image processing of the region of interest (e.g. contrast enhancement) and geometric manipulation (e.g. zooming) to optimize visual matching, initiates computation of ROI features on the radiologist's request for decision support, updates spreadsheet and database for newly retained region of interest scores.

[0143] The summation device requires the individual score values of the regions of interest (and feature values) as input values and computes and outputs a total overall score of the radiation image.

[0144] It may further compute sums of features which are output as an index (as will be explained further on).

[0145] The total score output by the summation device may be subjected to LUT (look up table) based transforms (the exact transform being dependent on the patient identification data) to convert the overall score to a clinical score value.

[0146] A report may finally be generated.

[0147] Comparison of ROI's with Reference Stages

[0148] Referring again to FIG. 1 (numeral 4), in the following step a radiologist or radiology operator visually compares each ROI with reference stages so as to find the most optimal matching stage. This procedure is performed successively for every ROI.

[0149] To this end a data base (FIG. 1, numeral 5) is composed in advance. The database comprises for each individual ROI a number of reference stage images corresponding with different developmental stages, each of these reference images having an associated score.

[0150] To facilitate this comparison, a graphical user interface has been developed. FIGS. 6 and 7 show different embodiments of the organization of a graphical user interface on a computer screen used for performing the scoring procedure on each ROI.

[0151] In the user interface a number of reference images having an associated score number are displayed for each of a number of stages.

[0152] In one embodiment all reference images according to a certain stage are displayed on the same row (referred to as the row-wise arrangement). Reference images corresponding with different stages are displayed in different rows. If the number of stages (hence the number of rows) exceeds the display capacity of the screen, a scroll able window can be used. In the illustrated embodiment the leftmost two columns depict a set of reference images, two images per stage. Up to 8 stages (stage B to I) are used in the bone age assessment.

[0153] This embodiment of a user interface according to the present invention further comprises means for displaying one or more regions of interest ROI of an image to be scored. The display of the region of interest is situated in close proximity to the display of the reference image(s). In the displayed embodiment the third column is reserved for the display of the actual ROI to be classified.

[0154] In the illustrated embodiment a fourth column is provided which depicts a sketch of the bone with emphasis of particular anatomic features.

[0155] A fifth column may also be provided which displays a list of salient anatomical feature(s) in textual format, one or more for each of the different stages.

[0156] Means may be provided for displaying a tick close to the textual information for ticking off the textual information so as to indicate the fact that the radiologist perceives the feature in the actual ROI of the skeletal region.

[0157] It will be clear that the columns may be interchanged without substantially changing the classification process. For example, the actual ROI image may be placed in between the two reference images, instead of being placed between the reference image and the sketch image. Also, more than two reference images per stage may be used to achieve broader coverage of the grey value appearance of a certain stage.

[0158] Instead of a vertical arrangement of the reference data, a horizontal arrangement may be adopted, and the actual ROI may be slided horizontally through the scoring screen. Because all reference elements of a certain stage are positioned in the same column, this disposition is called the column-wise arrangement.

[0159] Two modes are available for displaying the ROI's in the scoring screen.

[0160] A first mode is the static mode in which each ROI is displayed in juxtaposition with all reference stage images.

[0161] A second mode is the dynamic mode in which a single ROI is displayed juxtaposed to reference stage images. This ROI is shifted up and downwards by user interaction e.g. by mouse movement, and is juxtaposed to the next or previous reference stage images.

[0162] Selection of Matching Reference Stage

[0163] The reference stage which most optimally matches with the displayed ROI is selected (FIG. 1, numeral 7) and the score number associated with the selected reference stage is retained (numeral 8) for generating the total score number of the total image (numeral 9). The total score can be generated as the sum of the individual scores of the ROI's. Additionally an index can be calculated, being a more complex accumulation of feature values pertaining to individual ROI's (e.g. SDI—spinal deformity index).

[0164] The interactive comparison of only (a) region(s) of interest of the hand-wrist X-ray image has major advantages over the comparison of the entire hand-wrist X-ray image with a reference image.

[0165] Because the actual region of interest only comprises the skeletal site to be classified without any extraneous anatomical detail visual comparison and stage assignment is made easier than in prior art methods. Also because in the present user interface a region of interest can be juxtaposed to the reference images and the sketch, the visual comparison and stage assignment is greatly facilitated over the prior art method.

[0166] Maturity Indicator

[0167] A radiographic feature of a skeletal site that assists in the assessment of the skeletal maturity is called a maturity indicator. A maturity indicator must be determined during the maturation of every child to be useful in the assessment of maturity. By definition, the grades of a maturity indicator appear in a fixed sequence for each bone, irrespective of stature, weight, health or other socio-economic factors.

[0168] This characteristic of human growth can be exploited in the user interface according to the present invention in the following way.

[0169] Stages and corresponding scores can be recalled from the archived data and displayed during assessment of the maturity of the current skeletal site. In the assumption that the scores previously assigned to a specific skeletal site are correct, the score to be assigned to a ROI of this site under current evaluation cannot be less than the previously assigned score.

[0170] Likewise in rheumatoid arthritis previous damage to the skeletal site cannot be undone. The score number of the last evaluation is carried forward to the present evaluation.

[0171] This property can be implemented in a specific embodiment of the user interface according to the present invention by disabling all reference stages that are less mature or that correspond to a less deteriorated stage, thus narrowing the search in a plausible manner. Such stages can be disabled e.g. by graying them out.

[0172] Still further narrowing the set of assignable stages can be achieved by marking, e.g. ticking off a characteristic feature in the textual information section on the scoring screen of the user interface. The user interface is then organized such that whenever such a feature of a certain stage is deemed to be present, no less mature stages can be selected. Reversely, when a feature of a certain stage is believed not to be present, said stage and all more mature stages are put to a non-selectable state.

[0173] Furthermore, to clarify the decision support, the time sequence of the archived regions of interests for a particular skeletal site can be displayed along with the chronological time at which each radiograph has been taken, so that the evolution over time can be assessed.

[0174] The duration of each maturity indicator can be determined and displayed graphically, thus showing how long the bone remained at the same level of maturity.

[0175] In order not to miss stages, schedules of serial (follow-up) radiographs can be planned relative to the inception of maturity indicators as opposed to classical chronological age planning. This feature is termed the longitudinal maturity study and the screen layout is depicted in FIG. 8. Time t₁ is called baseline and is the point in time at which the first scoring was made. Distances along the time axis are proportional to the time of actual exposure of the radiographs. Time t_n+1 is a time point in future at which a new scoring is planned, depending on indications derived from the visual analysis of the ROI sequences. The screen is made scroll able to allow for a large number m of ROI's or a large number of scoring studies over time.

[0176] While the statement, that the sequence of maturity indicators is always the same, is true for the grades of a single indicator for a particular bone, it is not accurate if-applied to several indicators of various bones. Precisely because each skeletal site is assessed separately, this characteristic of local differences in human bone growth does not influence the precision of the TW2 method, in contrast to the global method of Greulich and Pyle where all skeletal sites must be assessed simultaneously.

[0177] The availability of the reference data in digital format enables more easily customisation towards the ethnic type. Currently, the TW2 atlas has been established for the Caucasian type. Differentiation towards ethnic type is achieved either by supplying reference pictures and associated scores according to the patient's ethnic type, or by substituting new value of the TW2 scores only and keeping the classical reference pictures. This type of customisation obviates the need for establishing different atlases in the current operation of the TW2 method.

[0178] A TW3 atlas has recently been introduced. The operational principle is identical. The calculation of the total score number has been changed relative to the TW2 atlas in order to match better with the characteristics of the present population.

[0179] In the prior art, the scores needed be written down in the patient dossier or added manually into the appropriate database record. In one embodiment of the present invention no recording of the score is needed because the score can be automatically entered according to the row position of the ROI on the screen. On exit of the screen and continuation to the next screen of the next skeletal site, the score can be entered in the spreadsheet and database, running in a background process. The assessor may navigate through the screens to review previously accorded stages, and on leave of the program, an overview of all assigned scores, total score and corresponding skeletal age can be displayed. Graphs, depicting the progression of skeletal maturity of each skeletal site, and overall bone age progression with respect to chronological age, complement the skeletal maturity report.

[0180] The embodiments of the scoring methods according to the present invention may have other features distinguishing them from the conventional film-based operation.

[0181] Each of the stages of the scoring methods is examined as to their prominent image features. The appearance of these features may be enhanced in a feature-specific way.

[0182] The application of ROI specific image processing is shown in FIG. 4 which figure gives a more detailed overview of a specific embodiment of the present invention.

[0183] In U.S. Pat. No. 5,805,721 image enhancement based on multi-scale image decomposition is disclosed.

[0184] An image is decomposed into detail images at different resolution levels and a residual image. The detail images are then modified by applying at least one non linear conversion according to a conversion function which is non-linear, monotonically increasing, odd, and has a slope that gradually decreases with increasing argument values. Next modified detail images and the residual images are recombined to yield a reconstructed, processed image.

[0185] Using a suitable transformation of Laplacian pyramid coefficients, different visual aspects of the regions of interest can be enhanced.

[0186] Latitude reduction can e.g. be applied for enhanced soft-tissue swelling visualization (RA Larsen scoring) or for enhanced visualization of the onset of ossification (bone age scoring).

[0187] Edge enhancement can e.g. be applied prior to width measurement between edges (RA scoring and osteoartritis scoring) or to make assessment more reliable as to whether a border is convex or concave (bone age scoring).

[0188] Multi scale contrast enhancement can e.g. be applied to assess the presence of erosions (RA scoring) or to make assessment more reliable as to whether an epiphyses caps the metaphysics (bone age scoring).

[0189] If there are more criteria per stage, all applicable processing of the ROI can be performed and displayed simultaneously. Let m be the total number of ROI's, and i denoting the sequence number of a ROI, then a ROI-specific number n(i) of multi scale processing can be performed as depicted in FIG. 4.

[0190] These optimally processing of the ROI's are input to the scoring system.

[0191] Two modes of displaying the processed ROI's in the scoring screen can be made available:

[0192] Static display mode (FIG. 6): all different processing of a ROI are displayed simultaneously in juxtaposition with the reference pictures. User interaction is minimal in this mode (no cursor movements). Selection of the matching stage is achieved by clicking on a row entity (either the most similar reference picture or the particular processed ROI). In this figure ROI_a1 refers to a region of interest processed to enhance feature 1 of stage a, ROI_a2 refers to a region of interest processed to enhance feature 2 of stage a etc.

[0193] Dynamic display mode (FIG. 7): the displayed processing of a ROI is changed in accordance to the cursor up and down movements. The attributed stage corresponds to the row position of the ROI on leave of the screen. In this figure ROI_c1 refers to a region of interest processed to enhance feature 1 of stage c, ROI_c2 refers to a region of interest processed to enhance feature 2 of stage c.

[0194] The user may navigate back and forth through the screens corresponding to the different skeletal sites to be scored, e.g. to revise the score attributed to a previously examined skeletal site.

[0195] To achieve optimal matching performance, the reference images are preferably also subjected to multi scale processing (identical processing conditions as those applied to the ROI) so as to enable comparison under identical image quality conditions between reference pictures and the processed ROI.

[0196] A zooming functionality, to magnify the ROI (and corresponding reference images) may be implemented to achieve better visibility of small details, such as e.g. the corner of the epiphyses when it is believed to cap the metaphysics (bone age scoring) or to assess the presence or absence of an erosion (RA).

[0197] In this way a drawback of the conventional atlas-based method is overcome, namely that in the conventional atlas-based method the illustrations are small, implying that it difficult to observe e.g. the erosive and joint space abnormalities characterizing the different stages of rheumatoid arthritis.

[0198] ROI-specific processing is generally better than first globally optimizing the image quality of the entire image and next cutting out the ROI's. By first cutting out the ROI's from the unprocessed images and next subjecting them to image enhancement, the ROI image quality is higher because the image statistics used in the enhancement procedure are based on pixels of the ROI only.

[0199] Evidently, such stage-specific processing is impossible with the prior art using film images. Furthermore, it will be clear that the conventional atlas-based comparison method as used in the prior art is unable to provide the aforementioned improved functionalities.