Detection of Agricultural Contraband in Baggage

This invention relates in general to radiant energy imaging systems and more particularly, to a novel method and apparatus for selectively enhancing the image of objects or voids having circular cross section, such as agricultural and like-shape objects of voids, to distinguish them from objects or voids having rectangular cross section (manufactured objects or cracks).

A common method of dissemination of plant or animal disease vectors or pests such as the Mediterranean fruit fly to areas not yet infested is by the transport of agricultural items such as infected fruit, vegetables, plants, packaged meats and birds. In particular airline travelers pose a hazard in that large distances are rapidly traversed promoting the introduction of a pest into a region devoid of its natural enemies. Such dissemination can have serious economic impact on agricultural production. Currently, the primary methods for detecting agricultural contraband in passenger baggage are by passenger questioning, and manual baggage search. Such methods are unreliable and time consuming.

In a few locations, X-ray imaging systems used for inspection of passenger baggage for security reasons and digital X-ray scanners have been introduced to inspect baggage for agricultural contraband. These methods have the disadvantage that they rely on the ability of a human operator to recognize contraband items objects solely on the basis of shadow shape and density. Thus while agricultural objects such as pineapples, pears and papayas which have distinctive shapes, can be readily distinguished from manufactured objects, other agricultural contraband such as tangerines, apples, mangos, sausages and the like which have image shapes similar to non-contraband objects cannot be readily detected. Furthermore, even in those cases where it is possible to distinguish images of agricultural objects by their shape, such an inspection requires intense operator effort resulting in operator fatigue. Furthermore, such a procedure is time-consuming. This is a disadvantage for to be successful in a commercial setting, the monitoring system must monitor a piece of luggage in about 2-4 seconds.

U.S. Pat. No. 3,808,444 discloses an automatic high energy X-ray monitoring system for detecting dense metallic objects such as guns or bullets in baggage. Objects in the density range of agricultural objects are not detected by this method.

In medical studies, X-ray imaging is commonly used to image internal organs or other objects contained in human or animal bodies. A single X-ray projection minimizes the dose and apparatus cost but does not allow depiction of the cross sectional shape of the imaged organ. Such shape information is desirable but can generally only be obtained through multiple angled exposures and reconstruction (tomography), which is expensive and exposes the patient to undesirable radiation doses. Alternatively, a very extensive analysis of the density of a single exposure is time consuming and difficult.

In industrial inspection of materials for cracks and voids X-rays are the method of choice. Single exposures are used for speed and economy, but the image does not reveal the cross sectional shape of the cracks and voids, nor does it allow automatic recognition and counting of such cross sectional shapes which are desired for quality control.

I have discovered a novel method and apparatus using a computerized radiant energy imaging system to rapidly and easily distinguish images of objects having circular cross section such as agricultural and like-shape objects from objects having rectangular cross section such as manufactured objects, the objects being contained in a material having a different density-absorption coefficient product. Since a void has a different density-absorption coefficient product from the material in which it is present, the term void is included within the term objects and the method and apparatus may be used in the same way for voids as for objects.

The method of the invention to distinguish objects having circular cross section from objects having rectangular cross section comprises directing a beam of radiant energy through the objects, measuring the spatially resolved transmitted intensity of the radiation, calculating the gradient image of the spatially resolved transmitted intensity, eroding the gradient image one or more times to preferentially remove the edges of images of objects having rectangular cross section relative to the edges of images of objects having circular cross section and presenting the image. The apparatus of the invention comprises means for practicing the above method.

One important advantage of the invention is the provision of a means for rapidly and easily detecting agricultural contraband in baggage and parcels and/or automatic activation of an alarm if the amount of contraband exceeds a set value. Use of the invention enables an operator to visually detect such contraband without undue strain or fatigue and in the 2-4 seconds time allotment required for optimal commercial operation. Alternately, or in addition, the alarm system can automatically alert an operator to the presence of contraband.

Another advantage of the invention is the prevention of agricultural losses and spread of disease due to the prevention of dissemination of plant or animal disease vectors or pests contained in agricultural or biological commodities.

Another advantage of the invention is that it allows three-dimensional shape information of organs and objects contained or embedded in human and animal bodies from a single X-ray exposure.

Another advantage of the invention is that it allows rapid and automatic recognition of circular voids in the presence of cracks during industrial inspection of materials by X-ray.

Further objects and advantages of the invention will be evident from the following description.

FIG. 1 is a pictorial representation of a conventional method of measuring the spatially resolved transmitted intensity of objects contained in a piece of baggage.

FIG. 2 is a pictorial representation of the steps of the method of the invention depicted as a section at constant x through FIG. 1.

FIG. 3 is a block diagram illustrating the invention.

FIG. 4 is a photograph of the image of a piece of baggage containing agricultural contraband obtained in the manner of FIG. 1.

FIG. 5 is a computer display of a sub-area of FIG. 4.

FIG. 6 is a computer display of the region of FIG. 5 after applying the method of the invention.

My method and apparatus to detect agricultural contraband and objects of like-shape are based on three-dimensional shape, particularly the cross section of an object and presence or absence of sharp (step) edges. Because of biological growth or method of preparation, agricultural objects are either ellipsoidal in shape (for example, most fruits and vegetables and other biological objects such as birds) or long cylinders (for example, bananas, sausages and plant cuttings). When imaged by a radiant energy source, these agricultural objects have rounded (circular) cross sections. In contrast, manufactured objects, particularly those in the same density range as agricultural objects, are rarely ellipsoidal and tend to be either rectangular in shape; or cylinders (particularly short cylinders such as cold cream jars, and deodorant sticks); or to have distinctive shapes (for example shoes, electric razor, and coat hangers). Because short cylinders are generally packed or shipped straight up, manufactured objects--when imaged by a radiant energy source--have a rectangular cross section and sharp (step) edges, or, if rounded, exhibit distinctive overall shape. Other manufactured objects such as bullets or guns are readily detected in such imaging systems by their density and distinctive shape.

In the method of the invention, objects having circular cross section such as agricultural contraband are selectively enhanced at the edges while objects having rectangular cross section are not so-enhanced. The enhanced edge image is then directly displayed to an operator or added to the original image, that is, the image obtained by the conventional use of a radiant energy imaging system as described in further detail below, to present the operator with an image wherein objects having circular cross section are selectively highlighted. Alternately, the enhanced edge image is summed and set to activate an alarm if a certain level of agricultural and like-shape objects (objects having circular cross section) are detected. In another alternate embodiment, the enhanced edge image ia further enhanced by changing it to another color (false color transform) and/or by multiplying it by a constant to increase the intensity prior to being displayed or added to the original image.

The steps of the method of the invention are as follows: first, a beam of radiant energy such as an X-ray beam, ultrasonic radiation or nuclear radiation is directed through the objects to be distinguished, for example agricultural contraband contained in passenger baggage or parcels. The radiant energy beam passes through the objects and is attentuated by them, that is, the intensity of the radiation is decreased by each object depending on its thickness, absorption coefficient and density. In the case of a void or other region of lesser density-absorption coefficient product, the attenuation of the radiation is diminished, and hence the transmitted radiation increased in like amount as by an object of corresponding greater density-absorption coefficient. Next, a detector located behind the objects and facing the radiation source measures the spatially resolved transmitted intensity of the radiation and a digital image is constructed.

Referring to FIG. 1, a convenient way to measure the spatially resolved transmitted intensity is to place baggage 4 on moving belt 6 in the xy plane, moving in the x-direction as shown. Using flat X-ray beam 1 essentially in the z-direction, baggage 4 containing objects 2 and 3 is detected by detector 10 shown in FIG. 1 as a linear array aligned in the y-direction and an image f(x,y) constructed. Other methods of imaging can be used such as a pencil beam scanning the object with motion by the object and/or the beam, a conical beam and a two-dimensional detector array or an X-ray phosphor with video camera. Such radiation and detection devices are well known in the art and are commercially available. Any radiant energy source may be used which is attenuated by the objects such that the transmitted intensity (i.e. the intensity received by the detector) is a function of the material transversed. An X-ray beam is the energy source of choice. It is preferred that the beam of radiant energy pass through the object substantially in the z-direction as shown in FIG. 1. Next, the image f(x,y) is optionally fully or partially gamma corrected (subjected to a logarithmic transformation) resulting in an image whose change of intensity is approximately proportional to the amount of material transversed. Conventionally, when imaging absorbing objects, a black/white reversal is applied, but is not necessary to the practice of the invention. In the following discussion, the transformed image is referred to as the original image p(x,y).

FIG. 2 depicts a section at constant x through FIG. 1 to illustrate the basis of the method of enhancement. In FIGS. 2.1 and 2.2 the vertical coordinate is -z, while in FIGS. 2.3-2.6 the vertical coordinate is the image intensity. The horizontal coordinate in FIG. 2 is always y, while the direction of x is into the paper. The original image p(y) is shown in FIG. 2.3. Because object 2 having circular cross section and object 3 having rectangular cross section are depicted as having the same maximum thickness, the maximum value of p(y) is the same for each; however the intensity of object 2 shows a lightening at the edges whereas the intensity of object 3 shows a sharp (step) edge.

In the next step, the gradient image, that is, the absolute value of the rate of change of the intensity, .vertline..gradient.p.vertline.=g, is calculated pixel by pixel. In one dimension, .gradient.p=dp/dy (FIG. 2.4). Since the two objects have the same maximum thickness, the integral under curves FIG. 2.4 are equal but the gradient image of object 3 is restricted to sharp narrow regions whereas the gradient image of object 2 shows broad sloping regions. If object 3 having rectangular cross section is slightly tilted or if the beam is slightly tilted from the z-direction, as will occur if the X-ray source in FIG. 1 is close to the object, the gradient edge of object 3 in FIG. 2.4 will spread somewhat, but will still be much narrower than that of object 2.

Next, the gradient image is eroded one or more times. One erosion cycle is as follows: for a given pixel, n adjacent pixels, where n is typically 4 or 8 and preferably 8, are considered. If at least m of the n adjacent pixels have an absolute gradient value less than a tolerance level, t, then the absolute gradient value of the central pixel is set equal to zero. If not, it is left unchanged (FIG. 2.5). Typically, the value of m is 3 and the value of t is 21/2 percent of the maximum value of p in the image. The optimum number of erosion cycles is that which erodes the gradient image of an object having rectangular cross section completely or almost completely as shown in FIG. 2.5B while allowing at a least a portion of the eroded gradient (enhanced edge) image of an object having circular cross section to remain as shown in FIG. 2.5A. This number depends primarily on the size of the smallest object having circular cross section which one wishes to detect, the width of the gradient image of the objects which one wants to set to zero--those objects with step edges as shown in FIG. 2.3B and those objects having confounding edges which are almost, but not quite step edges (e.g., rectangular objects which are slightly tilted with regard to the radiant energy beam). Where it is desired that objects having circular cross section such as small grapes (7 pixels, 1.5 mm per pixel) be detected two erosion cycles are preferred. Where the smallest object to be detected is typically hand fruit (e.g., tangerines) five cycles could be used.

If desired, thresholding can be applied to the gradient image before or after erosion to yield the binary image, FIG. 2.6. To additionally enhance the edges remaining after erosion, the eroded gradient image with or without thresholding can be transformed such as by changing it to another color (false color transform) and/or by multiplying the intensity by a constant.

The images obtained by the above method are next presented in one or more of the following ways to alert an operator to the presence of objects having circular cross section such as agricultural contraband. In one method, the eroded gradient image, that is, the image wherein the edges of objects having circular cross section are present and step edges are eroded away, is presented directly to the operator, with or without additional enhancement. Alternatively, the eroded gradient image with or without additional enhancement is added to the original image of spatially resolved transmitted intensity so that the operator views an image of the objects wherein the objects having circular cross section have edges of heightened (highlighted) intensity. In this way, fruit, vegetables, and the like or voids can be readily visually detected by shape, highlighted edges and shadow density. Alternatively, or in addition, the eroded gradient image with or without thresholding is integrated over a portion or all of the image to yield a measure of total edge of objects of circular cross section to activate an alarm if the amount detected exceeds a set value, i.e., that value which indicates that contraband or excessive voids may be present.

A block diagram illustrating the method and apparatus of the invention is given in FIG. 3. A digital spatial representation f(x,y) where x and y are the coordinates of the image of transmitted intensity on the image plane is received at detector 10. The image, with or without black/white reversal or gamma correction, is placed in frame buffer memory 11 in real time. The gradient of the image is computed and eroded as described in detail below. If this computation is done in real time, in the order of about 1 second, it is convenient to use array processor 12 and several frame buffers 13.

Once the eroded gradient image is created in frame buffers 13, it may be presented in one or more of the following ways. It may be sent to display buffer 14 by passing directly through arithmetic logical unit ALU 15 and after display formatting in digital-to-analog converter (DAC) 16 displayed on monitor 20 for viewing. Alternatively, the eroded gradient image may be increased in intensity or changed to another color in ALU 15 prior to displaying it. In another embodiment of the invention, the eroded gradient image in frame buffers 13, with or without increasing the intensity or changing to another color in ALU 15 may be added to the original image in frame buffer 11 using ALU 15, sent to display buffer 14, display formatted in DAC 16 and displayed on monitor 20 for viewing. Alternatively, or in addition, the enhanced edge image in 13 may be integrated over a portion or all of the image using integrator 17, and after comparison using comparator 18 with a trigger level contained in trigger register 19, set to activate alarm 21 if the trigger level is exceeded. Alternatively, the image in frame buffer 11 and/or the eroded gradient image or image with added eroded gradient image can be stored in a machine readable magnetic storage device 22 for later display, analysis or survey. All of the components depicted in FIG. 3 are individually available commercially. If 12 is an image array processor, the components 11-19 are commonly assembled as an image analysis system with frame buffers serving as units 11, 13 and 14. Alternatively, a generalized computer or floating point array processor can be used in place of units 11-13,15 and 17-19.

The instant invention provides a method of computations for the feedback loop of array processor 12 and frame buffer 13 in which the gradient is calculated and then eroded. Calculation of the gradient image can be performed by several methods such as the Prewitt gradient or the Sobel gradient. I have found the Prewitt gradient convenient which arises from fitting a plane to a 3.times.3 neighborhood of each pixel. If x,y are the position indices of the pixel, the computation using the Prewitt gradient is as follows:

and

from which the gradient g is found as

A number of erosion algorithms exist. I have found the following algorithm convenient to use: ##EQU1##

Here (x',y') ranges over the 8 pixels adjacent to (x,y) but not (x,y) itself, while m=3. The algorithm is recursive, the subscript on h indicating the erosion cycles, and image h.sub.o =g (gradient image). I have typically used h.sub.2 for the enhanced edge image, but h.sub.1 or h.sub.3 or greater may be used; use of h.sub.1 instead of h.sub.2 would enhance smaller objects but cause some false positives (i.e. enhancement of edges of objects not having circular cross section). Four instead of 8 adjacent pixels and other values of m, such as 2, can be used. It is also within the compass of the invention that an operator be given a switch to select h.sub.1, h.sub.2 or h.sub.3, or other h.sub.i as desired. The constant t will depend on the amount of contrast in the image. It can be set by histogramming an image or histogramming the neighborhood of h.sub.i-1 (x,y) using the image analysis system. For passenger baggage I have found the value t=0.025 max p convenient.

Addition of the eroded gradient image to the original image f or p in frame buffer 11 is given by b(f+ah.sub.i) or b(p+ah.sub.i) where i is chosen as above, a is a multiplier and/or false color transform and b is a normalization constant. The constants are chosen to obtain the desired appearance on the monitor. It has been found convenient to use a=b=1 and p.

The following listing of a computer program written in BASIC language for a Tektronix 4052 computer implements a typical embodiment of the invention that was described above. The main program link, statements 1-4400 remains in memory at all times; overlays are stored on files 6,7, and 8 and begin at statement 10000. Different algorithm steps are brought in by User Keys. Depressing User Key n transfers control to statement 4n. Arrays F1, F2, F3, and F4, are 32.times.32 frame buffers. These buffers are small in this prototype because of memory limitations. Gradient calculation starts at statement 950; F2.fwdarw.F3, F4. Erosion calculation starts at statement 3290; F2.fwdarw.F2. Image display in simulated gray scale is on file 7. Image input is on file 6. ##SPC1## ##SPC2## ##SPC3## ##SPC4##

The method and apparatus of the invention are particularly suitable for detecting agricultural contraband in baggage or parcels as described above. Another important application of the invention is in the field of medical X-ray imagery. Objects of circular cross section contained or embedded in human or animal bodies can be selectively enhanced using the invention and information obtained from the image shadow shape and shape of the cross section. In this way three-dimensional shape information of organs and objects in human or animal bodies can be obtained from a single X-ray exposure. In this case the threshold level t should be adjusted to be comparable to the amount of contrast present in the image.

The method and apparatus described above may also be used for industrial inspection of materials to distinguish voids from cracks. In this use, voids of circular cross section are selectively enhanced and thereby detected. Again the threshold level t is adjusted to be comparable to the amount of contrast present in the image.

The invention is further demonstrated by the following illustrative example wherein agricultural contraband contained in passenger baggage is detected. This is by way of illustration and not limitation.

A leatherette carry-on suitcase was filled with items commonly carried by travellers, i.e. clothing, shoes, coat hangers, glasses, electric razors, sunglasses, toilet kit, a souvenir wine glass and a jar of baby food. To this was added a number of items characteristic of agricultural contraband. These items included grapes, candy, three citrus fruits, a papaya, a banana and a sausage. Also added was a short round cylinder or similar size and density as the citrus fruits, to simulate a cold cream jar, packed end-on.

The suitcase was passed through a commercial X-ray imaging system (Astrophysics Line Scan 1 (Patent Application Ser. No. 185,561)) in the manner depicted in FIG 1. Referring to FIG. 3, the image from detector 10 was passed directly to ALU 15 (for gamma correction and black/white reversal), display buffer 14, DAC 16 and monitor 20, in that order. No array processors or other frame buffers were used. A photograph of the image of the objects in the suitcase obtained by this commercial system, i.e., without treatment with the invention, is shown in FIG. 4. At the same time that the data was sent to the display buffer, it was also captured in digital form on magnetic storage device (tape) 22.

The magnetic tape was processed by a Tektronix 4052 computer and treated by the computer code given above. After selecting a 58.times.58 pixel sub-area of FIG. 4 delineated by the curser box, this sub-area was divided into 4 overlapping sub-areas of 32.times.32 pixels each and each 32.times.32 sub-area was brought into computer array F2 using User Key 18 and file 6; displayed on the screen of the Tektronix 4052 using User Key 17 and file 7; a copy made by use of a Tektronix 4631 hardcopy device and the copy saved.

The gradient g of F2 was computed as described in file 4 by use of User Key 11; the resulting array F3 was moved to F2 by use of the statement F2=F3; F2 was eroded two times by using User Key 13 twice; using each time an erosion basis (m) of 3 and a tolerance (t) of 3 (out of 127 maximum). The resulting image h.sub.2 in F2 was then displayed and copied and saved as above. After repeating the above process for each of the four quadrants of the 58.times.58 sub-area, the quadrants of the original image were assembled to create FIG. 5 and the quadrants of the eroded edge image h.sub.2 to create FIG. 6. Because of inherent limitations in the display capacity of the Tektronix 4052 computer, no attempt was made to add h.sub.2 back to the original image. FIG. 5 shows the partial image of a coat hanger (30), shoe sole and heel (31), tangerine (32), simulated cold cream jar placed end-on (33), papaya (34), and three verical suitcase seams (35,36,37). In the eroded gradient image what remained was the edge of the papaya, the tangerine (including the hole in the middle), a very thin outline of part of the step edge of the cold cream jar and shoe sole and heel and a small section of the coathanger. Thus the method selectively enhanced the two pieces of fruit and completely or almost completely eliminated everything else in the image. Use of the image of FIG. 6 or a thresholded version thereof to enhance FIG. 5 would cause just the contraband fruit objects to be highlighted.