Large depth of field line scan camera

[0002] This application is a continuation of PCT/US01/09058, filed Mar. 21, 2001, which claims the benefit of U.S. Provisional application No. 60/190,796, filed Mar. 21, 2000.

[0003] The present invention relates generally to optical scanning systems. More particularly, this invention relates to a scanning system having a large depth of field which is capable of focusing on and scanning a plurality of targets over a wide range of distances.

[0004] Coded symbologies are being used in a vast array of applications. The ability to track a large amount of items quickly and efficiently has led coded symbologies to be used in applications such as retail checkout, warehousing, inventory control and document tracking. As the volume of items tracked by coded symbologies has increased, the need for optical scanners which operate at high speeds has likewise increased.

[0005] Various optical scanning systems have been developed for reading and decoding coded symbologies. Such optical scanning systems typically incorporate a non-coherent light source and a light detecting means, such as a CMOS- or CCD-based image sensor. A portion of the light which is reflected from the coded symbology is detected by the image sensor and converted into an electrical signal, which is the basis for the digital image of the coded symbology that has been scanned. The digital image is then processed and decoded according to the specific type of coded symbology.

[0006] One disadvantage with all current CMOS- and CCD-based scanning systems is that they require a mechanical focusing apparatus in order to focus the light reflected from the coded symbology onto the image sensor. Since a mechanical focusing apparatus typically operates slower than the image sensor and other electronics within the scanner, the focusing apparatus tends to be the limiting factor in the speed of operation of the scanning system. Although there are scanning systems that are available which do not require a moveable focusing apparatus, these scanning systems typically have a very small depth of field.

[0007] Accordingly, there exists a need for an efficient and inexpensive scanning system that can accurately read and decode coded symbologies over a large depth of field without requiring a moving mechanical apparatus to focus light on the image sensor.

[0008] The present invention is a scanning system which utilizes a randomly addressable CMOS area array sensor which is selectively positioned at the Scheimpflug angle in the image plane in order to detect focused light reflected from an object. Light reflected from the object is focused onto the CMOS area array sensor through an objective lens. Since the CMOS area array sensor is mounted at the Scheimpflug angle, each strip within the depth of field of the object plane has a corresponding string of pixels on the CMOS area array sensor which are in focus. The system may include a means for determining the distance between the scanner and the object. Once the distance between the scanner and the object is determined, the system reads out only those pixels which are in focus.

[0009] Objects and advantages will become apparent to those skilled in the art after reading the detailed description of a presently preferred embodiment.

[0010] FIG. 1 is a coded symbology scanning system made in accordance with the present invention.

[0011] FIG. 2 is the camera made in accordance with the present invention.

[0012] FIG. 3A illustrates the image of an object in the image plane relative to the position of the object in the object plane.

[0013] FIG. 3B shows the spatial relationship between the object plane, the optics and the image plane.

[0014] FIG. 3C shows the object plane of FIG. 3B in greater detail.

[0015] FIG. 3D shows the image plane of FIG. 3B in greater detail.

[0016] FIG. 4 is a flow diagram of the method of the present invention.

[0017] FIG. 5A is an alternative embodiment of the coded symbology scanning system of the present invention.

[0018] FIG. 5B is another alternative embodiment of the coded symbology scanning system of the present invention that does not require a separate height detection means.

[0019] FIG. 5C shows the light plane being projected upon those portions of the object plane O which are in focus.

[0020] FIG. 6 is a perspective view of an object in the object plane being scanned.

[0021] FIG. 7 is a cross sectional view of the same object of FIG. 6 as seen in the image plane.

[0022] FIG. 8 is a view of the object as detected by the CMOS area array sensor.

[0023] The present invention will be described with reference to the drawing figures wherein like numerals represent like elements throughout.

[0024] Referring to FIG. 1, a scanning system 10 made in accordance with the present invention is shown. The scanning system 10 is able to scan any type of object. For simplicity, however, the scanning system 10 will be explained with reference to scanning a coded symbology, and in particular a bar code symbol. The scanning system 10 includes a light source 12, a light curtain 15 and a camera 17 located within a housing 14. In the preferred embodiment of the present invention, the camera 17 includes a randomly addressable CMOS-based area array image sensor. However, those skilled in the art should know that any randomly addressable image sensor may be utilized without departing from the spirit and scope of the present invention.

[0025] The light source 12 facilitates detection of a subject bar code symbol 18 by illuminating the bar code symbol 18 located on an object 8. Although the light source 12 in FIG. 1 is shown as a non-coherent light source, in the present embodiment the light source 12 may compromise any type of coherent or non-coherent illumination means. Preferably, the object 8 is supported by a moving transport system 7. The scanning system 10 is aligned such that light reflected from the bar code symbol 18 passes through the front of the housing 14 and is detected by the camera 17. As will be explained in detail hereinafter, the rows of the area array sensor within the camera 17 are optically aligned parallel to the surface of the transport system 7 and perpendicular to the motion of the transport system 7. It should be recognized by those skilled in the art that FIG. 1 is illustrative only and is not drawn to scale.

[0026] The first embodiment will be described as including a light curtain 15. However, it should be recognized by those of skill in the art that any mechanism for detecting the height of an object 8, such as a proximity sensor, dimensioning system or the like, may also be utilized. Further, although the first embodiment requires the use of a light curtain 15 as a height detection means, at least one alternative embodiment, as will be described herein, does not require the use of a light curtain 15 or any other height detection means.

[0027] The light curtain 15 includes a plurality of light sources 7_a-nand a plurality of corresponding detectors 9_a-nlocated on the opposite side of the transport system 7 from the light sources 7_a-n. As those skilled in the art would clearly recognize, as the object 8 is conveyed along the transport system 7 it blocks at least one of the light beams emanating from the light sources 7_a-nfrom being detected by its corresponding detector 9_a-nThe absence of such light is detected by the light detectors 9_a-n. Depending upon the number of light beams that have been blocked, the light curtain 15 can determine the instantaneous height x of the object 8. This information is provided to the scanning system 10 and is utilized as will be explained in detail hereinafter.

[0028] A tachometer 19 monitors the speed of the surface of the transport system 7 and the object 8, and provides this information to a processor 32 (shown in FIG. 2) to control the read-out rate of the CMOS area array sensor 16. Those skilled in the art should know that there are several other methods of monitoring the speed of the transport system 7 and the object 8 and providing this information to the processor 32 to control the read-out rate of the array that may be used as well.

[0029] Referring to FIG. 2, the camera 17 includes fixed focusing optics 31, a randomly addressable CMOS area array sensor 16, a processor 32 and an output means 34. The CMOS area array sensor 16 includes a plurality of rows of pixels 26_a-26_nand a plurality of columns of pixels 28_a-28_n. As is well understood by those skilled in the art, light 29 reflected from the object 8 enters the camera 17 via the focusing optics 31. The focusing optics 31 focus the light 29 onto the CMOS area array sensor 16. The CMOS area array sensor 16 detects the reflected light 29 and each pixel within the CMOS area array sensor 16 converts the incident light energy into a stored electrical charge, and ultimately an electrical signal. The electrical signal comprises a value which represents the amount of light detected by a pixel. If the pixel's analog signal is digitized to have eight-bit resolution, the number of gray scale levels will be 255 (2⁸-1).

[0030] The information stored within the CMOS area array sensor 16 eventually results in a digitally quantized image of the object within the field of view of the camera 17. It should be appreciated by those skilled in the art that the current randomly addressable CMOS area array sensors contain greater than 800 columns and 600 rows. This resolution, though, is certain to increase as the technology advances. The rows 26_a-26_nor columns 28_a-28_nof the CMOS area array sensor 16 can be considered a series of parallel lines when projected into the object plane. Each pixel in any specific column 28_a-28_nwill have a corresponding unique focus point in the object plane due to the CMOS area array sensor 16 being positioned at the Scheimpflug angle in the image plane. The system 10 of the present invention utilizes the addressability and selective positioning of the CMOS area array sensor 16, and inputs from the light curtain 15 and the tachometer 19 in order to selectively read out information pertaining to the objects within the depth of field that are in focus.

[0031] A processor 32 receives the time sampled height data from the light curtain 15, and object motion velocity information from the tachometer 19 (shown in FIG. 1), and selectively accesses pixels within the CMOS area array sensor 16 to obtain the desired information. This information is then output to the output means 34 for further processing. The output means 34 may be a digital display, an image processing system, a positioning system, or any other type of system that will accept an output from an imaging camera.

[0032] In the first embodiment of the present invention shown in FIG. 3A, the CMOS area array sensor 16 is tilted at an angle with respect to the median plane 33 through the optics 31. The angle of tilt, (hereinafter, the Scheimpflug angle), is determined using the Scheimpflug condition. The Scheimpflug angle is dependent on the focal length of the optics 31 and the desired working range defining the depth of field (DOF) in object space. The CMOS area array sensor 16 must be positioned such that an object positioned within the DOF is imaged in focus onto the active portion of the CMOS area array sensor 16. Those skilled in the art know that the Scheimpflug angle may be calculated using the following relationship (i.e. the Scheimpflug condition): $\begin{array}{cc}\frac{1}{f}=\frac{1}{s}+\frac{1}{s^{\prime }}& \mathrm{Equation}1\end{array}$

[0033] where f equals the focal length of the optics, s equals the distance from the optics 31 to the object 8, and s′ equals the corresponding distance from the optics 31 to the image plane I, where a string of pixels on the CMOS area array sensor 16 is in focus. This relationship provides that each horizontal line in the object plane O has a corresponding horizontal line in the image plane I which is in focus. For example, each row 26_a-26_nin the CMOS area array sensor 16 in the image plane I corresponds to a specific distance in the object plane O; and each such distance in the object plane O corresponds to a line parallel to the surface of transport system 7 and perpendicular to the direction of motion of the transport system 7.

[0034] Referring to FIG. 3B, the spatial relationship between the object plane O, the optics 31 and the image plane I is shown. FIG. 3C shows the object plane O of FIG. 3B in greater detail. FIG. 3D shows the image plane I of FIG. 3B in greater detail.

[0035] As shown in FIGS. 3C and 3D, an object O₁in the object plane O at distance D₁from the optics 31 will be in focus in the image plane I on the randomly addressable CMOS area array sensor 16 at the string of pixels I₁. Likewise, an object O₂in the object plane O at distance D₂will be in focus in the image plane I at the string of pixels I₂, and an object O₃in the object plane O at distance D₃will be in focus in the image plane I at the string of pixels I₃. Accordingly, for any plane parallel to the surface of the transport system 7 in the object plane O within the DOF, a corresponding string of pixels on the randomly addressable CMOS area array sensor 16 will be in focus. From the distance (i.e. D₁, D₂, D₃or height x of a package), the row 26_a-26_nassociated with the height x may be calculated and the corresponding string of pixels which is in focus in the image plane I can be determined. For this embodiment, since the light curtain 15 detects a single constant height (per time sample) of the object 8, (i.e. it does not detect the profile of the object 8), data is read out of the CMOS area array sensor 16 as a complete row 26_a-26_n. The data from the string of pixels is selected by the processor 32 and read out to the output means 34 for further processing decoding, storage or display as is well known by those skilled in the art. It should be noted that new data is periodically read out of the CMOS area array sensor 16 as will be described in greater detail hereinafter.

[0036] In operation, the scanning system 10 executes the bar code symbol reading and decoding procedure 200 shown in FIG. 4. A light source 12 illuminates a subject bar code symbol 18 (step 210). Light is reflected from the bar code symbol 18 toward the camera 17 (step 220). The reflected light enters the camera 17 via the focusing optics 31 (step 230). The focusing optics 31 focus the light 29 onto the CMOS area array sensor 16 (step 240). Each pixel within the CMOS area array sensor 16 converts the light into an electrical signal (step 250).

[0037] As will be explained in greater detail, the calculated distance to the object 8, derived from the output of the light curtain 15, is used to determine the pixels within the CMOS area array sensor 16 which are in focus. In the preferred embodiment, the pixels are a string of pixels in a single row 26_a-26_n. Although in alternative embodiments described hereinafter, the pixels need not comprise a string of pixels in a single row 26_a-26_n. The light curtain 15 provides the raw height data (step 225). The distance to the object 8 is then calculated by the processor 32 (step 255) using the light curtain data, which then selects the string of pixels which is in focus (step 270).

[0038] The information in the CMOS area array sensor 16 is read out at a periodic rate. This periodic rate is determined by processor 32 using speed information of the object 8, (preferable determined by the tachometer 19), and the desired sample density. This image data is then forwarded to the output means 34 for decoding, storage or display (step 280) using methods that are well known to those of skill in the art.

[0039] An alternative embodiment 200 of the present invention is shown in FIG. 5A. This embodiment utilizes a dimensioning system 210 that makes a calculation of the surface contour of the object 208 prior to scanning. The dimensioning system 210 replaces the light curtain 15 or its equivalent. For purposes of this embodiment, the dimensioning system disclosed in U.S. Pat. No. 5,661,561 is used, but as those skilled in the art should know there are many systems which act as dimensioning systems to determine the profile of an object and can provide the necessary input to the processor 32.

[0040] Referring to FIG. 5A, the dimensioning system 210 provides a profile of the object 208 on the transport system 223. This profile is provided periodically to the processor 32, at the desired periodic rate. Since the CMOS area array sensor 16 has a plurality of columns 28_a-28_n, the width of the transport system 223 is segmented into a corresponding plurality of segments; one segment for each column 28_a-28_nof the CMOS area array sensor 16. As the object 208 passes the dimensioning system 210, an object profile is obtained. The profile is used to determine a height position (row 26_a-26_n) for each segment (column 28_a-28_n) across the width of the transport system 223. The height position is used to calculate a distance from the surface profile of the object 208 within that segment to the camera 230. This string of pixels derived from the profile information is created by selecting an appropriate pixel, written each row 26_a-26_nbased upon the distance for each column 28_a-28_n. Accordingly, one pixel is selected by the processor 32 for each segment across the width of the transport system 223, based upon the height of the object 8 within that segment.

[0041] Since the pixels of the CMOS area array sensor 16 are individually addressable, the information in the selected pixels is read out of the CMOS area array sensor 16 using the processor 32, which reads the pixels and forwards the information in the selected pixels to the output means 34 for decoding, storage or display. It should be recognized by those of skill in the art that, in contrast to the embodiment described with reference to FIG. 1, the pixels read out to the processor 32 need not comprise pixels from a single row 28_a-28_n. In this embodiment, since the dimensioning system 210 provides a profile of the object 208, the profile information is used to determine a specific pixel within each column 28_a-28_nwhich will be in focus. (Of course, a tolerance level may be determined whereby several pixels in the rows adjust to the “in focus” pixel may also be read out to improve accuracy).

[0042] It should be noted in this embodiment that the surface contour is preferably quantized to the number of rows 26_a-26_nand columns 28_a-28_n, of the CMOS area array sensor 16; whereby the rows 26_a-26_nprovide depth (or distance) information and the columns 28_a-28_nprovide width information. Thus, a single pixel of a single row 26_a-26_nwill be selected in each column. The pixels that are read out for each time sample will contain one pixel for each column 28_a-28_nin the CMOS area array sensor 16. Finally, it should be noted that the light source 212 used with this embodiment of the invention may be structured or non-structured.

[0043] The use of more accurate segmented height values as an input to the processor 32 eliminates the inflexibility associated with light curtains for multiple objects or contoured surfaces. It should also be noted that the addressability of the rows 26_a-26_nand columns 28_a-28_nis interchangeable. Accordingly, those sections of the specification which refer specifically to the functionality of a row 26_a-26_n, should be understood to refer to the same functionality in a column 28_a-28_n, and vice versa.

[0044] Another alternative embodiment of the present invention will be described with reference to FIG. 5B. This system 115 includes a profile detection-based camera 130 which does not require the use of a separate height detector as required by the prior two embodiments. The system comprises a structured light source 126, a CMOS area array sensor-based camera 130 and a transport system 123, (including a tachometer or other speed detection means (not shown)). As should be known by those skilled in the art, the structured light source 126 may comprise a bank of LEDs, a set of laser diodes or any such equivalent which creates a plane of structured light. The structured light source 126 is aligned to be co-planar with the points in the object plane O that are in focus, as determined by the positioning of the CMOS area array sensor 16 at the Scheimpflug angle in the image plane I.

[0045] This embodiment differs from the prior embodiments in that the in-focus pixel information to be read out is determined automatically, either in hardware or in software, by detecting the brightest pixel in each column 28_a-28_n. If the detection is performed in hardware, a customized CMOS area array sensor 16 which includes intensity detection circuitry to detect the pixels with the greatest amount of incident light, (or highest value) in each column 28_a-28_nis used. If the detection is performed in the software, the processor 32 will determine the pixels, or several adjacent pixels, within each column 28_a-28_nhaving the greatest value. The information in these pixels is representative of an object as located by the in-focus profile. The pixels in each row 28_a-28_nare monitored and the amount of light detected is used to identify the centroid of pixels having the greatest value, (corresponding to the most incident return light). In order to account for any error, for each column position 28_a-28_nthe processor 32 will also extract ± several pixels from the rows 26_a-26_nadjacent to the pixel having the greatest value.

[0046] As shown in FIG. 5C, the structured light source 126 is mounted off-axis at an angle O in relation to the camera 130. The structured light source creates a light plane 125 which is projected onto the transport system 123. The light plane 125 is aligned at the angle O such that it is co-planar with the in-focus rows 26_a-26_nof the CMOS area array sensor in the image plane I. Accordingly, the light plane 125 only illuminates these portions of the object plane O which are in focus. As a result, the brightest pixels will be the pixels that are in focus. Of course, a separate such image is taken for each time slice.

[0047] In essence, the system will detect a “peak” or centroid of higher values, representing higher intensity light, as indicated by one or more pixels in a particular row 26_a-26_nor column 28_a-28_n. It is the information from these pixels that is read out and further processed. It should be understood by those skilled in the art that since the profile information by definition represents data that is in focus and at a known distance, this embodiment does not require any type of height detection input or means.

[0048] As shown in FIG. 6, the light plane 125 strikes the surface of the object 118 and creates a profile of the object 118, which is as shown in FIG. 7. Accordingly, the image plane I will contain a like image of the object 118. As shown in FIG. 7, a cross-sectional profile of the object 118 is generated in the image plane I.

[0049] FIG. 8 illustrates the in-focus cross-sectional profile of the object 118 as it is detected by the CMOS area array sensor 16 in the image plane I using a properly aligned structured light source. The image plane I is also the surface of the CMOS area array sensor 16. Upon detection of the incident light, and hence the object 118, the information collected by the CMOS area array sensor 16 is then examined. Since the illuminated profile of the object 118, by definition, comprises those points in the object plane O which are in focus, only those pixels need to be further examined.

[0050] Those skilled in the art should realize that this embodiment may also be utilized for object dimensioning. Along with the ability to determine the cross-sectional height profile of an object, this embodiment also provides the information needed to calculate the length and width of the object.

[0051] As disclosed above, the present invention has the ability to accurately image multiple side-by-side cartons of varying height being transported on a conveyor system, and to accurately image odd package surfaces such as round and triangle tubes intermixed side-by-side with rectangular cartons. This invention has the ability to accurately image any complex surface contour, limited only by the quantization limits as defined by the resolution of the CMOS area array sensor 16.

[0052] While specific embodiments of the present invention have been shown and described, many modifications and variations could be made by one skilled in the art without departing from the spirit and scope of the invention. For example, although a stationary scanner has been utilized in the above embodiments, the embodiments and principles described herein may also be applied to mobile or hand-held scanners. The above description serves to illustrate and not limit the particular form in any way.