Sample processing apparatus with a vision system

[0001] 1. Field of Invention

[0002] This invention relates to a sample processing apparatus with a vision system.

[0003] 2. Discussion of Related Art

[0004] Sample processing apparatus are used to automatically manipulate sample holders and precise amounts of fluid. These devices enable testing and analysis of samples in many fields, such as genomics, drug discovery, proteomics, and diagnostics. For example, a blood sample may be aspirated by a pipettor or other liquid handling device, moved and dispensed at another location. The sample may be mixed with other compounds, filtered, centrifuged or otherwise processed as part of an analysis protocol.

[0005] In one aspect of the invention, a sample processing apparatus comprises a vision system that allows for the verification at least one aspect of a sample handling process involving a change in state of a sample with respect to a sample holder or other fluidic chamber. For example, the vision system may allow for the verification of a pipette aspiration/dispensing operation, a sample filtering operation or other with respect to a sample.

[0006] In another aspect of the invention, a vision system associated with a sample processing apparatus may capture and analyze images that are each associated with steps in a sample handling procedure. For example, an image of a sample holder supporting a liquid sample may be captured before a process, e.g., to verify that the liquid sample was present. Another image of the sample holder may be captured after the process has been completed, e.g., to verify that at least some portion of the liquid sample has been moved relative to the sample holder. For example, during a pipetting operation, a first image may be used to verify that a first volume of liquid sample is present in a pipette tip. After a dispensation operation, a second image may be used to verify that a second volume of the liquid sample (smaller than the first volume) remains in the tip. Thus, in this example, the vision system may be used to verify a change in volume of the sample with respect to the sample holder, i.e., verify a change in state of the sample with respect to a fluidic chamber. In some embodiments, such an arrangement may reduce or eliminate a need for individual sensors for each sample holder (such as capacitive sensors for detecting fluid in each of a plurality of pipette tips) to determine the presence, absence, movement, color, position, or other change in state of a sample. Instead, one or more images captured by the vision system may be used to assess multiple samples associated with multiple sample holders. In some embodiments, the vision system may be used for multiple purposes in addition to capturing and/or analyzing images relating to liquid samples. For example, the vision system may be used to control the operation of a robotic device, such as providing position feedback for a robotic manipulator.

[0007] In yet another aspect of the invention, a machine vision system may capture images of at least a portion of a sample processing apparatus during a procedure and analyze the captured image. The analysis may provide a user with information necessary to determine if and when an error occurred and/or to verify that the procedure was performed correctly.

[0008] In still another aspect of the present invention, images captured by a vision system associated with a sample processing apparatus may be tagged with an identifier or otherwise associated with a particular identity. For example, one or more images captured by a vision system may be associated with processing of a particular liquid sample that was taken, at least in part, from a particular person. The images may be associated with an identifier (e.g., a unique string of alphanumeric text) that has been provided for the liquid sample. Thus, the images may be later associated with the liquid sample and the person from whom the sample was taken. This may allow the person or some other entity to verify that particular processes were performed on the sample.

[0009] One aspect of the invention provides a method for monitoring a sample handling process including associating a liquid sample with a fluidic chamber (such as a pipette tip or well in a multi-well plate), and performing a sample handling process with the liquid sample with respect to the fluidic chamber. The sample handling process may include a change in state of the liquid sample with respect to the fluidic chamber, such as a change in color, position, volume or other characteristic of the sample. A machine vision system may be used to acquire a 2-dimensional image of at least a portion of the fluidic chamber before, during or after performance of the sample handling process, and the acquired image may be stored to allow later verification of the change in state of at least a portion of the liquid sample during the sample handling process.

[0010] In another aspect of the invention, a sample handling apparatus may include at least one fluidic chamber adapted to hold a liquid sample and allow for a change in state of the liquid sample relative to the fluidic chamber. A machine vision system may have an image acquisition device positioned to acquire a 2-dimensional image of at least a portion of the fluidic chamber, and be adapted to allow verification of the change in state of the liquid sample relative to the at least one fluidic chamber based on the image.

[0011] These and other aspects of the invention will be apparent from the following description and claims.

[0012] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures is represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. In the drawings:

[0013] FIG. 1 is a perspective view of a sample processing apparatus according to one embodiment of the present invention;

[0014] FIG. 2 is a perspective view of a support and sample holders according to one embodiment of the present invention;

[0015] FIG. 3 is a perspective view of the support and sample holders depicted in FIG. 2, where one of the supports has been lowered;

[0016] FIG. 4 is a perspective view of a set of sample holders on a support according to another embodiment of the present invention;

[0017] FIG. 5 is a perspective view of the sample holders and support depicted in FIG. 4, where the support has rotated;

[0018] FIG. 6 is a perspective view of a sample processing apparatus according to another embodiment of the present invention;

[0019] FIG. 7 is a frontal view of a sample graphical user interface according to one embodiment of the present invention; and

[0020] FIG. 8 is a block diagram of an operating procedure according to one embodiment of the present invention.

[0021] This invention is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The invention is capable of other embodiments and of being practiced or of being carried out in various ways. Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having,” “containing,” “involving,” and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.

[0022] As discussed above, sample processing apparatus are used to manipulate fluid samples so that various processes may be performed using the samples, such as pipetting, centrifuge processing, mixing, filtering, electrophoretic gel separation, passage through microfluidic channels, and so on. These processes may be performed automatically and concurrently on a plurality of samples, enabling high throughput and rapid testing protocols. However, sometimes a device may malfunction during the sample processing. For example, a pipette may not aspirate or dispense a sample, may clog, or may aspirate an air bubble. A sample tray may be placed in a suboptimal location, a centrifuge may not spin long or fast enough, or a plate may not be properly washed. Any of these errors may lead to inaccurate or misleading results.

[0023] One aspect of the present invention relates to a sample processing apparatus with an associated machine vision system that may be used to verify the proper operation of a sample process, such as a transfer of a liquid sample from one sample holder to another. Some examples of sample holders (or fluidic chambers) include pipettes, tubes (e.g., centrifuge tubes, sample tubes, and capillary tubes), syringes, fluidic channels and plates (e.g., filter plates and sample well plates). The machine vision system may include an image acquisition device arranged to capture a 2-dimensional image that includes at least a portion of a sample holder. After the image acquisition device captures an image of the sample holder, the image can be stored and/or analyzed to confirm that movement or other change in state of the sample was performed properly. In order to facilitate identification and retrieval, an image can be associated with one or more samples when the image is stored or at the time of capture, e.g., associated with an identifier that corresponds to the sample. Alternatively or additionally, data extracted from an image (or learned from other analysis of the image) may be tagged with an identifier and stored. Image analysis may include cutting portions of an image, annotating an image, comparing two or more images to each other, assessing a physical condition represented in the image (such as estimating a volume of sample in a holder using image information), or other. Image analysis may be performed in a fully automatic way, such as by image analysis software, or by interaction of a user with the image analyzer. For example, the image analyzer may present one or more images for review by a user, who may assess one or more conditions shown in the image(s) and provide information to the image analyzer regarding the assessment.

[0024] The machine vision system can be configured to extract different types of data from an image. For example with respect to a pipette tip, the machine vision system may verify proper aspiration, complete dispense, the volume aspirated, the absence of bubbles and/or clots in the pipette, the presence of a filter during aspiration, proper positioning of a sample well tray, etc. This flexibility allows a machine vision system to replace or supplement a number of instruments and/or sensors. Instead of requiring a pressure sensor to detect clots and a UV absorbance or fluorescence measurement system to determine concentration, a single machine vision system may be able to extract both pieces of data. This may reduce the space requirements and cost of a sample processing apparatus without sacrificing accuracy.

[0025] Additionally, the machine vision system may easily accommodate many different fluidic chambers, regardless of shape, size, or orientation. The vision system may be able to capture and analyze one or more images of a centrifuge tube, a pipette (regardless of size), a sample well plate, or other, in some cases without the need to make significant alteration to the system. Alternatively or additionally, the vision system can include image analysis capabilities to determine different characteristics in different fluidic chambers, such as the volume in a centrifuge tube, absence of clots in a pipette, proper dispense into a sample well, and presence of sample on an agarose gel.

[0026] The ability to store an image and analyze it later permits a user to retrieve additional information even after the sample handling process is complete. For example, a sample processing apparatus may image a sample holder after aspiration and verify that some sample has been aspirated. However, after the procedure is complete, a user may decide that he wants to calculate the volume aspirated, check for bubbles and/or clots in the sample, and confirm the presence of a filter. By recalling the image taken during aspiration, the user may be able to extract the additional desired information even though sample processing is complete.

[0027] In this way, machine vision can provide an accurate way to determine whether a sample handling error occurred and where it occurred when the result of a test or procedure seems erroneous. Instead of having to retest samples whenever results are called into question, the vision system may automatically keep a visual record of each sample tested at different stages in processing. Additionally, the machine vision system may give a user the flexibility to choose the timing and number of image acquisitions. A user may choose to record a single time point or multiple time points during a single sample handling process. Furthermore, the ability to tag the image and/or data with an identifier enables a user to quickly group together results from a single patient, sample, and/or record, and to quickly search obtain desired information.

[0028] As shown in FIG. 1, a sample processing apparatus 1 according to one embodiment of the present invention includes one or more fluidic chambers or sample holders, such as a pipette tip 10, a well in a multi-well plate 18, or other device arranged to support a liquid sample in some way. Sample holders may be manipulated for use with the apparatus 1 in any suitable way. For example, a multi-channel pipetting head 12 may be arranged to support and/or actuate some or all of the pipette tips 10, e.g., such that a plurality of the tips 10 can be moved together from a first position to a second position and actuated to aspirate and/or dispense a liquid sample. The pipetting head 12 may itself be manipulated by a robotic device or other arrangement that is adapted to support, move and/or actuate the pipetting head 12. Other arrangements may be incorporated into the apparatus 1 for manipulating sample holders 10, such as a robotic manipulator 14 that is capable of moving a sample holder from a storage location to an active location. For example, the robotic manipulator 14 may move a sample well plate 18 from a storage rack 15 to a location 17 where sample can be placed in/removed from the sample well plate 18. The robotic manipulator 14 may include a robotic arm or other suitable apparatus and may have one or more degrees of freedom. In addition, the robotic manipulator 14 may be adapted to engage with and manipulate the pipetting head 12 or other tools in the apparatus 1.

[0029] The sample processing apparatus 1 may include a controller 20 that coordinates or otherwise controls operation of various portions of the sample processing apparatus 1, such as the pipetting head 12, manipulator 14, or other portions of the apparatus 1. Additionally, the controller 20 may have a machine vision system that includes an image acquisition device 11 and an image analyzer 21. The image acquisition device 11 may be constructed and arranged to image at least a portion of one or more sample holders that contains a sample during processing of one or more samples associated with the holders. In one embodiment, the image acquisition device may include a camera, e.g., include a CCD imaging device, or other device suitable for capturing an image using visible or other electromagnetic radiation. The image acquisition device 11 may be capable of capturing images in rapid succession to record a movie clip, and/or may be capable of capturing still images.

[0030] In one embodiment, the image analyzer 21 may be used to analyze an image captured by the image acquisition device 11, e.g., used to detect any of a number of conditions and/or characteristics of the image. For example, the image analyzer 21 may be used to determine the presence or absence of sample in a sample holder. The image analyzer 21 may also be used to estimate the volume of sample in a sample holder by interpolating indicia on a sample holder or calculating the volume based on the height of a meniscus. The image analyzer 21 may also be used to detect clots or bubbles in a liquid sample, such as by observing a darker or lighter areas in a sample holder. Alternatively or additionally, the image(s) may be used to verify that a process was done correctly. For example, an image may be used to verify that a filter was used during aspiration. Such image analysis may be done automatically, e.g., by a software module operating on a computer or other data processing apparatus, and/or by inputting information from a user in response to review of a displayed image. In addition to providing information regarding image analysis, a user can provide other information that is associated with the image and/or a corresponding sample, such as providing annotations to otherwise unrecorded processing conditions. This information may be stored in a memory 22 with the image and/or with other information relating to the sample.

[0031] In some applications, the sample holders may be arranged in large groups which may make them difficult to image. For example, sample holders in the form of pipette tips 10 may be arranged in a plurality of columns and rows, as shown in FIG. 2. In this case, the image acquisition device 11 may image a subset of the sample holders. For example, pipetting head 12 may have a plurality of supports 12A, 12B, each of which may carry at least one pipette tip 10. As shown in FIG. 3, the supports 12A, 12B may be moved independently of each other. A first support 12A may lower its respective subset of pipettes while the remainder of the pipettes (e.g., those mounted to a second support 12B) remain in the raised position. If the image acquisition device 111 is positioned to acquire an image of pipettes in the lowered position, the sample processing apparatus 1 may lower the first support 12A so that the image acquisition device 11 may acquire an image of those sample holders 10. Then, the sample processing apparatus 1 may retract the first support 12A and lower the second support 12B to acquire an image of the sample holders connected to the second support 12B. The vision system can thus operate such that all pipette tips 10 having an associated sample are imaged. As will be understood by those of skill in the art, images may alternatively or additionally be acquired when the supports 12A, 12B are in the raised position, or any other position, as long as the image acquisition device 11 can capture a suitable image. Even though FIGS. 2 and 3 depict two supports 12A, 12B, any number of supports 12 may be used. Additionally, FIGS. 2 and 3 depict two offset rows of sample holders 10 on a single support 12A, 12B. However, as will be apparent to one of skill in the art, a single row of sample holders or any number of rows of sample holders may be arranged on a single support, as long as the desired image acquisition can be obtained.

[0032] Alternative sample holder configurations are also within the scope of the present invention, e.g., sample holders may be arranged in a polygon or a circle like that shown in FIGS. 4 and 5. In order to image all the sample holders, an image acquisition device 11 may image half of the sample holders at a time. For example, a screen 31 may to obstruct the view of a subset of sample holders 10F so as to capture a clear image of the remaining sample holders 10E. After an image is captured of a subset of the sample holders 10E, the support 12 could be rotated as shown in FIG. 5. Once the support has been rotated, the image acquisition device 11 can capture the remaining subset of the sample holders 10F. As will be apparent to one of skill in the art, other sample holder and imaging configurations are possible.

[0033] The image acquisition device 11 may be movable to capture images of different areas of the sample processing apparatus 1. For example, the image acquisition device 11 may be movable to acquire an image of one or more sample holders that could otherwise not be obtained without movement of the sample holder. Referring to the illustrative embodiments of FIGS. 2-5, instead of moving the sample holders 10 for imaging, the image acquisition device 11 may be moved or otherwise operated so as to obtain the desired images. The capability to acquire images of different areas may be accomplished by using an image acquisition device with a large field of view or, as shown in FIG. 1, by having a manipulator 16 to position the image acquisition device 11 appropriately. The manipulator 16 may be able to tilt the image acquisition device 11 as shown by arrows 160, raise and lower the image acquisition device as shown by arrows 161, or otherwise move the image acquisition device 11 in any suitable way. The manipulator 16 may be of any design, as long as it can move the image acquisition device 11 in the desired direction(s). Alternatively or additionally, mirrors, lenses, or other optical devices (not shown) may be used to change the field of view of the image acquisition device 11. For example, the sample processing apparatus may move a lens up to the image acquisition device's field of view in order to change the focal length of the image acquisition device 11. (Alternately, the image acquisition device 11 may have a built-in zoom function.) Also, multiple image acquisition devices may be used to obtain visual images of different areas, especially if simultaneous images are desired. Other means to change the field of view for the image acquisition device would be apparent to one of skill in the art and are also within the scope of the present invention. Despite the above, the image acquisition device 11 may be made stationary with a fixed (or variable) field of view.

[0034] The machine vision system can also be used to control the operation of portions of the sample processing apparatus 1 other than the image acquisition device 11. For example, as shown in FIG. 6, the machine vision system can be used to appropriately position two sets of sample holders relative to each other for transfer of samples from one holder to another. In this illustrative embodiment, a sample tray 18 may be placed on a surface with a grid 40 on it, and a camera 11A that is part of the image acquisition device 11 may image the sample tray 18 and determine the location of the tray 18 using the grid 40 information. The machine vision system may include a second camera 11B to image the location of pipette tips 10 on the pipetting head 12. The location of the tray 18 and the tips 10 may be used by the sample processing apparatus to determine the relative position of the two so that the head 12 may be moved to suitably locate desired tips 10 relative to wells in the tray 18. The use of machine vision systems for such control is well known in the art, and is not described further herein. Of course, it will be understood that the machine vision system may be used to control other portions of the sample processing apparatus 1, including portions that are not associated with a sample holder.

[0035] Additionally or alternatively, the sample holder may include at least one light source, such as a laser, that is aimed at a precise location if the sample holder is in the desired location (not shown). For example, lasers could be configured to shine at the corners of a sample tray. The image acquisition device could capture an image of the tray when properly placed to record that the tray was correctly positioned.

[0036] The controller 20 may have a user interface so that a user can interact with the controller and the sample processing apparatus 1. In one embodiment, the controller may include a computer system, which provides a user interface including a keyboard, mouse or other user pointing device, touch screen, display, printer and/or other components. In one embodiment of the invention, the user interface may include a graphical user interface (GUI) by which the user may receive information from, and input information to, the controller 20. FIG. 7 shows an illustrative embodiment of a GUI which may be included in one embodiment of the present invention. In particular, the GUI may enable the user to activate one or more analysis features of the image analyzer with respect to a given image (or images) 52, or portion of an image, e.g., by clicking or otherwise selecting one or more function buttons 51. For example, if an image 52 shows that the volume of one sample holder 10C is relatively low, a user may control the image analyzer 21 to determine a volume of sample in a sample holder of interest 10C. Image analysis may be done using a single image, or using two or more images. Alternately, the image analyzer could be configured to automatically perform image analysis on captured images, e.g., automatically compute a sample volume for samples included in the image. Results of such analysis may be displayed along with other information 53 and the image 52. The displayed information 53 may include information about the image, samples shown in the image, a patient associated with one or more samples in the image, protocols used to process the samples, or other. A user may be permitted (e.g., by using the GUI) to alter and/or add to the information 53 as desired, e.g., to provide comments regarding a particular sample, processing conditions not recorded in the image, and so on. In addition, the user may be permitted to associate the image with one or more identifiers, such as a patient identity, a source of one or more samples shown in the image, a protocol used to test the sample, etc. The image 52 and associated information 53 may be stored in a database or other suitable store for later reference or other use.

[0037] A user may also be able to program the sample processing apparatus or otherwise alter operating instructions used by the controller through the GUI, e.g., specifying the actions of the pipetting head 12, robotic manipulator 14, and manipulator 16; the volume of sample aspirated and/or dispensed; and/or other actions. The operator may additionally be able to program when and what to capture with the image acquisition device, how to tag and/or store the images, and any image analysis to be performed. The user interface may additionally or alternatively comprise buttons, dials, or other non-GUI means that will be apparent to those of skill in the art.

[0038] As shown in FIG. 8, a user may configure a sample processing apparatus to perform a sample handling process, which may include filtering, aspirating, dispensing, centrifuging, microfluidic processes, cell plating, running a gel or other processes with respect to one or more liquid samples. As one example, the sample handling process may be used to test one or more samples for a particular protein or DNA sequence. In step S601, the controller of the sample processing apparatus may direct a machine vision system to acquire an image of a sample or sample holder at a certain point in the process. After the image is acquired, in step S603 the image may be analyzed. For example, an image analyzer may determine a change in state of the sample, e.g., estimate a sample volume in a sample holder, determine a change in volume of the sample, determine a color or change in color of the sample, determine a position or change in position of the sample, check for the presence or absence of sample in a sample holder, or evaluate some other characteristic. In step S605, the results of the analysis and/or the image may be tagged with an identifier so that the image may be associated with the proper sample(s). The identifier tag may comprise alphanumeric symbols, an optical code such as a bar code, or any other identifier, and the identifier may be unique as compared to at least some other identifiers used with other images. In step S606, the image analysis data and/or image may be stored for later retrieval. Alternatively, the image may be stored only if the analysis indicates that an error occurred.

[0039] In some applications, it may not be desirable to analyze the image immediately upon capture. In these cases, after the image is acquired, it may be tagged and stored. As described above, the image can be tagged with a batch number, sample number, patient number, or any other identifier.

[0040] Additionally or alternatively, the results of the image analysis may trigger some action by the sample processing apparatus, e.g., in step S604. For example, if the analysis reveals that a sample was improperly or incompletely aspirated, the sample processing apparatus 1 may stop, sound an alarm, and/or reattempt the aspiration. Failed sample processes may prompt a sample to be reprocessed or discarded. The sample processing apparatus 1 may alternatively or additionally log the error in a list to be later reviewed by an operator. As shown in FIG. 8, after an action is triggered in step S604, the sample processing apparatus may additionally tag and/or store the image and/or data.

[0041] For example, a user may configure the sample processing apparatus to aspirate samples from a pipetting tray and dispense them into a set of wells in a multi-well plate. A robotic arm may move the desired sample tray from a storage location to an active location. A pipetting head may then lower at least one pipette to the sample tray and commence aspirating a plurality of liquid samples, e.g., under the control of the machine vision system. During aspiration, the image acquisition device may capture an image of the pipette tips. After aspiration is complete, the sample processing apparatus may again acquire image(s) of the pipettes, tag them with an identifier and store them in memory. Thereafter, the sample processing apparatus may dispense the sample into a different set of wells, acquire another image of the pipettes, and further testing may be done with the dispensed samples. When the operator reviews the test results, he may notice that the results from one sample or a subset of samples are not what was expected. At that point, he could review the captured images (retrieving them based on an identifier associated with the samples in question) to determine whether the proper volume of sample was aspirated and dispensed. The sample processing apparatus could calculate the volumes in each of the pipettes before and/or after the dispensing step to indicate to the operator whether the pipettor operated properly.

[0042] The method described above is one of many possibilities for a sample processing apparatus with vision capability. A number of variations are also within the scope of the present invention and will be apparent to one of skill in the art.

[0043] Having thus described several aspects of at least one embodiment of this invention, it is to be appreciated various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the spirit and scope of the invention. Accordingly, the foregoing description and drawings are by way of example only.