Adaptively reducing offset in a thermal imaging camera

[0001] This invention relates generally to the field of infrared cameras and more specifically to adaptively reducing offset in a thermal imaging camera.

[0002] Infrared cameras detect infrared radiation emitted by the objects of a scene, and generate an image of the scene from the detected radiation. An infrared camera has a temperature detector that detects subtle temperature changes in the scene such as changes of less than one degree Fahrenheit. The detector may also be extremely sensitive to camera temperature changes that cause unintentional signal offsets.

[0003] The signal offsets may be subtracted from the signal. Typically, subtraction may be performed by factory calibration followed by real-time calibration. According to known techniques for real-time calibration, the detector may be blocked by a shutter in order to provide a constant reference from which samples may be collected. The samples may then be used to determine a signal offset. Dropping the shutter during collection, however, does not allow the detector to gain information about the scene.

[0004] In accordance with the present invention, disadvantages and problems associated with previous techniques for reducing signal offset in an infrared camera may be reduced or eliminated.

[0005] According to one embodiment, collecting samples to adjust a signal includes providing a shutter in an opened position to allow a detector of an infrared camera to detect infrared radiation and generate a signal corresponding to the radiation. A first touchup iteration is initiated by moving the shutter to a closed position, and samples of a reference frame are collected according to a collection instruction. The shutter is then moved to the opened position. If there is a change in a state of the infrared camera, the collection instruction is adjusted in response to the change. A second touchup iteration is initiated by moving the shutter to the closed position, and samples are collected according to the adjusted collection instruction. A modification of the signal is determined in accordance with at least some of the samples.

[0006] Certain embodiments of the invention may provide one or more technical advantages. A technical advantage of one embodiment may be that sample collection may be dynamically adjusted in response to changes at the infrared camera. Dynamically adjusting sample collection may provide for more effective and efficient sample collection.

[0007] Certain embodiments of the invention may include none, some, or all of the above technical advantages. One or more other technical advantages may be readily apparent to one skilled in the art from the figures, descriptions, and claims included herein.

[0011] Embodiments of the present invention and its advantages are best understood by referring to FIGS. 1 and 2 of the drawings, like numerals being used for like and corresponding parts of the various drawings.

[0012] FIG. 1 is a block diagram of one embodiment of a system 10 for generating an image of a thermal scene 12. According to the embodiment, system 10 performs a touchup procedure by moving a shutter to block a detector to allow the detector to collect samples of a reference frame. System 10 dynamically controls the collection of samples in response to changes in system 10, which may provide for more effective and efficient sample collection.

[0013] Thermal scene 12 includes objects that emit infrared rays 14 that travel to system 10. An infrared ray 14 refers to electromagnetic energy in a band with a wavelength longer than that of visible light, but shorter than that of microwave energy. Rays 14 include thermal information about thermal scene 12. Examples of objects of thermal scene 12 may include a human, a vehicle, a structure, other object that emits infrared rays 14, or any combination of the preceding. Thermal scene 12 may be part of any suitable application such as a fire fighting, surveillance, thermography, security, or automotive application.

[0014] System 10 operates as a thermal camera that receives infrared rays 14 and generates an image 16 representative of thermal scene 12 from infrared rays 14. According to the illustrated embodiment, system 10 includes optics 20, a shutter 22, a shutter controller 24, a detector 26, electronics 30, and a display 32 coupled as shown.

[0015] Optics 20 directs rays 14 towards detector 26, and may comprise an optical lens. Shutter 22 may be moved to an opened position that allows detector 26 to detect most or all rays 14 from thermal scene 12, and may be moved to a closed position that prevents detector 26 from detecting most or all rays 14. Shutter controller 24 may control the movement of shutter 22 in response to instructions from electronics 30.

[0016] Detector 26 detects rays 14 and generates a signal corresponding to rays 14. The signal corresponding to rays 14 includes at least some of the thermal information of rays 14. Detector 26 comprises an array of detecting elements, or pixels, where each element generates image data for thermal scene 12. For example, detector 26 may comprise a microbolometer focal plane array.

[0017] Electronics 30 receives the signal from detector 26 and processes the signal to generate image 16 on display 32. According to the illustrated embodiment, electronics 30 includes a processor 40, a memory 42, an analog-to-digital converter (A/D) 44, and an image processor 46. Processor 40 controls the operation of system 10. As used in this document, the term “processor” refers to any suitable device operable to execute instructions and manipulate data to perform operations.

[0018] Processor 40 moves shutter 22 by sending instructions to shutter controller 24 to move shutter 22. During a touchup procedure, processor 40 sends sample collection instructions to shutter controller 24 to move shutter 20 to a closed position to allow detector 26 to collect samples of reference frames. A frame may refer to video data for at least most pixels of detector 26. A reference frame may refer to a frame of a constant known scene, such as the scene presented to detector when shutter 22 is in a closed position. A reference sample may refer to a sample of a reference frame.

[0019] Sample collection instructions may include a shutter duration describing the amount of time that shutter 22 is in a closed position, a collection time describing an amount of time to collect samples, a settle time describing an amount of time between closing shutter 22 and collecting-samples, a delay time describing the time of the next touchup iteration, other instruction, or any combination of the preceding.

[0020] According to one embodiment, processor 40 dynamically adjusts sample collection instructions in response to changes that occur at system 10. Processor 40 may adjust the instructions according to a method described with reference to FIG. 2.

[0021] Memory 42 stores and facilitates retrieval of information used by processor 40. The information may include touch-up data and reference data. Valid touch-up data includes samples collected during one or more previous iterations that have not been purged. Valid reference data may refer to the last known valid signal adjustments used to reduce or remove signal offsets. Reference data may be purged in response to a gain transition or a temperature change that exceeds a specific threshold.

[0022] Memory 42 may include Random Access Memory (RAM), Read Only Memory (ROM), magnetic drives, disk drives, Compact Disk (CD) Drives, Digital Video Disk (DVD) drives, removable media storage, any other suitable data storage device, or a combination of any of the preceding.

[0023] Analog-to-digital converter 44 converts the signal from detector 26 from an analog form to a digital form. Image processor 46 processes the signal to generate image 16. According to one embodiment, image processor uses signal samples collected during sample collection to determine offset adjustments. Image processor 46 may then apply the adjustments to the signal to reduce or remove signal offsets. Display 32 displays image 16. Display 32 may have any suitable shape or size. For example, display 32 may comprise a screen of any suitable size that may be coupled to a computing device. The computing device may have any suitable embodiment, such as a desktop computer, a laptop, a handset, or a headset.

[0024] Modifications, additions, or omissions may be made to system 10 without departing from the scope of the invention. The components of system 10 may be integrated or separated according to particular needs. Moreover, the operations of system 10 may be performed by more, fewer, or other modules. For example, the operations of processor 40 and image processor 46 may be performed by one module, or the operations of image processor 46 may be performed by more than one module. Additionally, operations of system 10 may be performed using any suitable logic comprising software, hardware, other logic, or any suitable combination of the preceding. As used in this document, “each” refers to each member of a set or each member of a subset of a set.

[0025] FIG. 2 is a flowchart illustrating one embodiment of a method for reducing signal offset in an infrared camera. The method may be used with any suitable system, for example, system 10 of FIG. 1.

[0026] The method begins at step 100, where detector 26 of system 10 receives rays 14 from scene 12. A touchup is scheduled at step 102. The time of the touch-up may be determined according to the state of system 10. The state of system 10 may refer to conditions at system 10 that create signal offsets. For example, a condition may refer to the temperature of detector 26 or other component of system 10. As another example, a condition may refer to the response of detector 26, a gain controller, or other component of system 10 to thermal scene 12. As yet another example, a condition may refer to the detection of a latent image.

[0027] A changing state may refer to a state during which conditions, such as the temperature or the response of detector 26, change significantly from frame to frame. An example of a changing state may be an initialization state when the power of system 10 initialized. Another example of a changing state occurs when movement within thermal scene 12 causes variation in the response of detector 26. Yet another example of a changing state occurs when changes of the brightness within thermal scene 12 causes variation in the response of a gain controller. Yet another example of a changing state occurs when environmental or operational factors cause variations in the temperature of system 10. A steady state may refer to a state during which conditions do not change significantly from frame to frame.

[0028] Touch-ups may be scheduled more frequently during a changing state than during a steady state to take into account the variation. A delay time may refer to the time period between the movement of shutter 20 to an open position at one iteration and the movement of shutter 20 to a closed position at the next iteration. The next touch-up during a changing state may be scheduled to occur after a delay time of x seconds, and the next touch-up during a steady state may be scheduled to occur after a delay time of y seconds, where x is less than y. Value x may be in the range of 10 to 20 seconds such as approximately 15 seconds, while value y may be in the range of 20 to 40 seconds such as approximately 30 seconds.

[0029] A touch-up may be dynamically scheduled in response to a change of a condition. A touch-up may be scheduled to occur after a short delay time in response to a detected change in temperature, gain, or detector 26. For example, a touch-up may be scheduled if the temperature has changed by 10 degrees Celsius. As another example, a touch-up may be scheduled to occur after a short delay time if a latent image is detected.

[0030] The shutter duration is calculated at step 104. The shutter duration may refer to the amount of time that shutter 22 is in the closed position. The shutter duration t_sd may be expressed by Equation (1):
t_sd=t_settle+t_collection+2×t_{shuttertransition}  (1)
where t_settle is the time for detector 26 to settle to a noise equivalent temperature difference (NETD), t_collection is the time it takes to collect a predetermined number of frames, and t_{shuttertransition} is the shutter transition time to move shutter 20 from one position to another position and may be approximately 25 milliseconds.

[0031] Settle time t_settle may be calculated in accordance with the temperature of the pixels of detector 26 as indicated by pixel values. The higher the temperature of the pixels, the longer the settle time. For example, higher temperature pixels may correspond to a settle time of more than 20 milliseconds, while cooler pixels may correspond to a settle time of less than 20 milliseconds. The temperature of the pixels may be calculated using the values of most of the pixels, for example, greater than 90% of the pixels, such as greater than 95% of the pixels. The values of these pixels may be used in a decay time equation to determine the time it would take the pixels to settle to an operating temperature. For example, settle time t_settle may be calculated according to Equation (2):

[0032] $\begin{array}{cc}{t}_{\mathrm{settle}}={\tau }_{\mathrm{th}}\ln \left(\frac{1}{\beta }\frac{T_{\mathrm{scene}}-T_{\mathrm{shutter}}}{\mathrm{NETD}}\right)& \left(2\right)\end{array}$
where τ_th represents the combined thermal and electrical time constant of the detector, β represents the factor relating the desired settling to NETD, β<1, T_scene represents the highest signal temperature in the field of view for most of the pixels, NETD is approximately 0.05 degrees, and |T_scene−T_shutter|≧(NETD×β).

[0033] Collection time t_collection may be calculated in accordance with the state of system 10. A changing state may use a longer collection time such as in the range of 100 to 800 msec, such as approximately 200 msec. A steady state may use a shorter collection time such as in the range of 25 to 100 msec, such as approximately 33 msec. The collection time may also be measured in terms of the number of samples collected. For example, a shorter collection time may correspond to one to four samples, such as approximately one sample, and a longer collection time may correspond to four to sixteen samples, such as approximately eight samples. Collection time t_collection may also be calculated in accordance with the detection of a latent image. If a latent image is detected, a longer collection time may be used.

[0034] Examples of shutter duration may include approximately 100 milliseconds for a steady state condition where |T_scene−T_shutter| is substantially equivalent to NETD×β, approximately 200 milliseconds for a steady state condition with a scene where |T_scene−T_shutter| is substantially equivalent to 10° Celsius with β=1, approximately 250 milliseconds for a steady state condition with a scene where |T_scene−T_shutter| is substantially equivalent to 600° Celsius with β=1, and approximately 500 to 600 milliseconds for a dynamic condition with a scene where |T_scene−T_shutter| is substantially equivalent to 600° Celsius with β=1.

[0035] Shutter 22 is moved to a closed position at step 106. Shutter 22 in the closed position prevents detector 26 from detecting rays 14. Reference samples are collected at step 108. Samples may be collected by electronics 30 through detector 26. The samples may be collected at any suitable interval, for example, at intervals of 16 to 40 milliseconds, such as approximately 33 milliseconds.

[0036] Shutter 22 is moved into an opened position at step 112. The method determines if there is previous valid touch-up data at step 116. Valid touch-up data may refer to samples collected during one or more previous iterations and that have not been purged. Processor 40 may check memory 42 to determine if there is previous valid touch-up data.

[0037] If there is previous valid touch-up data, the method proceeds to step 120, where the samples of the current touch-up data are combined with the samples of the previous touch-up data. The samples are combined to estimate the fixed noise of the signal. The estimate may be used to determine an adjustment of the signal to reduce or remove the fixed noise.

[0038] The samples of the data may be combined in any suitable manner. For example, the samples may be combined by averaging the samples, such as by simply or recursively averaging the samples. The samples of current touch-up data may be combined with the samples of previous touch-up data, by simple or recursive averaging, in order to provide a broader sample of reference frame. Any suitable number of previous samples may be combined with any suitable number of current samples. As an example, a greater number of previous samples may be used if the infrared camera is in a steady state than if the infrared camera is in a changing state. Previous samples may be weighted with respect to the current samples. As an example, previous samples may be given a higher weighting if the infrared camera is in the steady state than if the infrared camera is in the changing state. If there is no previous valid touch-up data at step 116, the method proceeds to step 124, where the samples of current touch-up data are combined.

[0039] The method determines if there is valid reference data at step 128. Processor 40 may check memory 42 to determine if there is valid reference data. Valid reference data may refer to the last known valid signal adjustments, and data may be purged in response to a gain transition or a temperature change that exceeds a specific threshold.

[0040] If there is valid reference data at step 128, the method proceeds to step 134, where the current touch-up data is compared with the reference data to determine if there is a latent image. For example, processor 40 may do a pixel by pixel comparison of the touch-up data with the reference data to determine if there are any differences between the pixels that indicate a latent image. The method determines if there is a latent image at step 128. Differences may indicate a latent image if the differences exceed a latency correction threshold. Differences that do not exceed the latency correction threshold may be designated as either not indicating a latent image or as indicating a latent image too subtle to correct.

[0041] If there is no latent image that needs to be corrected at step 138, the method proceeds to step 140, where reference data is saved. The reference data may be saved in memory 42. If there is a latent image that is to be corrected at step 138, the method proceeds to step 142. Typically, reference data resulting from a latent image is not saved.

[0042] The signal is adjusted using the current touch-up data at step 142. For example, image processor 46 may use the touch-up data to adjust the signal to reduce distortion of the signal. If the method is to continue at step 146, the method returns to step 102, where the next touchup is scheduled. If the method is not to continue at step 152, the method terminates.

[0043] Modifications, additions, or omissions may be made to the method without departing from the scope of the invention. The method may include more, fewer, or other steps. Additionally, steps may be performed in any suitable order without departing from the scope of the invention.

[0044] Certain embodiments of the invention may provide one or more technical advantages. A technical advantage of one embodiment may be that sample collection may be dynamically adjusted in response to changes at the infrared camera. Dynamically adjusting sample collection may provide for more effective and efficient sample collection.

[0045] While this disclosure has been described in terms of certain embodiments and generally associated methods, alterations and permutations of the embodiments and methods will be apparent to those skilled in the art. Accordingly, the above description of example embodiments does not constrain this disclosure. Other changes, substitutions, and alterations are also possible without departing from the spirit and scope of this disclosure, as defined by the following claims.