Motion detection in an image capturing device

[0001] The present invention relates generally to an image capturing device, and more particularly to an image capturing device that detects motion in images.

[0002] Image capturing devices, such as cameras, are typically used to capture scenes, persons, settings, occasions, etc. Image capturing devices may be used to capture images of persons, or groups of persons, animals and wildlife, etc.

[0003] A typical problem encountered in photographing moving subjects, especially children, is capturing an image when no one is moving. The movement of a subject may result in blurring, improper focus, or the appearance of not everyone being ready for a photograph. An additional problem is when the photograph is being taken through the use of a timer, wherein there is no photographer urging the subjects to hold still, look at the camera, smile, etc.

[0004] In the prior art, the typical approach to the problem of motion in an image is that the photographer must monitor everyone to be included in an image. The photographer therefore must make sure no one is moving before pressing the shutter button. Consequently, the photographer must watch through the viewer and attempt to snap the photo when no movement is observed.

[0005] This prior art approach relies on luck and the ability of the photographer to monitor and control the subjects. However, the photographer may be too busy to monitor the movement of the subjects. In addition, there may be a time lag between checking for movement and pressing the shutter button, and the photographer may still capture some movement in the photo. Additionally, if motion of the subjects stop, it may be brief, and the photographer may miss it. If the photographer is using a timer, the photographer may not have control over when the shutter is actually activated and a good photo is still dependent on luck.

[0006] In another prior art approach, the prior art has approached the problem of motion through the use of high speed shuttering. However, this prior art approach limits the light, and may result in an underexposed picture.

[0007] Therefore, there remains a need in the art for improvements in image capturing devices.

[0008] An image capturing device comprises an electronic image sensor, a memory, and a processor. The memory includes a motion detect routine, a predetermined image interval, and at least one predetermined motion threshold. The processor conducts the capturing of images separated by the predetermined image interval, compares a current image to one or more previous images, determines when motion between the current image and the previous image is below the at least one predetermined motion threshold, and stores the current image as a final image when the current image is stable.

[0009] FIG. 1 is a schematic of an image capturing device according to one embodiment of the invention;

[0010] FIG. 2 is a flowchart of an image capturing method according to another embodiment of the invention; and

[0011] FIG. 3 is a flowchart of an image capturing method according to yet another embodiment of the invention.

[0012] FIG. 1 is a schematic of an image capturing device 100 according to one embodiment of the invention. The image capturing device 100 may include a lens apparatus 102, an image sensor 105, a processor 110, a shutter button 111, and a memory 140.

[0013] In one embodiment, the image capturing device 100 may be a digital camera. In another embodiment, the image capturing device 100 may be a digital video camera capable of capturing still images.

[0014] The image sensor 105 may be any type of electronic image sensor capable of capturing images, such as a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor, for example.

[0015] The processor 110 may be any type of general purpose processor. The processor 110 executes a control routine contained in the memory 140. In addition, the processor 110 receives inputs and conducts image capturing operations.

[0016] The memory 140 may be any type of digital memory. The memory 140 may store, among other things, a motion detect routine 143, a number of (image sensor) regions variable 146, a main object designation 151, a sampling pattern 154, an image interval 158, one or more motion thresholds 163, a motion detect variable 169, and a hold timeout timer 168. In addition, the memory 140 may store captured images and software or firmware to be executed by the processor 110.

[0017] The motion detect routine 143 compares captured images and detects motion by detecting changes between pixels in the captured images. The motion detect routine 143 may compare all pixels in the images, may compare pixels of image regions, or may sample representative pixels. The motion detect routine 143 may compare pixel values between successive image frames and may produce a difference value for each compared pixel. In the absence of motion in the images, the pixels should change very little. Therefore, the difference values generated by the motion detect routine 143 may be compared to a threshold or thresholds in order to detect motion in the images.

[0018] The number of regions variable 146 is an optional variable and is included when an image is divided into regions. Each region may be concurrently processed in order to detect motion. If an unacceptable level of motion exists in any region, the image capturing operation may be delayed until the motion ceases (or at least drops to an acceptable level). It should be noted that individual regions may optionally include their own motion thresholds.

[0019] The main object designation 151 may be an optional feature where instead of comparing the whole image, the image capturing device 100 compares a designated object or region between image frames. The object or region may be, for example, a region in the center of the image, i.e., if the center of the image does not change then an image is captured. Alternatively, the designated object may be a designated person. The user of the image capturing device 100 may use crosshairs, etc., to designate a certain region or object. The region may encompass a person, a group of persons, or a designated set of regions.

[0020] The sampling pattern 154 may designate predetermined pixels to be sampled for the motion detection process. The sampling pattern 154 may be used for an entire image comparison, a designated object comparison, a regions comparison, etc. By using the sampling pattern 154, the image capturing device 100 may sample only designated pixels. This reduces processing time and processing requirements while still providing a reliable motion determination.

[0021] The image interval 158 is a predetermined time interval between images, i.e., successive images are separated by the predetermined time interval. If motion is detected, the image capturing device 100 waits for the image interval 158 to expire and then captures another image to compare to the previous image.

[0022] The image interval 158 is optionally user-settable. The image interval 158 may be chosen to accommodate the particular image capture environment. If the image interval 158 is small, then the motion detection is of a finer resolution, but more processing will be required.

[0023] The one or more motion thresholds 163 are predetermined acceptable motion values. If the detected amount of pixel change is greater than a corresponding motion threshold 163, the image capturing device 100 determines that there is still an undesirable level of motion occurring in the most recently captured image. Conversely, if the detected amount of pixel change is less than or equal to the motion threshold 163, then the image capturing device 100 may determine that the level of motion is acceptable and may complete the image capturing operation.

[0024] It should be understood that one or more motion thresholds 163 may be used in an image capturing device embodiment that compares multiple image regions. In one embodiment a single threshold may cover all regions. In another embodiment, each region may include a corresponding threshold. In yet another embodiment, some regions may have individual thresholds and some regions may share a common threshold.

[0025] The hold timeout timer 168 may be loaded with a predetermined hold timeout period (i.e., a maximum hold time). Although the image capture and image comparison may be indefinitely repeated, the hold timeout timer 168 may function as an overall timer. Therefore, the hold timeout timer 168 may cause an image capture at some fixed point in time regardless of motion, i.e., at some point in time the image capturing device 100 gives up and captures an image, regardless of detected motion. The hold timeout timer 168 may be user-settable.

[0026] The motion detect variable 169 controls the motion detect mode. When the motion detect variable 169 is set to an enable state, the image capturing device 100 is in the motion detect mode. This variable may be user controlled, wherein the motion detect may be turned on and off.

[0027] In operation, the image capturing device 100 captures a current image and compares it to one or more previously captured images. If significant motion is detected, the process loops and repeats; otherwise the current image (i.e., the most recently captured image) is stored as the final image (see FIGS. 2 and 3 below and accompanying discussion).

[0028] FIG. 2 is a flowchart 200 of an image capturing method according to another embodiment of the invention. In step 201, a shutter button press is detected. The shutter button press signals the start of the image capturing process.

[0029] In step 206, an initial image is captured.

[0030] In step 208, another (current) image is captured.

[0031] In step 212, the current image is compared to one or more previously captured images. For example, the comparison may be between the current image and the previous image. Alternatively, the current image may be compared to more than one previous image. The comparison may be a comparison of entire images, may be a comparison of a designated object, may be a comparison of a designated region, or may be a region-by-region comparison.

[0032] In step 213, the hold timeout timer 168 is checked. The hold timeout timer 168 may ensure that the wait for a motionless image does not go on indefinitely, and limits the amount of time the image capturing device 100 will wait for motion to cease. If the hold timeout timer 168 has expired, the method branches to step 218; otherwise it proceeds to step 215.

[0033] In step 215, the method determines whether the current image is stable. This is done by comparing pixels (or sampled pixels) and comparing the amount of pixel change to one or more motion thresholds 163. The current image is stable if the amount of pixel change from the previous image is below the motion threshold 163 (i.e., the image does not include an unacceptable amount of motion). If the current image is not stable, the method branches to step 222. If the image is stable, the method proceeds to step 218.

[0034] In step 218, the current image is stored as the final image of the image capturing operation, as a result of the shutter button press. If the hold timeout timer 168 did not expire, the stored image will include motion that is less than the one or more motion thresholds 163. If the hold timeout timer 168 did expire, the stored image contains motion in excess of a motion threshold. The motion detection of this embodiment may optionally be enabled or disabled by the user.

[0035] In step 222, the method waits for the image interval 158 to expire and then another image is captured for a purpose of comparison. The image interval 158 therefore regulates the elapsed time between successive image captures.

[0036] FIG. 3 is a flowchart 300 of an image capturing method according to yet another embodiment of the invention. In step 303, an image is captured, as previously described.

[0037] In step 308, another (current) image is captured, as previously described.

[0038] In step 314, the current image is compared to one or more previously captured images, as previously described. For example, the comparison may be between the current image and the previous image. As before, the comparison may be a comparison of entire images, may be a comparison of a designated object, may be a comparison of a designated region, or may be a region-by-region comparison.

[0039] In step 325, it is determined whether the current image is stable. This is done by comparing pixels or sample pixels and comparing the amount of change between pixels to one or more motion thresholds 163, as previously discussed. If the current image is stable, the method proceeds to step 337; otherwise it branches to step 332.

[0040] In step 332, the image interval 158 is allowed to expire and another image is captured and a new comparison is performed. The image interval 158 therefore regulates the elapsed time between successive image captures.

[0041] In step 337, the image is stable and the method waits for a shutter button press. If the shutter button 111 is pressed, then the method proceeds to step 350. If the shutter button 111 is not pressed immediately after stability is achieved, the method proceeds to step 345.

[0042] In step 345, the hold timeout timer 168 (a hold time that waits for a shutter button press) is checked. If the hold timeout timer 168 has not expired, the method loops back to step 337. If the shutter button is not pressed before the hold timeout timer 168 expires, the method branches back to step 308, wherein a new current image is captured and the motion detect is reprocessed. This is done so that a significant amount of time does not elapse between the capturing of a stable image and a shutter button press.

[0043] In step 350, the most recently captured current image is saved as the final image of the image capturing operation.

[0044] This method may continuously monitor for image motion and may be ready for a shutter button press in advance. The flowchart 300 therefore shows a motion detect mode wherein images are periodically captured and processed in order to detect motion. If an acceptable level of motion or no motion is detected, the image capturing device 100 is ready for a shutter button press. However, if a shutter button press does not occur within the predetermined hold time, a new image is captured and processed to detect motion. As before, the motion detection of this embodiment may optionally be enabled or disabled by the user.

[0045] In addition, the image capturing method 300 may be in operation whenever the image capturing device 100 is placed in an image capturing mode, such as if a lens door is opened, for example. Alternatively, the image capturing method 300 may be in operation whenever the user at least partially depresses the shutter button 111 (such as to charge a flash or to determine a focus distance, for example). In addition, the image capturing method 300 may also be in operation during a preview mode, wherein images are continuously updated and shown on a display of the image capturing device 100 and the user can elect to press the shutter button 111 at any moment.

[0046] The motion detection according to the invention may be employed by any manner of digital still camera. In addition, the motion detection may be employed by any manner of video camera capable of capturing still images.

[0047] The invention differs from the prior art in that the prior art focuses on detecting motion in order to adjust shutter speed to accommodate motion. The prior art does not capture successive images and does not compare them and capture an image when motion in the images drops below a threshold.

[0048] In contrast, the image capture according to the invention is automatically performed once the motion falls below the threshold. Furthermore, the invention may continuously capture images, perform comparisons, and be ready for an image capture in advance of a shutter button press.

[0049] The invention provides several benefits. The photographer does not have to try to capture an image when no one is moving. Another benefit is that the photographer does not have to watch the subject closely and may focus on framing the image, lighting, arrangement, etc. Yet another benefit is that the motion detect according to the invention may render a timer unnecessary, as the photographer can place the camera in a motion detect mode and join the group. As a result, the image capture occurs when the photographer is in front of the camera and no motion exists.