High speed multi-focal camera

[0001] The present invention relates in general to the field of information handling system cameras, and more particularly to an information handling system high speed multi-focal camera.

[0002] As the value and use of information continues to increase, individuals and businesses seek additional ways to process and store information. One option available to users is information handling systems. An information handling system generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes thereby allowing users to take advantage of the value of the information. Because technology and information handling needs and requirements vary between different users or applications, information handling systems may also vary regarding what information is handled, how the information is handled, how much information is processed, stored, or communicated, and how quickly and efficiently the information may be processed, stored, or communicated. The variations in information handling systems allow for information handling systems to be general or configured for a specific user or specific use such as financial transaction processing, airline reservations, enterprise data storage, or global communications. In addition, information handling systems may include a variety of hardware and software components that may be configured to process, store, and communicate information and may include one or more computer systems, data storage systems, and networking systems.

[0003] Portable information handling systems integrate processing components, a display and a power source in a portable housing to support mobile operations. Portable information handling systems allow end users to carry a system between meetings, during travel, and between home and office locations so that an end user has access to processing capabilities while mobile. Tablet configurations typically expose a touchscreen display on a planar housing that both outputs information as visual images and accepts inputs as touches. Convertible configurations typically include multiple separate housing portions that couple to each other so that the system converts between closed and open positions. For example, a main housing portion integrates processing components and a keyboard and rotationally couples with hinges to a lid housing portion that integrates a display. In clamshell configuration, the lid housing portion rotates approximately ninety degrees to a raised position above the main housing portion so that an end user can type inputs while viewing the display. After usage, convertible information handling systems rotate the lid housing portion over the main housing portion to protect the keyboard and display, thus reducing the system footprint for improved storage and mobility.

[0004] One common function of portable information handling systems is to support videoconferencing with a camera included in the information handling system housing. A typical integrated camera has a limited size to fit into the housing so that the lens is typically placed relatively close to the image sensor. At longer distances to an object, such as greater than two meters, the camera can typically focus to infinity and capture the object with clarity. At shorter distances, an actuator, such as a step motor or voice coil motor (VCM), moves the camera lens relative to the camera image sensor to focus the camera on an object. Conventional focus logic uses analysis of a captured image with adjustments to the lens position to determine when an object is in focus so that an image can be captured. One technique for autofocus is contrast detection auto focus (CDAF), which analyzes contrast at object borders to find focus on the object. A difficulty with CDAF is that the time to find focus can exceed several seconds while the camera image signal processor (ISP) analyzes the captured image information for contrast. Another technique for autofocus is phase detection auto focus (PDAF), which analyzes whether multiple images generated by prisms line up or are in phase with each other. PDAF determines focus more quickly than CDAF but has an increase cost and size due to the prisms that split the image for phase analysis. Both CDAF and PDAF have delays in sensing focus in part because the lens actuator has to increment through different focus ranges to compare images captured at the different focus ranges.

[0005] Therefore, a need has arisen for a system and method which automatically focuses a camera in a rapid manner.

[0006] In accordance with the present invention, a system and method are provided which substantially reduce the disadvantages and problems associated with previous methods and systems for automatically focusing a camera. A camera detects distance to an object with a sensor, such as a time-of-flight sensor, and applies the distance to a focal point lookup table to set an actuator position based on the distance that focuses the camera for the distance.

[0007] More specifically, an information handling system processes information with a processor and memory that cooperate to execute instructions, such as videoconferencing application. A camera captures visual images with a lens that focuses a camera field of view on an image sensor managed by an image signal processor (ISP). A distance sensor, such as a time-of-flight sensor, detects a distance from the camera to an object in the camera field of view and the ISP uses the distance to look up an actuator position for focus of the lens based upon the distance. The actuator is commanded to the actuator position associated with focus so that a visual image is captured in a rapid manner without having to perform image-based focus analysis, such as CDAF or PDAF.

[0008] The present invention provides a number of important technical advantages. One example of an important technical advantage is that a camera is rapidly focused on an object in the camera field of view by reference to a sensed distance so that delays associated with image-based analysis are avoided or reduced. Validation of the sensed distance and associated focus actuator position help to ensure that quality images are captured in a rapid and accurate manner.

[0016] An information handling system camera rapidly focuses on an object by applying distance information to the object determined by a distance sensor, such as a time-of-flight sensor. For purposes of this disclosure, an information handling system may include any instrumentality or aggregate of instrumentalities operable to compute, classify, process, transmit, receive, retrieve, originate, switch, store, display, manifest, detect, record, reproduce, handle, or utilize any form of information, intelligence, or data for business, scientific, control, or other purposes. For example, an information handling system may be a personal computer, a network storage device, or any other suitable device and may vary in size, shape, performance, functionality, and price. The information handling system may include random access memory (RAM), one or more processing resources such as a central processing unit (CPU) or hardware or software control logic, ROM, and/or other types of nonvolatile memory. Additional components of the information handling system may include one or more disk drives, one or more network ports for communicating with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, and a video display. The information handling system may also include one or more buses operable to transmit communications between the various hardware components.

[0017] Referring now to FIG. 1, an exploded perspective view of an information handling system 10 depicts various embodiments of cameras 40 that capture visual images. In the example embodiment, information handling system 10 is built in a portable housing 12 having a main portion 14 rotationally coupled with a lid portion 16 by a hinge 18 to rotate between open and closed positions. A cover portion 20 couples over main portion 14 to support a keyboard 22 and touchpad 24 that accept end user inputs. A central processing unit (CPU) 26 couples to a motherboard in main portion 14 to execute instructions that process information in cooperation with a random access memory (RAM) 28 that stores the instructions and information. A solid state drive (SSD) 30 has persistent storage to store information and instructions during power down states, such as an operating system and applications that are retrieved to RAM 28 for execution by CPU 26. An embedded controller 32 manages physical operating conditions at the information handling system, such as application of power, thermal constraints and interactions with input/output devices like keyboard 22 and touchpad 24. A graphics processing unit (GPU) 34 processes information to generate visual images defined by an array of pixel values for presentation at an integrated display 38 included in lid portion 16 and a peripheral display 42. A wireless network interface controller 36 manages network and peripheral communications, such as through WIFI and BLUETOOTH. In the example embodiment, a camera 40 couples to lid portion 16 to capture visual images of an end user viewing integrated display 38. Another camera 40 couples to peripheral display 42 to capture visual images of an end user viewing the peripheral display. A peripheral camera 40 operates separate from information handling system 10 and peripheral display 42 to capture visual images based upon its position in a work area. In alternative embodiments, other types of cameras may be used as described below.

[0018] Referring now to FIG. 2, a block diagram depicts operational elements of a camera that cooperate to capture visual images with a rapid focus response. In the example embodiment, an image signal processor (ISP) 50 executes instructions stored in a non-transitory flash memory 60 to manage capture of visual images of light accepted through a lens 58 and directed at an image sensor 51. ISP 50 processes the image sensor information to define the captured visual image as an array of pixel values having a resolution set by the amount of pixels of image sensor 51. The light directed through lens 58 is focused by adjusting a distance between lens 58 and image sensor 51 with an actuator, such as a voice coil motor (VCM) or a step motor. ISP 50 includes logic stored in the non-transitory memory that executes to perform image-based focus, such as CDAF and/or PDAF. Image based focus analyzes image uses the image information to determine when objects are in focus. For instance, contrast detection (CDAF) compares the contrast at different focuses to have a sharp contrast when an object is in focus relative to when the object is not in focus. Both CDAF and PDAF compare visual images captured at increments to find the increment with the best focus result; however, the incremental image capture and processing tend to slow the autofocus process. To reduce the time needed for focus of lens 58 by actuator 56, ISP 50 retrieves distance information from a distance sensor, such as a time-of-flight sensor 52, and applies the distance to an object to a focal point table 62 that has a lens actuator position for the distance detected by time-of-flight sensor 52. ISP 50 also accepts a temperature measurement from a thermistor 54 or other thermal sensor to adjust the actuator position selected from the focal point table 62 to correct for the sensed temperature.

[0019] In one example embodiment, ISP 50 detects a distance from time-of-flight sensor 52 and commands capture of a visual image by setting lens actuator 56 to an actuator position without any image based analysis, such as CDAF or PDAF. The direct command to an actuator position and capture of a visual image without any image-based analysis provides a rapid autofocus and image capture with a lens focus set by distance information from a different sensor. Once the image is captured, ISP 50 can perform additional analysis on the image data if desired, such as to determine whether the image quality is adequate and if additional time is available to perform image-based analysis that could yield a better image quality. In addition to image analysis, other validation of the focus distance may be performed to ensure adequate image quality. Validation can use a variety of factors based upon the context at the camera when the image is captured, such as the amount of the focal view included in the distance measured by the time-of-flight sensor and variations in the field of view that indicate facial features. For example, the time-of-flight sensor produces an infrared illumination with a field of view that is more narrow than the camera field of view and is scanned to different positions within the camera field of view. Variations in the timing of reflections can indicate facial features, such as nose, eye sockets, ears etc. . . . In contrast, a flat surface might indicate a reflective surface and a weak or non-existent infrared reflection might indicate a distance of greater than the sensing capability of the time-of-flight sensor. In one embodiment, user presence is detected by a time-of-flight sensor that detects slight changes in distance to an object, which shows a user in present and moving in position slightly relative to the camera. A user presence detection may be used as a context that allows a focus distance to be set without reference to image analysis. The camera context to validate focus distance can include previous visual images captured by the camera and distances sensed with the capture of those visual images. For instance, an image of a face captured by the camera that has a similar distance measurement in a limited time period validates subsequent pictures taken by the camera. When the sensed distance is not validated, the camera can instead rely upon image-based analysis and then use the sensed distance for validation of subsequent pictures. In one alternative embodiment, other types of distance sensors may be used, such as LIDAR sensors.

[0020] Referring now to FIG. 3, an example focal point table defines actuator positions associated with focus for distances detected by a time-of-flight sensor. In the example embodiment, five predefined object distances of 0.2, 0.4, 0.5, 1.0 and 1.9 M are used to cover a focal field of 0.18 M to 5.8 M. These example settings encompass normal operational conditions of webcam use cases, such as to support a videoconference with a camera included in a portable information handling system housing lid portion. The predefined lens shift values are translated to VCM step data and stored as a mapping table in camera nonvolatile memory. The VCM jumps to the step associated with a distance when the distance is detected by a time-of-flight or similar distance sensor. Thermal adjustments to the VCM steps are defined for different thermal conditions that alter the focus characteristics of the camera lens. Although the example embodiment uses five predefined VCM step values for an f1.8 camera, fewer step values may be used with a smaller lens aperture, such as f2.7, and a greater number of step values may be used with a larger lens aperture.

[0021] Referring now to FIG. 4, a flow diagram depicts a process for managing camera focus to capture a visual image with distance information and without image-based processing. The process starts at step 70 with camera initialization and at step 72 the ISP is ready to process visual images. At step 74 a time-of-flight sensor is initialized to detect distances to objects in the camera field of view. In one embodiment, the time-of-flight sensor may monitor the camera field of view to prepare the camera for focus before the camera receives a picture command from an end user. The monitoring may vary based upon context so that more active monitoring is performed when camera context indicates an increased probability of a picture being taken, such as an orientation of the housing detected by an accelerometer. At step 76 the object distance is check periodically to detect changes in the object distance. When an object distance is detected, the process continues to step 78 for the ISP to reference the focal point table from the nonvolatile memory and determine a focus actuator position. At step 80, the ISP checks the lens thermal shift table in the focal point table to determine an adjustment for the thermal conditions measure in the camera. At step 82, the ISP instructs the focus actuator, such as a VCM, to move the lens to a predefined actuator location based upon the distance and the thermal conditions. At step 84 the lens actuator, such as a VCM, moves the actuator to the determined actuator position so that the camera is ready to take the picture when the end user commands without performing image-based analysis. At step 86 optional fine tuning may be performed to improve the quality of the visual image. For instance, at sensed distances of less than 60 CM contrast detection autofocus may be employed to sharpen the visual image. Once focus is set, if the distance changes, the process returns to step 76 to adjust focus for the new distance.

[0022] Referring now to FIG. 5, a flow diagram depicts a process for managing focus of a camera by camera actuator position associated with a distance sensed by a distance sensor, such as a time-of-flight sensor. The process starts at step 100 with validation of range information detected by the distance sensor to an object in the camera field of view. The validation may involve a comparison of sensed distance information against predefined indications of a valid object in the field of view, such as a size of the object, a location of the distance scan, movement of the object similar to a user presence indication, or other context. In some instances, the validation may involve a comparison of distance information with visual image information, such as correspondence of the distance scan location and an object shown in the visual image or a change of contrast in the visual image from an initial capture and an image capture at a focus based upon the distance. At step 102 a command is received from an end user to capture the visual image with the camera. At step 104 the focal point of the camera is set based upon the distance in the focal point table and its associated actuator position. At step 104 an image may be completed without image based analysis, however, additional rapid steps may be taken to help confirm the camera focus. At step 106, an estimate of the depth of field range position for the commanded actuator position is considered versus the actual actuator and/or lens position associated with complete focus. At step 107, the focus is confirmed with the image sensor based logic by incrementing focus towards an in-focus actuator position, as is illustrated below.

[0023] Referring now to FIG. 6, a camera depth of field is depicted to reference for adjustment of a camera lens actuator position with image based analysis. An object in focus with a lens has a range of a depth of field in which slight inaccuracies of focus do not have a meaningful impact on the quality of an image captured by the camera. The depth of focus range is wider as distance to the object increases so that image-based analysis is more likely to have a meaningful increase in the quality of the image captured at shorter distances, such as 60 cm or less. FIG. 6 shows three objects at different distances and the size of the depth of focus range for each object. At greater distances such as d3c, an infinite camera focus captures visual images with an acceptable quality from d3a to d3b. The ideal focus distance for each object, shown as d1c, d2c and d3c, has a smaller range with shorter distances so that accurate focus is more important. To accomplish rapid focus as described above and achieve an accurate image, the initial actuator position is set within the depth of field but on the near side of in focus, such as at d1a when the distance for focus indicates d1c. An image is captured and the actuator is incremented in the direction of an expected better focus so that an image captured at the incremented actuator position confirms improved focus. In various embodiments, other types of incrementing may be performed around the expected in focus position to confirm the distance-selected actuator position before a visual image is captured.

[0024] Although the present invention has been described in detail, it should be understood that various changes, substitutions and alterations can be made hereto without departing from the spirit and scope of the invention as defined by the appended claims.