US 12,619,126 B2Grant

Dual-aperture zoom digital camera with automatic adjustable tele field of view

Issue Date:2026-05-05

•7 Claims

•13 Drawing Sheets

Abstract

Disclosed herein are digital cameras including an upright Wide camera configured to provide a Wide image with a Wide field of view (FOV_W) and a folded Tele camera configured to provide a Tele image with a Tele field of view (FOV_T), wherein the Tele camera includes a rotating OPFE operative to provide a folded optical path between an object or scene and a Tele sensor, wherein rotation of the OPFE moves FOV_Trelative to FOV_W.

Metadata

Assignee

Corephotonics Ltd.

Inventors

Gal Shabtay
Ephraim Goldenberg
Eran Kali
Noy Cohen
Gil Avraham
Ruthy Katz

Application Information

Application Number:US 18/335,192

Filing Date:2023-06-15

Priority Date:2015-12-29

Art Unit:2878

Classifications

IPC:

G03B3/06G02B13/06G02B26/10G03B17/17H04N3/08H04N23/45H04N23/55H04N23/58H04N23/63H04N23/67H04N23/69H04N23/90H04N23/951

Field of Search:

G03B 3/06G03B 17/17G03B 2217/002G02B 26/105G02B 13/06H04N 3/08H04N 23/45H04N 23/55H04N 23/58H04N 23/63H04N 23/631H04N 23/69H04N 23/90H04N 23/951H04N 23/675

Patent Drawings (13 sheets)

Description

Cross Reference to Related Applications

[0001] This application is a continuation of US patent application 17,882,640 filed Aug. 8, 2022, which was a continuation of US patent application 17,846,203 filed Jun. 22, 2022 (now U.S. Pat. No. 11,599,007), which was a continuation of U.S. patent application Ser. No. 17/145,309 filed Jan. 9, 2021 (now U.S. Pat. No. 11,392,009), which was a continuation of U.S. patent application Ser. No. 16/699,577 filed Nov. 30, 2019 (now U.S. patent Ser. No. 10/935,870), which was a continuation of U.S. patent application Ser. No. 15/525,059 filed May 6, 2017 (now U.S. patent Ser. No. 10/578,948), which was a 371 application from international patent application PCT/IB2016/057366 filed Dec. 5, 2016 and claims priority from U.S. Provisional Patent Applications No. 62/272,367 filed on Dec. 29, 2015 and 62/361,150 filed on Jul. 12, 2016, both of which are expressly incorporated herein by reference in their entirety.

Field

[0002] Embodiments disclosed herein relate in general to digital cameras and in particular to thin zoom digital cameras.

Background

[0003] Host devices or “personal computing and/or communication devices” (such as smartphones) having two back cameras (also referred to as “dual-camera” or “dual-aperture camera”) are known, see e.g. U.S. Pat. No. 9,185,291. The two back cameras have two image sensors (or simply “sensors”) operated simultaneously to capture an image, and have lenses with different focal lengths. Even though each lens/sensor combination is aligned to look in the same direction, each will capture an image of the same scene but with two different fields of view (FOV).

[0004] Dual-aperture zoom cameras in which one camera has a “Wide” FOV (FOV_W) and the other has a narrow or “Tele” FOV (FOV_T) are also known, see e.g. U.S. Pat. No. 9,185,291. The cameras are referred to respectively as Wide and Tele cameras that include respective Wide and Tele sensors. These sensors provide respectively separate Wide and Tele images. The Wide image captures FOV_Wand has lower spatial resolution than the Tele image that captures FOV_T. As used herein, “FOV” is defined by the tangent of the angle between a line crossing the lens and parallel to the lens optical axes and a line between the lens and any object that is captured on the respective image corner. The images may be merged (fused) together to form a composite image. In the composite image, the central portion is formed by combining the relatively higher spatial resolution image taken by the lens/sensor combination with the longer focal length, and the peripheral portion is formed by a peripheral portion of the relatively lower spatial resolution image taken by the lens/sensor combination with the shorter focal length. The user selects a desired amount of zoom and the composite image is used to interpolate values from the chosen amount of zoom to provide a respective zoom image. Hereinafter, the use of “resolution” in this description refers to image spatial resolution, which is indicative to the resolving power of a camera as determined by the lens focal length, its aperture diameter and the sensor pixel size.

[0005] Dual-aperture cameras in which one image (normally the Tele image) is obtained through a folded optical path are known, see e.g. co-invented and co-owned U.S. patent application Ser. No. 14/455,906, which teaches zoom digital cameras comprising an “upright” (with a direct optical axis to an object or scene) Wide camera and a “folded” Tele camera, see also FIG. 2B below. The folded camera has an optical axis substantially perpendicular (orthogonal) to an optical axis of the upright camera. The folded Tele camera may be auto-focused and optically stabilized by moving either its lens or by tilting an optical path folding (reflecting) element (e.g. a prism or minor and referred to also as “OPFE”) inserted in an optical path between its lens and a respective sensor. For simplicity, the optical path folding element is referred to hereinafter in this description generically as “prism”, with the understanding that the term may refer to any other optical path folding (reflecting) element that can perform the function of folding an optical path as described herein.

[0006] For example, PCT patent application PCT/IB2016/056060 titled “Dual-aperture zoom digital camera user interface” discloses a user interface for operating a dual-aperture digital camera included in host device, the dual-aperture digital camera including a Wide camera and a Tele camera, the user interface comprising a screen configured to display at least one icon and an image of a scene acquired with at least one of the Tele and Wide cameras, a frame defining FOV_Tsuperposed on a Wide image defined by FOV_W, and means to switch the screen from displaying the Wide image to displaying the Tele image. The user interface further comprises means to switch the screen from displaying the Tele image to displaying the Wide image. The user interface may further comprise means to acquire the Tele image, means to store and display the acquired Tele image, means to acquire simultaneously the Wide image and the Tele image, means to store and display separately the Wide and Tele images, a focus indicator for the Tele image and a focus indicator for the Wide image.

[0007] Object recognition is known and describes the task of finding and identifying objects in an image or video sequence. Many approaches have been implemented for accomplishing this task in computer vision systems. Such approaches may rely on appearance based methods by using example images under varying conditions and large model-bases, and/or on feature based methods comprising of a search to find feasible matches between object features and image features, e.g., by using surface patches, corners and edges detection and matching. Recognized objects can be tracked in preview or video feeds using an algorithm for analyzing sequential frames and outputting the movement of targets between the frames.

[0008] The problem of motion-based object tracking can be divided into two parts:

- [0009] (1) detecting moving objects in each frame. This can be done either by incorporating an object recognition algorithm for recognizing and tracking specific objects (e.g., human face) or, for example, by detecting any moving object in a scene. The latter may incorporate a background subtraction algorithm based on Gaussian mixture models with Morphological operations applied to the resulting foreground mask to eliminate noise. Blob analysis can later detect groups of connected pixels, which are likely to correspond to moving objects.
- [0010] (2) associating the detections corresponding to the same object over time, e.g., using motion estimation filters such as the Kalman filter.

Summary

[0011] In exemplary embodiments, there are provided digital cameras comprising an upright Wide camera configured to provide a Wide image with a Wide image resolution, the Wide camera comprising a Wide image sensor and a Wide lens with a Wide field of view (FOV_W); a folded Tele camera configured to provide a Tele image with a Tele image resolution higher than the Wide image resolution, the Tele camera comprising a Tele image sensor and a Tele lens with a Tele field of view (FOV_T); and a rotating OPFE (e.g. prism) operative to provide a folded optical path between an object or scene and the Tele sensor, wherein rotation of the prism moves FOV_Trelative to FOV_W.

[0012] In an embodiment, the Wide and Tele image sensors have a substantially rectangular shape defined by a respective height dimension and a respective width dimension and are in orthogonal planes and with their respective height dimensions orthogonal to each other such that FOV_Tis rotated at 90 degrees to FOV_W.

[0013] In an embodiment, the movement of FOV_Trelative to FOV_Wis performed in a scanning mode that provides a plurality of partially-overlapping or adjacent non-overlapping Tele FOVs within FOV_W.

[0014] In an embodiment, the prism rotation has a range of up to ±15 degrees around a zero prism position in which FOV_Tis centric to FOV_W.

[0015] In an embodiment, the digital camera may be included in a host device having a user interface for operating the digital camera, the user interface comprising a screen configured to display at least one icon and an image of the object or scene acquired with at least one of the Tele and Wide cameras and to display a frame defining FOV_Twithin FOV_W. The host device may have a user interface for operating the digital camera, the user interface comprising a screen configured to display at least one icon and an image of the object or scene acquired with at least one of the Tele and Wide cameras and to display a frame defining FOV_Twithin FOV_W. In an embodiment, the user interface may further comprise means for moving FOV_Trelative to FOV_W. In an embodiment, the user interface may further comprise means for scanning FOV_Tacross FOV_W. In an embodiment, the user interface may further comprise means for switching the screen from displaying the Wide image to displaying the Tele image. In an embodiment, the user interface may further comprise means to acquire the Tele image. In an embodiment, the user interface may further comprise means to acquire simultaneously the Wide image and the Tele image.

[0016] In an embodiment, the user interface may further comprise means to automatically (autonomously) move the FOV_Trelative to FOV_Wto track object of interest. In such cases, the camera may also be referred to as an “autonomous” camera.

[0017] In an embodiment, the user interface may further comprise means to acquire video streams of the Wide and Tele camera simultaneously.

[0018] In an embodiment, Tele images representing are plurality of adjacent non-overlapping Tele FOVs are stitched together to form a stitched Tele image used in the fusion with the Wide image.

[0019] In an embodiment, at least one Tele image includes a plurality of consecutive Tele images stitched together to form a stitched Tele image used in a fusion procedure with the Wide image to provide a composite image.

[0020] In an embodiment, the Wide and Tele images or video streams can be fused (or combined or stitched) on the device or in a cloud environment (referred to simply as “cloud”).

[0021] In an embodiment, the digital camera is further configured to form a composite video stream in which each frame is based on either a processed Wide image or a processed Tele image, the processed Wide and Tele images acquired during the autonomous FOV_Ttracking.

Brief Description of the Drawings

[0022] Non-limiting examples of embodiments disclosed herein are described below with reference to figures attached hereto that are listed following this paragraph. Identical structures, elements or parts that appear in more than one figure are generally labeled with a same numeral in all the figures in which they appear. The drawings and descriptions are meant to illuminate and clarify embodiments disclosed herein, and should not be considered limiting in any way.

[0023] FIG. 1A shows schematically an image reflecting the Wide FOV of a scene;

[0024] FIG. 1B shows schematically an image reflecting the Tele FOV of the scene in FIG. 1A;

[0025] FIG. 2A shows schematically a dual-aperture camera comprising a first upright camera and a second folded camera, with a prism folding an optical path to the folded camera in a zero position;

[0026] FIG. 2B shows a composite image obtained with a dual-aperture camera as in FIG. 2A, with the Tele FOV resulting from the zero position of the prism;

[0027] FIG. 3 shows schematically an arrangement enabling tilt of a prism in a folded optical path;

[0028] FIG. 4A shows schematically the dual-aperture camera of FIG. 2A with the prism rotated to a first position;

[0029] FIG. 4B shows a composite image obtained with a dual-aperture camera as in FIG. 4A, with the Tele FOV resulting from the first prism position;

[0030] FIG. 5A shows schematically the dual-aperture camera of FIG. 2A with the prism rotated to a second position;

[0031] FIG. 5B shows a composite image obtained with a dual-aperture camera as in FIG. 5A, with the Tele FOV resulting from the second prism position;

[0032] FIG. 6A shows schematically a dual-aperture camera comprising a first upright camera with a sensor rotated by 90 degrees relative to that in the camera of FIG. 2A and with a prism folding an optical path to a folded camera in a zero position;

[0033] FIG. 6B shows a composite image obtained with a dual-aperture camera as in FIG. 6A, with the Tele FOV resulting from the zero position of the prism;

[0034] FIG. 7A shows schematically the dual-aperture camera of FIG. 6A with the prism rotated to a first position;

[0035] FIG. 7B shows a composite image obtained with a dual-aperture camera as in FIG. 7A, with the Tele FOV resulting from the first prism position;

[0036] FIG. 8A shows schematically the dual-aperture camera of FIG. 6A with the prism rotated to a second position;

[0037] FIG. 8B shows a composite image obtained with a dual-aperture camera as in FIG. 8A, with the Tele FOV resulting from the second prism position;

[0038] FIG. 9A shows a smartphone and user interface incorporating a dual-aperture camera with Tele FOV positioning capability disclosed herein in a zero, centered position on the smartphone screen;

[0039] FIG. 9B shows the smartphone and user interface of FIG. 9A with the Tele FOV moved to a down position within the screen;

[0040] FIG. 9C shows the smartphone and user interface of FIG. 9A with the Tele FOV moved to an up position within the screen;

[0041] FIG. 9D shows the smartphone and user interface of FIG. 9A with the Tele FOV rotated by 90 degrees relative to the Wide FOV and moved to a right position within the screen;

[0042] FIG. 9E shows the smartphone and user interface of FIG. 9A with the Tele FOV rotated by 90 degrees relative to the Wide FOV and moved to a left position within the screen;

[0043] FIG. 10 shows a smartphone and user interface as in FIG. 9D or 9E used in a scanning mode of the Tele FOV.

[0044] FIG. 11A shows a smartphone and user interface incorporating a dual-aperture camera with automatic Tele FOV tracking disclosed herein with a first Tele FOV position on the smartphone screen;

[0045] FIG. 11B shows the smartphone and user interface of FIG. 11A with a second Tele FOV position on the smartphone screen;

[0046] FIG. 11C shows the smartphone and user interface of FIG. 11A with a third Tele FOV position on the smartphone screen.

Detailed Description

[0047] FIG. 1A shows schematically an image (or frame of a video stream) 102 reflecting the Wide FOV of a scene 106. FIG. 1B shows schematically an image (or frame of a video stream) 104 reflecting the Tele FOV of scene 106. Only part of the scene of FIG. 1A is seen in FIG. 1B. The two images or video streams are obtained simultaneously with a dual-aperture camera having an upright Wide camera and a folded Tele camera of the type for example disclosed in U.S. patent application Ser. No. 14/455,906. The two cameras may, for example, be two back cameras included in a smartphone or in another personal communication device.

[0048] FIG. 2A shows schematically a dual-aperture camera 200 comprising a first upright camera 202 and a second folded camera 204. An XYZ coordinate system as shown is used in the description of this and all following “camera” drawings. For example, upright camera 202 is a Wide camera that includes a Wide lens assembly (or simply “lens”) 206 with an optical axis 207, and a Wide sensor 208. For example, folded camera 204 is a Tele camera that includes a Tele lens 210 with an optical axis 211, and a Tele sensor 214. An OPFE (e.g. prism) 212 may be part of the folded Tele camera or may be a separate element that folds an optical path parallel to axis 207 into an optical path parallel to axis 211 into the Tele lens. The Wide and Tele sensors lie in respective orthogonal planes, respectively the XZ plane and the XY plane. For example, the Wide and Tele sensors are substantially rectangular with respective height (H) and width (W) dimensions. Thus, Wide sensor 208 has a height H_Wand a width W_Wand Tele sensor 214 has a height H_Tand a width W_T. The H/W ratio in both sensors is typically (although not necessarily) 9/16 or ¾. In the folded camera, positioning of sensor 214 with its height in the Y direction is advantageous in that it allows a smaller host device (e.g. a smartphone) thickness. In camera 200 in FIG. 2A (as well as cameras 200 in FIGS. 3, 4A and 5A), the width dimensions of sensors 208 and 214 are parallel to each other and to the X-axis.

[0049] The operation of a camera such as camera 200 is described in more detail in U.S. patent application Ser. No. 14/455,906. In particular, the prism can perform a tilt (rotation) movement 216 around the X-axis as shown. The rotation motion may be caused by a stepping motor 302 shown schematically in FIG. 3. The prism rotation angle may range between −15 degrees and +15 degrees around a “zero” position (see below), corresponding to ±30 degrees FOV_Tshift. Cameras 202 and 204 are used for example to take respectively the Wide and Tele images of FIGS. 1A and 1B.

[0050] FIG. 2B shows an image 230 identical with Wide image 102 with a frame 232 that indicates the position of the Tele image FOV. The camera can have high resolution in this framed FOV either by fusing the Wide and Tele images or by capturing and saving the Tele image. For reference purposes, the position of the prism that causes the Tele FOV and resulting image to be centric to the Wide FOV and Wide image is called here for example and in a non-limiting way a “zero” position of the prism.

[0051] The rotation of prism 212 around the X-axis moves the Tele FOV relative to the Wide FOV, causing other portions of scene 106 to become a “Tele image” with higher resolution. Thus, FIG. 4A shows prism 212 rotated counter-clockwise (as indicated by a curved arrow 402 when viewed in the —X direction) around the X-axis from its zero position to a new, first position. The counter-clockwise rotation causes the Tele FOV, indicated by frame 232, to move to a new, “down” position relative to the Wide FOV indicated by frame 230. FIG. 5A shows prism 212 rotated clockwise (as indicated by a curved arrow 502 when viewed in the —X direction) around the X-axis from its zero position to a new, second position. The clockwise rotation causes the Tele FOV, indicated by frame 232, to move to another new, “up” position relative to the Wide FOV indicated by frame 230. While FIGS. 4A, 4B, 5A and 5B show two discrete prism rotation movements and two discrete Tele FOV positions relative to the Wide FOV, there is clearly an entire range of practically continuous positions that the Tele FOV may occupy, depending on the degree and direction of prism rotation.

[0052] As mentioned, in FIGS. 2A, 3, 4A and 5A, both Wide sensor 208 and Tele sensor 214 are positioned with their longer side (“width”) in the X direction. Their shorter side (“height”) is in the Z direction for sensor 208 and in the Y direction for sensor 214. FIG. 6A shows schematically a dual-aperture camera 200′ comprising an upright camera 206 with a Wide sensor 208′ rotated by 90 degrees relative to sensor 208 in camera 206 of FIG. 2A. As also shown in FIGS. 7A and 8A, Wide sensor 208′ now has its height in the X direction and its width in the Z direction. Note that W_Wis now also orthogonal to the H_T. This 90 degree in-plane rotation of the Wide sensor provides certain advantages in terms of the positioning and movement of FOV_Trelative to FOV_Wand consequently in terms of the capture, processing and display of Tele and Wide images. One major advantage is that a larger percentage of FOV_Wcan have high resolution by setting a prism rotation angle. FIG. 6B shows schematically the position of FOV_T232′ (now rotated by 90 degrees relative to FOV_W230) in a centered position caused by a zero position of prism 212, after the camera itself was rotated by 90 degrees (not shown) compared to the camera of FIG. 2B.

[0053] FIG. 7A shows schematically the dual-aperture camera of FIG. 6A with prism 212 rotated counter-clockwise (as indicated by a curved arrow 702 when viewed in the —X direction) around the X-axis from its zero position to a new position. As shown in FIG. 7B, this prism rotation causes Tele FOV 232′ to move to a “right” position relative to the Wide FOV. FIG. 8A shows prism 212 rotated clockwise (as indicated by a curved arrow 802 when viewed in the −X direction) around the X-axis from its zero position to a second position, As shown in FIG. 8B, this prism rotation causes Tele FOV 232′ to move to a “left” position relative to the Wide FOV. While FIGS. 7A, 7B, 8A and 8B show two discrete prism rotation movements and two discrete Tele FOV positions relative to the Wide FOV, there is clearly an entire range of practically continuous positions that the Tele FOV may occupy, depending on the degree and direction of prism rotation.

[0054] Note that a similar FOV_Trelative positioning effect to that described above may be obtained by in-plane rotating Tele sensor 214 by 90 degrees and by leaving Wide sensor 208 unchanged from its original position in FIG. 2A. However, the positioning of sensor 214 with W_Tin the Y direction may disadvantageously increase a camera height (and therefore a host device thickness).

[0055] When a dual-aperture camera described above is included for example in a smartphone, the Tele image (i.e. the part of scene 106 viewed and acquired by the Tele camera) is bound by a frame 932 visible on the smartphone screen. FIG. 9A shows an exemplary smartphone 900 that includes on a back side (not shown) a Wide camera and a Tele camera as in FIG. 2A or 6A. The Wide and Tele cameras have known fields of view with a known ratio M=FOV_T/FOV_Wbetween them. In general, M may have any value between ¼ and ¾. For example, M may have values of ½, 9/16 or ¾. Consequently, frame 932 includes almost exactly the image seen by FOV_Tand has a size that is a fraction M of the entire screen (which includes the image seen by FOV_W). Note that for camera configuration as the one shown in FIG. 6A and for Wide and Tele image sensors with 4:3 aspect ratio, selecting M=¾ will result in FOV_Tthat will overlap the short dimension of FOV_Win its entirety and will enable complete scanning of the Wide FOV by tilting the prism. The same argument is applicable for image sensors having 16:9 aspect ratio with M= 9/16.

[0056] In still mode, scene 106 is acquired by both cameras, with the Wide camera providing the entire image seen (i.e. the Wide image) and the Tele camera providing the part of scene 106 bound by frame 932. Smartphone 900 further includes, on a front side opposite to the back side, a screen or display 902 displaying a view of scene 106. Screen 902 may display icons or text “A”, “B”, “C”, etc., that provide indications and/or are selectable to perform various operations of the phone and/or the cameras. Such icons or text may be indicative of flash setting, video or stills selection, back or front camera selection, etc. The square boxes surrounding “A”, “B” and “C” are merely illustrative and may have different shape or be removed altogether in some cases. Note that the fact that only three icons are shown is not meant to be limiting, and that more or fewer icons may be displayed and/or selectable at any time during or prior to image acquisition by the cameras and/or during display of acquired images. In an embodiment of the dual-aperture camera as in FIG. 2A, the “zero” position of the prism provides the composite image seen in FIG. 9A, where frame 932 is centered on the screen.

[0057] In various embodiments and as described in more detail in PCT/IB2016/056060, smartphone 900 may have a user interface that includes a single camera icon (or “button”) 908 and a “two-camera” button 910. The two-camera button may appear on the screen when the FOV of the scene (FOV_scene) is greater or equal to FOV_T. As described in detail in PCT/IB2016/056060, the user interface displays visually the almost exact FOV_Tand enables simple acquisition of the image within FOV_T, thereby providing a Tele image with the highest resolution enabled by the Tele camera. The user interface also enables simultaneous acquisition (with a single input through the user interface, i.e. using two-camera button 910) of a Wide image and a Tele image. Image fusion of the Wide and Tele images or video streams can take place on the capturing device or in a cloud environment.

[0058] The present inventors have determined that, advantageously, a user interface as described above can be used to “drag” frame 932 (and FOV_T) on screen 902 to bring different parts of the scene into FOV_T. That is, the dragging of frame 932 is translated into rotation of the “folded” path prism, such that the higher resolution Tele image “moves” to different parts of the scene. The dragging may be performed by a firm touch of the screen by a finger 920 and movement of the finger across the screen. In FIG. 9A, the finger touches the screen in the general area of “zero” positioned (centered) frame 932. The finger may drag the frame (and FOV_T) to a “down” position as in FIG. 9B, to an “up” position as in FIG. 9C, or to any intermediate position (not shown) between the down and up positions. The touch and drag actions are relayed to a camera controller (not shown), which, in turn, controls the prism movement. The dragging of frame 932 to the down position in FIG. 9B, indicated schematically by an arrow 934, is equivalent to rotation of the prism to its position in FIG. 4A. The dragging of frame 932 to the up position in FIG. 9C is equivalent to rotation of the prism to its position in FIG. 5A. Similarly, with a dual-aperture camera with a 90 degree rotated Wide sensor as in FIGS. 6A-8A, a frame 932′ (now also rotated by 90 degrees relative to frame 932 in FIGS. 9A-9C) may be dragged from a zero position to a “right” position as in FIG. 9D or a “left” position as in FIG. 9E. The dragging of frame 932′ to the right position in FIG. 9D is equivalent to rotation of the prism to its position in FIG. 7A. The dragging of frame 932′ to the up position in FIG. 9E is equivalent to rotation of the prism to its position in FIG. 8A.

[0059] As described in detail in PCT/IB2016/056060, in terms of image acquisition, a user may press two-camera button 910 to simultaneously acquire two images, the Wide image of scene 106 at its respective (lower) image resolution and the Tele image of region (frame) 932 (or 932′) at its respective (higher) image resolution. The two images may be stored in an on-board storage (such as “camera roll” in an iPhone) and may be displayed or downloaded for further use as known in the art. The user may press single camera button 908 to acquire only the Wide image, which can further be stored, displayed and downloaded for further use. The user may choose for display on screen 902 only the Tele image by, for example, double-tapping or pressing at any point on the screen within frame 932 (or 932′). This action leads to display of the Tele image on the entire screen. The Tele image (only) can then be chosen for acquisition by pressing on single camera button 908. The acquired Tele image can then be stored, displayed and downloaded for further use as above. The two images can also be fused (on the camera hosting device or in a cloud) into a composite image with a portion marked by a frame 932 or 932′ formed by the higher-resolution Tele image, and with a peripheral portion formed by a peripheral portion of the relatively lower resolution Wide image.

[0060] Clearly, frame 932′ (and the Tele FOV) may be dragged to any intermediate position (not shown) between the right and left positions. In other words, the Tele FOV may be moved laterally on the screen to a number of partially overlapping or non-overlapping (but touching) positions, from a right-most position (at a right screen edge 940) to a left-most position (at a left screen edge 942) or vice-versa, and an entire image of the scene may be “stitched” together from the partially overlapping or non-overlapping Tele images. For example, as shown in FIG. 10, the ratio M of the Tele and Wide FOVs may be such that screen 902 includes, and is substantially covered by, four adjacent frames 932′a-d. Note that the screen may include fewer adjacent frames than the number required to substantially fill in the entire screen. For example, in an embodiment, there may be only two or three adjacent frames (instead of the four shown). An optional “Scan” icon or button 1002 may then be used to scan (i.e. move the FOV_Tframe) automatically from a right-most position on the screen (i.e. from frame 932′a) to a left-most position (i.e. to frame 932′d) or vice-versa. Alternatively, a “Scan” command may be given by voice control. The scan function, similarly to the dragging action, provides partially overlapping or non-overlapping Tele images that can be stitched together into a high-resolution Tele image of the entire scene. A scan command, whether through icon 1002 or vocally, may also lead to acquisition of each Tele image defined by a frame 932′ and to its storage for further processing. A Wide image can be captured during the scan function, allowing further processing that may include image fusion of the Wide image with the various Tele images (or the stitched Tele images) to form a high resolution image with a FOV that is larger than FOV_T.

[0061] Note that the direction of prism rotation and the consequent movement of FOV_Trelative to FOV_Won a smartphone (or any other host device) screen depends on the geometry of the assembly of the dual-aperture camera in the host device. The description above relates to one particular such geometry. In a different geometry, the prism rotation directions and the resulting FOV_Tmovement may be in opposite directions to those described above.

[0062] The devices, used interface and associated methods disclosed above may be used for automatic movement or “automatic adjustment” of the Tele FOV for e.g. tracking a subject in an autonomous manner. We refer to a camera mode that performs automatic Tele FOV movement to track an object or subject of interest as “autonomous Tele FOV tracking”. The autonomous Tele FOV movement is in response to recognition (through e.g. the smartphone camera) of the object or subject of interest, and the Tele image focuses on and displays the object or subject of interest. The object recognition may be performed using any of the methods known in the art. An example of autonomous Tele FOV tracking is shown in FIGS. 11A-11C.

[0063] FIG. 11A shows a smartphone and user interface incorporating a dual-aperture camera with automatic Tele FOV tracking disclosed herein with a first Tele FOV position on the smartphone screen. FIG. 11B shows the smartphone and user interface of FIG. 11A with a second Tele FOV position on the smartphone screen. FIG. 11C shows the smartphone and user interface of FIG. 11A with a third Tele FOV position on the smartphone screen. In each of these figures, the object of interest is a runner 1102. The decision to track the runner is taken either by the user (e.g., by touching the runner's image) or automatically (e.g., using face detection). It is assumed that the Tele camera can change its FOV by tilting the prism to track the object of interest. Ideally, the camera will track the object such that it is as close as possible to the center of the adjustable Tele FOV as seen in 1032a, 1032b and 1032c.

[0064] While the smartphone shown in FIGS. 11A-11C displays icons 908, 910 and 1002, one or more other icons (not shown) may replace and/or be used in addition to or instead of icons 908, 910 and 1002 during operation in the Tele FOV tracking mode.

[0065] Wide and Tele images and/or video streams can be recorded during the automatic tracking mode and fused together to form a composite image or composite video stream. This fusion can be applied on the camera hosting device or alternatively, Wide and Tele images or video streams can be uploaded to the cloud for applying this fusion operation. Each composite image may also be formed with FOV_Wby scanning with the Tele camera, stitching a plurality of Tele images to provide a “stitched” Tele image, then fusing the stitched Tele image with the Wide image. This is advantageous in that the Wide image captures the entire scene simultaneously, while the Tele images to be stitched together are consecutive, so one can overcome motion or occlusions in the scene if required. The stitching of the Tele images and/or the fusion of the stitched Tele image with the Wide image may also be performed in a cloud.

[0066] While this disclosure has been described in terms of certain embodiments and generally associated methods, alterations and permutations of the embodiments and methods will be apparent to those skilled in the art. The disclosure is to be understood as not limited by the specific embodiments described herein, but only by the scope of the appended claims.

Claims

What is claimed is:

1. A mobile device, comprising:

a screen on a front side of the mobile device;

an upright Wide camera having a first field-of-view (FOV1), a first resolution, and a first image sensor that lies in a plane parallel to the screen; and

a folded Tele camera having a second field-of-view (FOV2), a second resolution, a Tele lens module, an optical folding element (OPFE), and a second image sensor,

wherein FOV2<FOV1,

wherein the OPFE is a prism having a reflective surface, the prism configured to receive light from a first direction and redirect the light to a second direction different from the first direction,

wherein the reflective surface of the OPFE has an angle α with the screen that fulfills 30≤α≤60 degrees, and

wherein the upright Wide camera and the folded Tele camera are disposed on a back side of the mobile device, the back side being opposite to the front side.

2. The mobile device of claim 1, wherein the folded Tele camera is an auto-focus camera.

3. The mobile device of claim 1, wherein the first resolution is different from the second resolution.

4. The mobile device of claim 1, wherein the mobile device is configured to display a first icon and a second icon on the screen, wherein the mobile device is configured to display a first scene captured by the upright Wide camera when the first icon is selected, and wherein the mobile device is configured to display a second scene captured by the folded Tele camera when the second icon is selected.

5. The mobile device of claim 4, wherein the first and second icons are displayed along a same axis on one side of the screen.

6. The mobile device of claim 5, wherein a ratio FOV2/FOV1 is between ¼ and ¾.

7. The mobile device of claim 6, wherein the mobile device is a smartphone.

Patent Citations (446)

Patent	Date	Inventor	Cited By
US3085354(A)	1963-04-01	Rasmussen et al.	Applicant
US3584513(A)	1971-06-01	Gates	Applicant
US3941001(A)	1976-03-01	LaSarge	Applicant
US4199785(A)	1980-04-01	McCullough et al.	Applicant
US4792822(A)	1988-12-01	Akiyama et al.	Applicant
US5005083(A)	1991-04-01	Grage et al.	Applicant
US5032917(A)	1991-07-01	Aschwanden	Applicant
US5041852(A)	1991-08-01	Misawa et al.	Applicant
US5051830(A)	1991-09-01	von Hoessle	Applicant
US5099263(A)	1992-03-01	Matsumoto et al.	Applicant
US5248971(A)	1993-09-01	Mandl	Applicant
US5287093(A)	1994-02-01	Amano et al.	Applicant
US5331465(A)	1994-07-01	Miyano	Applicant
US5394520(A)	1995-02-01	Hall	Applicant
US5436660(A)	1995-07-01	Sakamoto	Applicant
US5444478(A)	1995-08-01	Lelong et al.	Applicant
US5459520(A)	1995-10-01	Sasaki	Applicant
US5502537(A)	1996-03-01	Utagawa	Applicant
US5657402(A)	1997-08-01	Bender et al.	Applicant
US5682198(A)	1997-10-01	Katayama et al.	Applicant
US5768443(A)	1998-06-01	Michael et al.	Applicant
US5892855(A)	1999-04-01	Kakinami et al.	Applicant
US5926190(A)	1999-07-01	Turkowski et al.	Applicant
US5940641(A)	1999-08-01	McIntyre et al.	Applicant
US5982951(A)	1999-11-01	Katayama et al.	Applicant
US6101334(A)	2000-08-01	Fantone	Applicant
US6128416(A)	2000-10-01	Oura	Applicant
US6148120(A)	2000-11-01	Sussman	Applicant
US6201533(B1)	2001-03-01	Rosenberg et al.	Applicant
US6208765(B1)	2001-03-01	Bergen	Applicant
US6211668(B1)	2001-04-01	Duesler et al.	Applicant
US6215299(B1)	2001-04-01	Reynolds et al.	Applicant
US6222359(B1)	2001-04-01	Duesler et al.	Applicant
US6268611(B1)	2001-07-01	Pettersson et al.	Applicant
US6320610(B1)	2001-11-01	Van Sant	Examiner
US6549215(B2)	2003-04-01	Jouppi	Applicant
US6611289(B1)	2003-08-01	Yu et al.	Applicant
US6643416(B1)	2003-11-01	Daniels et al.	Applicant
US6650368(B1)	2003-11-01	Doron	Applicant
US6680748(B1)	2004-01-01	Monti	Applicant
US6714665(B1)	2004-03-01	Hanna et al.	Applicant
US6724421(B1)	2004-04-01	Glatt	Applicant
US6738073(B2)	2004-05-01	Park et al.	Applicant
US6741250(B1)	2004-05-01	Furlan et al.	Applicant
US6750903(B1)	2004-06-01	Miyatake et al.	Applicant
US6778207(B1)	2004-08-01	Lee et al.	Applicant
US7002583(B2)	2006-02-01	Rabb, III	Applicant
US7015954(B1)	2006-03-01	Foote et al.	Applicant
US7038716(B2)	2006-05-01	Klein et al.	Applicant
US7199348(B2)	2007-04-01	Olsen et al.	Applicant
US7206136(B2)	2007-04-01	Labaziewicz et al.	Applicant
US7248294(B2)	2007-07-01	Slatter	Applicant
US7256944(B2)	2007-08-01	Labaziewicz et al.	Applicant
US7305180(B2)	2007-12-01	Labaziewicz et al.	Applicant
US7339621(B2)	2008-03-01	Fortier	Applicant
US7346217(B1)	2008-03-01	Gold, Jr.	Applicant
US7365793(B2)	2008-04-01	Cheatle et al.	Applicant
US7411610(B2)	2008-08-01	Doyle	Applicant
US7424218(B2)	2008-09-01	Baudisch et al.	Applicant
US7509041(B2)	2009-03-01	Hosono	Applicant
US7533819(B2)	2009-05-01	Barkan et al.	Applicant
US7619683(B2)	2009-11-01	Davis	Applicant
US7738016(B2)	2010-06-01	Toyofuku	Applicant
US7773121(B1)	2010-08-01	Huntsberger et al.	Applicant
US7809256(B2)	2010-10-01	Kuroda et al.	Applicant
US7880776(B2)	2011-02-01	LeGall et al.	Applicant
US7918398(B2)	2011-04-01	Li et al.	Applicant
US7964835(B2)	2011-06-01	Olsen et al.	Applicant
US7978239(B2)	2011-07-01	Deever et al.	Applicant
US8115825(B2)	2012-02-01	Culbert et al.	Applicant
US8149327(B2)	2012-04-01	Lin et al.	Applicant
US8154610(B2)	2012-04-01	Jo et al.	Applicant
US8238695(B1)	2012-08-01	Davey et al.	Applicant
US8274552(B2)	2012-09-01	Dahi et al.	Applicant
US8390729(B2)	2013-03-01	Long et al.	Applicant
US8391697(B2)	2013-03-01	Cho et al.	Applicant
US8400555(B1)	2013-03-01	Georgiev et al.	Applicant
US8439265(B2)	2013-05-01	Ferren et al.	Applicant
US8446484(B2)	2013-05-01	Muukki et al.	Applicant
US8483452(B2)	2013-07-01	Ueda et al.	Applicant
US8514491(B2)	2013-08-01	Duparre	Applicant
US8547389(B2)	2013-10-01	Hoppe et al.	Applicant
US8553106(B2)	2013-10-01	Scarff	Applicant
US8587691(B2)	2013-11-01	Takane	Applicant
US8619148(B1)	2013-12-01	Watts et al.	Applicant
US8752969(B1)	2014-06-01	Kane et al.	Applicant
US8803990(B2)	2014-08-01	Smith	Applicant
US8896655(B2)	2014-11-01	Mauchly et al.	Applicant
US8976255(B2)	2015-03-01	Matsuoto et al.	Applicant
US9019387(B2)	2015-04-01	Nakano	Applicant
US9025073(B2)	2015-05-01	Attar et al.	Applicant
US9025077(B2)	2015-05-01	Attar et al.	Applicant
US9041835(B2)	2015-05-01	Honda	Applicant
US9137447(B2)	2015-09-01	Shibuno	Applicant
US9185291(B1)	2015-11-01	Shabtay et al.	Applicant
US9215377(B2)	2015-12-01	Sokeila et al.	Applicant
US9215385(B2)	2015-12-01	Luo	Applicant
US9270875(B2)	2016-02-01	Brisedoux et al.	Applicant
US9286680(B1)	2016-03-01	Jiang et al.	Applicant
US9344626(B2)	2016-05-01	Silverstein et al.	Applicant
US9360671(B1)	2016-06-01	Zhou	Applicant
US9369621(B2)	2016-06-01	Malone et al.	Applicant
US9413930(B2)	2016-08-01	Geerds	Applicant
US9413984(B2)	2016-08-01	Attar et al.	Applicant
US9420180(B2)	2016-08-01	Jin	Applicant
US9438792(B2)	2016-09-01	Nakada et al.	Applicant
US9485432(B1)	2016-11-01	Medasani et al.	Applicant
US9578257(B2)	2017-02-01	Attar et al.	Applicant
US9618748(B2)	2017-04-01	Munger et al.	Applicant
US9681057(B2)	2017-06-01	Attar et al.	Applicant
US9723220(B2)	2017-08-01	Sugie	Applicant
US9736365(B2)	2017-08-01	Laroia	Applicant
US9736391(B2)	2017-08-01	Du et al.	Applicant
US9768310(B2)	2017-09-01	Ahn et al.	Applicant
US9800798(B2)	2017-10-01	Ravirala et al.	Applicant
US9851803(B2)	2017-12-01	Fisher et al.	Applicant
US9894287(B2)	2018-02-01	Qian et al.	Applicant
US9900522(B2)	2018-02-01	Lu	Applicant
US9927600(B2)	2018-03-01	Goldenberg et al.	Applicant
US2002/0005902(A1)	2002-01-01	Yuen	Applicant
US2002/0030163(A1)	2002-03-01	Zhang	Applicant
US2002/0054214(A1)	2002-05-01	Yoshikawa	Applicant
US2002/0063711(A1)	2002-05-01	Park et al.	Applicant
US2002/0075258(A1)	2002-06-01	Park et al.	Applicant
US2002/0122113(A1)	2002-09-01	Foote	Applicant
US2002/0167741(A1)	2002-11-01	Koiwai et al.	Applicant
US2003/0030729(A1)	2003-02-01	Prentice et al.	Applicant
US2003/0093805(A1)	2003-05-01	Gin	Applicant
US2003/0156751(A1)	2003-08-01	Lee et al.	Applicant
US2003/0160886(A1)	2003-08-01	Misawa et al.	Applicant
US2003/0202113(A1)	2003-10-01	Yoshikawa	Applicant
US2004/0008773(A1)	2004-01-01	Itokawa	Applicant
US2004/0012683(A1)	2004-01-01	Yamasaki et al.	Applicant
US2004/0017386(A1)	2004-01-01	Liu et al.	Applicant
US2004/0027367(A1)	2004-02-01	Pilu	Applicant
US2004/0061788(A1)	2004-04-01	Bateman	Applicant
US2004/0141065(A1)	2004-07-01	Hara et al.	Applicant
US2004/0141086(A1)	2004-07-01	Mihara	Applicant
US2004/0239313(A1)	2004-12-01	Godkin	Applicant
US2004/0240052(A1)	2004-12-01	Minefuji et al.	Applicant
US2005/0013509(A1)	2005-01-01	Samadani	Applicant
US2005/0046740(A1)	2005-03-01	Davis	Applicant
US2005/0134697(A1)	2005-06-01	Mikkonen et al.	Applicant
US2005/0141390(A1)	2005-06-01	Lee et al.	Applicant
US2005/0157184(A1)	2005-07-01	Nakanishi et al.	Applicant
US2005/0168834(A1)	2005-08-01	Matsumoto et al.	Applicant
US2005/0185049(A1)	2005-08-01	Iwai et al.	Applicant
US2005/0200718(A1)	2005-09-01	Lee	Applicant
US2005/0248667(A1)	2005-11-01	Schweng et al.	Applicant
US2006/0054782(A1)	2006-03-01	Olsen et al.	Applicant
US2006/0056056(A1)	2006-03-01	Ahiska et al.	Applicant
US2006/0067672(A1)	2006-03-01	Washisu et al.	Applicant
US2006/0102907(A1)	2006-05-01	Lee et al.	Applicant
US2006/0125937(A1)	2006-06-01	LeGall et al.	Applicant
US2006/0126737(A1)	2006-06-01	Boice et al.	Applicant
US2006/0170793(A1)	2006-08-01	Pasquarette et al.	Applicant
US2006/0175549(A1)	2006-08-01	Miller et al.	Applicant
US2006/0181619(A1)	2006-08-01	Liow et al.	Applicant
US2006/0187310(A1)	2006-08-01	Janson et al.	Applicant
US2006/0187322(A1)	2006-08-01	Janson et al.	Applicant
US2006/0187338(A1)	2006-08-01	May et al.	Applicant
US2006/0227236(A1)	2006-10-01	Pak	Applicant
US2007/0024737(A1)	2007-02-01	Nakamura et al.	Applicant
US2007/0126911(A1)	2007-06-01	Nanjo	Applicant
US2007/0127040(A1)	2007-06-01	Davidovici	Applicant
US2007/0159344(A1)	2007-07-01	Kisacanin	Applicant
US2007/0177025(A1)	2007-08-01	Kopet et al.	Applicant
US2007/0188653(A1)	2007-08-01	Pollock et al.	Applicant
US2007/0189386(A1)	2007-08-01	Imagawa et al.	Applicant
US2007/0257184(A1)	2007-11-01	Olsen et al.	Applicant
US2007/0285550(A1)	2007-12-01	Son	Applicant
US2008/0017557(A1)	2008-01-01	Witdouck	Applicant
US2008/0024614(A1)	2008-01-01	Li et al.	Applicant
US2008/0025634(A1)	2008-01-01	Border et al.	Applicant
US2008/0030592(A1)	2008-02-01	Border et al.	Applicant
US2008/0030611(A1)	2008-02-01	Jenkins	Applicant
US2008/0084484(A1)	2008-04-01	Ochi et al.	Applicant
US2008/0088942(A1)	2008-04-01	Seo	Applicant
US2008/0106629(A1)	2008-05-01	Kurtz et al.	Applicant
US2008/0117316(A1)	2008-05-01	Orimoto	Applicant
US2008/0129831(A1)	2008-06-01	Cho et al.	Applicant
US2008/0218611(A1)	2008-09-01	Parulski et al.	Applicant
US2008/0218612(A1)	2008-09-01	Border et al.	Applicant
US2008/0218613(A1)	2008-09-01	Janson et al.	Applicant
US2008/0219654(A1)	2008-09-01	Border et al.	Applicant
US2009/0086074(A1)	2009-04-01	Li et al.	Applicant
US2009/0102948(A1)	2009-04-01	Scherling	Applicant
US2009/0109556(A1)	2009-04-01	Shimizu et al.	Applicant
US2009/0122195(A1)	2009-05-01	Van Baar et al.	Applicant
US2009/0122406(A1)	2009-05-01	Rouvinen et al.	Applicant
US2009/0128644(A1)	2009-05-01	Camp et al.	Applicant
US2009/0168135(A1)	2009-07-01	Yu et al.	Applicant
US2009/0200451(A1)	2009-08-01	Conners	Applicant
US2009/0219547(A1)	2009-09-01	Kauhanen et al.	Applicant
US2009/0234542(A1)	2009-09-01	Orlewski	Applicant
US2009/0252484(A1)	2009-10-01	Hasuda et al.	Applicant
US2009/0295949(A1)	2009-12-01	Ojala	Applicant
US2009/0324135(A1)	2009-12-01	Kondo et al.	Applicant
US2010/0013906(A1)	2010-01-01	Border et al.	Applicant
US2010/0020221(A1)	2010-01-01	Tupman et al.	Applicant
US2010/0060746(A9)	2010-03-01	Olsen et al.	Applicant
US2010/0097444(A1)	2010-04-01	Lablans	Applicant
US2010/0103194(A1)	2010-04-01	Chen et al.	Applicant
US2010/0134621(A1)	2010-06-01	Namkoong et al.	Applicant
US2010/0165131(A1)	2010-07-01	Makimoto et al.	Applicant
US2010/0196001(A1)	2010-08-01	Ryynänen et al.	Applicant
US2010/0202068(A1)	2010-08-01	Ito	Applicant
US2010/0238327(A1)	2010-09-01	Griffith et al.	Applicant
US2010/0246024(A1)	2010-09-01	Aoki et al.	Applicant
US2010/0259836(A1)	2010-10-01	Kang et al.	Applicant
US2010/0265331(A1)	2010-10-01	Tanaka	Applicant
US2010/0283842(A1)	2010-11-01	Guissin et al.	Applicant
US2010/0321494(A1)	2010-12-01	Peterson et al.	Applicant
US2011/0058320(A1)	2011-03-01	Kim et al.	Applicant
US2011/0063417(A1)	2011-03-01	Peters et al.	Applicant
US2011/0063446(A1)	2011-03-01	McMordie et al.	Applicant
US2011/0064327(A1)	2011-03-01	Dagher et al.	Applicant
US2011/0080487(A1)	2011-04-01	Venkataraman et al.	Applicant
US2011/0121666(A1)	2011-05-01	Park et al.	Applicant
US2011/0128288(A1)	2011-06-01	Petrou et al.	Applicant
US2011/0164172(A1)	2011-07-01	Shintani et al.	Applicant
US2011/0221599(A1)	2011-09-01	Högasten	Applicant
US2011/0229054(A1)	2011-09-01	Weston et al.	Applicant
US2011/0234798(A1)	2011-09-01	Chou	Applicant
US2011/0234853(A1)	2011-09-01	Hayashi et al.	Applicant
US2011/0234881(A1)	2011-09-01	Wakabayashi et al.	Applicant
US2011/0242286(A1)	2011-10-01	Pace et al.	Applicant
US2011/0242355(A1)	2011-10-01	Goma et al.	Applicant
US2011/0285714(A1)	2011-11-01	Swic et al.	Applicant
US2011/0298966(A1)	2011-12-01	Kirschstein et al.	Applicant
US2012/0014682(A1)	2012-01-01	David et al.	Applicant
US2012/0026366(A1)	2012-02-01	Golan et al.	Applicant
US2012/0044372(A1)	2012-02-01	Cote et al.	Applicant
US2012/0062780(A1)	2012-03-01	Morihisa	Applicant
US2012/0069235(A1)	2012-03-01	Imai	Applicant
US2012/0075489(A1)	2012-03-01	Nishihara	Applicant
US2012/0105579(A1)	2012-05-01	Jeon et al.	Applicant
US2012/0124525(A1)	2012-05-01	Kang	Applicant
US2012/0154547(A1)	2012-06-01	Aizawa	Applicant
US2012/0154614(A1)	2012-06-01	Moriya et al.	Applicant
US2012/0196648(A1)	2012-08-01	Havens et al.	Applicant
US2012/0229663(A1)	2012-09-01	Nelson et al.	Applicant
US2012/0249815(A1)	2012-10-01	Bohn et al.	Applicant
US2012/0287315(A1)	2012-11-01	Huang et al.	Applicant
US2012/0320467(A1)	2012-12-01	Baik et al.	Applicant
US2013/0002928(A1)	2013-01-01	Imai	Applicant
US2013/0016268(A1)	2013-01-01	Nakata	Examiner
US2013/0016427(A1)	2013-01-01	Sugawara	Applicant
US2013/0063629(A1)	2013-03-01	Webster et al.	Applicant
US2013/0076922(A1)	2013-03-01	Shihoh et al.	Applicant
US2013/0093842(A1)	2013-04-01	Yahata	Applicant
US2013/0094126(A1)	2013-04-01	Rappoport et al.	Applicant
US2013/0113894(A1)	2013-05-01	Mirlay	Applicant
US2013/0135445(A1)	2013-05-01	Dahi et al.	Applicant
US2013/0148215(A1)	2013-06-01	Mori et al.	Applicant
US2013/0148854(A1)	2013-06-01	Wang et al.	Applicant
US2013/0155176(A1)	2013-06-01	Paripally et al.	Applicant
US2013/0163085(A1)	2013-06-01	Lim et al.	Applicant
US2013/0182150(A1)	2013-07-01	Asakura	Applicant
US2013/0201360(A1)	2013-08-01	Song	Applicant
US2013/0202273(A1)	2013-08-01	Ouedraogo et al.	Applicant
US2013/0229544(A1)	2013-09-01	Bando	Applicant
US2013/0235224(A1)	2013-09-01	Park et al.	Applicant
US2013/0250150(A1)	2013-09-01	Malone et al.	Applicant
US2013/0258044(A1)	2013-10-01	Betts-LaCroix	Applicant
US2013/0258048(A1)	2013-10-01	Wang et al.	Applicant
US2013/0270419(A1)	2013-10-01	Singh et al.	Applicant
US2013/0278785(A1)	2013-10-01	Nomura et al.	Applicant
US2013/0286221(A1)	2013-10-01	Shechtman et al.	Applicant
US2013/0321668(A1)	2013-12-01	Kamath	Applicant
US2014/0009631(A1)	2014-01-01	Topliss	Applicant
US2014/0049615(A1)	2014-02-01	Uwagawa	Applicant
US2014/0118584(A1)	2014-05-01	Lee et al.	Applicant
US2014/0160311(A1)	2014-06-01	Hwang et al.	Applicant
US2014/0192224(A1)	2014-07-01	Laroia	Applicant
US2014/0192238(A1)	2014-07-01	Attar et al.	Applicant
US2014/0192253(A1)	2014-07-01	Laroia	Applicant
US2014/0218587(A1)	2014-08-01	Shah	Applicant
US2014/0313316(A1)	2014-10-01	Olsson et al.	Applicant
US2014/0362242(A1)	2014-12-01	Takizawa	Applicant
US2014/0376090(A1)	2014-12-01	Terajima	Applicant
US2014/0379103(A1)	2014-12-01	Ishikawa et al.	Applicant
US2015/0002683(A1)	2015-01-01	Hu et al.	Applicant
US2015/0002684(A1)	2015-01-01	Kuchiki	Applicant
US2015/0042870(A1)	2015-02-01	Chan et al.	Applicant
US2015/0070781(A1)	2015-03-01	Cheng et al.	Applicant
US2015/0085096(A1)	2015-03-01	Smits	Examiner
US2015/0092066(A1)	2015-04-01	Geiss et al.	Applicant
US2015/0103147(A1)	2015-04-01	Ho et al.	Applicant
US2015/0110345(A1)	2015-04-01	Weichselbaum	Applicant
US2015/0124059(A1)	2015-05-01	Georgiev et al.	Applicant
US2015/0138381(A1)	2015-05-01	Ahn	Applicant
US2015/0145965(A1)	2015-05-01	Livyatan et al.	Applicant
US2015/0154776(A1)	2015-06-01	Zhang et al.	Applicant
US2015/0162048(A1)	2015-06-01	Hirata et al.	Applicant
US2015/0195458(A1)	2015-07-01	Nakayama et al.	Applicant
US2015/0198464(A1)	2015-07-01	El Alami	Applicant
US2015/0215516(A1)	2015-07-01	Dolgin	Applicant
US2015/0237280(A1)	2015-08-01	Choi et al.	Applicant
US2015/0242994(A1)	2015-08-01	Shen	Applicant
US2015/0244906(A1)	2015-08-01	Wu et al.	Applicant
US2015/0253543(A1)	2015-09-01	Mercado	Applicant
US2015/0253647(A1)	2015-09-01	Mercado	Applicant
US2015/0261299(A1)	2015-09-01	Wajs	Applicant
US2015/0271471(A1)	2015-09-01	Hsieh et al.	Applicant
US2015/0281678(A1)	2015-10-01	Park et al.	Applicant
US2015/0286033(A1)	2015-10-01	Osborne	Applicant
US2015/0296112(A1)	2015-10-01	Park et al.	Applicant
US2015/0316744(A1)	2015-11-01	Chen	Applicant
US2015/0334309(A1)	2015-11-01	Peng et al.	Applicant
US2016/0044250(A1)	2016-02-01	Shabtay et al.	Applicant
US2016/0070088(A1)	2016-03-01	Koguchi	Applicant
US2016/0127652(A1)	2016-05-01	Park	Examiner
US2016/0154066(A1)	2016-06-01	Hioka et al.	Applicant
US2016/0154202(A1)	2016-06-01	Wippermann et al.	Applicant
US2016/0154204(A1)	2016-06-01	Lim et al.	Applicant
US2016/0212358(A1)	2016-07-01	Shikata	Applicant
US2016/0212418(A1)	2016-07-01	Demirdjian et al.	Applicant
US2016/0238834(A1)	2016-08-01	Erlich et al.	Applicant
US2016/0241751(A1)	2016-08-01	Park	Applicant
US2016/0291295(A1)	2016-10-01	Shabtay et al.	Applicant
US2016/0295112(A1)	2016-10-01	Georgiev et al.	Applicant
US2016/0301840(A1)	2016-10-01	Du et al.	Applicant
US2016/0301868(A1)	2016-10-01	Acharya et al.	Applicant
US2016/0342095(A1)	2016-11-01	Bieling et al.	Applicant
US2016/0353008(A1)	2016-12-01	Osborne	Applicant
US2016/0353012(A1)	2016-12-01	Kao et al.	Applicant
US2016/0381289(A1)	2016-12-01	Kim et al.	Applicant
US2017/0001577(A1)	2017-01-01	Seagraves et al.	Applicant
US2017/0019616(A1)	2017-01-01	Zhu et al.	Applicant
US2017/0070731(A1)	2017-03-01	Darling et al.	Applicant
US2017/0085764(A1)	2017-03-01	Kim	Examiner
US2017/0094187(A1)	2017-03-01	Sharma et al.	Applicant
US2017/0124987(A1)	2017-05-01	Kim et al.	Applicant
US2017/0150061(A1)	2017-05-01	Shabtay et al.	Applicant
US2017/0187962(A1)	2017-06-01	Lee et al.	Applicant
US2017/0214846(A1)	2017-07-01	Du et al.	Applicant
US2017/0214866(A1)	2017-07-01	Zhu et al.	Applicant
US2017/0219749(A1)	2017-08-01	Hou et al.	Applicant
US2017/0242225(A1)	2017-08-01	Fiske	Applicant
US2017/0276954(A1)	2017-09-01	Bajorins et al.	Applicant
US2017/0289458(A1)	2017-10-01	Song et al.	Applicant
US2018/0013944(A1)	2018-01-01	Evans, V et al.	Applicant
US2018/0017844(A1)	2018-01-01	Yu et al.	Applicant
US2018/0024329(A1)	2018-01-01	Goldenberg et al.	Applicant
US2018/0059379(A1)	2018-03-01	Chou	Applicant
US2018/0081149(A1)	2018-03-01	Bae	Examiner
US2018/0109660(A1)	2018-04-01	Yoon et al.	Applicant
US2018/0109710(A1)	2018-04-01	Lee et al.	Applicant
US2018/0120674(A1)	2018-05-01	Avivi et al.	Applicant
US2018/0150973(A1)	2018-05-01	Tang et al.	Applicant
US2018/0176426(A1)	2018-06-01	Wei et al.	Applicant
US2018/0184010(A1)	2018-06-01	Cohen et al.	Applicant
US2018/0198897(A1)	2018-07-01	Tang et al.	Applicant
US2018/0241922(A1)	2018-08-01	Baldwin et al.	Applicant
US2018/0295292(A1)	2018-10-01	Lee et al.	Applicant
US2018/0300901(A1)	2018-10-01	Wakai et al.	Applicant
US2018/0329281(A1)	2018-11-01	Ye	Applicant
US2018/0368656(A1)	2018-12-01	Austin et al.	Applicant
US2019/0100156(A1)	2019-04-01	Chung et al.	Applicant
US2019/0121103(A1)	2019-04-01	Bachar et al.	Applicant
US2019/0121216(A1)	2019-04-01	Shabtay et al.	Applicant
US2019/0130822(A1)	2019-05-01	Jung et al.	Applicant
US2019/0213712(A1)	2019-07-01	Lashdan et al.	Applicant
US2019/0215440(A1)	2019-07-01	Rivard et al.	Applicant
US2019/0222758(A1)	2019-07-01	Goldenberg et al.	Applicant
US2019/0228562(A1)	2019-07-01	Song	Applicant
US2019/0297238(A1)	2019-09-01	Klosterman	Applicant
US2020/0103726(A1)	2020-04-01	Shabtay et al.	Applicant
US2020/0104034(A1)	2020-04-01	Lee et al.	Applicant
US2020/0134848(A1)	2020-04-01	El-Khamy et al.	Applicant
US2020/0221026(A1)	2020-07-01	Fridman et al.	Applicant
US2020/0264403(A1)	2020-08-01	Bachar et al.	Applicant
US2020/0389580(A1)	2020-12-01	Kodama et al.	Applicant
US2021/0180989(A1)	2021-06-01	Fukumura et al.	Applicant
US2021/0286193(A1)	2021-09-01	Kwon	Examiner
US2021/0333521(A9)	2021-10-01	Yedid et al.	Applicant
US2022/0252963(A1)	2022-08-01	Shabtay et al.	Applicant
CN101276415(A)	2008-10-01		Applicant
CN201514511(U)	2010-06-01		Applicant
CN102215373(A)	2011-10-01		Applicant
CN102739949(A)	2012-10-01		Applicant
CN102982518(A)	2013-03-01		Applicant
CN103024272(A)	2013-04-01		Applicant
CN203406908(U)	2014-01-01		Applicant
CN103841404(A)	2014-06-01		Applicant
CN205301703(U)	2016-06-01		Applicant
CN105827903(A)	2016-08-01		Applicant
CN105847662(A)	2016-08-01		Applicant
CN107608052(A)	2018-01-01		Applicant
CN107682489(A)	2018-02-01		Applicant
CN109729266(A)	2019-05-01		Applicant
EP1536633(A1)	2005-06-01		Applicant
EP1780567(A1)	2007-05-01		Applicant
EP2523450(A1)	2012-11-01		Applicant
JPS59191146(A)	1984-10-01		Applicant
JP4211230(A)	1992-08-01		Applicant
JPH07318864(A)	1995-12-01		Applicant
JP8271976(A)	1996-10-01		Applicant
JP2002010276(A)	2002-01-01		Applicant
JP2003298920(A)	2003-10-01		Applicant
JP2003304024(A)	2003-10-01		Applicant
JP2004056779(A)	2004-02-01		Applicant
JP2004133054(A)	2004-04-01		Applicant
JP2004245982(A)	2004-09-01		Applicant
JP2005099265(A)	2005-04-01		Applicant
JP2005122084(A)	2005-05-01		Applicant
JP2005321592(A)	2005-11-01		Applicant
JP2006237914(A)	2006-09-01		Applicant
JP2006238325(A)	2006-09-01		Applicant
JP2007228006(A)	2007-09-01		Applicant
JP2007306282(A)	2007-11-01		Applicant
JP2008076485(A)	2008-04-01		Applicant
JP2008271026(A)	2008-11-01		Applicant
JP2010204341(A)	2010-09-01		Applicant
JP2011055246(A)	2011-03-01		Applicant
JP2011085666(A)	2011-04-01		Applicant
JP2011138407(A)	2011-07-01		Applicant
JP2011203283(A)	2011-10-01		Applicant
JP2012132739(A)	2012-07-01		Applicant
JP2013101213(A)	2013-05-01		Applicant
JP2013106289(A)	2013-05-01		Applicant
JP2016105577(A)	2016-06-01		Applicant
JP2017146440(A)	2017-08-01		Applicant
KR20070005946(A)	2007-01-01		Applicant
KR20090058229(A)	2009-06-01		Applicant
KR20100008936(A)	2010-01-01		Applicant
KR20110080590(A)	2011-07-01		Applicant
KR20130104764(A)	2013-09-01		Applicant
KR1020130135805(A)	2013-11-01		Applicant
KR20140014787(A)	2014-02-01		Applicant
KR101428042(B1)	2014-08-01		Applicant
KR101477178(B1)	2014-12-01		Applicant
KR20140144126(A)	2014-12-01		Applicant
KR20150118012(A)	2015-10-01		Applicant
KR20170105236(A)	2017-09-01		Applicant
KR20180120894(A)	2018-11-01		Applicant
KR20130085116(A)	2019-06-01		Applicant
WO2000027131(A2)	2000-05-01		Applicant
WO2004084542(A1)	2004-09-01		Applicant
WO2006008805(A1)	2006-01-01		Applicant
WO2010122841(A1)	2010-10-01		Applicant
WO2014072818(A2)	2014-05-01		Applicant
WO2017025822(A1)	2017-02-01		Applicant
WO2017037688(A1)	2017-03-01		Applicant
WO2018130898(A1)	2018-07-01		Applicant

Non-Patent Literature (16)

Statistical Modeling and Performance Characterization of a Real-Time Dual Camera Surveillance System, Greienhagen et al., Publisher: IEEE, 2000, 8 pages.Applicant
A 3MPixel Multi-Aperture Image Sensor with 0.7 μm Pixels in 0.11 μm CMOS, Fife et al., Stanford University, 2008, 3 pages.Applicant
Dual camera intelligent sensor for high definition 360 degrees surveillance, Scotti et al., Publisher: IET, May 9, 2000, 8 pages.Applicant
Dual-sensor foveated imaging system, Hua et al., Publisher: Optical Society of America, Jan. 14, 2008, 11 pages.Applicant
Defocus Video Matting, McGuire et al., Publisher: ACM SIGGRAPH, Jul. 31, 2005, 11 pages.Applicant
Compact multi-aperture imaging with high angular resolution, Santacana et al., Publisher: Optical Society of America, 2015, 10 pages.Applicant
Multi-Aperture Photography, Green et al., Publisher: Mitsubishi Electric Research Laboratories, Inc., Jul. 2007, 10 pages.Applicant
Multispectral Bilateral Video Fusion, Bennett et al., Publisher: IEEE, May 2007, 10 pages.Applicant
Super-resolution imaging using a camera array, Santacana et al., Publisher: Optical Society of America, 2014, 6 pages.Applicant
Optical Splitting Trees for High-Precision Monocular Imaging, McGuire et al., Publisher: IEEE, 2007, 11 pages.Applicant
High Performance Imaging Using Large Camera Arrays, Wilburn et al., Publisher: Association for Computing Machinery, Inc., 2005, 12 pages.Applicant
Real-time Edge-Aware Image Processing with the Bilateral Grid, Chen et al., Publisher: ACM SIGGRAPH, 2007, 9 pages.Applicant
Superimposed multi-resolution imaging, Carles et al., Publisher: Optical Society of America, 2017, 13 pages.Applicant
Viewfinder Alignment, Adams et al., Publisher: Eurographics, 2008, 10 pages.Applicant
Dual-Camera System for Multi-Level Activity Recognition, Bodor et al., Publisher: IEEE, Oct. 2014, 6 pages.Applicant
Engineered to the task: Why camera-phone cameras are different, Giles Humpston, Publisher: Solid State Technology, Jun. 2009, 3 pages.Applicant