Wireless hand-held digital camera

[0002] The invention relates to digital cameras and more specifically to a digital camera system that provides remote storage and image enhancement combined with an in-camera display device.

[0003] A digital camera captures an image of a particular subject using an imaging system that includes an optical lens and a photosensor array, such as a charged coupled device (CCD). The imaging system “captures” a set, or frame, of digital image data that represents a particular subject at the time that the frame is captured. Each frame of digital image data can be stored in a memory for subsequent viewing on a display, printing onto paper, or downloading to a computer system. Because of the inherent space limitations of a hand-held digital camera, memory capacity for storing frames of digital image data within the camera is limited.

[0004] In addition to having storage capability, hand-held digital cameras are able to process digital image data to enhance the resolution and overall quality of an image that is presented to a viewer. In order to perform increasingly complex image enhancements, additional processing capability is required on-board a hand-held digital camera, but because of the inherent space limitations of hand-held digital cameras, processing capability is limited.

[0005] Hand-held digital cameras are available in which a still image of a subject can be viewed from a display device located within the hand-held digital camera. In hand-held digital cameras with image display capability, the problems of memory and processing limitations are magnified.

[0006] One technique used to overcome the memory limitations of hand-held digital cameras is disclosed in U.S. Pat. No. 5,534,921, entitled “CCD Digital Camera System With Selectable Memories,” issued to Sawanobori. In the Sawanobori patent, a detachable memory card is inserted into a hand-held digital camera and digital image data is transferred to the memory card. When the storage capacity of the memory card is reached, the memory card can be removed from the camera and replaced with a new memory card. A concern with the Sawanobori technique is that the memory cards themselves have limited storage capacity. A second concern is that the memory cards must be manually replaced when memory capacity is reached, creating a delay and a distraction to the operator of a hand-held digital camera.

[0007] Another technique that is employed to address the memory limitations of hand-held digital cameras is disclosed in U.S. Pat. No. 5,402,170 entitled “Hand-Manipulated Electronic Camera Tethered To A Personal Computer,” issued to Parulski et al. (hereinafter '170). In the '170 patent, an electronic camera is linked to a computer by a cable and an interface protocol is utilized to allow the camera to transmit image data over the cable and into a memory device of the computer. Disadvantages of the technique are that the hand-held digital camera must be “tethered” to the computer by a cable in order to accomplish data storage and that the technique only allows data transfer in one direction, from the camera to the computer, eliminating the possibility of viewing previously stored images on the display device of the hand-held digital camera.

[0008] A technique for overcoming the memory limitations involving the wireless transmission of electronic image data from an electronic hand-held camera is disclosed in U.S. Pat. No. 5,666,159, entitled “Electronic Camera System With Programable Transmission Capability,” issued to Parulski (hereinafter '159). In the '159 patent, an electronic camera is detachably coupled to a portable computer that has a display module and a radio-frequency transmitter module. The system is able to transmit electronic image data to a selected base unit using standard wireless communications techniques. While the technique is intended to be used to send facsimile images of a subject to a fax machine, the technique can be adapted to allow corresponding facsimile image data to be stored in a properly equipped computer. A disadvantage of the technique is that image data is transferred in only one direction, from the camera/computer combination to a base unit.

[0009] Another example of a wireless transmission technique used to transmit electronic image data is disclosed in U.S. Pat. No. 5,475,441, entitled “Electronic Camera With Memory Card Interface To A Computer,” issued to Parulski et al (hereinafter '441). In the '441 patent, electronic image data is transmitted from an electronic hand-held camera with no display capability to a computer via a wireless transmission link. The computer that receives the data is relied upon to perform image processing, image data storage, and image display. While the '441 patent provides a high speed interface for transferring image data from a hand-held camera to a computer, a disadvantage of the '441 patent is that the hand-held camera does not have image display capability and the interface does not allow image data that is stored on the computer to be transmitted back to the hand-held camera for viewing.

[0010] What is needed is a hand-held digital camera system that has a built-in image display device and high capacity image data storage. In addition, what is need is a hand-held digital camera system that can display high quality images for viewing immediately after an image is captured.

[0011] A method and system allow a hand-held digital camera to have continuous access to large volumes of digital image data utilizing a bi-directional wireless communications link between a host computer and the camera. In an embodiment of the invention, imaging optics and a photo-sensor array capture image data that represents an image of a subject. A transceiver integrated into the hand-held digital camera transmits the image data to a host computer via a wireless communications link. The host computer stores the image data, or a copy of the image data, and retransmits the image data back to the hand-held digital camera via the wireless communications link. Once the image data is received by the hand-held digital camera, an electronic image representative of the image data is displayed by a display device that is integrated into the camera. The communications link between the hand-held digital camera and the host computer can transfer data at a sufficiently high rate to provide virtually real-time feedback to a camera operator.

[0012] In a preferred embodiment of the invention, when the image data is received at the host computer, the image data is processed by an image processor within the host computer in order to enhance the quality of the image data that was initially captured by the hand-held digital camera. Enhancements in the image data may include applying mathematical algorithms to generate greater pixel density or adjusting color balance, contrast, and/or luminance. Once the improved image data is generated by the image processor of the host computer, the host computer transmits the improved image data to the hand-held digital camera. The hand-held digital camera receives the improved image data and forms an enhanced electronic image that is displayed on the display device of the camera. The bi-directional communications link and the processing speed of the host computer enable virtually real-time feedback to a camera operator. The improved data may consist of image correction data or it may contain an entire array of image data. In addition, the improved image data may be made responsive to user-defined criteria that may include morphing, distortion, changes in perspective, and other artistic enhancements.

[0013] In addition to displaying images that have just been captured by the hand-held digital camera, the camera can display previously captured images by accessing image data that is stored in the host computer. For example, the hand-held digital camera can access and display an image that was captured the previous day, since the data is stored in the host computer.

[0014] An advantage of the hand-held digital camera is that the camera can display just-captured images in nearly real-time, while having continuous and wireless access to a large image data storage capacity that allows viewing of previously captured images. The hand-held digital camera also provides the capability of comparing captured images to images in an image database and displaying certain stored images that meet user-specified similarity criteria. In addition, stored global positioning system (GPS) data concerning camera position and even orientation can enable the display of previously captured images having similar associated attributes.

[0015] Another advantage is that processing performance required to display high quality images is not limited by the processing capability of the hand-held digital camera, but by the processing capability of the host computer. The processing capability of the host computer allows the host computer to improve captured image data well beyond the practical capabilities of a hand-held digital camera. Another advantage of the hand-held digital camera is that the camera does not require removable memory cards which must be frequently exchanged and the camera is not “tethered” to a host computer by a data cable.

[0016] FIG. 1 is an illustration of a hand-held digital camera in wireless communication with an adapted host computer in accordance with the present invention.

[0017] FIG. 2 is a block diagram of the components of the hand-held digital camera and the adapted host computer of FIG. 1 in accordance with the present invention.

[0018] FIG. 3 is a flow diagram of a method for managing image information through a hand-held digital camera that includes storing image data in a host computer in accordance with the present invention.

[0019] FIG. 4 is a flow diagram of a method for managing image information through a hand-held digital camera that includes processing the image data to enhance image resolution in accordance with the present invention.

[0020] FIG. 1 is an illustration of a hand-held digital camera 40 that is linked to a portable computer 10 by a wireless communications link 20 which may include segments of wire, cable, optical fiber, or other physical connections to complete the wireless link. The components of the hand-held digital camera 40 and the host computer 10 are shown in FIG. 2. Integrated into a housing 42 of the hand-held digital camera are imaging optics 44 that may include a single lens or a lens array positioned to collect optical energy representative of a subject or scenery, and to focus the optical energy onto a photosensor array 48. In addition to a lens or lens system, color filters may be placed between the imaging optics and the photosensor array to achieve desired color manipulation.

[0021] The photosensor array 48 defines a matrix of photosensitive pixels. Each photosensitive pixel generates an electrical signal that is representative of the optical energy that is directed at the pixel by the imaging optics. The resolution of an image that can be produced from a photosensor array directly relates to the density of photosensitive pixels in the photosensor array. The electrical signals that are output by the photosensor array are characterized as image data or digital image data, wherein each image or “picture” that is captured is considered one “set” or “frame” of digital image data. In the preferred embodiment, the photosensor array is a charged coupled device (CCD).

[0022] A processor 56 that is in data transfer communication with the photosensor array 48 is also integrated into the hand-held digital camera 40. The processor performs data management functions within the hand-held digital camera and is also able to perform limited processing on raw digital image data in order to e quality of the raw digital image data. Processing of the image data may include performing data compression in order to decrease the quantity of data that is transmitted or stored and data decompression to display previously compressed data. This processing may include run length encoding and/or delta modulation.

[0023] Memory 52 that is in data transfer communication with the processor 56 is also integrated into the hand-held digital camera 40. In the preferred embodiment, the camera memory is electronic memory, such as flash memory, that can be used to store frames of digital image data. The camera memory can be used to buffer digital image data before the digital image data is displayed and/or processed. The memory may have a capacity of, for example, 2 megabytes, allowing storage of dozens of pictures. In alternative embodiments, the built-in memory of the hand-held digital camera may be augmented by removable memory, or the camera may have removable memory completely substituted for built-in memory. In other alternative embodiments, the memory may be other than electronic memory, for example magnetic memory.

[0024] A power source 60 is contained within the hand-held digital camera 40 to operate the electronic components of the camera. The power source may be conventional alkaline batteries, such as AA size batteries, or rechargeable nickel-cadmium batteries.

[0025] A control interface 64 is included in the hand-held digital camera 40 to allow an operator to control the functions of the camera. Interface functions of the hand-held digital camera may include an on/off switch, an image capture switch, a battery status readout, image display option controllers, and controllers for lens focus, exposure, and shutter speed. The control interface may also include selections for requesting desired image enhancements from the host computer.

[0026] A display device 68 is also integrated into the hand-held digital camera 40. The display device may be, for example, a liquid crystal display (LCD) or a thin-film transistor (TFT) display, and the resolution of the display device is related to the number of pixels in the fixed display area. The display device allows a camera operator to view an image that has been captured, so that the camera operator has real-time feedback on the imaging process. In addition to displaying a single image, the display device may be able to display more than one image at a time, allowing the operator to compare multiple images side-by-side. In an alternative embodiment, the display device need not be integrated into the camera housing 42 but may be external to the camera, such as viewing goggles or a remote display.

[0027] In an alternative embodiment of the hand-held digital camera 40, image data displayed on the display device 68 is dynamically dithered to allow the displayed image to appear to have higher resolution and quality than would otherwise be expected from the display device.

[0028] A camera transceiver 72 is also associated with the hand-held digital camera 40, preferably integrated into the hand-held digital camera housing 42, and is connected such that the transceiver can transfer data to and from the processor 56, the memory 52, and the display device 68 of the camera. The camera transceiver has a transmitter portion 74 and a receiver portion 76 for transferring data over a wireless communications link between the camera transceiver and a compatible transceiver. The camera transceiver may include an infrared transmission system or a radio-frequency (RF) transmission system. In an RF system, the full 1.25 megahertz (MHz) band-width of the IS-95 Mobile Radio Standard can be used. The IS-95 Mobile Radio Standard is a service using Code Division Multiple Access (CDMA) and Quadrature Phase Shift-Keying (QPSK)/Bipolar Phase Shift-Keying (BPSK) modulation on a carrier frequency of 824-994 MHz or 1.8-2.0 gigahertz (GHz). Emerging systems with much higher bandwidths on carriers upwards of 60 GHz and higher may also be used. In order to increase the bandwidth of the wireless transfer of image data between compatible transceivers, parallel code and/or parallel transmission channels may be used simultaneously. Alternatively, the camera transceiver can receive broadcast transmissions of information. For example, the host computer 10, or a network of host computers, may broadcast signals that designate the availability of the host computer. The hand-held digital camera 40 may include an annunciator device such as an indicator light showing when the host computer is within communications range.

[0029] The host computer 10 is preferably a conventional computer system that is adapted for use with a hand-held digital camera 40. The host computer may be a portable computer, a desktop computer, a super computer, or even a network of computers. As an alternative embodiment, the host computer may be a specially designed single-use computer. In any case, the host computer is remote from the hand-held digital camera in that the host computer is not physically connected to the camera.

[0030] An added component to the host computer 10 is a computer transceiver 14. The computer transceiver is compatible with the camera transceiver 72 and enables the transfer of electronic data between the hand-held digital camera 40 and the host computer over a communications link that is at least partially a wireless communications link. The computer transceiver may be internal or external to the host computer as long as the computer transceiver is in data-transfer communication with a memory 16 and an image processor 18 of the computer.

[0031] The memory 16 of the host computer 10 can be any conventional computer memory, including electronic memory and/or magnetic memory. Electronic memory may be in the form of RAM computer chips, and magnetic memory may be in the form of a magnetic disk drive. The computer memory can be used to store large quantities of image data in comparison to the storage capacity of the hand-held digital camera memory 52. Image data stored in the computer memory can be transferred to and from the computer transceiver 14 to allow rapid communication between the host computer and the hand-held digital camera 40. The memory can also store image data in a relational database format, enabling the database to be rapidly searched for related data which may include the original image data. For example, if the database is presented with an image of a horse, the database could be searched for other images of horses. In addition, images could be searched by other criteria, such asa GPS marker or an identified name.

[0032] The image processor 18 of the host computer 10 is a combination of computer hardware and software. The hardware may include a single or multiple processors and various other support circuits, while the software may include image enhancement software that operates to improve or custom tailor digital image data to create, for example, a higher resolution, better quality, or stylized image of a subject. The processor may also enable database searches for image data that is related to initially captured image data. The image processor is connected to the host computer such that digital image data can be transferred between the computer memory and the computer transceiver 14.

[0033] The image processor 18 may operate on image data by, for example, adjusting color balance, gamma and luminance, filtering device limiting fixed pattern noise, filtering noise using a Wiener filter, changing zoom factors, recropping, applying enhancement filters, applying smoothing filters, applying subject-dependent filters, and applying coordinate transformations.

[0034] Operation of a first embodiment of the invention is described with the reference to FIGS. 2 and 3. In the embodiment, the hand-held digital camera 40 is used to capture image data that represents an image of a subject. The image data is captured at step 100 by directing the camera at the subject of interest and activating an image capture button. The captured image data may be compressed by the processor 56 and is then transferred, either directly or indirectly, to the camera transceiver 72, where the image data is processed for wireless transmission to the host computer 10.

[0035] In step 102, the camera transceiver 72 transmits the image data to the host computer 10 via the at least partially wireless communications link 20 that is established between the camera transceiver and the computer transceiver 14 located within the host computer 10. The computer transceiver immediately retransmits or “echos” related image data back to the hand-held digital camera 40, where the related image data is received at step 104 by the camera via the communications link established with the camera transceiver. In response to the proper command, at step 106 the hand-held digital camera forms a related image from the related image data and displays the related image of the subject on the display device of the camera. Rapidly transmitting the related image data, via a wireless communications link, to a host computer memory enables the hand-held digital camera to store a large quantity of image data without requiring exchanges of removable memory diskettes and without having the camera physically connected to a supporting computer by a cable or a memory card.

[0036] In an alterative embodiment, the related image data that the host computer 10 sends to the hand-held digital camera 40 consists of nothing more than the original image data. In another embodiment, when the host computer initially receives the image data from the camera transceiver 72, the host computer forms a copy of the image data to store in the computer memory 16. In addition to displaying images on the camera display device, images captured by the camera may also be remotely displayed, for example, on a big screen display monitor. Remote display may allow many people to view captured images simultaneously.

[0037] Operation of a second embodiment of the invention is described with reference to FIGS. 2 and 4. In this embodiment, the hand-held digital camera 40 is used to capture image data that represents an image of a subject or scene. The image data is captured at step 120 by directing the camera at the subject or scene of interest and activating an image capture button. The captured image data is transferred, either directly or indirectly, to the camera transceiver 72 where the image data is processed for wireless transmission.

[0038] At step 122, the camera transceiver transmits the image data to the host computer 10 via the at least partially wireless communications link 20 that is established between the camera transceiver 72 and the computer transceiver 14 located within the host computer. The image processor 18 within the host computer receives the transmitted image data and at step 124 processes the image data into enhanced image data. The process of enhancing the image data is executed in continuity with the previous steps and may include performing software algorithms that operate to generate more detailed pixel information than was provided by the original image capture. Upon completion of the enhancement processing, a copy of the image data is stored in the memory 16 of the host computer. The computer transceiver 14 then transmits the enhanced image data to the hand-held digital camera, where the enhanced image data is received at step 126 by the camera via the communications link established with the camera transceiver 72. Once at the hand-held digital camera, and in response to the proper command, at step 128 an enhanced image is formed from the enhanced image data and the enhanced image is displayed on the display device of the camera.

[0039] Transmitting the raw image data to the host computer allows the host computer to perform more extensive image improvements on the image data in less time than could be practically accomplished by the internal processor of the hand-held digital camera. The improved image data allows the display device of the hand-held digital camera to display a higher resolution image. In addition to greater processing capability, the host computer can store large amounts of image data, allowing an operator to capture more images without being interrupted by memory card changes and without being burdened by the limitations of a physical data cable connected to the hand-held digital camera.

[0040] In addition to displaying images that have just been captured, the hand-held digital camera system can display previously captured images. For instance, the hand-held digital camera is able to display an image that was captured by the camera the previous day or the previous week by manipulating the control interface 64 of the camera to access the image data that is stored in the host computer 10. The hand-held digital camera and the host computer may also work together to display, on the camera display, prerecored images based on other captured data parameters such as GPS and/or camera orientation data or the date or time of image capture. The camera and computer combination can also operate together to allow a camera operator to browse a remote image database through the camera.

[0041] Although the hand-held digital camera system is described with reference to still pictures, the system can also be applied to video data. For example, the hand-held digital camera may have video capture capability, with the captured video data being transmitted to the host computer via the wireless communications link. The host computer can then store and/or process the video data in a similar fashion to the still image data.

[0042] In an alternative embodiment of the invention, a microphone is located within the hand-held digital camera. The microphone allows the recording of sound along with the capturing of images. Voice recording can be used to annotate images as the images are captured. Voice data that is recorded by the hand-held digital camera is managed in the same manner as the image data and can be considered equivalent to the image data, except that it is collected from a microphone and output from a speaker. That is, voice or other audio data is captured using the microphone and is transmitted to the host computer, with “improved” audio data that is echoed to the hand-held digital camera being used to provide the output from the speaker or similar transducer of the hand-held digital camera. In accordance with the invention, voice data can be transmitted to the host computer for storage and/or improvement along with the image data.