Memory-enhanced image sensor

[0001] The present invention relates to a memory enhanced image sensor, in particular to a digital image sensor with integrated memory functions.

[0002] A digital image sensor is a semiconductor device comprising an array of pixels, each of which comprises a photosensitive element (e.g. a photodiode) for converting incident light into a photocurrent. The generated photocurrent is gathered for an exposure time before being converted to a voltage, digitized and read out. Different architectures of digital image sensor are defined by the processes used in their manufacture. Two of the most common types of digital image sensor are the charge coupled device (CCD) and the Complementary Metal Oxide Semiconductor (CMOS) image sensor. CMOS image sensors are particularly applicable for applications which require low power operation at low cost.

[0003] A digital image sensor is formed as an integrated circuit (IC), and an imaging device typically comprises the digital image sensor IC together with optical components (e.g. a lens) and other device components.

[0004] A digital image sensor comprises a number of well known volatile memory components, i.e. memory components that only maintain the stored data for as long as a power supply is maintained In particular, for a CMOS sensor, each pixel comprises capacitors which store charge for subsequent readout. The readout operation is controlled with vertical and/or horizontal shift registers. Line memories are required for functions such as analog to digital conversion (ADC), digital filters, and defect correction algorithms. A frame store can be provided which stores image data for correlated double sampling or for schemes which combine images to extend dynamic range. This is not an exhaustive list of the types of volatile memory, or of the uses to which each is put.

[0005] For the permanent storage of data, non-volatile memory is used. Non-volatile memories are typically used in read-only memory (ROM), flash memory, magnetic computer storage devices such as hard disks, floppy disk drives, or in optical disc drives

[0006] Most non-volatile memory cells rely on a trapped charge stored on a floating gate in a field effect transistor (FET), and so the contents of the memory are erased if exposed to incident light. Because of this, non-volatile memory is not generally thought of as being suitable for use with an image sensor. If any extra memory storage is required for a given image sensing device, the non-volatile memory needs to be stored on a separate IC, remote from the image sensor IC. The use of a multiple chipset package increases the size and manufacturing costs of an image sensing device. U.S. Pat. No. 6,879,340 to Chevallier discloses a CMOS image sensor that incorporates a non-volatile memory, which is used as a frame store. In order to enable the use of a non-volatile memory, the disclosure of Chevallier provides a layer of opaque protective material which covers the non-volatile memory. This represents an extra processing step during sensor manufacture, and an increase in manufacturing complexity such as this increases the time of manufacture and decreases device yield and so from a cost and profitability perspective the avoidance of this step is desirable.

[0007] As described above, volatile memory is commonly used in image sensors for the performance of the sensor's image processing tasks U.S. Pat. No. 6,879,340 to Chevallier discloses an image sensor IC which comprises a non-volatile memory which is also used for the sensor's image processing tasks. There is a need for a non-volatile memory provided on an image sensor IC which can perform other tasks.

[0008] According to a first aspect of the present invention, there is provided an image sensor integrated circuit (IC) comprising an antifuse non-volatile memory. The IC further may comprise a volatile memory means or a volatile memory, for mapping, or a mapper, the contents of the non-volatile memory to the volatile memory means. The volatile memory means may comprise an I2C data bus space. The IC may further comprise a switcher for selectively powering the antifuse non-volatile memory. The switcher is arranged to power the antifuse non-volatile memory upon power-up of the IC and to switch off the antifuse non-volatile memory when its contents have been copied to the volatile memory.

[0009] Furthermore, the antifuse non-volatile memory is encoded with data for part-tracking. The data for part-tracking may comprise at least one of: sensor ID, date of manufacture, sales region ID, tester ID, test result, wafer ID, and die ID. The antifuse non-volatile memory may be encoded with data for defect correction of image data. The data for defect correction comprise the locations of defective pixels. The data for defect correction may comprise couplet locations. The antifuse non-volatile memory may be encoded with data for color filter array reconstruction. The data for color filter array reconstruction may comprise at least one of: data for anti-vignetting, color channel gains, range scaling parameters, demosaicing algorithms, range clipping parameters, and gamma correction data. The antifuse non-volatile memory may be encoded with data defining embedded firmware for microprocessor or DSP control.

[0010] According to a second aspect of the invention, there is provided an imaging apparatus comprising the IC of the first aspect, together with a co-processor associated therewith. The imaging apparatus may further comprise an auxiliary device connected to at least one of the IC or the co-processor. The auxiliary device may comprise a motor for driving a lens carriage, and the antifuse non-volatile memory may be encoded with operational parameters for driving the motor. The motor may comprise a piezoelectric motor. The imaging apparatus may further comprise a lens position sensor arranged in feedback with the motor. The operational parameters may comprise one or more of a first command signal frequency for driving the motor in a first direction, a second command signal frequency for driving the motor in a second direction, and a hard-stop lens position defining an infinity focus.

[0011] According to a third aspect, there is provided a mobile (cellular) telephone comprising the imaging apparatus of the second aspect. According to a fourth aspect, there is provided an optical mouse comprising the imaging apparatus of the second aspect. According to a fifth aspect, there is provided a webcam comprising the imaging apparatus of the second aspect. According to a sixth aspect, there is provided an imaging apparatus comprising an IC comprising a non-volatile memory, a co-processor associated with said IC, and a motor for driving a lens carriage connected to at least one of the IC or the co-processor; characterized in that the non-volatile memory is encoded with operational parameters for driving the motor. The motor may comprise a piezoelectric motor. The imaging apparatus may further comprise a lens position sensor arranged in feedback with the motor. The operational parameters comprise one or more of a first command signal frequency for driving the motor in a first direction, a second command signal frequency for driving the motor in a second direction, and a hard-stop lens position defining an infinity focus. The non-volatile memory may be an antifuse non-volatile memory.

[0012] According to a seventh aspect, there is provided an image sensor integrated circuit (IC) comprising a non-volatile memory which is encoded with data for part-tracking. The data for part-tracking may comprise at least one of: sensor ID, date of manufacture, sales region ID, tester ID, test result, wafer ID, and die ID. The non-volatile memory may be an antifuse non-volatile memory.

[0013] According to a eighth aspect, there is provided an image sensor integrated circuit (IC) comprising a non-volatile memory which is encoded with data for defect correction of image data.

[0014] The data for defect correction may comprise the locations of defective pixels.

[0015] The data for defect correction may comprise couplet locations. The non-volatile memory may be an antifuse non-volatile memory.

[0016] According to a ninth aspect, there is provided an image sensor integrated circuit (IC) comprising a non-volatile memory which is encoded with data for color filter array reconstruction. The data for color filter array reconstruction may comprise at least one of: data for anti-vignetting, color channel gains, range scaling parameters, demosaicing algorithms, range clipping parameters, and gamma correction data. The non-volatile memory may be an antifuse non-volatile memory.

[0017] According to a tenth aspect, there is provided an image sensor integrated circuit (IC) comprising a non-volatile memory which is encoded with data defining embedded firmware for microprocessor or DSP control. The non-volatile memory may be an antifuse non-volatile memory.

[0018] It is to be appreciated that the embodiments of the sixth to tenth aspects can also include any of the features listed above as being properties of the first aspect, and can be incorporated with an image sensing apparatus that comprises any of the features of the second aspect.

[0024] Referring to FIG. 1, a digital image sensor IC 11 comprises an image sensing pixel array 10, analog to digital conversion means (ADC) 12 and a number of line memories 16, each of which functions in a well known manner. The IC 11 is further provided with a non-volatile memory 18 which functions as described below.

[0025] It is to be noted that the IC 11 illustrated in FIG. 1 is for the purposes of illustration only, and that the scope of the invention is in no way intended to be limited to any one particular design of IC. The arrangement and sizes of the various components may vary arbitrarily.

[0026] In a first preferred embodiment, the non-volatile memory 18 is an antifuse non-volatile memory. An antifuse memory component employs a non-conducting barrier of amorphous material which is sandwiched between two conductors. When a sufficiently high voltage is applied across the non-conducting barrier, it is converted to a conductor, so forming an electrical connection. Suitable materials for forming a non-volatile memory antifuse are amorphous silicon sandwiched between two metal conductors. When the higher voltage is applied, the amorphous silicon is converted to a polycrystalline silicon-metal alloy, which is conductive. Other materials with suitable properties may be used as the insulator or conductor in an antifuse.

[0027] Antifuse non-volatile memories are known in products such as programmable logic devices (PLD's), PROMS and digital storage products, such as MMC cards. The common link between these devices is that they are used as dedicated “memory” chips, that is, IC's that have the sole or primary function of acting as memory storage devices.

[0028] However, because an antifuse non-volatile memory does not rely on a trapped charge stored on an FET floating gate, the memory elements do not need to be shielded from incident light. This means that an antifuse non-volatile memory, traditionally used for dedicated memory storage IC's, can be used as part of an image sensing IC, without having to make special provisions for shielding the memory elements.

[0029] An on-chip non-volatile memory 18 (either a shielded floating gate non-volatile memory or an antifuse non-volatile memory) can be used for a number of purposes, which will now be discussed.

[0030] FIG. 2 illustrates an IC 30 according to a second preferred embodiment, wherein components shared between the IC 30 and the IC 11 of FIG. 1 are illustrated with like reference numerals.

[0031] In the IC 30, the contents of the non-volatile memory 18 can be copied to a volatile memory 20. The volatile memory 20 defines a data bus space where the contents of the non-volatile memory 18 can be mirrored, by mapping into the appropriate register and data space. FIG. 2 illustrates an inter-integrated circuit (I²C) space, which reads from and/or writes to serial data (SDA) and serial clock (SCL) pins 22, 24. It will be appreciated that different types of data buses may require or define different pin configurations. Furthermore, the data in the non-volatile memory 18 can be mapped into a volatile memory 20 for use with any type of data bus, as chosen by the manufacture. The SMIA specification defines a common camera interface (CCI), which provides a manufacturer with the option to define his own register functions.

[0032] In a preferred method of operation, the contents of the non-volatile memory 18 are copied into the volatile memory 20 at each power-on of the device. Thus, the data stored in the non-volatile memory 18 can be readily read and overwritten while always safely retaining the original contents on the non-volatile memory. This configuration also means that there is no need to exercise the non-volatile memory 18 during subsequent device operation and so it may be configured into a low power state so enhancing platform battery life. The speed of access to the relevant data is also improved, as the volatile memory 20 can be read from more quickly than the non-volatile memory 18, due to the non-volatile memory's serial shifting and access routines.

[0033] The availability of the data in a volatile format enables a closed loop control to be implemented. FIG. 3 shows an embodiment where the IC 30 of FIG. 2 is connected to a co-processor 32 which operates for the control functions of an image sensing device. On power up of the system, the sensor maps the contents of the non-volatile memory 18 to the volatile memory 20. The co-processor 32 can read the data and write a copy of the data back into a different region of the volatile memory 20 in order to change the functionality of the system. That is, the co-processor 32 can be used to change the software or firmware processing tasks of a system linked to the digital image sensor chip 30.

[0034] The closed loop control can be used to set the operating parameters of an auxiliary device 34, as shown in FIG. 4. The non-volatile memory 18 can define a number of registers in the volatile memory 20 to represent the operational parameters of an auxiliary device 34, and the co-processor 32 can obtain the data from the auxiliary device 34 to be fed back into the volatile memory 20 so that command signals for the operation of the auxiliary device 34 are interpreted in the correct manner for operation of the auxiliary device. In this way, the non-volatile memory 18 provides a generic foundation for the operation of different types of auxiliary devices, for example, those from different manufacturers, therewith.

[0035] The “auxiliary device” may be any device which has a functional interdependence with the image sensor 30 or the output thereof. It could for example be a mechanical component, an optical element, or an electrical circuit, any of which has a function in the formation or presentation of an image to a user.

[0036] Some of the specific uses of a memory enhanced image sensor will now be discussed.

[0037] Firstly, the auxiliary device 34 shown in FIG. 4 may be an actuator for moving a lens carrier to adjust the focus of an optical imaging assembly which comprises the digital sensor IC as the image capturing element. Such an actuator may take the form of a piezoelectric motor. The piezoelectric motor is driven by command signals that are pulse width modulated according to specific frequencies. These command frequencies can be encoded on the non-volatile memory 18. In such a motor, the number of command pulses may not dependably correlate with a set variation in distance moved—this motion is not strictly linear, and may also vary according to different temperatures, data transmission errors, or positioning tolerances. The initial position of the lens carrier may also introduce an uncertainty. To deal with this, a lens position sensor forms part of the autofocus system control loop. It is proposed that the required ADC gains and/or offsets required for conversion of the fed back voltage are encoded into the non-volatile memory 18. The hard stop position of the lens, which corresponds to an infinity focus, can also be encoded in the non-volatile memory 18.

[0038] The non-volatile memory 18 can also be used for part tracking. Data such as the sensor ID, manufacturing data for traceability, day, year, sales region id, tester, test result (pass/fail), and even wafer and individual die references can be encoded into the non-volatile memory 18. This enables the origin of a chip to be tracked to a high degree of accuracy, which is useful for tracing the origin of faults or for investigating the history of chips which have been illegally re-sold or imported.

[0039] Another use of the non-volatile memory 18 is for defect correction. One particular problem in defect correction is the occurrence of couplet defects, that is, neighboring pixels in the pixel array or in the color domain that exhibit defective pixel values. Couplets are difficult to deal with using regular defect correction algorithms, as a first defective pixel is typically used as one of the inputs to correct for the defect in a second defective pixel, giving an inaccurate result. However, the X,Y positions of couplets can be encoded into the non-volatile memory 18. This allows for a downstream defect correction map based algorithm to be implemented (as opposed to an on the fly algorithm). Locations of any other types of defects may also be stored in the non-volatile memory 18.

[0040] The non-volatile memory 18 can also be used for color science purposes. For example, anti-vignetting data to correct for lens shadowing effects, color matrices which correct for the differing photon transmission differences between the three color filter materials in a color filter array, and setup data for statistics gathering such as color channel gains for white balancing could be encoded into the non-volatile memory 18.

[0041] The non-volatile memory 18 can also be used to encode embedded firmware for microprocessor or DSP control. When an antifuse non-volatile memory 18 is used, the memory elements which have not been “popped” provide a measure of reprogramming capability. Various improvements and modifications can be made to the above without departing from the scope of the invention.