Color Field Sequential Imaging Apparatus Having Color-Misregistration Correction Function Based on Correlations in Units of Colors

1. Field of the Invention

The present invention relates to a color field sequential imaging apparatus for imaging an image in units of three colors, i.e., red, green, and blue.

2. Description of the Related Art

Such an imaging apparatus is disclosed in, e.g., "Solid-state Imaging Device Beginning to be Used in Endoscope by Utilizing Its Compact Structure", Nikkei Electronics, May 5, 1986 (No. 394) pp. 145-158. In this reference, when a charge transfer type imaging device such as a CCD is used in a conventional color field sequential imaging operation, if field data for one color is read out in 1/90 sec or 1/60 sec, 1/30 sec or 1/20 sec are required to expose three colors, i.e., red, green, and blue (to be abbreviated to as RGB hereinafter).

If 1/20 sec is required to constitute one color image, color misregistrations of RGB colors occur for an object which moves quickly, as shown in FIG. 23A. In a conventional imaging apparatus, the scanning time of field data is shortened to eliminate such color misregistrations. However, sufficient effects have not been obtained.

It is therefore an object of the present invention to provide a field sequential imaging apparatus, which can achieve high-precision color-misregistration correction with low cost by correcting color misregistrations on the basis of correlations in units of colors.

In order to achieve the above object, the field sequential imaging apparatus of the present invention comprises imaging means for imaging an object in units of colors by sequentially switching a plurality of color filters, and correction means for correcting color misregistrations on the basis of correlations of image signals from the imaging means in units of colors.

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate presently preferred embodiments of the invention, and together with the general description given above and the detailed description of the preferred embodiments given below, serve to explain the principles of the invention.

FIG. 1 is a block diagram showing a field sequential imaging apparatus according to the first embodiment of the present invention;

FIG. 2 is a block diagram showing the second embodiment of the present invention applied to an endoscope;

FIG. 3 is a block diagram showing the details of a given circuit portion of FIG. 1;

FIG. 4 is a chart for explaining operations of a 3DFM and a 3DFL;

FIG. 5 is a view showing an arrangement of a rotary color filter of the first embodiment;

FIG. 6 is a block diagram showing another arrangement of the 3DFM;

FIG. 7 is a block diagram showing an arrangement for realizing another interpolation method;

FIG. 8 is a block diagram showing the third embodiment of the present invention that can eliminate read noise;

FIG. 9 is a view showing an arrangement of a rotary color filter according to the third embodiment;

FIG. 10 is a chart showing a state wherein image data is read out in a non-interlace read mode;

FIG. 11 is a block diagram showing a circuit arrangement of a 3DFM of the third embodiment;

FIG. 12 is a chart showing the relationship between time and position in a non-interlace mode;

FIG. 13 is a block diagram showing arrangements of a 3DFM and a 3DFL according to the fourth embodiment of the present invention;

FIG. 14 is a chart showing the relationship between time and input and output images for a non-interlaced image;

FIG. 15 is a block diagram showing an arrangement according to the fifth embodiment of the present invention;

FIG. 16 is a block diagram showing an arrangement of an interpolation circuit shown in FIG. 15;

FIGS. 17A and 17B are views for explaining a position shift detection method;

FIG. 18 is a view showing deviation vectors;

FIG. 19 is a block diagram showing an arrangement of an interpolation coefficient calculation circuit;

FIG. 20 is a block diagram showing an arrangement of a correlation circuit shown in FIG. 19;

FIG. 21 is a view for explaining an interpolation calculation;

FIG. 22 is a block diagram showing an arrangement of a coefficient calculation circuit;

FIGS. 23A and 23B are views for explaining an operation of the fifth embodiment;

FIG. 24 is a block diagram showing an arrangement according to the sixth embodiment of the present invention;

FIG. 25 is a schematic view showing an arrangement of a microscope according to the seventh embodiment of the present invention;

FIG. 26 is a block diagram showing an arrangement of an interpolation coefficient calculation circuit;

FIG. 27 is a view showing an example of a displayed image;

FIG. 28 is a view showing an example of a displayed image;

FIG. 29 is a view showing a reference area according to the eighth embodiment of the present invention;

FIG. 30 is a view showing motion vectors; and

FIG. 31 is a block diagram showing arrangements of a correlation circuit and a coefficient calculation circuit of the eighth embodiment.

The first embodiment of the present invention will be described below with reference to FIG. 1. In the first embodiment, RGB-exposure operations are performed within a period (1/60 sec for NTSC scheme) of a display system so as to reduce color misregistrations. In order to perform the RGB-exposure operations within the period, a device capable of performing a high-speed reading operation is used as an imaging device. As a high-speed imaging means, a charge modulation device (to be referred to as a CMD hereinafter), a multi-line reading CCD, and the like are known.

In the following description, a CMD is used as the high-speed imaging device. A rotary color filter to be used is arranged, as shown in FIG. 5. The rotary color filter is rotated about a point O. In FIG. 5, reference symbols RL, GL, and BL denote exposing portions of the corresponding colors; and Rr, Gr, and Br, light-shielding portions for reading.

Light input through a lens 1 and an R filter portion of a rotary color filter 2 is photoelectrically converted by a CMD 3. An exposure operation is performed during a period corresponding to the portion RL, and when the CMD 3 is shielded by the light-shielding portion Rr of the rotary color filter, the CMD 3 outputs a signal. The output from the CMD 3 is amplified by an amplifier (AMP) 4, and the amplified signal is analog-to-digital (A/D)-converted by an ADC 5. The CMD 3 is controlled by a control unit 7. The output from the ADC 5 is selected by a data selector 6, and is recorded on a three-dimensional field memory (to be abbreviated to as a 3DFM hereinafter) 8r. In this case, the data selector 6 is controlled by the control unit 7. The 3DFM comprises a plurality of field memories, and time-elapsed data can be read out therefrom.

The data from the 3DFM 8r is subjected to interpolation processing by a three-dimensional filter circuit (to be abbreviated to as a 3DFL hereinafter) 9r. The timings of the 3DFM and 3DFL are controlled by the control unit 7. Thereafter, the output from the 3DFL is digital-to-analog (D/A)-converted by a DAC 10r, and the analog data is displayed on a monitor 11.

A B component is similarly fetched, and is displayed on the monitor 11. As will be described later, since interpolation processing is performed with reference to a G component, a G signal passes through only a 3DFM 20 so as to adjust the timing. In this embodiment, the 3DFM 20 comprises field memories (to be abbreviated to as FMs hereinafter) 31 and 32, as shown in FIG. 11. Data read out from the 3DFM 20 is D/A-converted, and the analog data is displayed on the monitor 11.

The operations of the 3DFM and 3DFL will be described below with reference to FIG. 4. FIG. 4 illustrates an operation while paying attention to a given pixel and its upper and lower pixels. When image data is read out from the CMD 3 in correspondence with an interlace display system, the relationship between the vertical positions of pixels and time is as shown in FIG. 4.

In this embodiment, a G image including many luminance components is used as an image of interest, and R and B images are time-interpolated at the time position of the G image. When an interpolation method employs a method of calculating a color of interest on the basis of colors before and after the color of interest, an equation therefor is defined as follows. In FIG. 4, R' is interpolated and displayed at the position of G.sub.N+1.sup.2.

Similarly, B' is interpolated, as indicated by the following equation (2):

The above-mentioned R' calculation can be realized by a circuit shown in FIG. 3. In order to obtain immediately preceding and next field image data, FMs 12 and 13 are used. The outputs from these FMs are connected to the 3DFL 9. In the interpolation of a B image, another FM 30 is required in a 3DFM 8b.

In FIG. 3, the output from the FM 12 is supplied to a 1HDL 15 and an adder 16 to calculate R.sub.N+2.sup.1 +R.sub.N+2.sup.2. The output from the adder 16 is connected to a multiplier 17 to calculate 1/3.times.{1/2.times.(R.sub.N+2.sup.1 +R.sub.N+2.sup.2)} in equation (1). More specifically, the control unit 7 outputs a coefficient "1/6", and the multiplier 17 performs a multiplication.

On the other hand, a multiplier 18 calculates 2/3.times.R.sub.N+1.sup.2 in equation (1), and outputs the product. More specifically, the control unit 7 outputs a coefficient "2/3", and the multiplier 18 performs a multiplication. An adder 19 adds the products from the two multipliers, and outputs R'.

In this embodiment, data are read out from the imaging device in correspondence with the interlace system. In a non-interlace system, however, image data are read out, as shown in FIG. 10. R' and B' are interpolated at the time position of G.sub.N+1.sup.1, as indicated by the following equations (3) and (4):

When image data are displayed through the above-mentioned time-interpolations, color misregistrations can be eliminated. When image data are read out in correspondence with the non-interlace system, frame memories are used in place of field memories.

FIG. 2 shows the second embodiment wherein the present invention is applied to an endoscope. When the present invention is applied to an endoscope, a rotary color filter 2 is arranged in front of a light source 21. Light emerging from the rotary color filter 2 is radiated through an optical fiber 22 in an RGB sequential manner, and images reflected by an object 23 are picked up. Since the following circuit operations are the same as those in the first embodiment, a detailed description thereof will be omitted.

In the above embodiments, R and B are interpolated in correspondence with G. Alternatively, G and B may be interpolated and substituted at the position of R, or R and G may be interpolated and substituted at the position of B. In place of interpolating two colors at the position of a given color, three colors may be time-interpolated at a time position where no image signals are present, as indicated by an alternate long and short dashed line vertical A in FIG. 4, and the interpolated colors may be displayed on the monitor. A circuit in this case is arranged, as shown in FIG. 7, and a 3DFM 8 and a 3DFL 9 are inserted in each path of RGB color signals.

The order of color filters is not limited to RGB, but may be arbitrarily changed. For example, when R and B are interpolated at the time position of G, if the order of RBG is employed, an interpolation can be performed using only data in the (N+1)th and (N+2)th fields. For this reason, field memories for two fields need only be prepared, and the FM 30 shown in FIG. 6 can be omitted.

FIG. 8 shows the third embodiment for eliminating fixed pattern noise (to be abbreviated to as FPN hereinafter). In this embodiment, a rotary color filter is arranged, as shown in FIG. 9, and operates to read out FPN in a dark state, and to subtract FPN from color signals. In FIG. 9, portions RL, Rr, GL, Gr, BL, and Br are the same as those in FIG. 5, and a detailed description thereof will be omitted. Reference symbol FL denotes a light-shielding portion for storing FPN; and Fr, a portion for reading out FPN.

In this embodiment, storage and reading operations of R, G, B, and FPN components are performed at high speed during one field period. FPN data read out from a CMD is selected by a data selector 6, and is recorded on an FM 24. The FPN data is read out from the FM 24 in correspondence with the read timing of each color, and is subtracted from the corresponding color signal by a corresponding one of subtracters 25r, 25g, and 25b. The reason why the FPN storage time is assured by the portion FL is that the FPN caused by a dark current tends to increase along with the storage time, and the FPN storage time must be set to be equal to that of each color signal.

Interpolation equations of this embodiment will be described below. In a non-interlace system, since the relationship between time and position is as shown in FIG. 12, interpolation equations of R and B at the time position of G.sub.N+1.sup.1 are expressed by:

According to this embodiment, when FPN is subtracted from image signals, the S/N ratio of the image signals can be increased.

A conventional device announced in, e.g., an academic society has been exemplified as the CMD. When such conventional CMD is applied to the present invention, the light-shielding portions are required, as shown in FIG. 5. However, as described in Japanese Patent Application No. 59-124590, when a CMD, which has light-shielding portions in the device, and uses them for transferring data, is used, since signals need not be read out during the light-shielding portions, each light-shielding portion can have a small width. Therefore, an exposure time can be effectively assured.

In the above embodiments, CMD imaging devices capable of performing a high-speed read operation have been exemplified. However, the present invention is also effective when the method of the present invention is executed at an exposure timing of one color per field (one color per frame in a non-interlace system) in place of high-speed rotation. The fourth embodiment will be described below. Since the circuit arrangement of this embodiment is the same as that shown in FIG. 7, only a difference will be described below.

FIG. 14 shows the relationship between time and input and output images for a non-interlaced image in this embodiment. An output image is output to be delayed from an input image by one frame. Interpolation equations of R are expressed as the following equations (7) and (8):

Similarly, G and B are expressed as follows:

In general, interpolation equations of R before and after 3n frames (n is an integer) can be expressed as:

Since the same applies to G and B, a detailed description thereof will be omitted.

The arrangements of a 3DFM 8 and a 3DFL 9 for executing the above-mentioned equations are as shown in FIG. 13. In this embodiment, frame memories are used in place of field memories since a non-interlace system is employed. Therefore, "FM" means a frame memory. The same circuit arrangement shown in FIG. 13 is prepared for each of RGB colors. The operation will be described below with reference to FIG. 14 using R as an example. An R input image is selected by a selector 33, so that an image R.sub.0 is input to an FM 12, and an image R.sub.1 input three frames later is input to an FM 13. In the fourth frame shown in FIG. 14, the output from the FM 12 is selected by and output from a selector 34. In the fifth and sixth frames, interpolation calculations given by equations (7) and (8) are performed using multipliers 17 and 18 and an adder 19, and the calculation results are selected by and output from the selector 34. In the seventh frame, the output from the FM 13 is selected by and output from the selector 34. In the seventh frame, since an image R.sub.2 is input, it is selected by the selector 33, and is recorded on the FM 12.

When the above-mentioned operation is repetitively executed, color misregistrations can be eliminated even with an exposure timing of one color per frame. According to this embodiment, a special imaging device capable of performing a high-speed read operation need not be used, and a commercially available CCD can be used. In FIG. 13, the selectors are used. However, as shown in FIG. 3, FMs may be connected in series with each other.

In each of the above embodiments, a linear interpolation between two signals has been described as a time interpolation. However, the number of data to be used in an interpolation may be increased to improve precision. Alternatively, a spline function may be used to attain an interpolation method having further desirable reproducibility.

When a plurality of colors are exposed and read out during one field period, a decrease in sensitivity may occur due to a short exposure time. However, when an amplifying type imaging device such as a CMD is used, since an amplification effect can be provided in units of pixels, such decrease in sensitivity will not occur. When a field sequential camera is used in a microscope, color misregistrations upon movement of a stage are limited to two-dimensional parallel misregistrations or rotational misregistrations. In this case, another processing for eliminating color misregistrations can be performed.

The fifth embodiment of the present invention for eliminating color misregistrations of the above-mentioned kind will be described below.

FIG. 15 is a block diagram showing the overall arrangement according to the fifth embodiment of the present invention. Reference numeral 1 denotes a photographing optical system for photographing object light; and 2, a rotary color filter having R, G, and B filters in the order named. Reference numeral 3 denotes an imaging device which comprises, e.g., CMD solid-state imaging devices, and is driven by a control unit. Reference numeral 4 denotes a preamplifier (AMP) for amplifying outputs from the imaging devices; 5, an ADC for converting the output from the AMP 4 into a digital signal; and 6, a data selector for selecting an input signal. Reference numerals 37r, 37g, and 37b denote frame memories for storing image signals. Reference numerals 38r, 38g, and 38b denote interpolation circuits each for interpolating color images other than an input color image with respect to the input color image. Each interpolation circuit has an arrangement, as shown in FIG. 16.

In FIG. 16, reference numeral 381 denotes a data reading circuit for reading out pixel values V.sub.b, V.sub.c, V.sub.d, and V.sub.e at four positions from the corresponding frame memory 37 according to corresponding coordinates (IC.sub.x, IC.sub.y) of an image read out from a coefficient memory. Reference numeral 382 denotes a linear interpolation circuit. In this circuit, multipliers 383, 384, 385, and 386 multiply the data V.sub.b, V.sub.c, V.sub.d, and V.sub.e with interpolation coefficients C.sub.b, C.sub.c, C.sub.d, and C.sub.e read out from the coefficient memory, and an adder 387 adds these products. That is, an interpolation calculation is performed by:

The coefficients will be described in detail later. Note that coefficients used in the interpolation are calculated by an interpolation coefficient calculation circuit 35, as will be described later. Each of data selectors 39r, 39g, and 39b switches between an interpolated signal and a non-interpolated signal. In switching control, for example, if an image input from the CMD is an R image, the R image signal is directly output without being interpolated, and G and B signals are interpolated and output. The outputs from the data selectors 39r, 39g, and 39b are respectively converted into analog signals by DACs 10r, 10g, and 10b, and the analog signals are output to a monitor 11. Reference numeral 7 denotes a control unit for controlling the CMD 3, the data selectors 6, 39r, 39g, and 39b, and the like.

The interpolation coefficient calculation circuit 35 will be described below. As shown in FIG. 19, the interpolation coefficient calculation circuit 35 is constituted by correlation circuits 40 (40A and 40B) and coefficient calculation circuits 41 (41A and 41B). Prior to the detailed description of the arrangement, a flow for detecting an interpolation coefficient will be explained below.

A method of detecting a position shift will be described below. In the following description, a G image is exemplified as an input image. The position shift is separately detected for translation and rotation components at each given position in an image. As shown in FIG. 17A, reference areas a1, a2, a3, and a4 for detecting a position shift are set in the input image (the central positions of these areas are designated by p1, p2, p3, and p4). The areas are present at symmetrical positions about a position c, and each area is separated from the position c by a distance corresponding to k pixels. In an image other than the input image, search areas b1, b2, b3, and b4 for searching positions corresponding to the areas a1, a2, a3, and a4 of the input image are set. Deviation vectors V.sub.1, V.sub.2, V.sub.3, and V.sub.4 corresponding to the areas a1, a2, a3, and a4 are detected. The deviation vectors are expressed as follows by a rotation vector r and a translation vector s at the position pl having the position c as the center (see FIG. 18).

where r.sub.-90 and r.sub.+90 are the vectors obtained by rotating r through -90.degree. and +90.degree., respectively. The rotation vector r is expressed by:

where .theta. is the rotational angle. From equations (16a) to (16d) , we have:

In this manner, the deviations of translation and rotation can be detected. Note that the rotational angle .theta. of the rotation is given by:

The correlation circuit 40 detects s and .theta., and has an arrangement, as shown in FIG. 20. Reference numerals 401, 402, 403, and 404 denote correlators for performing correlation calculations between the reference area a1 and the search area b1, between the reference area a2 and the search area b2, between the reference area a3 and the search area b3, and between the reference area a4 and the search area b4 so as to detect the deviation vectors v.sub.1, v.sub.2, v.sub.3, and V.sub.4, respectively. As for the correlation calculation, various methods have been conventionally proposed. In this case, a calculation for comparing sums of the absolute values of differences is performed. Reference numeral 405 denotes an SR detector for detecting the translation vector s and the rotation vector r according to equations (16a) to (16d), (18), and (19). Reference numeral 406 denotes a .theta. detector for detecting the rotational angle .theta. using equations (17) and (20).

The coefficient calculation circuit 41 for calculating the coefficients C.sub.b, C.sub.c, C.sub.d, and C.sub.e used in interpolation calculations on the basis of r and .theta. will be described below. A linear interpolation calculation will be described below with reference to FIG. 21. A case will be examined below wherein the value of a pixel A is obtained from pixels B, C, D, and E whose values are already known, as shown in FIG. 21. If intersections between a line passing the pixel A and extending perpendicularly to line segments BC and DE, and the line segments BC and DE are respectively represented by F and G, and BF : FC=DG : GE=m : n, and FA : AG=p : q, a value V.sub.f of a pixel F is given by:

A pixel value V.sub.g of a pixel G is given by:

Therefore, the value Va is given by:

Since m+n=p+q=1 for an inter-pixel distance=1, v.sub.a can be calculated by: ##EQU1## Upon comparison with equation (1), the interpolation coefficients are respectively given by:

If the coordinates of the pixel A are represented by (C.sub.x, C.sub.y), the coordinates of the pixels B, C, D, and E are respectively expressed by:

where IC.sub.x is the integral part of C.sub.x, and IC.sub.y is the integral part of C.sub.y.

Positions X.sub.r and X.sub.b on images other than an input image (R and B images in this case) corresponding to a position X.sub.g of the input image (G image in this case) are respectively given by:

where S.sub.r, S.sub.b, .theta..sub.r, and .theta..sub.b are respectively the translation vectors and the rotational angles of the R and B images, and X.sub.r, X.sub.g, and X.sub.b are the two-dimensional vectors having the x- and y-coordinates as elements. In addition, R(.theta.) is given by: ##EQU2##

FIG. 22 shows the arrangement of the coefficient calculation circuit 41. In FIG. 22, reference numeral 411 denotes a coordinate conversion circuit for calculating equations (27) and (28), and outputting corresponding coordinates C.sub.x and C.sub.y of the R and B images. Note that C.sub.x and C.sub.y are real numbers. Reference numerals 412 and 413 denote integer converters for outputting the coordinates IC.sub.x and IC.sub.y in FIG. 21. Reference numerals 414 and 415 denote subtracters for respectively outputting m (=C.sub.x -IC.sub.x) and p (=C.sub.y -IC.sub.y). Reference numeral 416 denotes a coefficient calculator for calculating the interpolation coefficients on the basis of equation (25), and outputting the interpolation coefficients C.sub.b, C.sub.c, C.sub.d, and C.sub.e.

The operation of the present invention with the above arrangement will be described below. In the following description, a G image is exemplified as an input image.

When a photographing operation is started, focusing adjustment of the photographing optical system 1 and adjustment of the exposure time of the imaging device 3 are performed by a distance measurement system and a photometric system (neither are shown). Object light is transmitted through the rotary color filter 2 in an RGB sequential manner, and is converted into an image signal by the imaging device 3. The image signal is amplified by the AMP 4 to keep white balance, and the amplified signal is then converted into a digital signal by the ADC 5.

The digital signal is selected by the data selector 6, and is stored in the FM 37g. Simultaneously with this storage operation, the correlation circuit 40A receives reference areas a1, a2, a3, and a4 of a G image, and search areas b1, b2, b3, and b4 are read out from the frame memory 37r (R image), thereby detecting a translation vector S.sub.r and a rotational angle .theta..sub.r between the G and R images. S.sub.r and .theta..sub.r are supplied to the coefficient calculation circuit 41A, thereby calculating the corresponding coordinates IC.sub.x and IC.sub.y of the R image corresponding to the coordinate X.sub.g of the G image and the interpolation coefficients C.sub.b, C.sub.c, C.sub.d, and C.sub.e. Then, IC.sub.x and IC.sub.y, and C.sub.b, C.sub.c, C.sub.d, and C.sub.e are detected for a predetermined range of the G image, and are output to the interpolation circuit 38r. The predetermined range means an overlapping range (photographing area) of R, G, and B images, as shown in FIG. 23B. The photographing area is pre-set by predicting the moving range of an object.

Similarly, in the interpolation coefficient calculation circuit 35, the position shift amounts of the G and B images are detected by the correlation circuit 40B and the coefficient calculation circuit 41A, and the corresponding coordinates IC.sub.x and IC.sub.y of the B image in the photographing area and the interpolation coefficients C.sub.b, C.sub.c, C.sub.d, and C.sub.e are calculated. These data are output to the corresponding interpolation circuit. Thereafter, R image data used in the interpolation calculation is read out from the frame memory 37r, and the interpolation calculation is performed by the interpolation circuit 38r, thereby calculating the pixel value of the R image corresponding to the designated G image position. Similarly, the pixel value of the B image corresponding to the designated G image position is calculated. These pixel values are output to the corresponding DACs 10r, 10g, and 10b. The pixel values are converted into analog signals, and the analog signals are displayed. As described above, the interpolation coefficients for the photographing area where R, G, and B image overlap each other are calculated.

In the above description, a G image is exemplified as an input image. When an R image is an input image, G and B images can be interpolated; when a B image is an input image, R and G images can be interpolated. The inputs to the correlation circuits are switched by a data selector 42, as shown in FIG. 19, and the data selector 42 is controlled by a selector controller 44. The outputs are switched by a data selector 43.

In this embodiment, the two interpolation coefficient calculation circuits are used to calculate the coefficients for correction. Alternatively, only one interpolation coefficient calculation circuit may be used, and images may be switched. In this embodiment, the four areas a1, a2, a3, and a4 are selected as the reference areas, as shown in FIG. 17A. Alternatively, only two areas, i.e., a1 and a3, or a2 and a4 may be selected, or five or more areas may be selected. The interpolation circuit 38 uses a linear interpolation calculation. However, the present invention is not limited to this. For example, a high-order correlation calculation such as a spline interpolation, an SINC interpolation, or the like may be used.

The sixth embodiment of the present invention will be described below.

In this embodiment, interpolation coefficients are calculated based on information stored in a frame memory. In order to obtain a B image at the position of G.sub.1 in the sixth field in FIG. 28, the positional relationship between B.sub.0 two frames before and G.sub.1 must be calculated. This operation may undesirably decrease a photographing area shown in FIG. 23A for an object which moves quickly. In this embodiment, a B image at the position of G.sub.1 is interpolated from B.sub.1 (see the seventh frame in FIG. 27) unlike in the above embodiment. In the following description, a G.sub.1 image shown in the seventh frame in FIG. 27 is exemplified as an input image.

FIG. 24 is a block diagram showing the arrangement of this embodiment. A description of the same arrangement as in the fifth embodiment will be omitted.

Assume that a G.sub.1 image is stored in a frame memory (to be abbreviated to as an FM hereinafter) 48. The correlation between the G.sub.1 image and a B.sub.1 image output from an ADC 5 is calculated by a correlation circuit 40A shown in FIG. 26. At the same time, an R.sub.1 image is read out from an FM 37r, and a correlation circuit 40B calculates the correlation between the B.sub.1 and G.sub.1 image. Furthermore, at the same time, the G.sub.1 image is written in an FM 37g. With the above-mentioned operations, the FM 48 stores the B.sub.1 image, the FM37r stores the R.sub.1 image, the FM 37g stores the G.sub.1 image, and an FM 37b stores a B.sub.0 image. Thereafter, the output from the FM 37r is input to a DAC 10r through an interpolation circuit 38B, the output from the FM 37g is directly input to a DAC 10g, and the output from the FM 48 is input to a DAC 10b through an interpolation circuit 38A. The output images are those shown in the seventh frame in FIG. 27. Data selectors 49 and 39 select images to be supplied to the interpolation circuits in correspondence with sequentially input color images.

With the above-mentioned operations, a position interpolation can be realized by images at adjacent time positions.

According to this embodiment, when 1/30 sec are required to fetch an R image, and 1/10 sec are required to fetch all the colors, since R, G, and B color images are obtained at 1/30-sec intervals while being position-interpolated, images with less color misregistrations can be obtained. In the fifth embodiment, the interpolation circuits are arranged for all the R, G, and B colors. However, as in the sixth embodiment, when the data selectors are inserted at the input and output sides of the interpolation circuits, one interpolation circuit can be omitted. In addition, like in the first embodiment, for example, R and B images may be corrected with reference to a G image by performing a high-speed photographing operation. In this case, an interpolation circuit for the G image can be omitted.

The seventh embodiment of the present invention will be described below. An application for a case wherein a portion of an object moves will be described below. For example, in a microscope, when a stage is moved, color misregistrations are corrected by a position interpolation, and when a portion of an object moves, color misregistrations are corrected by a time interpolation. Means for discriminating movement of the stage and movement of the object will be described below.

As shown in FIG. 25, switches are provided to dials 46 and 47 for moving the stage. When one of these dials is operated, the position interpolation is performed. As for color misregistrations caused by other factors, the time interpolation is performed.

As another embodiment, the movement is predicted from a pattern of deviation vectors corresponding to the four search areas shown in FIG. 18. When only the stage is moved, the deviation vectors can be expressed by the translation vector s and the rotation vector r, as indicated by equations (16a) to (16d). However, when a portion of an object moves, the motion vector of the object is added to one of these deviation vectors. More specifically, a deviation vector V cannot be expressed as equations (18) and (19). Thus, for the purpose of discrimination, the following evaluation value .epsilon. is defined:

When a portion of the object moves, .epsilon. is increased. Thus, when .epsilon. is larger than a threshold value .epsilon..sub.T, the time interpolation is performed; otherwise, the position interpolation is performed.

The eighth embodiment of the present invention will be described below. The above-mentioned position interpolation is a correction method used when the entire object moves in a uniform direction like in a case wherein the stage is moved. The embodiment which uses the time interpolation when a portion of an object moves has been proposed. However, discrimination of movement of the stage or movement of a portion of an object requires cumbersome operations. Thus, an embodiment for correcting color misregistrations by the position interpolation even when a portion of an object moves will be described below.

In this embodiment, motion vectors at a large number of positions in an image are obtained, and a motion vector at a position where no motion vector is obtained initially is obtained by interpolating the already obtained motion vectors. As shown in FIG. 29, e.g., m.times.n reference areas are selected from an input image. The central coordinates of these areas are represented by P.sub.11, P.sub.21, . . . , P.sub.mn. Then, a case will be examined below wherein corresponding coordinates in R and B images at a position x.sub.g of a G image are calculated when the G image is an input image. A motion vector V.sub.g at the position x.sub.g is calculated from motion vectors V.sub.a, V.sub.b, V.sub.c, and V.sub.d obtained from reference areas closest to the coordinate position x.sub.g, as shown in FIG. 30. V.sub.g can be calculated by the following equation as in the above-mentioned coordinate interpolation:

Since the method of calculating C.sub.va, C.sub.vb, C.sub.vc, and C.sub.vd are the same as that as described above, a detailed description thereof will be omitted. A coordinate position x.sub.r corresponding to x.sub.g can be calculated by:

The arrangement of this embodiment is common to those of the fifth and sixth embodiments, except for a portion of an interpolation coefficient calculation circuit. That is, a correlation circuit 40 and a coefficient calculation circuit 41 in an interpolation coefficient calculation circuit 352 have arrangements shown in FIG. 31. Reference numerals 40.sub.11, 40.sub.21, . . . 40.sub.mn denote correlators for performing correlation calculations between reference areas a.sub.11, a.sub.21, . . . a.sub.mn, and corresponding search areas b.sub.11, b.sub.21, . . . b.sub.mn, and outputting motion vectors V.sub.11, V.sub.21 = . . . =V.sub.mn. When only the stage is moved, V.sub.11 =V.sub.21 = . . . =V.sub.mn. However, when a portion of an object moves, or when an object expands or contracts, the motion vectors are different from each other. The motion vectors V.sub.11, V.sub.21, . . . V.sub.mn are stored in a motion vector memory 40.sub.7, and are used for interpolating a motion vector. Reference numeral 41.sub.7 denotes a motion vector interpolation circuit for calculating the motion vector V.sub.g corresponding to the position x.sub.g of an input image according to equation (31). Reference numeral 41.sub.8 denotes an adder for calculating equation (32), and outputting corresponding coordinates (C.sub.x, C.sub.y).

The subsequent circuit outputs interpolation coefficients C.sub.b, C.sub.c, C.sub.d, and C.sub.e used in interpolation circuits 38r, 38g, and 38b, and addresses IC.sub.x and IC.sub.y of data to be read out like in the fifth and sixth embodiments.

As described above, according to this embodiment, since the motion vectors at respective positions of an input image are calculated, even when a portion of an object moves, color misregistrations can be corrected by the position interpolation. Of course, when the stage is moved, color misregistrations can be similarly corrected.

As described in detail above, in each of the embodiments of the field sequential imaging apparatus according to the present invention, a position shift between successive ones of sequentially picked-up image signals in units of colors is detected, and the position shift of the image signal is corrected according to the detected position shift, thereby attaining easy and satisfactory color-misregistration correction with high precision.

Additional advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details, and representative devices shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.