Image processing apparatus and recording medium, and image processing apparatus

[0001] This is a continuation of Application No. PCT/JP01/05915 with an international filing date of Jul. 6, 2001, which was published under PCT Article 21(2) in Japanese, and the complete disclosure of which is incorporated into this application by reference.

[0002] The present invention relates to an image processing method, a recording medium, and an image processing apparatus for smoothing digital data for an input image and for removing noise.

[0003] For an image input device such as a digital still camera (hereinafter referred to as digital cameras) or a scanner that use image input unit, including a CCD, the image input unit sometimes outputs not only an electric signal corresponding to the intensity of the light received, but also an unwanted electric signal. This unwanted electrical signal is included as noise in the digital data constituting an image. The occurrence of this noise tends to be especially remarkable in a low intensity luminous portion, i.e., a dark image portion.

[0004] Generally, in order to remove such noise included in an image, such a method is employed that smoothes all the digital data constituting an image, or a method is employed that uses edge information to remove the included noise.

[0005] For the removal of noise while maintaining the edge information, a method (a) for smoothing image data or for emphasizing contours in accordance with the slopes of the edges of an image; and a method (b) for detecting the direction of edges and changing smoothing ranges are presently available. Herein the slopes of the edges are the slopes of the luminance of the pixels that constitute an image.

[0006] However, according to the method (a) for smoothing the image data or for emphasizing the contours in accordance with the slopes of the edges, noise present in a flat portion wherein the amount of noise is comparatively less is also emphasized, and the image quality is deteriorated. On the other hand, according to the method (b) for detecting the direction of an edge so as to change a smoothing range, conditional branching is employed to detect the direction of the edge. Therefore, when angles used for detecting the direction of the edge are set in detail in order to increase the accuracy of the detection angles, the number of conditions imposed for the performance of the branching is increased. Therefore, it is difficult to perform fast processing such as pipeline processing. In addition, when the angles for the detection of the direction of an edge are roughly set in order to increase the processing speed, errors occur more frequently during the determination processing performed for conditional branching, and the edge tends to be blurred.

[0007] Further, the methods (a) and (b) are performed as independent functions, and there is no method for jointly implementing the functions of the two methods.

[0008] It is, therefore, one objective of the present invention to provide an image processing method and a recording medium therefor, and an image processing apparatus for quickly removing noise without blurring the edges of images or emphasizing the noise present in flat portions.

• [0009] (1) An image processing method for smoothing digital data for an input image and for removing noise included in the digital data, the method comprising:

[0010] an edge information calculation step of extracting an edge of the image from the digital data, and calculating edge information including a grade and a direction of a slope of the edge;

[0011] a filter information selection step of selecting filter information that is set in advance based on the edge information obtained at the edge information calculation step; and

[0012] a processing step of smoothing the digital data based on the filter information selected at the filter information selection step.

• [0013] (2) The image processing method according to (1), wherein, at the filter information selection step, when the grade of the slope is small, filter information is selected whose smoothing range describes a circular shape, and when the grade of the slope is large, filter information is selected whose smoothing range describes an elliptical shape.
• [0014] (3) The image processing method according to (1), wherein, at the filter information selection step, filter information is selected for which the direction of the slope corresponds to the inclination of the smoothing range.
• [0015] (4) The image processing method according to (1) further comprising:

[0016] a smoothing strength calculation step of calculating luminance using the digital data, and calculating a smoothing strength using the luminance,

[0017] wherein, at the filter information selection step, filter information is selected that corresponds to the edge information obtained at the edge information calculation step and the smoothing strength obtained at the smoothing strength calculation step.

• [0018] (5) An image processing method, for smoothing digital data for an input image and removing noise included in the digital data, the method comprising:

[0019] an edge information calculation step of extracting an edge of the image from the digital data, and calculating edge information that includes a grade and a direction of a slope of the edge;

[0020] a filter information generation step of generating filter information based on the edge information obtained at the edge information calculation step; and

[0021] a processing step of smoothing the digital data based on the filter information generated at the filter information generation step.

• [0022] (6) A recording medium storing a computer program for causing an image processing apparatus, which includes an input unit for entering image data, a processor for processing digital data output by the input unit and a recording unit for recording filter information used to process the digital data, to smooth digital data for an input image and to remove noise included in the digital data, the image processing apparatus being caused by the computer program to perform:

[0023] an edge information calculation step of extracting an edge of the image from the digital data received from the input unit, and calculating edge information including a grade and a direction of a slope of the edge;

[0024] a filter information reading step of reading out specific filter information stored in the recording unit based on the edge information obtained at the edge information calculation step; and

[0025] a processing step of smoothing the digital data based on the filter information read at the filter information reading step.

• [0026] (7) The recording medium according to (6), wherein the filter information read at the filter information reading step is so set that when the grade of the slope is small, a smoothing range describes a circular shape, and when the grade of the slope is large, the smoothing range describes an elliptical shape.
• [0027] (8) The recording medium according to (6), wherein filter information read at the filter information reading step is so set that a inclination of the smoothing range corresponds to the direction of the slope.
• [0028] (9) The recording medium according to (6), wherein the image processing apparatus is caused by the computer program to further perform:

[0029] a smoothing strength calculation step of calculating luminance using the digital data, and of calculating a smoothing strength using the luminance,

[0030] wherein, at the filter information reading step, specific filter information is read out from the recording unit based on the edge information obtained at the edge information calculation step and the smoothing strength obtained at the smoothing strength calculation step.

• [0031] (10) A recording medium storing a computer program for causing an image processing apparatus, which includes an input unit for entering image data and a processor for processing digital data output by the input unit, to smooth digital data for an input image and to remove noise included in the digital data, the image processing apparatus being caused by the computer program to perform:

[0032] an edge information calculation step of extracting an edge of the image from the digital data, and calculating edge information that includes the grade and the direction of a slope of the edge;

[0033] a filter information generation step of generating filter information based on the edge information obtained at the edge information calculation step; and

[0034] a processing step of smoothing the digital data based on the filter information generated at the filter information generation step.

• [0035] (11) An image processing apparatus comprising:

[0036] an image input unit for receiving image data and outputting the image data as digital data;

[0037] an edge calculation unit for extracting an edge of an image from the digital data output by the image input unit, and for calculating edge information that includes the grade and the direction of a slope of the edge;

[0038] a recording unit for storing filter information that is set in correlation with the edge information;

[0039] a filter information selection unit for, based on the edge information calculated by the edge information calculation unit, selecting filter information stored in the recording unit; and

[0040] a smoothing unit for smoothing the digital data based on the filter information selected by the filter information selection unit.

• [0041] (12) An image processing apparatus according to (11), further comprising:

[0042] a smoothing strength calculation unit for calculating luminance using the digital data, and calculating a smoothing strength using the luminance,

[0043] wherein, on the recording unit, filter information is stored that is set in correlation with the edge information obtained by the edge information calculation unit and the smoothing strength obtained by the smoothing strength calculation unit.

• [0044] (13) An image processing apparatus comprising:

[0045] an image input unit for receiving image data and outputting the image data as digital data;

[0046] an edge calculation unit for extracting an edge of an image from the digital data output by the image input unit, and calculating edge information that includes the grade and the direction of a slope of the edge;

[0047] a filter information generation unit for generating filter information stored in the recording unit based on the edge information calculated by the edge information calculation unit;

[0048] a smoothing unit for smoothing the digital data based on the filter information generated by the filter information generation unit; and

[0049] a recording unit for storing the smoothed digital data.

[0050] According to (1), (6) or (11), digital data is smoothed using filter information corresponding to an edge. Further, since the filter information is set in advance and is selected based on the edge information that is obtained, a complicated conditional branching process need not be performed, and pipeline processing can be used to perform the smoothing process quickly. Therefore, the noise removal process can be performed rapidly without blurring the edges of images or emphasizing the noise present in flat portions.

[0051] According to (2) or (7), the filter information is so set that as the slope of an edge is increased, the range for smoothing the filter information is changed from a circle to an ellipse. Therefore, when the slope of an edge is steep, i.e., there is a great change in the luminance, the range for smoothing the filter information is changed to an elliptical shape, and the smoothing can be performed along the edge. Therefore, without emphasizing the noise present in a flat portion, along an edge, only the noise near the edge is removed.

[0052] According to (3) or (8), the filter information is so set that the direction of a slope matches the inclination of the smoothing range. Thus, even when the shape of the edge is changed, the smoothing range can be changed in accordance with the shape of the edge.

[0053] According to (4), (9) or (12), luminance is calculated based on the digital data, and the smoothing intensity is calculated using the luminance. The filter information is selected based on the smoothing intensity and the edge information. Thus, the smoothing intensity can be changed using the luminance of the edge, and noise near the edge can be extensively removed. As a result, the noise near the edges can be removed without emphasizing the noise present in the flat portions.

[0054] According to (5), (10) or (13), digital data is smoothed by using a filter that corresponds to an edge. The filter information is created based on the obtained edge information that is obtained. Therefore, complicated conditional branching processing need not be performed, and pipeline processing can be used to perform the smoothing processing quickly. Further, the filter information to be smoothed is changed in accordance with an image. And therefore, noise can be thoroughly removed in accordance with an image, without the edges of the image becoming blurred and noise present in flat portions being emphasized.

[0067] A plurality of embodiments according to the present invention will now be described based on the drawings.

[0068] FIG. 2 is a diagram showing a digital camera 1 for which an image processing apparatus according to a first embodiment of the present invention is applied.

[0069] As is shown in FIG. 2, the digital camera 1 comprises: a controller 10, an image input unit 20, a recording unit 30, a display unit 40 and an interface 50.

[0070] The controller 10 is an electric circuit for processing digital data output by the image input unit 20. The controller 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12 and a processor 60. In the ROM 12, a computer program is recorded that is executed by the CPU 11 and the processor 60 of the controller 10.

[0071] As is shown in FIG. 3, the processor 60 includes an edge information calculation unit 61, a filter information selection unit 63, a smoothing unit 64 and a writing unit 65.

[0072] As is shown in FIG. 2, an input unit for receiving an entry from a user is connected to the controller 10. The input unit includes a shutter button 71 for receiving an imaging instruction from the user, and a plurality of input buttons 72 for receiving instructions for the performance of various functions of the digital camera 1.

[0073] The image input unit 20 includes a condensing lens 21, a CCD 22 and an A/D converter 23. The condensing lens 21 collects light from an object that is then transmitted to the CCD 22. The CCD 22 includes a plurality of imaging elements. These imaging elements are arranged horizontally and vertically in the shape of a matrix, and each imaging element constitutes one pixel.

[0074] Color filters are arranged on the light receiving face of the imaging element. The color filters are complementary Cy (Cyan), Mg (Magenta), Ye (Yellow) and G (Green) filters. The complementary filters on the CCD 22 are arranged as shown in FIG. 4. Of the primary three light colors, i.e., red (R), green (G) and blue (B), the color filters pass light having the following colors: Cy=G+B, Mg=B+r, and Ye=G+R. That is, one filter passes light for two colors.

[0075] The light entering the individual imaging elements of the CCD 22 is converted into an electric signal, and the electric signal is then output. Since the electric signal output by the CCD 22 is an analog signal, the A/D converter 23 converts the analog signal into digital data.

[0076] The recording unit 30 includes a RAM (Random Access Memory) 31 and a flash memory 32. A DRAM (Dynamic RAM) having a self-refresh function is used as the RAM 31. The flash memory 32 is a writable recording medium on which stored contents can be retained even in the non-active state, and is either incorporated into the digital camera 1, or is detachable from the digital camera 1.

[0077] The RAM 31 is used to temporarily store the digital data processed by the controller 10 or output from the A/D converter 23. The flash memory 32 is used to accumulate and store the digital data that is temporarily stored in the RAM 31. Further, filter information, which will be described later, is stored in the flash memory 32.

[0078] The display unit 40 includes a liquid crystal display device (LCD) 41 and a VRAM (Video RAM) 42. The LCD 41 is used to display an image based on digital data stored in the flash memory 32 or output by the A/D converter 23. The VRAM 42 is used to store display data that is created from the digital data in order to display an image on the LCD 41.

[0079] An interface 50 outputs digital data stored in the flash memory 32 to an external apparatus, such as a personal computer.

[0080] The detailed processing performed by the controller 10 will now be described.

[0081] As previously described, a processor 60 that is provided for the controller 10 is a dedicated arithmetic unit for performing image processing. The processor 60 performs predetermined processing using a computer program stored in the ROM 32.

[0082] The processing performed by means of the processor 60 will be described by referring to FIGS. 1 and 3. The pipeline processing is employed by means of the processor 60 without the CPU 11 using used.

Edge Information Calculation Unit

[0083] The edge information calculation unit 61 performs edge extraction processing for the digital data for an image output by the image input unit (S101). An extracted edge is employed for calculations performed to determine the slope and the direction of the edge.

[0084] For the extraction of the edge, a Prewitt operator is employed for obtaining the differential value of a pixel. The Prewitt operator is represented by the following equations. When the target pixel is defined as P(i, j), as is shown in FIG. 5,
Δfx={P(i+1, j−1)−P(i−1, j−1)}+{P(i+1, j)−P(i−1, j)}+{P(i+1, j+1)−P(i−1, j+1)}
Δfy={P(i−1, j+1)−P(i−1, j−1)}+{P(i, j+1)−P(i, j−1)}+{P(i+1, j+1)−P(i+1, j−1)}

[0085] In these equations, edge extraction is performed based on the digital data that is output from pixels neighboring the target pixel which constitute a matrix 80 (3×3 matrix around the target pixel P in this embodiment) and include the target pixel P in the center as shown in FIG. 5 and include the target pixel P in the center. During the edge extraction process, the edge elements in the directions X and Y, indicated by arrows, are detected. When the output is 8 bits, 256 tone data levels, 0 through 255, are output by the pixels as digital data.

[0086] Next, edge information, including the slope and the direction of the edge, is calculated using the edge extracted by the equations (S102). The grade grad of the slope of the edge is calculated using the following equation (A), and the direction θg of the slope of the edge is calculated using the following equation (B).
G(φ)=√{square root over (Δƒ_x²+Δƒ_y²)}  (A)

[0087] $\begin{array}{cc}{\Theta }_g={\tan }^{-1}\left(\frac{\Delta f_y}{\Delta f_x}\right)& \left(B\right)\end{array}$

[0088] Using the equations (A) and (B), the edge information calculation unit 61 calculates, as edge information, the grade grad and the direction θg of the slope of the edge.

[0089] Further, instead of equation (A), equation (A1) or (A2) may be employed to calculate the grade grad of the slope of the edge. When the equation (A1) or (A2) is employed, the grade grad of the slope of the edge can be quickly obtained.
grad(φ)=|Δf_x|+|Δf_y|  (A1)
grad(φ)=max(Δf_x, Δf_y)  (A2)

[0090] The edge is a portion whereat the tone of the digital data output by the pixel is greatly changed, i.e., the boundary portion between the objects included in the image. In this embodiment, the tone of the digital data output by the pixel is greatly changed on both sides of the edge. The change in the tone of the digital data is defined as the grade grad of the slope, and the direction in which the tone of the digital data is changed is defined as the direction θg of the slope. Therefore, as is shown in FIG. 6, the direction of the edge E is perpendicular to the direction θg of the slope of the edge.

Smoothing Strength Calculation Unit

[0091] Not only the edge information, but also the smoothing strength is calculated using the digital data output by the image input unit 20 (S103).

[0092] The smoothing strength calculation unit 62 calculates smoothing strength σ based on the information for luminance Y included in the digital data that is output by each pixel of the CCD 22. The smoothing strength σ is obtained by solving the following equation (C):

[0093] $\begin{array}{cc}\sigma =N\left(Y\right)={\left(\frac{eY}{n}\right)}^2\exp \left(-\frac{2Y}{n}\right)& \left(C\right)\end{array}$

[0094] In (C), e denotes a natural logarithmic number, and n denotes a luminance at which the noise reaches the maximum. The luminance n is changed in accordance with the imaging condition, and in this embodiment, n=50 when the output by the pixel is 8 bits, 256 tones.

[0095] The smoothing range is calculated using the smoothing strength σ obtained by (C). The Gaussian distribution, which will be described later, is employed to obtain the smoothing range.

Filter Information Selection Unit

[0096] Based on the edge information obtained by the edge information calculation unit 61 and the smoothing strength, the filter information selection unit 63 selects filter information stored in the flash memory 63. From among multiple sets of stored filter information, the filter information selection unit 63 selects the filter information that corresponds to the obtained edge information and the smoothing strength (S104).

[0097] The filter information is the one obtained by, based on the edge information, modifying a smoothing filter that is set using the Gaussian distribution. The principle will now be described by using an example. In this embodiment, a 5×5 matrix, including the target pixel P in the center, is defined as a smoothing range.

[0098] The Gaussian distribution is obtained by solving the following equation (D):

[0099] $\delta \left(x,y\right)=\frac{1}{2\pi {\sigma }^2}\exp \left(\frac{{n^2\left(x\cos {\theta }_g-y\sin {\theta }_g\right)}^2+{m^2\left(x\sin {\theta }_g+y\cos {\theta }_g\right)}^2}{-2{\sigma }^2}\right).$

[0100] Equat ion (D) represents a Gaussian distribution that is modified to describe an ellipse by scaling the short axial direction using 1/n and the long axial direction using 1/m, and is rotated to the right in a direction θg. That is, the elliptical shape is changed by m and n and the inclination of the ellipse is changed by θg. The m and n values are scaling parameters and depend on the grade grad of the slope obtained by equation (A).

[0101] As the grade grad of the slope of the edge is increased, the values of the scaling parameters m and n (mainly the value of n) are increased, and the Gaussian distribution shape is changed and describes a flat elliptical shape. Further, as the ellipse is rotated in accordance with the slope direction θg, the smoothing range A along the edge E is obtained, as is shown in FIG. 7. That is, when the values of the scaling parameters m and n are small, the smoothing range A1 describes a circle, as indicated by the broken line in FIG. 7. On the other hand, as the scaling parameters m and n, especially the value of n, are increased, the smoothing range A2 assumes a flat elliptical shape, as is indicated by the solid line in FIG. 7.

[0102] As a result of the calculation of the edge information for the target pixel P, it is found that the periphery of the target pixel P is a flat portion when the slope of the edge is small. Therefore, the digital data output by the target pixel P must be equally distributed throughout the peripheral pixels of the target pixel P. Thus, the smoothing range is shaped like a circle that includes the target pixel P in the center.

[0103] When the slope of the edge is large, the periphery of the target pixel P is an edge portion. Therefore, the digital data output by the target pixel P must be distributed throughout the peripheral pixels of the target pixel P along the edge. Thus, the smoothing range is shaped like an ellipse along the edge E along the target pixel P in the center. That is, as the slope of the edge is increased, the smoothing process must be performed along the edge E.

[0104] In this embodiment, the grade grad of the slope of the edge obtained by the edge information calculation unit 61 is correlated with the values of the scaling parameters m and n. Thus, when the grade grad of the slope is determined, scaling parameters m and n are determined that correspond to the grade grad of the slope.

[0105] FIG. 8 is a diagram showing the change in the smoothing range when the scaling parameter n is changed while the direction θg=45° of the slope and the smoothing strength σ=0.7 are maintained as an example. The matrix is not limited to the 5×5 dimension used in this embodiment, and can be an appropriately set dimension, such as 3×3 or 7×7.

[0106] It is apparent from FIG. 8 that when n=1 the smoothing range is uniform in all directions, and that as the value of n is increased, the axis along the direction θg of the slope is shortened and the smoothing range is concentrated along the edge E. Further, when the scaling parameter n is greater than n=3, the smoothing range is substantially unchanged. Therefore, when σ=0.7, the effective range of the scaling parameter n is from 1 to 3.

[0107] An explanation will now be given for the relationship between the smoothing strength and the scaling parameter. The smoothing range can be changed in accordance with the level of the smoothing strength σ using the equation (D) When the smoothing strength σ is small, the smoothing effect is small, and when the smoothing strength σ is large, the smoothing effect is great.

[0108] Since the smoothing range is increased as the smoothing strength σ becomes greater, the smoothing range must be reduced by increasing the scaling parameter n. Further, since the effects of smoothing the edge become too great if only the scaling parameter n in the shorter axial direction is changed, the scaling parameter m in the longer axial direction must also be changed.

[0109] FIG. 9 is a diagram showing filter information that provides the minimum smoothing effect and filter information that provides the maximum smoothing effect, which are obtained in case of changing the smoothing strength σ. The left column in FIG. 9 represents the filter information that provides the minimum smoothing effect, and the right column represents the filter information that provides the maximum smoothing effect.

[0110] Multiple filter information sets that correspond to the individual parameters are stored in the flash memory 32. These filter information sets include a smoothing range that is set by combining the scaling parameters m and n and the direction θg of the slope, which are determined by the grade grad of the slope, and the smoothing strength σ that is calculated using the luminance Y.

[0111] The filter information selection unit 63 reads out specific filter information from multiple filter information sets that are stored in the flash memory 32. That is, the values of the scaling parameters m and n are determined from the grade grad of the slope of the edge that is calculated by the edge information calculation unit 61. Then, based on the scaling parameters m and n that are thus determined, the direction θg of the slope, and the smoothing strength σ obtained by the smoothing strength calculation unit 62, the specific filter information is selected and is read out from the flash memory 32.

Smoothing Unit

[0112] When the filter information is selected by the filter information selection unit 63, the smoothing process is performed by using the digital data output by the target pixel P and the selected filter information (S105). The smoothing process is performed by multiplying the row data of the digital data output by the target pixel P by the numerical value of the filter information.

[0113] Assume that the scaling parameters m=1.25 and n=4 are determined from the grade grad of the slope obtained by the peripheral matrix 80 of the target pixel P, the inclination of the slope is θg=45°, and the smoothing strength σ=0.9 is obtained based on the luminance Y of the target pixel P. Then, filter information 90 shown in FIG. 10 is selected by the filter information selection unit 63. When the row data of the digital data output by the target pixel P is multiplied by the numerical value (the numerical value of a matrix/the sum) of the filter information in FIG. 8, the information for the target pixel P is distributed among specific pixels of the 5×5 matrix 80, including the target pixel P in the center. Since the information for the target pixel P is distributed among the specific pixels of the matrix 80, the smoothing process is performed for the digital data for the target pixel P.

[0114] This smoothing process is performed for all the pixels that constitute the image, i.e., all the pixels in the CCD 22. Since the smoothing process is performed for the 24 peripheral pixels of one pixel, the smoothing process is repeated 25 times for one pixel. The total of the data obtained by the smoothing process performed for the peripheral pixels becomes the digital data obtained by the smoothing.

Writing Unit

[0115] When the smoothing process is completed by the smoothing unit 64, the obtained digital data is written to the RAM 31 by the writing unit 65.

[0116] When the digital data for one image is written to the RAM 31 by the writing unit 65, the digital data stored in the RAM 31 is compressed in order to reduce the amount of data that is to be stored in the flash memory 32. When data for an image obtained by the digital camera 1 is to be compressed, a file form, such as JPEG (Joint Photographic Experts Group) or TIFF (Tagged Image File Format), is employed. The compressed digital data is recorded in the flash memory 32.

Operation of Digital Camera

[0117] The operation performed by the digital camera 1 according to the first embodiment will now be described.

[0118] (1) When the power switch (not shown) of the digital camera 1 is turned on, the digital camera 1 is set to the standby state so that the image pickup is always ready. At this time, the CCD 22 converts into an electric signal light that is condensed by the condenser 21 every tenth to several hundredths of a second. The obtained electric signal is then converted to digital data by the A/D converter 23. When a user employs the LCD 41 as a viewfinder, the digital data output by the A/D converter is transmitted to the VRAM 42, and the object is displayed on the LCD 41 as an animated picture.

[0119] (2) When the user depresses the shutter button 71 to the middle of the operating range and the shutter button 71 is set to the “half-depressed” state, the exposure and focus are set and fixed. The exposure fixed during the imaging can be changed when the CPU 11 of the controller 10 controls the aperture of the condensing lens 21 or the shutter speed, i.e., the charge accumulation period for the CCD 22. One of or both of a mechanical shutter for physically shielding light and a digital shutter for controlling the charge accumulation time for the CCD 22 are employed as the shutter of the digital camera 1.

[0120] (3) When the shutter button 71 is set to the “fully depressed state” in which the user depresses the button 71 to the limit of the operating range, the following process is performed. First, light measurement and focusing are precisely performed for the object. When the light measurement and the focusing have been completed, all the charges accumulated in the CCD 22 are temporarily discharged, and light from the object passes through the condenser 21 and enters the CCD 22. The CCD 22 then outputs an electric signal based on the strength of charges that correspond to the amount of incident light.

[0121] (4) The electric signal output by the CCD 22 is then converted into digital data by the A/D converter 23. And in order to increase the processing speed, DMA (Direct Memory Access) is employed to directly, temporarily store the digital data at a designated address in the RAM 31 without the data passing through the CPU 11 of the controller 10.

[0122] (5) The processor 60 performs the above processing for the digital data stored in the RAM 31, and regards the resultant data as digital data for an appropriate color image. In order to increase the number of images recorded in the flash memory 32, the obtained data is compressed to obtain digital data having a JPEG file form.

[0123] (6) After the digital data has been compressed, the digital data is copied from the RAM 31 to the flash memory 32 and is recorded therein.

[0124] As is described above, according to the image processing method performed by the digital camera 1 according to the first embodiment of the present invention, filter information is selected and read based on the scaling parameters n and m, which are determined by the grade grad of the slope of the edge that passes through the target pixel P, the direction θg of the slope of the edge, and the smoothing strength σ calculated using the digital data output by the target pixel P. Therefore, appropriate filter information corresponding to the edge can be selected without complicated conditional branching being required. In addition, the filter information is designated and stored in the flash memory 32, i.e., filter information appropriate for processing digital data output by the target pixel P is selected based on the obtained edge information and the smoothing strength σ. Therefore, the smoothing can be performed quickly using pipeline processing.

[0125] Thus, noise can be removed quickly, without the edge of an image being blurred or the noise present in a flat portion being emphasized.

[0126] Furthermore, the information for the smoothing range included in the filter information is so set that as the grade grad of the slope of the edge is increased, the smoothing range is changed from a circular into an elliptical shape. Thus, noise near the edge can be thoroughly removed without emphasizing noise present in the flat portion.

[0127] Further, when the filter information is selected using the edge information and the smoothing strength, a function for smoothing the data in accordance with the slope of the edge and for emphasizing the contour and a function for changing the smoothing range can be performed at the same time. Thus, a fast noise removal process can be performed.

[0128] A processor 100 for a digital camera according to a second embodiment of the present invention will now be described while referring to FIGS. 10 and 11. No explanation will be given for components and processing that are substantially the same as those in the first embodiment.

[0129] The second embodiment differs from the first embodiment in that filter information is generated each time the grade grad of the slope of the edge and the direction θg of the slope, both of which constitute the edge information.

[0130] As is shown in FIG. 11, the processor 100 for the digital camera in the second embodiment comprises: an edge information calculation unit 101, a smoothing strength calculation unit 102, a filter information generation unit 103, a smoothing unit 104 and writing unit 105.

[0131] The edge information calculation unit 101 and the smoothing strength calculation unit 102 are the same as those in the first embodiment. The filter information generation unit 103 generates filter information using equation (D), as explained in the first embodiment, based on the edge information obtained by the edge information calculation unit 101 and the smoothing strength obtained by the smoothing strength calculation unit 102. That is, each time the edge information and the smoothing strength are calculated based on the digital data output by the pixels that constitute an image, filter information is generated for the individual pixels.

[0132] In the second embodiment, the process (S201) for extracting an edge from the digital data output by the CCD 22, the process (S202) for calculating the edge information using the extracted edge and the process (S203) for calculating the smoothing strength are the same as those as in the first embodiment. In the second embodiment, the scaling parameters m and n are set based on the edge information and the smoothing strength obtained by the filter information generation unit 103. The filter information generation unit 103 generates filter information using equation (D) (S204). The smoothing unit 104 employs the filter information obtained by the filter information generation unit 103 to smooth the digital data output by the pixel (S205). And the smoothed digital data is stored by the writing unit 105 in the RAM 31 of the recording unit 30.

[0133] In the second embodiment, each time the edge information and the smoothing strength are calculated using the digital data output by the pixel, the scaling parameters m and n are designated, and filter information corresponding to each pixel is generated. Therefore, noise can be effectively removed without the edge being blurred.

[0134] In these embodiments of the invention, the luminance Y has been calculated after the edge information has been is obtained. However, the luminance Y may be calculated first, or the edge information and the luminance Y may be calculated in parallel, at the same time. Further, the smoothing range may be determined not only based on the luminance Y, but also based on the strength of the color element, such as RGB.

[0135] Furthermore, in the embodiments of the invention, the smoothing range is changed by altering the smoothing strength and the scaling parameters m and n. However, even when the smoothing strength φ is fixed, the smoothing range can be changed by changing the scaling parameters m and n. Therefore, when the value of the smoothing strength φ is fixed at a constant value, and the scaling parameters m and n are changed based on the grade and the direction of the slope of the edge, the filter information can more quickly be selected or generated.

[0136] Further, in the embodiments of the invention, the Prewitt operator is employed as a method for extracting the grade and the direction of the slope of the edge. However, another edge extraction method, such as the Sobel operator or the Kirsch operator, may be employed.

[0137] In addition, in the embodiments of the invention, a color image of 256 tones for which a complementary filter is used has been employed as an image to be input. However, a color image using the RGB primary color filter, a color image having 1024 or 4096 tones, a gray scale image or a binary monotone image may be employed as an input image.

[0138] Moreover, in the embodiments of the invention, an explanation is given for the processing of the digital data output by a CCD having a complementary filter. However, not only a complementary filter, but also a RGB primary filter can be employed.

[0139] In the embodiments of the invention, a digital camera is employed as an image processing apparatus. However, the present invention can be applied for image processing performed not only by a digital camera, but also by an image reading apparatus, such as a scanner or a copier, or for image processing for which software, such as a printer driver, is used.