Method and apparatus for expanding image data

[0002] 1. Field of the Invention

[0003] The present invention relates to a method and apparatus for expanding original image data to obtain expanded-image data composed of larger number of pixels than that of the original image data.

[0004] 2. Description of the Related Art

[0005] Conventionally, when expanding original image data to obtain expanded-image data composed of a larger number of pixels than that of the original image data, an interpolation processing, such as a linear interpolation, is executed. Thus, pixels are interpolated so that the expanded-image is generated. By performing the interpolation processing, a size of a digital image displayed on a display unit can be changed as required.

[0006] However, when performing the conventional interpolation processing including the linear interpolation, a variation of pixel value between pixels is not smooth, and the expanded-image data with a high-resolution can not be obtained. Namely, picture quality decreases in the process of expansion.

[0007] Therefore, an object of a present invention is to provide a method and apparatus for expanding original image data, while limiting the degradation of picture quality. An apparatus for expanding original image data according to the present invention expands the original image data. The original image data is arranged in a matrix and is composed of a determined number of pixels. The expanded-image data is partitioned into a plurality of blocks composed of a plurality of pixels. The apparatus has a magnifying power setting processor, a pixel arranger, a shifting processor, a magnifying power inspector and an expanded-image generating processor. The magnifying power setting processor sets a magnifying power regarding at least of a width direction and a length direction in the original image data. The pixel arranger arranges each pixel in said original image data at a position corresponding to a center position of each of said plurality of blocks in accordance with said magnifying power. The expanded-image generating processor generates the expanded-image data corresponding to the magnifying power by applying a fluency transform to each pixel in the original image data. Thus, the plurality of pixels in each block is generated at pixel generating positions. The magnifying power inspector inspects whether the magnifying power is odd numbered. Then, when the magnifying power is the odd numbered, the shifting processor relatively shifts the pixel generating positions with respect to each corresponding pixel of original image data by a shifting-amount, corresponding to the magnifying power, such that each arranged pixel of original image data is off-center with respect to the plurality of blocks. As the fluency transform is executed, the variation of pixel value between pixels becomes smooth, and the picture quality is maintained in the process of expansion processing. Further, the fluency transform is executed after the shifting process is performed when the magnifying power is odd numbered, so that the variation of pixel value between pixels becomes smooth.

[0008] The present invention will be better understood from the description of the preferred embodiment of the invention set forth below together with the accompanying drawings, in which:

[0009] FIG. 1 is a block diagram of an expanded-image generating apparatus of an embodiment.

[0010] FIG. 2 is a view showing an expansion processing.

[0011] FIGS. 3A, 3B, 3C and 3D are view showing fluency functions varying with parameter m.

[0012] FIG. 4 is a normalized fluency function at the parameter m=1.

[0013] FIG. 5 is a view showing an interpolation processing by a fluency transform.

[0014] FIG. 6 is a view showing the fluency transform along a horizontal direction.

[0015] FIG. 7 is a view showing a table T1 indicating 8 pixel values along the horizontal direction in a block.

[0016] FIG. 8 is a view showing the fluency transform along a vertical direction.

[0017] FIG. 9 is a view showing a table T2 representing 8 pixel values along the vertical direction in the block.

[0018] FIG. 10 is a view showing the fluency transform along the horizontal direction in the case in which the magnifying power is odd number.

[0019] FIG. 11 is a view showing a table T3 representing 3 pixel values along the vertical direction in the block.

[0020] FIG. 12 is a view showing pixel values along the horizontal direction in the block.

[0021] FIG. 13 is a view showing 3×3 pixel values including an unchanging pixel value in the block

[0022] FIG. 14 is a view showing a shift processing along the horizontal direction.

[0023] FIG. 15 is a view showing a table T4 representing pixel values obtained by shift processing.

[0024] FIG. 16 is a view showing a shift processing along the vertical direction.

[0025] FIG. 17 is a view showing a flowchart of the process for obtaining the expanded-image data.

[0026] Hereinafter, the preferred embodiment of the present invention is described with reference to the attached drawings.

[0027] FIG. 1 is a block diagram of an expanded-image generating apparatus of an embodiment of the present invention.

[0028] In a memory M1, original image data composed of luminance data Y and color difference data Cb, Cr is stored. In this embodiment, the original image data is obtained by being photographed by a digital still camera. The original image data is composed of a plurality of pixels, and the plurality of pixels are arranged in a matrix. The luminance data Y and color difference data Cb, Cr corresponding to one frame worth of the original image data are stored in an independent area of the memory M1, respectively. The luminance data Y and color difference data Cb, Cr are fed to an expanded-image generating apparatus 10, and are subjected to an expansion processing, separately.

[0029] The expanded-image generating apparatus 10 has a magnifying power setting processor 11, a shifting processor 12 and an expanded-image generating processor 13. Note that, a CPU (not shown) controls the expanded-image generating apparatus as a whole, and an input device (not shown) for selecting a magnifying power is provided. When the input device is operated, the magnifying power is set in the magnifying power setting processor 11. Note that, the magnifying power is an integer number.

[0030] In the shifting processor 12, an arrangement of each pixel in the original image data is performed in accordance with a value of the magnifying power. Further, in the shifting processor 12, a shift processing regarding a pixel position of arranged pixels of the original image data is executed when the magnifying power is odd numbered, on the other hand, the shift processing is not executed when the magnifying power is even numbered.

[0031] In the expanded-image generating processor 13, the fluency transform is executed to each pixel in the original image data, so that expanded-image data, which has a larger number of pixels than that of the original image data, is obtained. The expanded-image data is fed to a memory M2. The luminance data Y and color difference data Cb, Cr corresponding to one frame worth of the expanded-image data are fed to a display device (not shown), thus the expanded-image is displayed on the display device.

[0032] FIG. 2 is a view showing an expansion processing.

[0033] The original image data P is composed of “A×C” pixels, “A” pixels are arranged along a horizontal direction and “C” pixels are arranged along a vertical direction. A range of values of each pixel P_yxis 0 to 255. Note that, hereinafter, pixel values are also represented by “P_yx”. As shown in FIG. 2, x-y coordinates are defined with respect to the original image data P. A top-left corner of the original image data P is set to an origin, an x-axis is parallel to the horizontal direction and a y-axis is parallel to the vertical direction. A pixel P_tsshown in FIG. 2 is positioned at the s-th pixel along the x-axis and the t-th pixel along the y-axis.

[0034] Each pixel in the original image data P is subject to the fluency transform along the horizontal (x-axis) and vertical (y-axis) direction, in accordance with the magnifying power For example, the pixel P_tsis subject to the fluency transform along the horizontal direction, thus block BP composed of 3 pixels along the horizontal direction is generated. Then, the fluency transform along the vertical direction is executed to the 3 pixels obtained by the fluency transform along the horizontal direction, thus a block B0 composed of 3×3 (=9) pixels is generated. A size of the pixel J_yxis equal to that of the pixel P_yx.

[0035] When all pixels in the original image data P are subjected to the fluency transform, the expanded-image data J partitioned into a plurality of blocks B including the block B0, each of which is composed of 3×3 pixels, is generated. A position of the block B0 in the expanded-image data J corresponds to a position of the pixel P_tsin the original image data P, therefore, the block B0 is positioned at the s-th block along the horizontal direction and the t-th block along the vertical direction. In this way, the position of each of the blocks B in the expanded-image data J corresponds to the position of the corresponding pixel P_yxin the original image data P.

[0036] To indicate a position of a pixel in the block B0, an x′-axis and a y′-axis are defined with respect to the horizontal direction and the vertical direction respectively, and each pixel in the original image data P is represented by “I′_y′x′”. The pixel J_yxin the expanded-image data J and the pixel “I′_y′x′” satisfy the following formula.

[0037] J_yx=J_{t×3+y′,s×3+x′}=I′_y′x′  (1)

[0038] Each block B is a square block, and a pixel number along the x′-axis and the y′-axis are both of “3”. Therefore, a pixel number along the horizontal (width) direction and the vertical (length) direction in the original image data P are “3A” and “3C” respectively, and the pixel number of the expanded-image data J is “9×A×C”.

[0039] In this way, the original image data P composed of “A×C” pixels is transformed to the expanded-image data J composed of “3A×3C” pixels by the fluency transform. Note that, in this embodiment, the magnifying power means a ratio of the pixel number along the horizontal/vertical direction in the original image data P to the pixel number along the horizontal/vertical direction in the expanded-image data J. Therefore, a size of the expanded-image data J to a size of the original image data P is the square of the magnifying power. For example, in FIG. 2, the magnifying power is “3” and the expanded-image data has a size of 9 times of the original image data P.

[0040] Hereinafter, with reference to FIGS. 3 to 9, the fluency transform is explained. Since the fluency transform is based on a fluency function, the fluency function will be described before the explanation of the fluency transform.

[0041] The fluency function, named by professors Kamada and Toraichi, is known as a function, which represents various signals appropriately, for example, disclosed in a Mathematical Physics Journal (SURI-KAGAIU) No.363, pp8-12 (1993), published in Japan. To begin with, a fluency function space is defined as follows.

[0042] It is supposed that a function space composed of a staircase (scaling) function, which is obtained on the basis of a rectangular function represented by formula (2), is represented as shown in following formula (3), the fluency function space is defined by formula (4). $\begin{array}{cc}\chi \left(t\right)=\{\begin{array}{cc}1,& \left|t\right|\le 1/2\\ 0,& \mathrm{otherwise}\end{array}& \left(2\right)\\ {\hspace{0.17em}}^{1}S:=\left\{\begin{array}{cc}f:R\to C|f\left(t\right)=\sum _{n=-\infty }^{\infty }{b}_n\chi \left(t-n\right),& \left\{b_n\right\}\in l^2\end{array}\right\}& \left(3\right)\\ {\hspace{0.17em}}^{m}S:=\{\begin{array}{cc}\begin{array}{cc}h:R\to C|h\left(t\right)={\int }_{-\infty }^{\infty }f\left(t-\tau \right)g\left(\tau \right)d\tau ,& f\in {\hspace{0.17em}}^{m-1}S,\end{array}& g\in {\hspace{0.17em}}^{1}S\}\end{array}\left(m\ge 2\right)& \left(4\right)\end{array}$

[0043] The fluency function space of order m “^mS” is constructed by a function system, composed of an (m−2)-times continuously differentiable piecewise polynomial of degree m−1. The fluency function is a staircase (scaling) function when m is 1. The formula (2), which represents rectangular function, indicates a sampling basis at order m=1, and the formula (3) indicates the fluency function space at order m=1. The fluency function space ^mS is a series of a function space, which can connect from a staircase function space (m=1) to a Fourier band-limited function space (m=∞) Note that, a continuous differentiability m is regarded as a parameter.

[0044] The function system, characterizing the fluency function space ^mS and corresponding to an impulse response, is derived from a biorthogonal sampling basis theorem composed of a pair of a sampling basis and its biorthogonal basis. In this theorem, an arbitrary function fε^mS satisfies following formulae (5), (6) for a sampling value f_n·=f(n) $\begin{array}{cc}f\left(t\right)=\sum _{n=-\infty }^{\infty }{f}_{n\left[{\hspace{0.17em}}^{m}S\right]}\phi \left(t-n\right)& \left(5\right)\\ f_n={\int }_{-\infty }^{\infty }{f\left(t\right)}_{\stackrel{\_}{\left[{}^{m}S^{*}\right]}}\stackrel{\_}{\phi \left(t-n\right)}dt& \left(6\right)\end{array}$

[0045] where _[m]φε^mS satisfying ${\hspace{0.17em}}_{\left[{\hspace{0.17em}}^{m}S\right]}\phi \in {\hspace{0.17em}}^{m}S=\left(1/2\pi \right){\int }_{-\infty }^{\infty }{\left\{\sum _{q=-\infty }^{\infty }{\left[{\left(-1\right)}^q\left(1-q\left(2\pi /\omega \right)\right)\right]}^m\right\}}^{-1}\times \exp \left(i\omega t\right)d\omega$

[0046] on the other hand, _[mn]φε^mS satisfying ${\int }_{-\infty }^{\infty }{{\hspace{0.17em}}_{\left[{\hspace{0.17em}}^{m}S\right]}\phi \left(t-n\right)}_{\stackrel{\_}{\left[{}^{m}S^{*}\right]}}\stackrel{\_}{\phi \left(t-p\right)}dt=\{\begin{array}{c}1,\\ 0,\end{array}\begin{array}{c}p=n\\ p\ne n\end{array}$

[0047] The formula (5) indicates a function of the fluency transform derived from the sample value, and represents an expansion form, in which the sampling value sequences is a expansion coefficient. On the other hand, the formula (6) indicates a function of the fluency inverse transform derived from the function of the fluency transform, and represents an operator, which obtains the sampling value sequences from the function of the fluency transform in the form of a integral transform. Note that, p is an arbitrary integer, and a bar expressed above “φ” in the formula (6) represents a conjugated complex of φ.

[0048] Further, in the fluency function space ^mS, an orthogonal transform is derived from a fluency transform theorem. The orthogonal transform provide a generalization of a frequency concept in terms of agreement with Fourier transform at m=∞, and characterizes a harmonic structure. The theorem satisfies the following two formulae for an arbitrary function fε^mS. $\begin{array}{cc}f\left(t\right)=\left(1/2\pi \right){\int }_{-\pi }^{\pi }{F\left(u\right)}_{\left[{\hspace{0.17em}}^{m}O\right]}\phi \left(t,u\right)du& \left(7\right)\\ F\left(u\right)={\int }_{-\infty }^{\infty }{f\left(t\right)}_{\stackrel{\_}{\left[{\hspace{0.17em}}^{m}O\right]}}\stackrel{\_}{\phi \left(t-u\right)}dt& \left(8\right)\end{array}$

[0049] where $\begin{array}{c}{\hspace{0.17em}}_{\left[{\hspace{0.17em}}^{m}O\right]}\phi \left(t,u\right)=\left(1/2\pi \right){\int }_{-\infty }^{\infty }\left\{\sum _{p=-\infty }^{\infty }\delta \left(\omega -u+2\pi p\right)\right\}\times \\ {\left\{\sum _{q=-\infty }^{\infty }{\left(1-q\left(2\pi /\omega \right)\right)}^{-2m}\right\}}^{-1/2}\times \exp \left(i\omega t\right)d\omega \end{array}$

[0050] Note that, φ(u) is a function of the fluency function space ^mS and is expressed by using a Dirac delta function δ. “u” is an arbitrary variable.

[0051] The formula (7) is designated as a fluency orthogonal transform, and the formula (8) is designated as a fluency inverse orthogonal transform.

[0052] The fluency orthogonal transform expressed by the formula (7) can transform discrete sample values to continuous function values. Accordingly, in this embodiment, the fluency orthogonal transform is utilized for expanding the original image data. Namely, the fluency transform is executed to each pixel of the original image data P, and then the expanded-image data J is generated on the basis of the continuous function values.

[0053] Herein, some concrete fluency functions are explained. The order “m” of the fluency function space ^mS can be expressed as a parameter “m” of the fluency function, and when the parameter “m” is set to “1,2, . . . ” in order, the fluency function are represented as shown below.

[0054] The most simple function system in the fluency function is obtained by setting the function “f(t) ε¹S” in the formula (3) to the rectangular function, which is represented by “χ(t)” in the formula (2), in place of the staircase function, and setting the function gε¹S represented in the formula (4) to the above rectangular function “f(t)”. Namely, the function f(t) represented in the formula (3) becomes a rectangular function by applying a Delta (δ) function to an input function of the formula (3) in place of the rectangular function expressed by the formula (2). Then, the rectangular function f(t) is utilized for a function of the convolution integral in place of the function g(τ) represented in the formula (4), when transforming the function space ^m−1S to the function space ^mS.

[0055] An input value at the formula (3) becomes the δ function shown in FIG. 3A in place of the rectangular function χ(t) represented in the formula (2). The value of the δ function is 1 when variable t is τ, and is 0 when variable t is not τ. The fluency function f(t) with the parameter m=1 is represented as following formula, in place of the formula (3). $\begin{array}{cc}f\left(t\right)=\sum _{n=-\infty }^{\infty }{\chi }_n\chi \left(t-n\right)=\{\begin{array}{cc}1,& \tau -1/2<t<\tau +1/2\\ 0,& \mathrm{otherwise}\end{array}f\left(t\right)\in {\hspace{0.17em}}^{1}S& \left(9\right)\end{array}$

[0056] The fluency function f(t) in the formula (9) becomes a rectangular function, as shown in FIG. 3B. Then, the fluency function with the parameter m=2, denoted by “g(t)”, is found by executing the convolution integral on the basis of the rectangular function f(t), as shown in the following formula. $\begin{array}{cc}\begin{array}{c}g\left(t\right)={\int }_{-\infty }^{\infty }f\left(t-\tau \right)f\left(\tau \right)d\tau \\ =\{\begin{array}{cc}t-\tau +1,& \tau -1<t\le \tau \\ -t+\tau +1,& \tau <t<\tau +1\\ 0,& \mathrm{otherwise}\end{array}g\left(t\right)\in {\hspace{0.17em}}^{2}S\end{array}& \left(10\right)\end{array}$

[0057] The fluency function g(t) obtained by the formula (10) is, as shown in FIG. 3C, a triangular function.

[0058] When the parameter m is 3, 4, 5 . . . , the convolution integral is executed, similarly to the parameter m=2. Namely, based on the fluency function at the parameter “m−1” and the function f(t) represented in the formula (9), the convolution integral is executed, so that the fluency function with the parameter m is generated. For example, when the parameter m is 3, based on the function g(t) obtained by the formula (10) and the function f(t) in the formula (9), the convolution integral is executed, so that a fluency function h(t) shown in FIG. 3D, which is smooth and expressed by a curve, is generated. The fluency function h(t) is expressed as $\begin{array}{cc}\begin{array}{c}h\left(t\right)={\int }_{-\infty }^{\infty }g\left(t-\tau \right)f\left(\tau \right)d\tau h\left(t\right)\in {\hspace{0.17em}}^{3}S\\ =\{\begin{array}{cc}-\frac{1}{4}{\left(t-\tau +2\right)}^2,& \tau -2<t<\tau -\frac{3}{2}\\ \frac{3}{4}{\left(t-\tau +1\right)}^2+\frac{1}{2}\left(t-\tau +1\right),& \tau -\frac{3}{2}\le t<\tau -1\\ \frac{5}{4}{\left(t-\tau +1\right)}^2+\frac{1}{2}\left(t-\tau +1\right),& \tau -1\le t<\tau -\frac{1}{2}\\ -\frac{7}{4}{\left(t-\tau \right)}^2+1,& \tau -\frac{1}{2}\le t<\tau +\frac{1}{2}\\ \frac{5}{4}{\left(t-\tau -1\right)}^2-\frac{1}{2}\left(t-\tau -1\right),& \tau +\frac{1}{2}\le t<\tau +1\\ \frac{3}{4}{\left(t-\tau -1\right)}^2-\frac{1}{2}\left(t-\tau -1\right),& \tau +1\le t<\tau +\frac{3}{2}\\ -\frac{1}{4}{\left(t-\tau -2\right)}^2,& \tau +\frac{3}{2}\le t<\tau +2\\ 0,& \mathrm{otherwise}\end{array}\end{array}& \left(11\right)\end{array}$

[0059] In this way, the fluency function varies with the parameter m. The fluency functions, shown in FIGS. 3B to 3D, correspond to sampling bases regarding the fluency function space ^mS respectively, which are disclosed in the Mathematical Physical Journal described above in this embodiment, the fluency transform (orthogonal transform) to the original image data P is executed on the basis of the fluency function in the case of the parameter m=1, 2 and 3.

[0060] However, if the function f(t), obtained by the formula (9) and shown in FIG. 3B, is directly utilized at the convolution integral at the parameter m≧3, the value of the fluency function at t=τ does not coincide with 1. Accordingly, in this embodiment, normalized function shown in FIG. 4 is applied to the convolution integral in place of the function f(t) shown in FIG. 3B. The normalized function f(t) is normalized as for an area, which is formed based on the t-axis and the function f(t), such that the area of the function f(t) is always 1. For example, when the parameter m is 3, the normalized function f(t) is represented by $\begin{array}{cc}f\left(t\right)=\{\begin{array}{cc}-\frac{1}{3},& \tau -1<t\le \tau -\frac{1}{2}\\ \frac{4}{3},& \tau -\frac{1}{2}<t\le \tau +\frac{1}{2}\\ -\frac{1}{3},& \tau +\frac{1}{2}<t<\tau +1\\ 0,& \mathrm{otherwise}\end{array}& \left(12\right)\end{array}$

[0061] Thus, the value of the fluency function is always 1 at t=τ.

[0062] FIG. 5 is a view showing an interpolation processing by the fluency transform. Note that, for ease of explanation, the fluency transform is executed for only the horizontal direction, or x-axis. The magnification is “8”.

[0063] Herein, as one example, three pixels P_ts−1, P_ts, P_ts+1, adjoining each other in the original image data P, are subjected to the fluency transform by the formula (7) along the horizontal direction. Each pixel value of the three pixels P_ts−1, P_ts, P_ts+1is different respectively. The input function F(u) at the formula (7) corresponds to each pixel value of the three pixels P_ts−1, P_ts, P_ts+1.

[0064] When the input function F(u) at the formula (7) is a discrete value corresponding to δ function, the output function f(t) in the formula (7) corresponds to the fluency functions shown in FIGS. 3A to 3D. Therefore, by the fluency orthogonal transform, the output function f(t) having continuous values is generated. When the parameter m=1, an output function f_s, obtained by executing the fluency transform to the pixel value P_ts, is a rectangular function, as shown in FIG. 5. Each pixel value of 8 pixels along the horizontal direction (x-axis) in the block BP, represented by a bar “Q”, is equal.

[0065] Then, when the parameter m is 1, the range of the output function f_salong the x-axis corresponds to the range of the horizontal direction of the block BP (represented by “L” in FIG. 5). The range of the output functions f_s−1, f_s+1, obtained by executing the fluency transform to the pixel values P_ts−1, P_ts+1, does not overlap the range of the output function f_s.

[0066] On the other hand, when the parameter m is 2, each of the functions f_s−1, f_s, f_s+1, obtained by the formula (7) and corresponding to the pixel value P_ts−1, P_ts, P_ts+1respectively, is a triangular function. In the case of the parameter m=2, each range of functions f_s−1, f_s, f_s+1overlaps each other as the range L of the block BP is defined in accordance with the parameter m=1. The difference between the range of the function f(t) at the parameter m=2 and that of the function f(t) at the parameter m=2 is shown in FIGS. 3B and 3C.

[0067] Accordingly, each pixel value, represented by the bar Q, is obtained by adding each value of the functions f_s−1, f_s, f_s+1. Namely, each value of the functions f_s−1, f_s, f_s+1, corresponding to each position of 8 pixels in the horizontal direction of the block BP, is added, so that each pixel value of 8 pixels along the horizontal direction (x-axis) is obtained. For example, regarding the second pixel from the right end in the block BP, the pixel value Z3 is obtained by adding the pixel value Z2, which is the value of the function f_s, and the pixel value Z1, which is the value of the function f_s+1.

[0068] As the parameter “m” becomes large, the range of the function f(t) along the horizontal direction becomes larger, as shown in FIG. 3D. In this case, each pixel value of the block BP, corresponding to the pixel P_tsis calculated on the basis of the pixel value P_ts−1, P_ts+1, and other adjacent pixels in the original image data P.

[0069] After the fluency transform along the horizontal direction is executed, the fluency transform along the vertical direction (y-axis) is executed to the block BP, so that the block B0 composed of 8×8 pixels is generated. Namely, when executing the fluency transform along the x-axis and the y-axis for each pixel of the original image data P in order, the expanded-image data J is generated. Each pixel value of the expanded-image data J depends upon the parameter m.

[0070] With reference to FIGS. 6 to 9, a generation process of the expanded-image data J by the fluency transform is explained. Herein, the parameter m is one of 1 to 3, and adjacent pixels to the pixel P_ts, necessary for finding each pixel of the block BP, are represented by P_ts+2, P_ts−1, P_ts+1, P_ts+2, P_t−2s, P_t−1s, P_t+1s, P_t+2s. Note that, values of the pixels P_ts−2, P_ts−1, P_ts+1, P_ts+2, P_t−2s, P_t−1s, P_t+1s, P_t+2sare also expressed by “P_ts−2, P_ts−1, P_ts+1, P_ts+2, P_t−2s, P_t−1s, P_t+1s, P_t+2s”.

[0071] FIG. 6 is a view showing the fluency transform along the horizontal direction (x-axis). The positions of 8 pixels in the block BP are expressed by “0, 1, 2 . . . 7” in order.

[0072] Firstly, the pixel P_tsis arranged at the center of the generated block BP (between third position and fourth position), and then the fluency transform at the parameter m (=1, 2 or 3) is executed along the horizontal direction (x-axis). Similarly to the pixel P_ts, the pixels P_ts−2, P_ts−1, P_ts+1, P_ts+2, P_t−2s, P_t−1s, P_t+1s, P_t+2sare subject to the fluency transform in the horizontal direction, at the center of each block generated by the fluency transform. 8 pixels, represented by “I0, I1, I2, I3, I4, I5, I6, I7” in FIG. 6, are based on the pixels P_ts−2, P_ts−1, P_ts, P_ts+1, P_ts+2.

[0073] FIG. 7 is a view showing a table T1 indicating 8 pixel values along the horizontal direction in the block BP, obtained by the fluency transform. In the table T1, pixel values corresponding to the parameter 1, 2 and 3 respectively, are shown, and each pixel value is obtained on the basis of the formula (7). Herein, pixel values along horizontal direction are also represented by “I0, I1, I2, . . . I7”, which are identical to the 8 pixels along the horizontal direction.

[0074] For example, when the parameter m is 1, all of the pixel values I0 to I7 coincide with the pixel value P_ts. As described above using FIG. 5, the pixel values P_ts−2, P_ts−1, P_ts+1, P_ts+2except for the pixel value P_tsin the original image data P do not effect generation of the pixel values to I7 when the parameter m=1. On the other hand, when the parameter m is 2, as shown in FIG. 7, the pixel values I0 to I7 are generated on the basis of the pixel values P_ts−1, P_ts, P_ts+1and the addition of the value of the function f_s−1, f_s, f_s+1at the formula (7). Namely, the pixel values I0 to I7 are respectively generated by adding each value of the functions f_s−1, f_s, f_s+1, which corresponds to the pixel position, as shown in FIG. 5. When the parameter m is 3, the pixel values I0 to I7 are obtained on the basis of the pixel values P_ts−2, P_ts−1, P_ts, P_ts+1, P_ts+2. After the fluency transform in the horizontal direction is executed, secondly, the fluency transform in the vertical direction (y-axis) is executed to the block BP composed of 8 pixels I0 to I7.

[0075] FIG. 8 is a view showing the fluency transform along the vertical direction. Similarly to the horizontal direction, the positions of 8 pixels along the vertical direction are expressed by “0, 1, 2, . . . 7” in order.

[0076] When the fluency transform along the horizontal direction is executed to each pixel of the original image data P, blocks FP, GP, HP, KP, corresponding to the pixels P_t−2s, P_t−1s, P_t+1s, P_t+2srespectively, are obtained. Each pixel of the blocks FP, GP, HP, KP is designated by “f0, f1, . . . f7”, “g0, g1, . . . g7”, “h0, h1, . . . h7”, “k0, k1, . . . k7”, respectively. Note that, Each pixel of the blocks FP, GP, HP, KP is obtained on the basis of other adjacent pixels in the original image data P in addition to P_ts−2, P_ts−1, P_ts, P_ts+1, P_ts+2. For example, “f0, f1, . . . f7” of the block FP is obtained by “P_t−2s−2, P_t−2s−1, P_t−2s+1, P_t−2s+2.

[0077] The 8 pixels “I0, I1, I2, . . . I7” are subjected to the fluency transform along the vertical direction at the center of the block BP (between the third position and the fourth position), on the basis of the formula (7) The other pixels “f0, f1, . . . f7”, “g0, g1, . . . g7”, “h0, h1, . . . h7”, “k0, k1, . . . k7” are also subjected to the fluency transform along the vertical direction respectively, similarly to the 8 pixels I0 to I7. Then, 8×8 pixels of the block B0 are obtained by adding each value of the output values f(t) (at the formula (7)) based on the pixels “f0, f1, . . . f7” corresponding to the pixel positions “0, 1, 2, . . . 7”, the output functions f(t) based on the pixels “g0, g1, . . . g7” corresponding to the pixel positions “0, 1, 2, . . . 7”, the output values f(t) based on the pixels “I0, I1, . . . I7” corresponding to the pixel positions “0, 1, 2, . . . 7”, the output values f(t) based on the pixels “h0, h1, . . . h7” corresponding to the pixel positions “0, 1, 2, . . . 7” and the output values f(t) based on the pixels “k0, k1, . . . k7” corresponding to the pixel positions “0, 1, 2, . . . 7”. Herein, the pixel value of the block B0 is denoted by I′_y′x′ (0≦x′≦7, 0≦y′≦7).

[0078] FIG. 9 is a view showing a table T2 representing 8 pixel values I′_y′7(0≦y≦7), shown by diagonal line in FIG. 8. When the parameter m is 1, all of the pixel values I′_y77are I7, when the parameter m is 2, the pixel values I′_y7are obtained on the basis of the pixel values I7, g7, h7. When the parameter m is 3, the pixel values I′_y7are obtained on the basis of the pixel values f7, g7, I7, h7, k7. Other pixels I_y′x′ (0≦x′≦6, 0≦y′≦7) are also obtained on the basis of the pixels fx′, gx′, Ix′, hx′, kx′ (x′=0 to 6). In this case, the pixel values I_y′x′ (0≦x′6, 0≦y≦7) are obtained by substituting fx′, gx′, Ix′, hx′, kx′ (x′=0 to 6) for f7, g7, I7, h7, k7 represented in FIG. 9.

[0079] In this way, the block B0 corresponding to the pixel P_tsin the original image data P is generated by executing the fluency transform along the horizontal and the vertical direction. The fluency transform is executed for each pixel of the original image data P at the same time, so that the blocks B including the block B0 is generated, namely, the expanded-image data J is obtained. Note that, the range of the parameter m is not restricted 1 to 3, and may be set to an arbitrary range (for example, 1 to 5). In this embodiment, the parameter m is more than 2 so as to make the variation of pixel values smooth.

[0080] With reference to FIGS. 10 to 16, the shift processing is explained below. In FIGS. 10 to 12, the fluency transform without the shift processing in a case in which the magnifying power is odd numbered is shown, and in FIGS. 13 to 16, the fluency transform with the shift processing is shown.

[0081] FIG. 10 is a view showing the fluency transform along the horizontal direction. Note that, the magnifying power along the horizontal (or vertical) direction is 3. When the fluency transform is executed, as described above, each pixel P_yxin the original image data P is positioned at the center of the corresponding block. Therefore, when the magnifying power is 3 times, the pixel P_tsis arranged at the pixel position “1” along the horizontal direction. Similarly, the other pixels P_t−2s, P_t−1s, P_ts+1, P_ts+2are arranged at the center of the corresponding block.

[0082] FIG. 11 is a view showing a table T3 indicating 3 pixel values along the horizontal direction. As shown in the table T3, even when the parameter m=2 and 3, the pixel values P_tsdirectly become the pixel value I1. This is because, regarding the functions f(t) (at the formula (7)) based on the pixel P_ts−2, P_ts−1, P_{ts +1}, P_ts+2, a value corresponding to the pixel position “1” in the block BP, are all “0”. Other pixel values P_ts−2, P_ts−1, P_ts+1, P_ts+2also become pixel values at the center of the corresponding block respectively, similarly to the pixel P_ts. In FIG. 12, the pixel values along the horizontal direction, based on the parameter m=2, are shown with the function f_s−1, f_s, f_s+1, obtained by the formula (7).

[0083] FIG. 13 is a view showing the fluency transform along the vertical direction. As shown in FIG. 13, the block B0 is generated by executing the fluency transform along the vertical direction. The pixel value I′₁₁is equal to the pixel value I1, or the pixel value P_ts. Then, the pixel values I′₁₀, I′₁₂become equal to the pixel value I0, I2 respectively.

[0084] In this way, when the fluency transform is executed in a state in which the magnifying power is odd numbered, each pixel value P_yxin the original image data P becomes equal to the pixel value at the center of corresponding block B. Consequently, the pixel values equal to the pixel values P_yxexist in the expanded-image data J. Herein, the above pixel values are represented as “unchanging pixel vales”. It is due to a characteristic of the fluency transform, in other words, the fluency function shown in FIGS. 3A to 3D that unchanging pixel values occur.

[0085] Incidentally, the expandad-image data J is generated by interpolating pixels between pixels in the original image data P Accordingly, as a variation of the pixel value between pixels is smaller, namely, the pixel values vary smoothly between adjacent pixels, picture quality in the expanded-image data J is less degraded

[0086] However, when the unchanging pixel value exists in the expanded-image data J, the variation of the pixel value is not as smooth as compared with that of the expanded-image data J in which no unchanging pixel value exists. For example, the variation among the pixels I0, I1 and I2, shown in the table T2 in FIG. 11, is not as smooth compared with the variation of the pixel value among the pixels, I2, I3, I4 and I5 shown in the table T1 in FIG. 7.

[0087] Further, regarding the vertical direction, a similar problem occurs. For example, the pixel value I′₁₀becomes equal to the pixel value I0, so that the variation of the pixel values along the vertical direction is not as smooth similarly to the horizontal direction.

[0088] In this way, picture quality in the expanded-image data J in a case where the magnifying power is odd numbered decreases in comparison with the expanded image data J in the case where the magnifying power is even numbered. Accordingly, in this embodiment, position-relationship between the pixel-position of each block B and each corresponding pixel P_yxof the original image data P is changed, so as not to generate the unchanging pixel values, when the magnifying power is odd numbered.

[0089] FIG. 14 is a view showing the shift processing along the horizontal direction.

[0090] When the magnifying power is odd numbered, the position of each pixel P_yxin the original image data P is relatively shifted from the center position of the corresponding block by a shifting-amount ΔS along the horizontal and left direction, such that each pixel P_yxare arranged between the first pixel position and the second pixel position In the block BP. For example, the pixel P_tsis relatively shifted from the center position of the block to a position between a pixel position “0” and a pixel position “1”. When it is supposed that a magnitude of width/length of the each block B is “M”, the shifting-amount ΔS is “M/6”.

[0091] FIG. 15 is a view showing a table T4 indicating 3 pixel values along the horizontal direction “I0, I1 and I2”. The 3 pixel values are obtained by applying the fluency transform along the horizontal direction after the shift processing along the horizontal direction is executed. As can be seen in table T4, in comparison with the table T3 in FIG. 11, no unchanging pixel values (=P_ts) exist when the shift processing is executed. This is because, regarding the functions f(t) (at the formula (7)) based on the pixel P_ts−2, P_ts−1, P_ts+1, P_ts+2, a value of “0” in the block BP, is relatively shifted to between the pixel position “0” and the pixel position “1”.

[0092] FIG. 16 is view showing a shift processing executed at the fluency transform along the vertical direction. Note that, only pixels f0, g0, I0, h0, k0, positioned at “0” along the horizontal direction, are shown for explanation of the shift processing.

[0093] When the fluency transform along the vertical direction is executed, the pixels, generated by the fluency transform along the horizontal direction, are relatively shifted along the vertical and upper direction by the shifting-amount “Δs”. As the block is the square block, the shifting amount “ΔS” is “M/6”. For example, the pixel I0 is relatively shifted from the pixel position “1” to a position between the pixel-position “0” and the pixel-position “1”.

[0094] In this way, as the shift processing along the horizontal direction by “1/2M” is executed before the fluency transform along the horizontal direction, using the formula (7), and the shift processing along the vertical direction by “1/2M” is executed before the fluency transform along the vertical direction using the formula (7), no unchanging pixel values exist in the expanded-image data J, so that the variation of the pixel value becomes smooth.

[0095] Note that, in FIGS. 14 to 16, for ease of explanation, each pixel P_yxis relatively shifted with respect to each corresponding pixel-position of each block B in the generated expanded-image data J. In fact, when the shift processing is executed by the shifting processor 12, the pixel-positions of the generated expanded-image data J are relatively shifted with respect to each corresponding pixel P_yxin place of shifting each pixel P_yx. To be precise, when the fluency transform along the horizontal direction shown in FIG. 14 is executed, the pixel-positions “0 to 2” along the horizontal direction in the block BP are relatively shifted along the horizontal and right direction such that the pixel P_tsis between the pixel-position “0” and the pixel-position “1”. Other blocks are similar to the block BP. Then, when the fluency transform along the vertical direction shown in FIG. 16 is executed, the pixel-positions “0 to 2” along the vertical direction in the block B0 are relatively shifted along the vertical and lower direction such that each of the pixels I0 to I3 is between the pixel-position “0” and the pixel-position “1”. Other blocks are similar to the block B0. Hereinafter, the pixel-positions are represented by “pixel generating positions”.

[0096] FIG. 17 is a view showing a flowchart of the expansion processing. The size of the expanded-image data J is Z²times of the original image data P. Note that, herein the magnifying power is represented by “Z”.

[0097] In Step 101, each pixel P_yxis arranged at a position corresponding to the center position of each block. In Step 102, it is determined whether the magnifying power Z set in advance is odd numbered.

[0098] When it is determined that the magnifying power Z is odd numbered at Step 102, the process goes to Step 103. In Step 103, each pixel P_yxin the original image data P is shifted along the horizontal and left direction by the shifting-amount “ΔS”. In other words, the pixel generating positions are relatively shifted along the horizontal and right direction with respect to each corresponding pixel P_yx. As the magnitude of the width/length of each block B in the expanded-image data J is represented by “M” as described above, the shifting-amount ΔS becomes “M/2Z”. After the shirt processing along the horizontal direction is executed, the process goes to Step 104.

[0099] In Step 104, the fluency transform along the horizontal direction is executed to each shifted pixel P_yx, so that pixels along the horizontal direction, a number of which is Z times of “A”, is generated. After Step 104 is executed, the process goes to Step 105.

[0100] In Step 105, the pixels generated by the fluency transform along the horizontal direction are relatively shifted along the vertical and upper direction by the shifting-amount “ΔS (=M/2Z)”. In other words, the pixel generating positions are relatively shifted along the vertical and lower direction with respect to corresponding pixels, generated by the fluency transform along the horizontal direction Then, in Step 106, the fluency transform along the vertical direction is executed, so that the expanded-image data J is generated.

[0101] On the other hand, when it is determined that the magnifying power is not odd numbered, namely, is even numbered, the process goes to Step 107. The process in Step 107 is similar to the process in Step 103, and the process in Step 108 is similar to the process in Step 105. Namely, the fluency transform along the horizontal and vertical direction is directly executed to each pixel in the original image data P, so that the expanded-image data J is generated. After Step 108 is executed, the expanded-image generating process is terminated.

[0102] In this way, in this embodiment, the expanded-image data J is obtained by applying the fluency transform to the original image data P. Consequently, the variation of the pixel values between each pixel in the expanded-image data J becomes smooth, namely, the picture quality can be maintained in the process of expansion processing.

[0103] When the magnifying power is odd numbered, firstly, the pixel generating positions along the horizontal direction, corresponding to the pixel-position of the generated expanded-image data J, is relatively shifted along the horizontal direction by the shifting-amount “ΔS (=M/2Z)” at the fluency transform along the horizontal direction. Then, the pixel generating positions along the vertical direction are relatively shifted along the vertical direction by the shifting-amount “ΔS (=M/2Z)” with respect to each of the pixels aligned along the horizontal direction, obtained by the fluency transform along the horizontal direction. Thus, no unchanging pixel values exist in the expanded-image data J, the variation of pixel values between each pixel becomes smooth even when the magnifying power is odd numbered.

[0104] Note that, the fluency transform may be executed along the vertical direction and along the horizontal direction in order. Further, the original image data P may be expanded only along the horizontal (width) direction or the vertical (length) direction. In this case, the pixel generating positions along the horizontal direction or the vertical direction are shifted relative to each corresponding pixel P_yxsuch that each pixel P_yxis off-center of each corresponding block of the generated expanded-image data J.

[0105] Finally, it will be understood by those skilled in the art that the foregoing description is of preferred embodiments of the device, and that various changes and modifications may be made to the present invention without departing from the spirit and scope thereof.

[0106] The present disclosure relates to subject matters contained in Japanese Patent Application No.11-235960 (filed on Aug. 23, 1999) which is expressly incorporated herein, by reference, in its entirety.