Watermarking methods for digital images and videos

[0002] This application claims the benefit of Provisional application Ser. No. 60/087,189, filed May 29, 1998.

[0003] 1. Field of the Invention

[0004] The present invention relates to watermarking methods for digital images and videos. More particularly, this invention relates to watermarking methods in the wavelet transform domain.

[0005] 2. Description of the Prior Art

[0006] With the rapid development of information technology, electronic publishing, such as the distribution of digitized images/videos, is becoming more popular. One of the more important issues for the electronic publisher is the ability to obtain and enforce copyright protection. Watermarking is one of the common copyright protection methods that has recently received considerable attention. Watermarking for digital images basically consists of signing an image with a signature or copyright message such that the message is secretly embedded in the image and there is no visible difference between the original and the signed images.

[0007] There are two common methods of watermarking: frequency domain and spatial domain watermarks. The present invention focuses on the frequency domain watermarks. Conventional frequency domain watermarking methods are based on the discrete cosine transform (hereinafter “DCT”), where pseudo-random sequences, such as M-sequences, are added to the DCT coefficients at the middle frequencies as signatures. This approach, of course, matches the current image/video compression standards well, such as JPEG, MPEG1-2, and the like.

[0008] Moreover, it is known that wavelet image/video coding, such as embedded zero-tree wavelet (hereinafter “EZW”) coding, has potential to be included in future image/video compression standards, such as JPEG2000 and MPEG4, due to its excellent performance in compression. The basics of EZW can be found in J. Shapiro, “Embedded image coding using zerotrees of wavelet coefficients”, IEE Trans. On Signal Processing,Vol. 41, pages 3445-3462, December, 1993, which is hereby incorporated in its entirety by reference. The basic idea of EZW is to keep large coefficients with a quantization and throw away small coefficients in the DWT transform domain with a tree-structured addressing. The tree-structured addressing is due to the DWT pyramid decomposition as shown in FIG. 3. Consequently, it is important to study watermarking methods in the wavelet transform domain.

[0009] Discrete wavelet transform (hereinafter “DWT”) has been extensively studied in the last decade. There are numerous applications of wavelet transforms such as compression, detection, and communications. Basically, in the DWT for a one dimensional signal, a signal is split into two parts, one having high frequencies and one having low frequencies. The part with the high frequencies basically contains the edge components of the signal, while the part with the low frequencies basically contains the smooth areas of the signal. The part with low frequencies is split again into two parts of high and low frequencies. This process is continued an arbitrary number of times, which is usually determined by the application at hand. Furthermore, from these DWT coefficients, the original signal can be reconstructed. This process is called the inverse DWT (hereinafter “IDWT”).

[0010] The DWT and IDWT can be mathematically stated as follows:

[0011] Let H(w)=Σhke^−jkw, and G(w)=Σgke^−jkw, represent a lowpass filter and a highpass filter, respectively, which will satisfy certain conditions for reconstruction to be stated later. A signal x[n] can be decomposed recursively as shown in equations (1) and (2): $\begin{array}{cc}{c}_{j-1,k}=\sum _n{h}_{n-2k}{c}_{j,n};& \left(1\right)\\ d_{j-1,k}=\sum _n{g}_{n-2k}{c}_{j,n}& \left(2\right)\end{array}$

[0012] for j=J+1, J, . . . , J₀where c_j+_l+k=x[k], k ε Z, J+1 is the high resolution level index and J₀is the low resolution level index. The coefficients c_J0,k, d_J0,k, d_J0+l,k, . . . , d_J,kare called the DWT of signal x[n] where c_J0,kis the lowest resolution part of x[n] and d_j,fare the details of x[n] at various bands of frequencies. Furthermore, the signal x[n] can be reconstructed from its DWT coefficients recursively as shown in equation (3): $\begin{array}{cc}{c}_{j,n}=\sum _k{h}_{n-2k}{c}_{j-1,k}+\sum _k{g}_{n-2k}{d}_{j-1,k}.& \left(3\right)\end{array}$

[0013] The above reconstruction is called the IDWT of x[n]. To ensure the above relationship between IDWT and DWT, the following orthogonal condition on the filters H(w) and G(w) is needed:

[0014] |H(w)|²+|G(w)|²=1.

[0015] Examples of H(w) and G(w) are given by: $H\left(\omega \right)=\frac{1}{2}+\frac{1}{2}{e}^{-\mathrm{j\omega }}.\mathrm{and}G\left(\omega \right)=\frac{1}{2}-\frac{1}{2}{e}^{-\mathrm{j\omega }},$

[0016] and are known as the Haar wavelet filters.

[0017] The above DWT and IDWT for a one dimensional signal x[n] can be also described via two channel tree structured filterbanks as can be seen in FIG. 1.

[0018] The DWT and IDWT for two dimensional images x[m, n] can be similarly defined by implementing the one dimensional DWT and IDWT for each dimension m and n separately: DWT_n[DWT_m[x[m,n]]], as can be seen in FIG. 2.

[0019] An image can be decomposed into a pyramid structure, as shown in FIG. 3, and provide various band information, for instance, the low-low, low-high, and-high-high frequency bands. An example of this kind of decomposition with two levels is shown in FIG. 4, where the edges appear in all bands, except in the lowest frequency band, i.e., the corner part at the left and top.

[0020] Conventional watermarking methods have several limitations upon which the present invention seeks to improve. For instance, current watermarking methods for digital images and videos, such as the discrete cosine transform (hereinafter “DCT”) based approach, are not very robust, do not have multiresolution characteristics, and are not hierarchical in structure, which results in, among other things, high computer loads for distorted images. The present invention solves these problems.

[0021] It is, therefore, an object of the present invention to develop a watermarking method for digital images and videos which is based on DWT, and overcomes the limitations of conventional watermarking methods.

[0022] It is a further object of the present invention to develop a DWT based watermark which is more robust than conventional methods for common image distortions.

[0023] It is yet further another object of the present invention to develop a DWT based watermark which has multi-resolution characteristics.

[0024] It is also an object of the present invention to develop a DWT based watermark which is hierarchical and provides reduced computational loads.

[0025] These and other objects of the present invention can be better appreciated by referring to the following description and claims taken in conjunction with the accompanying drawings.

[0026] The present invention relates to a method of multiresolution watermarking for digital images, which utilizes wavelet transform based watermarking techniques by adding pseudo-random codes (or sequences) to large coefficients located at the high and middle frequency bands of the DWT of the digital image. A peak is then detected to signify a signature of the watermark.

[0027] There are three principal advantages with this approach. First, the watermarking method has multiresolution characteristics and is hierarchical. In the case when a received image is not distorted significantly, the cross correlations with the whole size of the image may not be necessary, and therefore, much of the computational load can be saved. Second, the human eyes are generally insensitive to small changes in the edges and textures of an image, but are very sensitive to small changes in the smooth parts of an image. With the DWT, the edges and textures are usually exploited very well in high frequency subands, such as HH, LH, HL, and the like. The large coefficients in these bands usually indicate edges in an image. Therefore, adding watermarks on these large coefficients would be difficult for the human eyes to perceive. Finally, the third advantage is that this approach matches the emerging image/video compression standards. Numerical results show that the watermarking method of the present invention is very robust to wavelet transform based image compressions, such as the EZW image compression scheme, as well as to other common image distortions, such as additive noise, halftoning, and rescaling.

[0028] FIG. 1 depicts a prior art flowchart of the DWT for one dimensional signals.

[0029] FIG. 2 depicts a prior art flowchart of the DWT for two dimensional signals.

[0030] FIG. 3 depicts a prior art DWT pyramid decomposition of an image.

[0031] FIG. 4 depicts another prior art DWT pyramid decomposition of an image which shows only two band levels.

[0032] FIGS. 5aand 5billustrate the encoding and decoding parts, respectively, of the inventive watermarking process in the DWT domain.

[0033] FIGS. 6a, 6b, 6c,and 6dshow cross correlations for watermark detections with low and high additive noises.

[0034] FIGS. 7aand 7bshow cross correlations for watermark detections with compressed images.

[0035] FIGS. 8aand 8bshow cross correlations for watermark detections with half toned images.

[0036] FIGS. 9aand 9bshow cross-correlations for watermark detections with rescaled images.

[0037] Watermarking in the DWT domain includes two parts: encoding and decoding. In the encoding part, an image is first decomposed into several bands with a pyramid structure as shown in FIGS. 3 and 4, and then pseudo-random sequences (e.g., Gaussian noise) are added to large coefficients located at high and middle frequency bands of the DWT, that is, to the large coefficients which are not located at the lowest resolution, for example, at the top left hand corner. Mathematical analysis of the encoding part lets y[m,n] denote the DWT coefficients which are not located at the lowest frequency band of an image x[n,m]. A Gaussian noise N[m,n] is added with a mean of 0 and a variance of 1 to y[m,n]. The resulting mathematical expression is equation (4):

[0038] {overscore (y)}[m,n]=y[m,n]+α(y[m,n])².V[m,n],

[0039] where ∝ is a parameter to control the level of the watermark and the squared y[m,n] indicates the amplification of the large DWT coefficients. The DWT coefficients at the lowest resolution are unchanged. Then, take the two dimensional IDWT of the modified DWT coefficients {overscore (y)} and the unchanged DWT coefficients at the lowest resolution and let {overscore (I)}[m,n] denote the IDWT coefficients. For the resultant image to fit within the 0 to 255 integer values, which is typical image data, it is modified as shown in equation (5): $\hat{x}\left[m,n\right]=\left[255\frac{\tilde{x}\left[m,n\right]-{\min }_{m,n}\left(\tilde{x}\left[m,n\right]\right)}{{\max }_{m,n}\left(\tilde{x}\left[m,n\right]-{\min }_{m,n}\left(\tilde{x}\left[m,n\right]\right)\right)}\right].$

[0040] The operation of equation (5) converts the two dimensional data {overscore (I)}[m,n] into an 8-bit level image. The resultant image {overscore (I)}[m,n] is the watermarked image of x[m,n]. The encoding part of the present invention is illustrated in FIG. 5(a).

[0041] The decoding part of the present invention is hierarchical and described as follows. First, a received image and the original image (it is assumed that the original image is known) is decomposed with DWT into four bands, for instance, low-low (LL₁) band, low-high (LH₁) band, high-low (HL₁) band, and high-high (HH₁) band, respectively. Then, the signature added in the HH₁band and the difference of the DWT coefficients in the HH₁bands of the received and original images are compared by calculating their cross correlations. If there is a peak in the cross correlations, the signature is determined to be detected. Otherwise, the signature added in the HH₁and LH₁bands is compared with the difference of the DWT coefficients in the HH₁and LH₁bands, respectively. If there is a peak, the signature is detected. Otherwise, one considers the signature added in the HL₁, LH₁, and HH₁bands. If there is still no peak in the cross correlations, one continues to decompose the original and the received signals in the LL₁band into four additional subbands, LL₂, LH₂, HL₂, and HH₂, and so on until a peak appears in the cross correlations. If numerous attempts (e.g., 6 times) to split the signals into subbands fails to produce a peak, then the signature is determined to be undetectable. The decoding part of the present invention is illustrated in FIG. 5(b).

[0042] The following example will further illustrate the present invention:

[0043] Consider the image of peppers with a size of 512×512. Two watermarking methods are implemented: one using the DCT approach and the other using the DWT approach taught by the present invention. The DWT approach advantageously includes the Haar DWT wavelet filters. A two step DWT is implemented and images are decomposed into seven (7) subbands. Watermarks (e.g., Gaussian noise) are added to six (6) subbands, but not to the seventh or lowest subband (the lowest frequency components). In the DCT approach, watermarks (e.g., Gaussian noise) are added to the DCT coefficients at the same positions as the ones in the above DWT approach. The levels of watermarks in the DWT and DCT approaches are the same.

[0044] The digital image is pyramidally decomposed and distorted. Four distortions are tested to identify and describe the watermarked image. The first distortion to test with the present invention's algorithm is additive noise. When the variance of the additive noise is not too large, the signature can be detected only by using the information in the HH₁band with the DWT approach. FIG. 6(a) shows the cross correlations for watermark detection for the DWT method with HH₁band and low additive noise; a peak can be clearly seen. In contrast, when the variance of the additive noise is large, the HH₁band information is not good enough with the DWT approach. FIG. 6(b) shows the cross correlations for watermark detection for the DWT method with HH₁band and high additive noise; no clear peak can be seen. However, the signature can still be detected by using the information in the HH₁and LH₁bands with the DWT approach. FIG. 6(d) shows the cross correlations for watermark detection for the DWT method with HH₁and LH₁bands and high additive noise; a peak can be clearly seen. Finally, the DCT approach was also implemented for the image with high additive noise. In this case, the signature with the DCT approach cannot be detected. FIG. 6(c) shows the cross correlations for watermark, detection for the DCT method for high additive noise; no peak can be clearly seen.

[0045] The second distortion to test is image compression. The two watermarked images with the DWT and DCT approaches are compressed by using the EZW coding algorithm scheme discussed in the Shapiro reference, supra. It is advantageous to include EZW wavelet coding in the present invention. The compression ratio is chosen as 64, for example, 0.125 bpp. The cross correlations for watermark detection for these two compressed images are shown in FIGS. 7(a) and 7(b), where FIG. 7(a) corresponds to the DWT approach and FIG. 7(b) corresponds to the DCT approach. A peak in the center of the drawing can be clearly seen in FIG. 7(a) with the DWT approach, but no clear peaks can be seen in FIG. 7(b) with the DCT approach.

[0046] The third distortion to test is the halftoning. The two watermarked images are both halftoned by rising the following standard method. Let x[m,n] be an image with 8-bit levels. To halftone it, execute nonuniform thresholding through the Bayer's dither matrix T: ${T_P\left(T_{j,k}\right)}_{4n-1}=14(\begin{array}{cccc}11& 7& 10& 6\\ 3& 15& 2& 14\\ 9& 5& 12& 4\\ 1& 13& 4& 14\end{array})$

[0047] Compare each disjointed 4×4 block in the image of x[m,n]. If x[m*4+j,n*4+k]≧T_j,k, then it is quantized to 1. Otherwise, it is quantized to 0. Both DWT and DCT watermarking methods were tested. Surprisingly, it was found that the watermarking method based on DWT as proposed by the present invention is more robust than the method based on DCT. The cross correlations for watermark detection for halftoned images are shown in FIGS. 8(a) and 8(b), where FIG. 8(a) corresponds to the DWT approach, while FIG. 8(b) corresponds to the DCT approach. One can clearly see a peak in the center of FIG. 8(a), while no clear peak can be seen in the middle of FIG. 8(b).

[0048] The last distortion to test is image scaling. The resolutions of the watermarked images are reduced by averaging four neighboring samples together and then down sampling them by 2 at each dimension. With this kind of distortion, the watermark method using DWT is more robust than the watermarking method using DCT. The cross correlations for watermark detection for rescaled images are shown in FIGS. 9(a) and 9(b), the former figure representing the DWT approach and the latter figure representing the DCT approach. One can clearly see the peak in the center of FIG. 9(a) for the DWT approach, but no peak can be seen in the center of FIG. 9(b) for the DCT approach.

[0049] The present invention has introduced new and improved multiresolution watermarking methods using a discrete wavelet transform (DWT). The watermarking method taught herein adds Gaussian random noise to the large coefficients in the DWT domain. The decoding is hierarchical. If distortion of a watermarked image is not serious, then only a few bands of information will be sufficient to detect the signature, which would result in a reduction of the required computational load. The examples illustrated herein show several kinds of distortions, such as additive noise, compressed imaging (with the wavelet approach, such as EZW), halftoning, rescaled imaging, and reduced resolution, all of which are applicable to the present invention. It has been found that the DWT based watermark approach of the present invention is robust to all of the above distortions, while the DCT approach is not very robust. In particular, the DCT approach is not robust with respect to the distortions of compression, additive noise with large noise variance, and resolution reduction.

[0050] Although the invention has been described with a certain degree of particularity, it is understood that the present invention has been made only by way of example and that numerous changes in the details of construction and the combination and arrangement of parts may be resorted to without departing from the spirit and scope of the invention.