Electronic watermarking system capable of providing image data with high secrecy

[0002] This invention relates to an electronic watermarking system of a type jointly using a scramble effect, particularly to an electronic watermarking system which inserts a watermark into data and adds an scramble effect thereinto for the purpose of protecting a copyright, or the like.

[0003] Recently, an electronic watermarking system comes to be used for authenticating copyright ownership, identifying copyright infringers, or transmitting a hidden message.

[0004] As a conventional example of such an electronic watermarking system in which electronic watermarking data are embedded into a digital image, a proposal is made for a method of inserting electronic watermarking data in Nikkei Electronics, No. 660 published Apr. 22, 1996, Page 13. In the proposal, an image is converted into frequency components and the electronic watermarking data are embedded into an area having strong frequency components in an image signal after frequency conversion. The method disclosed therein is such a method for inserting ID information into digital works of which an object is a moving picture, a static picture,. a photograph, or a voice. In the method, the ID information is inserted per every buyer of the digital work so as to be used in a case that a digital work illegally copied by a person other than the buyer should be prosecuted. Further, another conventional method is also proposed, in which an improvement is made particularly for image processing systems using MPEG.

[0005] However, as will later be described more in detail, it is not enough in the conventional methods to provide digital image data with high secrecy.

[0006] It is therefore an object of the present invention to provide an electronic watermarking system which is capable of providing image data with very high secrecy.

[0007] Other objects of the present invention will become clear as the description proceeds.

[0008] According to an aspect of the present invention, there is provided an electronic watermarking device which is for use in inserting an electronic watermark into an original image consisting of a plurality of primary blocks, comprising: first discrete cosine transform means for carrying out first discrete cosine transform of at least one of the a plurality of primary blocks; inserting means for inserting electronic watermarking data into an output of the first discrete cosine transform means; inverse discrete cosine transform means for carrying out inverse discrete cosine transform of an output of the inserting means; and first coordinate converting means which have first coordinate conversion data and which carry out first coordinate conversion of an output of the inverse discrete cosine transform means by the electronic watermarking data and the coordinate conversion data.

[0009] According to another aspect or the present invention, there is provided an electronic image reproducing device which is for use in reproducing an original image from an electronic watermarked image consisting of a plurality of secondary blocks, comprising: second discrete cosine transform means for carrying out second discrete cosine transform of at least one of the a plurality of secondary blocks; extracting means for extracting electronic watermarking data from an output of the second discrete cosine transform means; extracted data containing means for containing an output of the extracting means; correspondence detecting means which compare an output of the extracted data containing means with an original electronic watermarking data to detect whether or not the output of the extracted data containing means is corresponding with the original electronic watermarking data; output image switching means which output the electronic watermarked image when the output of the extracted data containing means is detected to be not corresponding with the original electronic watermarking data by the correspondence detecting means, and first inverse coordinate converting means which have first inverse coordinate conversion data and which carry out first inverse coordinate conversion of the output of the extracted data containing means with reference to the original electronic watermarking data and the first inverse coordinate conversion data when the output of the extracted data containing means is detected to be corresponding with the original electronic watermarking data by the correspondence detecting means.

[0010] According to still another aspect of the present invention, there is provided an electronic watermarking system which is for use in inserting an electronic watermark into an original image consisting of a plurality of primary blocks, comprising: inserting means for inserting electronic watermarking data into at least one of the a plurality of primary blocks; second coordinate converting means which have second coordinate conversion data and which carry out second coordinate conversion of an output of the inserting means by the electronic watermarking data and the second coordinate conversion data; extracting means for extracting electronic watermarking data from a part of the coordinate-converted electronic watermarked image; and second inverse coordinate converting means which have second inverse coordinate conversion data and which carry out second inverse coordinate conversion of the output of the extracting means with reference to the original electronic watermarking data and the second inverse coordinate conversion data when the output of the extracting means is corresponding with the original electronic watermarking data.

[0011] The electronic watermarking data may be identical with the original electronic watermarking data, the second coordinate conversion data may be identical with the second inverse coordinate conversion data.

[0012] The second coordinate converting means may calculate a coordinate of one of the primary blocks outputted from the inserting means by the electronic watermarking data and the second coordinate conversion data, the coordinate being occupied within a region where an image having the inserted electronic watermarking data is contained.

[0013] One primary block may consist of 8×8 elements, the second coordinate conversion being carried out by converting one block coordinate into another block coordinate in a frame.

[0014] FIG. 1 is a block diagram for showing a copyright protection system in which a conventional electronic watermarking method is used;

[0015] FIG. 2 is a block diagram for showing an electronic watermarking system according to a preferred embodiment of the present invention;

[0016] FIG. 3 is a block diagram for showing a method of reloading an image in which an electronic watermarking data are embedded by the electronic watermarking system illustrated in FIG. 2; and

[0017] FIG. 4 is a block diagram for showing a coordinate conversion device which is used in a reloading process in the method illustrated in FIG. 3.

[0018] Description is, at first, made about a conventional electronic watermarking method in order to facilitate an understanding of the present invention.

[0019] In the conventional electronic watermarking method, as mentioned in the preamble of the instant specification, the ID information is inserted per every buyer of the digital work so as to be used in a case that a digital work illegally copied by a person other than the buyer should be prosecuted. First, an original work is subjected to a frequency conversion by discrete cosine transform (DCT) or fast fourrie transform (FFT). The frequency-converted work is then subjected to inverse frequency conversion by spread spectrum using ID information of random numbers. Accordingly, a digital work having a hidden ID information is obtained. Thereafter, a difference is detected between one frequency spectrum obtained by frequency conversion for the digital work having the hidden ID information and another frequency spectrum obtained by frequency conversion for the original work. The difference is compared with an original ID information As a result, when the difference is not corresponding to the original ID information, the digital work is judged to be an illegally copied one. In this method, since the electronic watermarking data are embedded into frequency components, the electronic watermarking data are not deleted by image processing such as compression, filtering, or the like. Further, the method employs random numbers in accordance with normal distribution as the electronic watermarking data. As a result, the electronic watermarking data can be prevented from interference with each other. In addition, it becomes difficult to destroy the electronic watermarking data without giving a large influence to the whole of the image.

[0020] In this method, the electronic watermarking data are embedded into an image, as follows.

[0021] At first, an original image is converted into frequency components by the use of discrete cosine transform (DCT), or the like. Let n numbers of data f(1), f(2), . . . , f(n) each indicating high value in a frequency region be selected. Further, let n numbers of electronic watermarking data w(1), w(2), . . . , w(n) be selected from a normal distribution where average is 0, and dispersion is 1. Then, a solution in the following equation is calculated with respect to each i.

[0022] F(i)=f(i)+α×|f(j)|×w(i)

[0023] where α is a scaling element.

[0024] Finally, let the f(i) be replaced with F(i) to calculate a frequency component. Accordingly, from the frequency component in which the f(i) is replaced with the F(i), an image to which the electronic watermarking data are embedded is acquired.

[0025] Next, in the method, the electronic watermarking data are detected from an image, as follows. In this detection, the original Image must be already known. In addition, also materials for the electronic watermarking data w(i) [where i=1, 2, . . . , n] must be already known.

[0026] At first, an image including the electronic watermarking data is converted into frequency components by the use of DCT, or the like. In the frequency region, each value of elements corresponding to f(1), f(2), . . . , f(n), respectively in which the electronic watermarking data are embedded is defined as F(1), F(2), . . . , F(n). In view of f(i) and F(i), the electronic watermarking data W(i) is calculated to be extracted by the following equation.

[0027] W(i)=(F(i)−f(i))/f(i)

[0028] Then, statistical similarity C between w(i) and W(i) is calculated by the following equation with the use of inner product.

[0029] C=W×w/(WD×wD)

[0030] where W=(W(1), W(2), . . . , W(n)), w=(w(1), w(2), . . . , w(n)); p1 and where WD is an absolute value of vector W while wD is an absolute value of vector w.

[0031] Accordingly, when the statistical similarity C exceeds a predetermined value, the outstanding electronic watermarking data are determined to be embedded in the image.

[0032] Thus, by embedding the electronic watermarking data in the image, it is advantageous for an author owning an original image to carry out the detection processing against digital image data suspected of being an illegal copy.

[0033] However, the conventional method thus mentioned above needs an original image. As each user of an image reproducing device of each terminal does not have the original image, it is impossible for each user to carry out such a detection processing. Under the circumstances, another conventional method is proposed to improve the above-mentioned method for such a processing of a terminal user, especially for an MPEG system.

[0034] In this another conventional method, an original image is divided into a plurality of blocks each consisting of 8×8 pixels. In the another conventional method, each block is, as a unit, subjected to embedding or detecting process of electronic watermarking data.

[0035] In this method, the electronic watermarking data are embedded into an image, as follows.

[0036] At first, in a frequency region after DCT process in MPEG coding processes, let f(1), f(2), . . . , f(n) be determined, respectively, from a lower frequency one to a higher frequency one in AC components thereof. Further, let electronic watermarking data w(1), w(2), . . . , w(n) be selected from a normal distribution where average is 0, and dispersion is 1. Then, a solution in the following equation is calculated with respect to each i.

[0037] F(i)=f(i)+α×avg (f(i))×w(i)

[0038] where α is a scaling element and avg (f(i)) is a partially average value obtained by an average of absolute values of three points each near the f(i).

[0039] Then, let the f(i) be replaced with F(i) and thereafter the other MPEG coding processes are carried out.

[0040] Next, in the method, the electronic watermarking data are detected from an image, as follows. In this detection, the original image is not required. On the other hand, materials for the electronic watermarking data w(i) [where i=1, 2, . . . , n] must be already known.

[0041] At first, in a frequency region of a block after inverse quantization in MPEG expanding Processes, let F(1), F(2), . . . , F(n) be determined, respectively, from a lower frequency one to a higher frequency one. Further, let an average value of absolute values of three points each near the F(i) be defined as a partially average value avg (F(i)). The electronic watermarking data W(i) is calculated by the following equation.

[0042] W(i)=F(i)/avg (F(i))

[0043] Further, WF (i), that is a total amount of W(i) over one image is calculated with respect to each i. Then, statistical similarity C between w(i) and WF(i) is calculated by the following equation with the use of inner product.

[0044] C=W×w/(WFD×wD)

[0045] where W=(WF(1), WF(2), . . . , WF(n)), w=(w(1), w(2), . . . , w(n));

[0046] and where WFD is an absolute value of vector WF while wD is an absolute value of vector w.

[0047] Accordingly, when the statistical similarity C exceeds a predetermined value, the outstanding electronic watermarking data are determined to be embedded in the image.

[0048] Next, referring to FIG. 1, description is made as regards a copyright protection system using the above-mentioned conventional electronic watermarking technique.

[0049] As illustrated in FIG. 1, electronic watermarking data 402 indicating prohibition of copy are previously embedded by an owner of the copyright into an image which requires prohibition of illegal use. In the detection side, image data 401 including the embedded electronic watermarking data 402 are subjected to DCT per each k×k blocks thereof by the DCT device 403. By the use of a value after the DCT, the embedded electronic watermarking data 402 are detected by an electronic watermarking data detector 404 with reference to electronic watermarking data 405. When the result of detection requires that the copy should be prohibited, the controller 406 outputs image data 408 with a copy prohibition signal 409. On the contrary, when the result of detection states that the copy should be permitted, the controller 406 outputs the Image data 401 as it stands.

[0050] However, the image data 401 are, in the color sense, such a digital image as substantially identical with an original image before the electronic watermarking data are embedded. Therefore, if a bypass transmission route 410 is unfairly formed from the image data 401 through the controller 406, as depicted by a broken line in FIG. 1, the image data 401 are outputted with no relation to the embedded electronic watermarking data. In this case, contents of the image data 401 become known.

[0051] Now, referring to FIGS. 2 through 4, description will proceed to an electronic watermarking system according to a preferred embodiment of the present invention.

[0052] The electronic watermarking system according to the preferred embodiment comprises a first device which carries out processing for preventing an unfair use (shown in FIG. 2), and a second device for reproducing an image of which unfair use is prohibited (shown in FIG. 3).

[0053] In FIG. 2, illustrated is the first device which carries out an electronic watermarking processing and a scramble processing for a digital image to convert the digital image into such a digital image as being hardly used unfairly.

[0054] In FIG. 2, a digital image converting device comprises a DCT unit 103 which extracts a block 102 of k×k pixels to be subjected to DCT and which outputs data of the block 102 of k×k pixels after finishing the DCT, an electronic watermarking unit 104 which inserts the electronic watermarking data into the block 102 of k×k pixels by the use of electronic watermarking data 105, an IDCT unit 106 which makes the block 102 of k×k pixels having the embedded electronic watermarking data be subjected to IDCT, and a scramble device 111. The scramble device 111 includes a coordinate converter 107 which converts the coordinate of the block 102 of k×k Pixels by the use of the electronic watermarking data 105 and coordinate conversion data 108.

[0055] In FIG. 3, illustrated is a digital image reproducing device which reproduces a digital image converted by the digital image converting device illustrated in FIG. 2 as such a digital image as identical with the original image 101 in FIG. 2 in the color sense.

[0056] In FIG. 3, a digital image reproducing device comprises a DCT unit 203 which extracts a block 102 of k×k pixels from the image data 201 to be subjected to DCT and which outputs a position information of the extracted block 102 of k×k pixels and data after finishing the DCT, an extracted data containing unit 205 for containing the extracted data having the same data length as that of the electronic watermarking data, an electronic watermarking data extracting unit 204 which extracts the electronic watermarking data from frequency data of k×k pixels outputted by the DCT unit 203 and which makes the extracted electronic watermarking data be contained in the extracted data containing unit 205, a correspondence detector 206 which determine whether or not the data contained in the extracted data containing unit 205 is corresponding with the electronic watermarking data 207, an image output switch 208 which outputs the image data 201 to a reproduced image 211 or to a scramble decoder 213 in accordance with determination by the correspondence detector 206, and an inverse coordinate converter 209 which converts coordinates of every block 202 of k×k pixels in the image data 201 inputted from the image output switch 208 by the use of the electronic watermarking data 207 and inverse coordinate conversion data 210 and which outputs the the image data 201 to the reproduced image 211. The scramble decoder 213 is composed of the inverse coordinate converter 209 and the inverse coordinate conversion data 210.

[0057] In FIG. 4, illustrated is an example of the coordinate converter 107 in the scramble device 111. As illustrated in FIG. 4, the coordinate converter 107 comprises a coordinate calculation unit 303 which calculates a coordinate of a block 301 of k×k pixels outputted from the IDCT unit 106 by the use of the electronic watermarking data 304 and coordinate conversion data 305, and a controller 302 which locates the block 301 of k×k pixels into an output buffer 306 in accordance with the coordinate obtained by the coordinate calculation unit 303. In this embodiment, it is assumed that the electronic watermarking data 105 used by the electronic watermarking unit 104 have the same value as that of the electronic watermarking data 207 used by the correspondence detector 206. Further, the inverse coordinate conversion data 210 are automatically determined by the coordinate conversion data 108.

[0058] Referring to FIGS. 2 and 4, description will proceed to an operation of embedding or inserting the electronic watermarking data in the preferred embodiment of the present invention.

[0059] First, the block data 102 of k×k pixels are extracted from the original image 101 by the DCT unit 103 and then subjected to DCT. The electronic watermarking data 105 are inserted by the electronic watermarking unit 104 into frequency data of the block of k×k pixels outputted from the DCT unit 103. The data outputted from the electronic watermarking unit 104 are subjected to IDCT by the IDCT unit 106. As a result, frequency components of k×k pixels are converted into an image of k×k pixels. A new coordinate of the block of k×k pixels outputted from the IDCT unit 106 occupied in a region where an image having the inserted electronic watermarking data is contained is calculated by the coordinate converter 107 with reference to the electronic watermarking data 105 used in the electronic watermarking unit 104 and the coordinate conversion data 108. The block of k×k pixels is embedded in k×k pixels of the original image 101. Thereby, an electronic watermark is inserted into the original image 101 to obtain an image 109 including the electronic watermark.

[0060] FIG. 4 is a block diagram of the coordinate converter 107. In FIG. 4, a new coordinate of the block 301 of k×k pixels outputted from the IDCT unit 106 occupied in the output buffer is calculated by the coordinate calculation unit 303 with reference to the electronic watermarking data 304 used in the electronic watermarking unit 104 and the coordinate conversion data 305. This calculation is carried out so that the blocks each consisting of k×k pixels may have different coordinates from each other. In this coordinate calculation unit 303, an encoding operation of a scramble type is carried out. A scramble is performed in accordance with a certain rule. For example, when the coordinate conversion data is defined as A, the electronic watermarking data is defined as B, a coordinate of the block before coordinate conversion is defined as (x, y), and a coordinate of the block after coordinate conversion is defined as (X, Y) by the use of an encoding table, the following equation is available.

[0061] (X, Y)=f(x, y, A, B)

[0062] Accordingly, the new coordinate of the block of 8×8 pixels is determined by the above (X, Y).

[0063] Next, in a decoding operation, the coordinate of the block of 8×8 pixels is returned to the original coordinate thereof by a decoder under the following equation.

[0064] (x, y)=F(X, Y, A, B)

[0065] This functions as inverse coordinate conversion means.

[0066] As depicted above, the same coordinate conversion data “A” are used both in the equation of the encoder and in the equation of the decoder. Besides, it is not necessary for both the coordinate conversion data “A” to have the same values as each other, since it is enough to determine the original coordinate thereof. Thus, in this embodiment, not only a scrambled indistinguishable image but also an unscrambled distinguishable image can be detected by the same electronic watermark detector.

[0067] The block 301 of k×k pixels is located in the output buffer 306 by the controller 302 in accordance with a coordinate value outputted from the coordinate calculation unit 303. Besides, the coordinate conversion data 108 is the same element as the coordinate conversion data 305.

[0068] The above operations are carried out over the whole of a picture. The electronic watermarking data are inserted into every block of k×k pixels over the whole region of the picture. An image scrambled per each block of k×k pixels is outputted.

[0069] Referring to FIG. 3, description will proceed to an operation of reproducing the image in which the electronic watermarking data are inserted by the above-mentioned inserting operation and which is scrambled as described above.

[0070] First, the block 202 of k×k pixels is extracted from the image data 201 by the DCT unit 203 and then subjected to DCT so as to be converted into. frequency components. Components to be compared with the electronic watermarking data are extracted from frequency data outputted from the DCT unit 203 by the electronic watermarking data extracting unit 204. The extracted components are contained in the extracted data containing unit 205. It is judged by the correspondence detector 206 whether or not the extracted data contained in the extracted data containing unit 205 is corresponding with the electronic watermarking data 207. As the result of judgement by the correspondence detector 206, when the extracted data contained in the extracted data containing unit 205 is not corresponding with the electronic watermarking data 207, the image data 201 are outputted to the reproduced image 211 by the image output switch 208. Thereby, outputted are such the image data 201 that are scrambled to become a visually indistinguishable image. In this case, nothing is outputted through a route 215. On the other hand, when the extracted data contained in the extracted data containing unit 205 is corresponding with the electronic watermarking data 207, the image data 201 are outputted to the inverse coordinate converter 209 by the image output switch 208. In this case, nothing is outputted through a route 214.

[0071] With reference to the electronic watermarking data 207 and the inverse coordinate conversion data 210, coordinates are calculated by the inverse coordinate converter 209 so that the block 202 of k×k pixels may be fitted. The coordinates thus calculated are located in the reproduced image 211. These processing are repeatedly carried out on all of the blocks each consisting of k×k pixels within the image data 201.

[0072] By the above operations, in a case that the extracted data are corresponding with the electronic watermarking data, the reproduced image 211 becomes such an image as visually identical with the original image 201. On the contrary, in a case that the extracted data are not corresponding with the electronic watermarking data, the reproduced image 211 becomes such an indistinguishable image as scrambled and seems to be visually different from the original image 201.

[0073] As mentioned above, the electronic watermarking data are inserted into every block of k×k pixels. Consequently, even if a scramble operation is carried out with respect to a block of k×k pixels, as a unit, as carried out by the coordinate converter 107, it is possible to detect an electronic watermark either from. the image data 109 and the image data 211.

[0074] According to the preferred embodiment of the present invention, jointly using the scramble processing. the electronic watermarking system can prevent a user who does not have original electronic watermarking data from readily looking into image data.

[0075] Further, even if the image data were scrambled, similarly to that scrambly decoded, it is possible to detect an electronic watermark therefrom. Therefore, even after scramble decoding, it can be kept difficult that the embedded electronic watermarking data are removed.

[0076] Furthermore, the scramble processing brings different characteristics to each of the embedded electronic watermarking data, respectively from each other. It is therefore difficult to carry out scramble decoding unfairly.

[0077] Moreover, the electronic watermarking system according to the preferred embodiment of the present invention can maintain its function to prevent an unfair use, even though a bypass transmission route is formed after detecting the electronic watermarking data. Namely, in an inserting operation of the electronic watermark, electronic watermarking data are embedded, for example, by an owner of a copyright, and a digital image having the embedded electronic watermarking data is scrambled to provide such image data as completely different from original image data visually. On the other hand, in a decoding operation, the scrambled image data are descrambled to detect the electronic watermark, as decoded image data. Accordingly, as mentioned above, even after scramble decoding, it can be kept difficult that the embedded electronic watermarking data are removed. In addition, the scramble processing in the preferred embodiment of the present invention brings different characteristics to each of the embedded electronic watermarking data, respectively from each other. It is therefore difficult to carry out scramble decoding unfairly.

[0078] While the present invention has thus far been described in conjunction with only a preferred embodiment thereof, it will now be readily possible for those skilled in the art to put this invention into various other manners.

[0079] For example, in the above embodiment, description is mainly made about image data. However, the present invention is not restricted to image data but can be applied to the other digital data, such as a voice signal, text data, or the like. In a case of the voice signals, a part of the voice signal, that is too short time to hear, is subjected to DCT to be converted into frequency components. A specific frequency component is inserted, as electronic watermarking data, into the converted frequency components to obtain data. The obtained data are subjected to IDCT to be converted into original voice signal components. A short time which is too short to hear is embedded into the above-mentioned short time portion extracted from the original voice signal. The voice signal components are thereafter scrambled to be outputted as voice data similar to the original voice data. Thus, a voice signal having an electronic watermark is acquired. The voice signal having the electronic watermark can be decoded in a manner similar to that of the above-described preferred embodiment.

[0080] Further, in the above embodiment, the divided block is converted into frequency components by DCT. However, conversion into frequency components can be made by the other methods such as fast Fourier transform (FFT), orthogonal transformation, or the like. Besides, in the scramble processing, the electronic watermarking data as well as coordinate conversion data are inputted. In a reproducing process, a block of k×k pixels in which the electronic watermarking data are embedded is subjected to an inverse coordinate conversion after decoding the electronic watermarking data so as to obtain the original image. In this case, alternatively, at first, an image obtained as an output of encoder side may be subjected to inverse coordinate conversion. The inserted electronic watermarking data then may be decoded.