Systems and methods for etching digital watermarks

[0003] The present invention relates generally to systems and methods for etching digital watermarks and, more particularly, to systems and methods for etching digital watermarks with improved error tolerances.

[0004] The secure transmission of multimedia information is an important concern for multimedia service providers and multimedia content owners. Secure conditional access has conventionally been accomplished through the use of encryption. However, for widespread distribution of multimedia, conventional key-management mechanisms used in encryption techniques are often too cumbersome. In addition, encryption techniques do not address the issue of copyright protection. Once encrypted material has been decoded, the reproduction, manipulation, and redistribution of data streams can be almost effortless, especially since information transmitted in packet networks is in a digital format.

[0005] More recently, attention has been focused on supplementing encryption with digital watermarking to provide copyright protection and tamper-proofing for multimedia streams. Digital watermarking is a technique that hides information, or a message authentication code (MAC), in the audio, image, or video stream. Watermarking has advantages over traditional MAC systems in that watermarks are embedded directly into the multimedia information (e.g., into the coefficients of the image or video signal or into the tones of the audio signal). A traditional digital MAC can be removed easily because it is attached to the document and jointly transmitted or stored.

[0006] In addition, conventional cryptographic techniques can be inadequate if the communication channel is lossy. A single bit error can cause the received signal to fail authentication when conventional MACs are used. If a robust MAC scheme such as watermarking is used, minor distortions will be ignored so that decisions based on the watermark can be made more intelligently and can more accurately reflect the state of the content that the watermark is protecting.

[0007] A considerable amount of research has been performed to investigate robust watermarks that identify the owner and/or the purchaser of the audio track or image in question. The primary requirement for copyright protection watermarks is that they be survivable to compression and common signal distortions, thus, making it impossible to remove the watermark without destroying the value of the multimedia content that it protects.

[0008] In addition to copyright applications, watermarking can also be used to authenticate and tamper-proof multimedia content. For a video authentication application, a watermark must be robust to compression, common signal distortions, and the effects of error-prone channels, but it must further indicate any modification made to the stream. Therefore, the watermark should describe the content and the salient features of the video information. For example, video authentication MACs can conventionally be constructed from edge maps of video frames or from an image histogram of video frames.

[0009] Digital watermarks are added to compressed video (e.g., MPEG video) by modifying the encoded video information. If a watermark is meant to be extracted without the use of the original, as is the case in authentication applications, the etching process must use some video coefficients as a reference.

[0010] One conventional method embeds a watermark by sliding a non-overlapping 3×1 window over DC coefficients from luminance (Y) blocks in intraframes (I-frames). The coefficients contained in the window may be denoted b₁, b₂, and b₃to represent the coefficient values at the (i−1)th, (i)th and (i+1)th position of the DC sequence. The corresponding rank-ordered coefficients are denoted as b(1)≦b(2)≦b(3). The median coefficient is then modified depending on the values of b(1), b(3), and the watermark bit x to be etched. The interval between b(1) and b(3) is divided into levels 1_0, 1_1, . . . , 1_m where b(1) is defined to be at level zero 1_0 and b(3) is defined to be at level m 1_m. b(2) lies at an arbitrary level 1_k between b(1) and b(3). If k is even and x=0, b(2) remains unchanged, but if x=1, then b(2) is increased (decreased) by one level making it fall on an odd level. Likewise, if k is odd and x=1, b(2) remains unchanged, but if x=0, then b(2) is increased (decreased) by one level making it fall on an even level. This transformation forces the coefficient b(2) to fall on an even level if x=0 and on an odd level if x=1. The coefficients b(1) and b(3) remain unchanged so that the original image is not needed during the extraction process.

[0011] Another conventional method uses an approach similar to that just described, except it uses only the first coefficient of each I-frame macro-block as a reference. The first coefficient is defined at level zero, and a watermark bit is etched into the second coefficient depending on whether the second coefficient lies on an even or odd level relative to the first coefficient. The same rules used above to determine if the median coefficient should be modified are applied to the second coefficient. The method then continues for the third coefficient using the second as the new reference, and then for the fourth coefficient using the third as the new reference, and so on.

[0012] One of the main difficulties in constructing a watermarking system for copyright protection or authentication applications is its survivability to acceptable signal distortions. For a copyright protection application, the list of acceptable video distortions is quite extensive because the content owner is interested in claiming ownership regardless of the manipulation done to the original content. For example, watermarking for copyright protection may need to be robust to cropping, rotating, dithering, recompression, and the like. The acceptable distortions for the authentication application are more limited because the authentication MAC must still be able to identify tampering. Distortions caused by recompression or by minor channel errors do not indicate tampering, while cropping and rotating do. During recompression, the essence of the video information is maintained, but minor fluctuations in individual coefficient values, though not perceptible, can change the value of the MAC. Robustness to distortion can be built into the watermark etching algorithm or into the MAC itself.

[0013] Therefore, there exists a need for a system and method that improves the survivability of an etched watermark to signal distortions induced in multimedia data.

[0014] Systems and methods consistent with the present invention address this need by improving the error tolerance of watermark etching algorithms. The present invention builds in an error tolerance buffer when watermark bits are etched into digital data. Using systems and methods consistent with the present invention, minor modifications (e.g., distortions) to digital data values that fall within a specified error tolerance range will not cause an error in the watermark extraction process. Modifications to digital data values above the specified error tolerance range (whether due to excessive recompression or to actual tampering) may alter the extracted watermark bits at the receiver and may indicate tampering.

[0015] In accordance with the purpose of the invention as embodied and broadly described herein, a method of watermarking digital data includes obtaining a matrix of digital data values; selecting a plurality of data values from the matrix; specifying a parameter that indicates an acceptable amount of variation in the data values; and etching watermark bits in at least two of the plurality of data values using the specified parameter.

[0016] In another implementation consistent with the present invention, a method of extracting a watermark from digital data includes obtaining a matrix of digital data values containing an etched watermark; selecting a plurality of data values from the matrix; specifying a parameter that indicates an acceptable amount of variation in the data values; and extracting watermark bits from at least two of the plurality of data values using the specified parameter.

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, explain the invention. In the drawings,

[0018] FIG. 1 illustrates an exemplary system consistent with the present invention;

[0019] FIG. 2 illustrates exemplary components of a watermark etching/extraction device consistent with the present invention;

[0020] FIGS. 3-5 are flowcharts that illustrate exemplary system processing for etching a watermark in digital data consistent with the present invention;

[0021] FIG. 6 illustrates an exemplary data matrix consistent with the present invention; and

[0022] FIGS. 7-9 are flowcharts that illustrate exemplary system processing for extracting a watermark consistent with the present invention.

[0023] The following detailed description of the invention refers to the accompanying drawings. The same reference numbers in different drawings identify the same or similar elements. Also, the following detailed description does not limit the invention. Instead, the scope of the invention is defined by the appended claims.

[0024] Systems and methods consistent with the present invention provide mechanisms that improve the error tolerance of watermark etching algorithms. The present invention builds in an error tolerance buffer when watermark bits are etched into digital data. Thus, using systems and methods consistent with the present invention, minor modifications (e.g., distortions) to digital data values that fall within a specified error tolerance range will not cause errors in the watermark extraction process.

[0025] FIG. 1 illustrates an exemplary system 100, consistent with the present invention, for etching and extracting watermarks from digital data. System 100 may include a watermark etching subsystem 105 and a watermark extraction subsystem 110. Watermark etching subsystem 105 can include an analog signal input 115, a digital signal input 120, an analog-to-digital (A/D) converter 125, a switch 130, a compression coder 135, a watermark etching device 140, an optional data storage device 145, and an optional transmitter 150.

[0026] Analog signal input 115 connects to compression coder 135 through A/D converter 125 and switch 130. A/D converter 125 includes conventional circuitry for converting analog input signals to digital signal form. Digital signal input 120 connects to compression coder 135 through switch 130. Compression coder 135 can include any known digital compression coder such as, for example, a conventional MPEG compression coder for coding video data. Watermark etching device 140 receives compressed digital data from compression coder 135. Watermark etching device 140 etches digital watermarks into the data received from compression coder 135 in accordance with the present invention.

[0027] Data storage device 145 may include conventional devices for storing and retrieving digital data, including floppy diskette drives, CD ROM drives, DVD drives or the like. Transmitter 150 may receive data containing etched digital watermarks from watermark etching device 140. Transmitter 150 may include transmitter circuitry well known to one skilled in the art for transmitting the received data over link 155. Link 155 may include one or more wireless, wire-line or optical links.

[0028] Watermark extraction subsystem 110 can include an optional receiver 160, an optional data storage device 165, a watermark extraction device 170, a decompression decoder 175, a switch 180, a digital-to-analog (D/A) converter 185, an analog signal output 190, and a digital signal output 195.

[0029] Data storage device 165 may include conventional devices for storing and retrieving digital data, including floppy diskette drives, CD ROM drives, DVD drives or the like. Receiver 160 may include receiver circuitry well known to one skilled in the art for receiving data transmitted over link 150. Watermark extraction device 170 receives data containing etched digital watermarks from data storage device 165 or receiver 160. Watermark extraction device 170 extracts, in accordance with the present invention, digital watermarks from the digital data received from receiver 160.

[0030] Decompression decoder 175 receives, and decompresses, video data from watermark extraction device 170. Decompression decoder 175 can include any known digital decompression decoder such as, for example, a conventional MPEG video decompression decoder. Decompression decoder 175 provides decompressed video to analog signal output 190 via switch 180 and digital-to-analog converter 185. D/A converter 185 includes conventional circuitry for converting signals from digital to analog signal form. Decompression decoder 175 provides decompressed video to digital signal output 195 via switch 180.

[0031] FIG. 2 illustrates an exemplary watermark etching device 140, consistent with the present invention. Watermark extraction device 170 may be configured similarly. Watermark etching device 140 may include a communication interface 205, an input device 210, an output device 215, a processor 220, a Random Access Memory (RAM) 225, a Read Only Memory (ROM) 230, and a bus 235.

[0032] Communication interface 205 includes conventional mechanisms for connecting device 140 to devices, such as compression coder 135, data storage device 145, and transmitter 150. Input device 210 permits entry of data into device 140 and can include one or more user interfaces (not shown), such as a keyboard, a mouse or the like. Output device 215 permits the output of data in video, audio, or hard copy format.

[0033] Processor 220 may perform all data processing functions for inputting, outputting, and watermarking of video data. RAM 225 provides semi-permanent working storage of data and instructions for use by processor 220. ROM 230 provides permanent or semi-permanent storage of data and instructions for use by processor 220. RAM 225 and ROM 230 may include large-capacity storage devices, such as a magnetic and/or optical recording medium and its corresponding drive. Bus 235 interconnects the various components of device 140 to permit the components to communicate with one another.

[0034] FIGS. 3-5 are flowcharts that illustrate exemplary processing, consistent with the present invention, for etching digital watermarks in a matrix of digital data. As one skilled in the art will appreciate, the method exemplified by FIGS. 3-5 can be implemented as a sequence of instructions and stored in ROM 230 of device 140 for execution by processor 220.

[0035] To begin the watermark etching processing, device 140 receives a tuning parameter Te that specifies the amount of acceptable distortion that the data of the etched data matrix may undergo (e.g., due to excessive recompression or tampering) before the watermark is destroyed [step 305]. Device 140 may receive tuning parameter Te via input device 215. Though various values of Te may be specified, a Te of 16 is one example of an acceptable value consistent with the present invention. Device 140 calculates an interval parameter Se [step 310], used for etching a watermark, according to the following equation:

[0036] Se=2*Te+1  Eqn. (1)

[0037] Device 140 obtains a matrix of digital data of width (w) and height (h) into which the watermark will be etched [step 315]. An example of one such matrix is a matrix of luminance DC coefficients of a conventional MPEG I-frame. An exemplary data matrix 600 that can be obtained by device 140 is illustrated in FIG. 6. Data matrix 600 can comprise a plurality of 2×2 windows 605, 610, and 615, each including data values b₁, b₂, b₃, b₄. Device 140 then obtains a watermark that includes a binary sequence of bits of length (w*h)/2 [step 320] (FIG. 3). The watermark can be derived using any conventional digital watermark production technique.

[0038] Device 140 sets matrix displacement parameters r and c, which specify rows and columns of 2×2 windows in the data matrix, to a value of zero [step 325]. Device 140 obtains data values of the matrix (b₁, b₂, b₃, b₄) by sliding a non-overlapping 2×2 window over the coordinates (2r, 2c), (2r+1, 2c), (2r, 2c+1), and (2r+1, 2c+1) of the matrix, where 0≦r≦h/2 and 0≦c≦w/2 [step 330]. For example, if data matrix 600 includes an MPEG I-frame, then device 140 can obtain the DC coefficients associated with each of the matrix coordinates of the 2×2 window.

[0039] Device 140 then orders the data values in the 2×2 window in rank order such that b(1)<b(2)<b(3)<b(4) [step 335]. For example, if b₁=2016, b₂=127, b₃=683 and b₄=480, then b(1) is set equal to b₂, b(2) is set equal to b₄, b(3) is set equal to b₃, and b(4) is set equal to b₁. Thus, in this example, b(1)<b(2)<b(3)<b(4).

[0040] Device 140 determines the significance (SIGWINDOW) of the current 2×2 window [step 340] using the following equation:

[0041] SIGWINDOW=FLOOR[b(4)/Se]*Se−FLOOR[b(1)/Se]*Se  Eqn. (2)

[0042] where the function, FLOOR of [x], determines the largest integer less than or equal to x.

[0043] The significance of the current window indicates whether the region of data matrix 600, represented by the 2×2 window, is too smooth to hide watermark bits. Device 140 determines if the SIGWINDOW parameter is greater than, or equal to, two times the interval parameter Se [step 345]. If so, the current 2×2 window is too smooth to hide watermark bits and, thus, device 140 skips the 2×2 window and the (rw+2c)th and (rw+2c+1)th watermark bits [step 350]. Device 140 then proceeds to step 525 (see below). If SIGWINDOW is less than 2 times Se, device 140 modifies the data values b(1) and b(4) so that they each lie in the middle of the previously specified interval Se [step 405] (FIG. 4). To perform this modification, device 140 uses the following equation:

[0044] b(j)′=floor[b(j)/Se]*Se+Te; j=1,4  Eqn. (3)

[0045] Device 140 sets counter value i to 2 [step 410]. Device 140 then, using the counter value i, determines a value k [step 415] according to the following equation:

[0046] k=FLOOR[(b(i)−b(1)′)/Se  Eqn. (4)

[0047] Device 140 determines if b(i) is equal to b(4)′ [step 420]. If so, device 140 decrements k by 1 [step 425]. If not, device 140 obtains the (rw+2c+(i−2))th watermark bit x_{rw+2c+(i−2)}[step 430]. Device 140 transforms the data value b(i) [step 435] using the following equation:

[0048] b(i)′=b(1)′+[k+XOR(mod(k,2), x_{rw+2c+(i−2)}))]*Se  Eqn. (5)

[0049] Device 140 then determines if the counter value i is equal to three, indicating that both b(2) and b(3) have been modified by Eqn. (5) [step 440]. If not, device 140 increments i by 1 [step 445] and returns to step 415 to modify b(3).

[0050] If device 140 determines that the counter value i is equal to three, device 140 may correct the modified data values b(2)′ and b(3)′ to ensure the correct rank order. To do so, device 140 determines if b(3)′ is greater than, or equal to, b(2)′ [step 505]. If b(3)′ is greater than, or equal to, b(2)′, correction of the data values b(2)′ and b(3)′ is not required and processing proceeds to step 525 below. If b(3)′ is less than b(2)′, device 140 determines if b(1)′ subtracted from b(2)′ is greater than, or equal to, two times the interval parameter Se [step 510]. If so, 2 times Se is subtracted from b(2)′ [step 515]. If b(1)′ subtracted from b(2)′ is less than two times Se, then two times Se is added to b(3)′ [step 520].

[0051] At step 525, device 140 determines if the sliding 2×2 window has reached the last 2×2 column of the data matrix. To achieve this, device 140 determines if column parameter c is equal to w/2 [step 525]. If not, device 140 increments the column parameter c by 1 [step 530] and returns to step 325 (FIG. 3). If c is equal to w/2, device 140 resets c to a value of zero [step 535] and determines if the row parameter r is equal to h/2 [step 540]. If not, device 140 increments r by 1 [step 545] and returns to step 325 (FIG. 3). If r is equal to h/2, then processing completes.

[0052] FIGS. 7-9 are flowcharts that illustrate exemplary processing, consistent with the present invention, for extracting digital watermarks from, for example, video data. As one skilled in the art will appreciate, the method exemplified by FIGS. 7-9 can be implemented as a sequence of instructions and stored in ROM 230 of device 170 for execution by processor 220.

[0053] To begin watermark extraction processing, device 170 receives a tuning parameter Te that specifies the amount of acceptable distortion that the data of the etched data matrix may undergo (e.g., due to excessive recompression or tampering) before the watermark is destroyed [step 705]. Device 170 may receive tuning parameter Te via input device 215. Though various values of Te may be specified, a Te of 16 is one example of an acceptable value consistent with the present invention. Device 170 calculates an interval parameter Se [step 710], used for extracting the watermark, according to the following equation:

[0054] Se=2*Te+1  Eqn. (6)

[0055] Device 170 receives a matrix of watermark etched digital data of width (w) and height h) [step 715]. An example of one such matrix is a matrix of conventional luminance DC coefficients of an MPEG I-frame that contains a watermark etched in accordance with the present invention.

[0056] Device 170 sets matrix displacement parameters r and c, which specify rows and columns of 2×2 windows in the watermark etched data matrix, to a value of zero [step 720]. Device 170 obtains data values of the matrix (B₁, B₂, B₃, B₄) by sliding a non-overlapping 2×2 window over the coordinates (2r, 2c), (2r+1, 2c), (2r, 2c+1), and (2r+1, 2c+1) of the etched matrix, where 0≦r≦h/2 and 0≦c≦w/2 [step 725]. For example, if the etched data matrix includes an MPEG I-frame, then device 170 can obtain the DC coefficients associated with each of the matrix coordinates of the 2×2 window.

[0057] Device 170 then orders the data values in the 2×2 window in rank order such that B(1)<B(2)<B(3)<B(4) [step 730]. For example, if B₁=2000, B₂=100, B₃=650 and B₄=450, then B(1) is set equal to B₂, B(2) is set equal to B₄, B(3) is set equal to B₃, and B(4) is set equal to B₁. Thus, in this example, B(1)<B(2)<B(3)<B(4).

[0058] Device 170 determines the significance (SIGWINDOW) of the current 2×2 window [step 735] using the following equation:

[0059] SIGWINDOW=FLOOR[B(4)/Se]*Se−FLOOR[B(1)/Se]*Se  Eqn. (7)

[0060] where the function, FLOOR of [x], determines the largest integer less than or equal to x.

[0061] The significance of the current window indicates whether the region of the data matrix, represented by the 2×2 window, is too smooth to contain watermark bits. Device 170 determines if the SIGWINDOW parameter is greater than, or equal to, two times the interval parameter Se [step 740]. If so, the current 2×2 window is too smooth to contain watermark bits and, thus, device 170 skips the window and sets the (rw+2c)th and (rw+2c+1)th watermark bits to zero values [step 745]. Device 170 then proceeds to step 925 (see below). If SIGWINDOW is less than 2 times Se, device 170 modifies the data values B(1) and B(4) so that they each lie in the middle of the previously specified interval Se [step 805] (FIG. 8). To perform this modification, device 170 uses the following equation:

[0062] B(j)′=floor[B(j)/Se]*Se+Te; j=1,4  Eqn. (8)

[0063] Device 170 sets counter value i to 2 [step 810]. Device 170 then, using the counter value i, determines a value k [step 815] according to the following equation:

[0064] k=FLOOR[(B(i)−B(1)′)/Se  Eqn. (9)

[0065] Device 170 determines if B(i) is equal to B(4)′ [step 820]. If so, device 170 decrements k by 1[step 825]. If not, device 170 extracts the rw+2c+(i−2)th watermark bit x_{rw+2c+(i−2)}[step 830] from B(i) using the following equation:

[0066] x_{rw+2c+(i−2)}=arg min_z=(0,1)|B(i)−B(1)′−Se*(k+XOR(mod(k,2),z))|  Eqn. (10)

[0067] Device 170 then determines if the counter value i is equal to three, indicating that both watermark bits etched in B(2) and B(3) have been extracted [step 835]. If not, device 170 increments i by 1 [step 840] and returns to step 815 to extract the next watermark bit.

[0068] If device 170 determines that the counter value i is equal to three, device 170 then determines if the sliding 2×2 window has reached the last 2×2 column of the data matrix. To achieve this, device 170 determines if column parameter c is equal to w/2 [step 925] (FIG. 9). If not, device 170 increments the column parameter c by 1 [step 930] and returns to step 725 (FIG. 7). If c is equal to w/2, device 170 resets c to a value of zero [step 935] and determines if the row parameter r is equal to h/2 [step 940]. If not, device 170 increments r by 1 [step 945] and returns to step 725 (FIG. 7). If r is equal to h/2, processing completes.

[0069] Systems and methods consistent with the present invention provide mechanisms that improve the error tolerance of watermark etching algorithms. The present invention makes the watermark etching process more robust to errors by building in an error tolerance buffer when watermark bits are etched into digital data. Thus, using systems and methods consistent with the present invention, minor modifications (e.g., distortions) to digital data values that fall within a specified error tolerance range will not cause errors in the watermark extraction process.

[0070] The foregoing description of exemplary embodiments of the present invention provides illustration and description, but is not intended to be exhaustive or to limit the invention to the precise form disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of the invention. For example, while certain components of the invention have been described as implemented in hardware and others in software, other configurations may be possible. Also, while series of steps have been described with regard to FIGS. 3-5 and 7-9, the order of the steps may be varied in other implementations consistent with the present invention. No element, step, or instruction used in the description of the present application should be construed as critical or essential to the invention unless explicitly described as such. The scope of the invention is defined by the following claims and their equivalents.