Method and apparatus for authentication of data streams with adaptively controlled losses

[0001] This application claims the benefit of Provisional Application No. 60/495,787, filed Aug. 15, 2003. The present application incorporates the disclosure of this provisional application by reference.

[0002] 1. Field of the Invention

[0003] The present invention relates to data stream authentication, and more specifically to authentication schemes with adaptively controlled packet loss.

[0004] 2. Description of the Related Art

[0005] In many cases, it is desirable to append authentication information to a stream of data to assure a recipient that the data came from a specific source and was not modified en-route. For example, if the data is being provided to an application, then it would be important for the application that the data has not been corrupted either maliciously or by accident.

[0006] In cryptography, there are two traditional mechanisms for permitting such authentication:

[0007] 1. Message Authentication Codes (MAC)

[0008] 2. Digital Signatures

[0009] With a MAC, both the original source and the ultimate receiver must possess knowledge of a shared secret key. The sender applies a mathematical transformation involving the original data and secret key, and produces a tag. The receiver can then apply a similar transformation with the data, the tag, and the secret key to verify the origin and the integrity of the data.

[0010] With Digital Signatures, the key is split into two parts: a secret signing key and a public verification key. The public verification key can be used to verify anything signed using the secret signing key. The key is split in such a way that it is not possible to derive the private portion from the public portion. The sender applies a mathematical transformation involving the original data and secret signing key, and produces a signature. The recipient can then apply a similar transformation with the data, the signature, and the public verification key to ascertain the identity of the sender and the integrity of the data.

[0011] Digital signatures have a non-repudiation property that MACs do not. Namely, the signer cannot later deny having signed the document since the signing key is secret and was in the signer's possession. Of course, the signature owner can always claim that the secret signing key was stolen by some adversary.

[0012] Because of their nature, traditional authentication schemes do not tolerate any transformations to the data made by the source or by an intermediate. If a document is modified after it is signed, the verification step will so indicate, and will fail.

[0013] But for many applications, it is not only convenient, but sometimes necessary, to permit some specific types of modifications. For example, scalable video coding schemes, a high-level picture of the principle of which is shown in FIG. 1, have the property that a subset of the stream can be decoded and the quality is commensurate with the amount decoded. These schemes may encode video into a base layer and then zero or more “enhancement” layers. Just the base layer alone would be sufficient to view the stream. Enhancement layers are utilized to improve the overall quality.

[0014] Now, in an environment that is resource constrained, one might want to strip the enhancement layers and only send the base layers. If the entire stream has been digitally signed or authenticated in conventional ways, then by removing the enhancement layers, the original tag or signature becomes invalid. Thus the entire stream would have to be reauthenticated.

[0015] Alternatively, one may want to splice several streams of different qualities as in a simulcast situation. There may be one high-quality version of the stream, one medium-quality version of the stream, and one low-quality version of the stream. If network resources are available, then the high-quality stream may be sent, but if the network congestion goes up, then one may want to shift to the medium or low quality streams. In an alternate scenario, it could be the case that the receiver is mobile and is leaving one network environment and entering another that has different resource restrictions. The splicing situation can be considered a special case of a lossy situation where the quality of signal transmission is poor or otherwise is degraded, for example, by viewing the three data streams as one huge layered stream and imagining that two out of three frames are being discarded.

[0016] Yet another application is dynamic advertising. A source may include in a given slot a number of advertisements that can be displayed. An intermediary can then choose from among these choices which advertisement it would like to display. The choice can, for example, be based upon what the intermediary thinks will be the best advertisement for the target audience. The advertisements themselves can be created by an intermediary or some other party, and can be provided to the source either in their original form or may be hashed. The source would then include them when signing the stream.

[0017] Thus, signature schemes that can handle these types of losses in a secure manner are needed. Here, “secure” means that the ultimate end receiver can determine with overwhelmingly high confidence that the data it receives comes from a stream that was originally signed validly, but for which certain portions were removed. In addition, there is also a need for an intermediary that can adaptively and intelligently decide which blocks to drop.

[0018] One conventional solution to the controlled loss authentication problem is to authenticate each packet individually. This solution has two substantial drawbacks. First, in the case of using digital signatures, a fairly expensive computation must be performed for each packet. Second, in both the digital signature and MAC case, authentication information must be appended to each packet, which may not be feasible in consideration of efforts to remove portions of the stream stem to meet bandwidth constraints. levels. Then, the decoder 984 is designed to select one of the tone voltages of 64 IEEE/ACM Transactions on Networking, 7(4):502:513, August 1999, the authors propose a solution in which each data element is hashed, and then the resulting hashes are digested using a Merkle-tree. The root of the Merkle tree is authenticated. Then, with each data element, the co-nodes are sent, thereby allowing the receiver to authenticate without it. Since Wong and Lam deal with per-packet authentication, each packet contains authentication information. In particular, if |v| is the size, in bytes, of a Merkle tree node, h is the height of the Merkle tree, then each data element transmitted must be accompanied by |v|×|h| bytes. Thus, this approach does not deal with the controlled loss authentication problem, and is not bandwidth efficient.

[0019] In R. Johnson, D. Molnar, D. Song, and D. Wagner, Homomorphic Signature Schemes—RSA 2002, Cryptographer's Track, the authors propose a redactable signature scheme. It permits certain specific transformations on the data while still allowing the receiver to verify. It also allows arbitrary deletion of substrings in a signed document and has applications for censoring. Suppose n message blocks m=m₁, . . . , m_n are to be signed, and assume that n is a power of 2. The scheme starts with an initial secret key k and uses it to generate n keys k₁, . . . , k_n with the aid of a tree-like construction such as that of Goldreich, Goldwasser, and Micali (GGM), O. Goldreich, S. Goldwasser, and S. Micali, How to Construct Random Functions, Journal of the ACM, vol. 33, No. 4, 1986, pages 210-217. Then, to sign message m, the triplets (0, m₁, k₁), . . . , (0, m_n, k_n) are hashed in a Merkle-like tree and the root r is signed to produce the signature s. The difference between this tree and a regular Merkle tree is that the value 1 is pre-pended before the internal hashes are computed. With knowledge of k, anyone can verify s. However, in order to censor the data stream, the value of k is never published. Instead, only certain intermediate values of the GGM tree are published. These values correspond to the information needed to derive the final keys k_i corresponding to the data elements which are not censored. With uncensored blocks, the intermediate GGM values, and the co-nodes in the Merkle-like tree, the signature can be verified. However, the above Homomorphic Signature Scheme takes precautions, via a GGM tree, to protect the confidentiality of censored data and requires all uncensored message blocks, all co-nodes, and all keying information in order to permit verification, and thus is not efficient.

[0020] Accordingly, there has been a need for a secure authentication scheme that permits controlled removal of certain blocks in a stream without weakening the receiver's ability to verify the authentication information, and without requiring confidentiality of censored data.

[0021] In view of the foregoing, it is an object of the present invention to provide schemes for secure authentication under adaptive data loss both in the symmetric setting (with MAC) or in the asymmetric setting (with digital signatures), which are efficient with respect to the computation requirements of the sender, receiver, and intermediary, as well as the bandwidth requirements of the channels over which these parties communicate.

[0022] Briefly, the present invention addresses the following problems:

[0023] 1. adaptive loss (subsequence) authentication, wherein data chunks are removed arbitrarily;

[0024] 2. simulcast authentication, wherein several data streams are intertwined and only one data chunk is taken at a time from a given stream, and the data from the other streams is dropped; and

[0025] 3. adaptively lossy simulcast authentication, wherein sometimes the entire data chunk is dropped altogether.

[0026] The present invention provides the following schemes:

[0027] 1. Linear Scheme for Subsequence Authentication;

[0028] 2. Linear Scheme for Simulcast Authentication;

[0029] 3. Tree Scheme for Subsequence Authentication; and

[0030] 4. Tree Scheme for Simulcast Authentication.

[0031] Each of the above schemes may incorporate either a digital signature or a MAC. Therefore, the present invention implicitly provides 8 (=4×2) schemes.

[0032] The schemes use cryptographic hash functions to process the blocks of the original stream and create a short digest. A digital signature or MAC is then applied to the digest, thereby providing authentication information. If the receiver is given the entire stream, then it can recompute the digest and verify the signature. When specific portions of the stream need to be removed, the remover sends information that allows the receiver to efficiently compute the digest. The amount of information provided to the receiver in this setting is related to the output size of the cryptographic hash function and is otherwise independent of the actual data stream.

[0033] According to one aspect of this invention, Linear Scheme for Subsequence Authentication, the intermediary or source can remove arbitrary blocks (irrespective of their location) while still permitting the receiver to authenticate information. The scheme involves computing a two-layer hash chain and providing the recipient with various values in this chain. The scheme is online for the receiver in the sense that the receiver does not have to incur any delay in verifying the authentication information. In an optimization and generalization to this scheme, one second layer-hash is computed for every bundle of r first-layer hashes. When r=1, the scheme is the original linear scheme for subsequence authentication. In an improvement to this scheme, several first-layer hashes are aggregated before performing the second-layer hash. Consequently, fewer second-layer hashes need to be performed.

[0034] According to a second aspect of this invention, Linear Scheme for Simulcast Authentication, the intermediary or source is provided with multiple streams and can arbitrarily switch among which stream it transmits while still permitting the receiver to authenticate information. The scheme involves computing a multi-layer hash chain and providing the recipient with various values in this chain. The scheme is online for the tone power source lines by ½, ¼, ⅛, . . . . Then, the area occupied by the authentication information.

[0035] According to a third aspect of this invention, Tree Scheme for Subsequence Authentication, the intermediary or source can remove arbitrary blocks (irrespective of their location) while still permitting the receiver to authenticate information. The scheme involves computing a hash tree and providing the recipient with various values in this tree. In the case that some subset (of size greater than one) of dropped blocks constitute a subtree of the hash tree, the hashed scheme is more efficient with respect to bandwidth than the corresponding linear scheme. The scheme is not online for the receiver in the sense that the receiver must wait for all blocks before being able to verify the authentication information.

[0036] According to a fourth aspect of this invention, Tree Scheme for Simulcast Authentication, the intermediary or source is provided with multiple streams and can arbitrarily switch among which stream it transmits while still permitting the receiver to authenticate information. The scheme involves computing a hash tree and providing the receiver with various values in this tree. The scheme is not online for the receiver in the sense that the receiver must wait for all blocks before being able to verify the authentication information.

[0037] In all aspects of this invention, it is assumed that the sender has possession of all data to be signed at the onset. In most cases, such as when media is pre-recorded, this will not be a concern. In the case of a live stream, the present invention breaks the stream into smaller chunks and applies the schemes specified herein. Those skilled in the art will recognize that variations and modifications can be made without departing from the spirit of the invention.

[0038] The present invention permits a situation in which an intermediary may adaptively and intelligently decide which blocks are to be dropped. The schemes of the present invention readily adapt to any model for dropping blocks. Moreover, the intermediary is not required to know of any cryptographic keying material. Furthermore, if the source provides the intermediary with various hash values, then the intermediary can avoid having to do any cryptographic related computation. Instead, it just has to forward the blocks it desired together with the hash information for those blocks that are dropped.

[0039] All of the inventive schemes have the property that, given knowledge ahead of time that a given block will not be dropped, then the first layer hash on that block will not be performed. That is, the first layer hash for just that block can be replaced with the identity function (h(x)=x).

[0040] Both the linear and tree-based schemes can take advantage of correlation among blocks of data. For example, in the tree-based scheme, if a given subset of blocks has the behavior that all will be dropped or all will be kept, then these blocks can be placed as all the leaves of the same subtree. In the event that all packets in the given subset are dropped, only the root has to be transmitted. However, this concept applies even if the correlation is probabilistic. For example, if a given block being dropped makes it more likely that another block will be dropped, then these blocks should also be clustered. Likewise, in the linear schemes, if a given sequence of frames are to be all kept or dropped, these frames can be treated as a single block unit to be hashed. Then, if the entire sequence of frames is dropped only a single hash value needs to be sent.

[0041] The present invention is described herein with reference to the accompanying drawings, similar reference numbers being used to indicate functionally similar elements.

[0042] FIG. 1 shows a high-level depiction of a scalable coder.

[0043] FIG. 2 shows a block diagram of a sender or source.

[0044] FIG. 3 shows a block diagram of a receiver.

[0045] FIG. 4 shows a block diagram of an intermediary.

[0046] FIG. 5 shows a block diagram of a system including a sender, a receiver, and an intermediary.

[0047] FIG. 6 illustrates a Merkle Tree with eight leaves.

[0048] FIG. 7 illustrates a basic linear subsequence authentication scheme according to one embodiment of the present invention.

[0049] FIG. 8 illustrates an optimized linear subsequence authentication scheme with r=3 and with n a multiple of r, according to one embodiment of the present invention.

[0050] FIG. 9 illustrates a basic linear simulcast authentication scheme according to one embodiment of the present invention.

[0051] FIG. 10 illustrates a tree-based subsequence authentication scheme according to one embodiment of the present invention.

[0052] FIG. 11 illustrates a tree-based simulcast authentication scheme according to one embodiment of the present invention.

[0053] In the schemes of the present invention, an initial sender 200 in FIG. 2 is responsible for authenticating the data stream. As shown, each sender 200 includes a processor 201 in bidirectional communication with a memory 202. The processor 201 executes program code for carrying out the schemes of the present invention to generate, transmit or receive data streams. The memory 202 stores cryptographic keys, program codes, as well as intermediate results and other information used during execution of the schemes. A communications network 203 is provided over which the sender may communicate with receivers.

[0054] FIG. 3 shows a block diagram of a receiver which receives data streams from the sender or server or an intermediary over a communication network according to one embodiment of the present invention. The system of the present invention includes a number of receivers, which verify the received data. Each receiver 300 includes a processor 301 in bidirectional communication with a memory 302. The processor 301 executes program code for carrying out the schemes of the present invention to generate, transmit, and receive data streams. Program code may be created according to methods known in the art. The memory 302 stores cryptographic keys and the program code, as well as intermediate results and other information used during execution of the schemes.

[0055] A communications network 303 is provided over which the sender and the receivers may communicate. The communications network may be of various common forms, including, for example, a local area network (LAN), a wide area network (WAN), and/or a mobile telephone network. The network may permit either wired or wireless communications.

[0056] FIG. 4 shows a block diagram of an intermediary. There may be more than one intermediary; alternatively, the source and intermediary may be identical. If the intermediary and source are not identical, then the intermediary needs not have any cryptographic keying material. The data for the sender may pass through one or more intermediaries shown in FIG. 4 on its way to the sender or receiver. The intermediaries may choose to perform certain transformations on the data. Each intermediary 400 includes a processor 401 in bidirectional communication with a memory 402. The processor 401 executes program code for carrying out the schemes of the present invention to generate, transmit, and receive data streams. The memory 402 may store cryptographic keys.

[0057] FIG. 5 shows a block diagram of a system according to one embodiment of the present invention, including a sender 501, an intermediary 503, a receiver 505, and communication networks 502 and 504. As shown, the sender 501 transmits an original data stream with signature and optional helper information to the intermediary 503 via the communication network 502, which then transmits a reduced data stream with signature and relevant helper information to receiver 505 via communication network 504.

[0058] The above-mentioned transformations involve removing certain portions of the data. If an intermediary modifies the data stream, it will determine what information, if any, is required by the receiver to verify the authentication information associated with the stream.

[0059] M denotes a media stream that can be broken up into it blocks of length b: M=M₁M₂ . . . M_n, |M_i|=b, 1≦i≦n. H denotes a cryptographic compression function that takes as input a b-bit payload as well as a v-bit initialization vector or IV, and produces a v-bit output where typically v<b. These cryptographic compression functions are collision resistant, that is, it is hard to find two inputs m₁ and m₂ with m₁≠m₂ such that H(IV,m₁)=H(IV, m₂) for a fixed IV. It is assumed that there is a standard IV, called IV₀, that is fixed and publicly known. For notational simplicity, the description below will not explicitly list the IV as an argument in the hash function—though it should be thought of as being there implicitly.

[0060] Examples of such cryptographic compression functions are found in SHA-1 or MD5. The compression function in SHA-1 has an output and IV size of 160-bits whereas the compression function in MD5 works with 128-bit values. Both allow for a 512-bit payload size. When it is necessary to operate on data blocks that are larger than the payload size, application of the compression function is repeated. Functions that operate as such while still retaining the collision resistance property are termed cryptographic hash functions. For simplicity, this term is used below even if a data block that fits within the payload is dealt with.

[0061] For the schemes involving digital signatures, it is assumed that a public-key infrastructure exists, and that the sender has a key pair (Pk, Sk). Sk is the sender's private signing key—which can be used for appending a digital signature to a message, and Pk is the sender's public verification key which can be used to verify the authenticity of any signature issued using Pk. σ(Sk, M) denotes the digital signature algorithm on message M under signing key Sk, and v(Pk, M, σ) denotes the verification algorithm. The intermediate does not need to know either the signing or the verification key. For the schemes involving MAC, it is assumed that both the initial sender S and the ultimate receiver R share knowledge of a symmetric key, which need not be known by the intermediaries.

[0062] The schemes of the present invention make use of conventional constructs involving cryptographic compression functions. One such construct is an iterated hash function which is built from cryptographic compression functions as follows. Suppose a message M can be broken up into n blocks of length b, and H is a cryptographic compression function with a b-bit payload and a v-bit output. The iterated hash function defined by H is the value x_n where: $\begin{array}{c}{x}_1=H\left({\mathrm{IV}}_0,M_1\right)\\ x_2=H\left(x_1,M_2\right)\\ ⋮\\ x_n=H\left(x_{n-1},M_n\right)\end{array}$

[0063] Assuming that it is hard to find collisions in the compression function H, it is then hard to find collisions in the iterated hash. Typically, when one wants to digitally sign a message, an iterated hash is applied to the message, and the resulting output is signed. The methods, systems, and components of the present invention will involve similar constructions, but intermediate values will be provided to aid in verification.

[0064] Another conventional construct involving cryptographic compression functions is a Merkle tree. FIG. 6 shows a graphical depiction of a Merkle tree with eight leaves. Each leaf is the hash of the message below it. Each interior node represents the hash of its children. The root is signed. Suppose that M can be broken up into n blocks M=M₁ . . . M_n. For least 2K bits so that voltages of 4^K different voltage levels can be output from the incorporate powers other than 2. The Merkle tree associated with M under hash function H is a binary tree in which each node is associated with a specific value. There are n leaves, and each leaf l_i takes on the hash of M_i—that is, H(IV₀, M_i). Each interior (non-leaf) node then takes on the value associated with the hash of the concatenations of the values of its two children. That is, if vertex v has children v₁ and v₂ where v₁ has value x₁ and v₂ has value x₂, then the value associated with v is H(IV₀, x₁ x₂).

[0065] the selection circuit receives as input first through m-th (=2^K, where K is a root of the tree associated with the message M forming the digest is signed. If the underlying compression or hash function is collision resistant, then it will be hard to find two different messages whose Merkle root value is identical.

[0066] The present invention also makes use of the notion of the co-nodes for a given vertex in a Merkle tree. The co-nodes of a vertex v consist of the direct siblings of the vertices on the path from v to the root. Given a vertex v and its co-nodes, one can compute the sequence of hash functions that lead from v to the root.

1. Subsequence Authentication

[0067] The linear subsequence authentication scheme of the present invention allows stream authentication even when arbitrary blocks from the message are removed. As long as the blocks sent by an intermediate node are a proper subsequence of the original message, the receiver can authenticate the stream.

1.1 Signing

[0068] FIG. 7 illustrates a basic linear subsequence authentication scheme according to one embodiment of the present invention. Given a message M, signature generation follows a similar paradigm to an iterated hash, except that it uses “two hashing layers”.

[0069] Given a message M=M₁M₂ . . . M_n, in one embodiment, the present invention generates partial hash computations h₁, . . . , h_n as follows: $\begin{array}{cc}\begin{array}{c}{h}_0={\mathrm{IV}}_0\\ g_1=H\left(h_0,M_n\right)\\ h_1=H\left(h_0,g_1\right)\\ g_2=H\left(h_1,M_{n-1}\right)\\ h_2=H\left(h_1,g_2\right)\\ ⋮\\ g_n=H\left(h_{n-1},M_1\right)\\ h_n=H\left(h_{n-1},g_n\right)\end{array}& \left(1\right)\end{array}$

[0070] In the process of computing h₁, . . . , h_n, the scheme shown in FIG. 7 computes auxiliary hash values g₁, . . . , g_n which are not sent. The initial sender S transmits (M, σ_Sk(h_n)). The value of IV₀ can be used as the IV for the computation of all the g_i values.

[0071] Alternatively, the sender S may decide to transmit the hash values h_i along with the message blocks (M₁, h_n−1, σ_Sk(h_n)), (M₂, h_n−2) . . . (M_n,h₀).

1.2 Signature Update

[0072] If an intermediate node wants to strip off k arbitrarily located message blocks, the node generates a resulting “message” M′, identical to M but where k blocks have been removed. The receiver needs to be able to authenticate M′.

[0073] Given the received n-block message M, the intermediate node computes “new” blocks M₁′, . . . , M_n′. For each message block M_n−i+1 (starting from the end, i=1 to i=n), the intermediate node computes the corresponding auxiliary and partial hashes as follows:
g_i=H(h_i−1, M_n−i+1),
h_i=H(h_i−1, g_i)  (2)

[0074] Depending on whether the block will be forwarded or dropped, the intermediate node computes
M′_n−i+1=M_n−i+1, if block M_i is forwarded, or g_i, if block M_i is dropped  (3)

[0075] Let t be the index of the last message block that the intermediate node wants to send to the receiver, such that M_t′=M_t, and M_l′≠M_l for all l>t. The intermediate node finally transmits M₁′, . . . M_n′σ_Sk(h_n), h_n−t)

[0076] Some standard encoding is applied to the block contents to facilitate distinguishing between “message blocks” and “hashes”. Skilled artisans would appreciate that there are numerous ways to perform this encoding.

[0077] Alternatively, to enable on-line verification, the intermediate node transmits (M₁′,h_n−1,σ_Sk(h_n)), (M₂′,h_n−2), . . . (M_n′, h₀)

1.3 Verification

[0078] The receiver can verify the signature by computing h_n from M₁′, . . . , M_k′ and h_n−t as follows: for each message block M′_n−i+1 (starting from the end, i=1 to i=n), and depending on whether the received block is a “message block” or a “hash”, it computes
h_i=H(h_i−1, H(h_i−1, M′_n−i+1)), if M′_n−i+1 is a “message block” H(h_i−1, M′n−i+1)), if M′_n−i+1 is a “hash”  (4)

[0079] The receiver can then verify the signature on h_n as normal using the verification algorithm v.

[0080] The alternative on-line verification proceeds as follows: the receiver computes the partial hash h_n from (M′₁, h_n−1) using relation (4) and then it verifies the signature on the partial hash h_n. Afterwards, for i=2, . . . , n, it computes the partial hash h_i from (M′_i, h_n−i) using (4) and verifies that the so computed hash matches the hash value received in iteration i−1.

1.4 Security

[0081] As mentioned above, the iterated hash construction is collision resistant so long as the underlying hash function H is as well. In particular, if one finds a collision in the iterated construction, then at some point there is an internal collision, which means one can find a collision on the hash function H. If an adversary can come up with a non-subsequence forgery (that is, a message/signature pair that is not obtained by merely taking a subsequence of the original message), then it is possible to show that one can demonstrate either a collision in the hash function or a forgery on the underlying signature scheme. Therefore, as long as the signature scheme is not easily susceptible to forgery and the hash function is not easily susceptible to collisions, the scheme presented above is secure.

1.5 Performance

[0082] When the intermediary removes blocks, it only needs to compute the hash of the block being removed. This computation does not involve any public-key steps and is fairly efficient. In fact, the throughput of algorithms like SHA-1 is on the order of a few hundred megabits per second. Moreover, if the intermediate nodes are resource bounded with respect to computation, the source can follow the alternative approach and include the intermediate h_i values. In the case of SHA-1, each such value is 20-bytes long, so the bandwidth overhead will likely be quite small.

[0083] A tradeoff between bandwidth usage and buffering/computation is possible by sending some intermediate h_i values selectively. If the receiver can store up to b message blocks, then the intermediate node can send the hash value h_n−b only after b message blocks. Authentication can be done as described above starting from h_n−b. Then, the intermediate node sends a second “bundle” (next b message blocks and h_n−2b), which is authenticated by recomputing the partial hashes h_n−b, . . . , h_n−2b+1 and then verifying the recomputed hash value h_n−b matching the one received in the first bundle.

[0084] The computations of this embodiment do not require storing the entire stream in memory since only a single input block to the hash function is needed at any given time.

[0085] The scheme of the first embodiment permits the role of an intermediary which can adaptively and intelligently choose to remove any number of blocks without requiring knowledge of any cryptographic keying material. Moreover, the intermediary can be proximate to the receiver and can control the loss (and therefore the amount of hash information) dynamically. Furthermore, the authentication information can be verified in an online manner by the receiver. That is, the receiver can verify the authentication information as it receives the stream, and will not be required to do any form of extensive buffering. Also, the first layer hash computations are not required for any block that will not be dropped. For example, an MPEG I-frame or the base layer of a scalable coding scheme will not be intentionally dropped. For these blocks, only the second layer is required. In this instance, the first layer hash function for that block can be replaced with the identity function h(x)=x. In a similar spirit, if a given sequence of frames will either all be dropped or all be kept, then the above scheme is even more advantageous since it can cluster these as a single block before hashing.

2. An Efficiency Improvement to the Subsequence Authentication

[0086] The second embodiment of the present invention provides an efficiency improvement to the basic linear subsequence authentication, by aggregating several first layer hashes before performing the second layer hashes. As a result, the method according to the second embodiment performs fewer second layer hashes. For a typical compression function, such as the one accompanying SHA-1, the payload size is 64 bytes whereas the digest size is 20 bytes. As a result, in this situation, three digests can be concatenated together before the second layer function is called. In the second embodiment, it is assumed that r hashes are aggregated. In addition, for any decimal number a, └a┘ denotes the smallest integer greater or equal than a, and ┌a┐ denotes the largest integer less or equal than a.

2.1 Signing

[0087] For a message M, signature generation according to the second embodiment follows a similar paradigm to the scheme of the first embodiment, and uses “two hashing layers”. However, the scheme of the second embodiment involves fewer hashes than that of the first embodiment. FIG. 8 shows an improved linear subsequence authentication scheme according to one embodiment of the present invention, with r=3 and with n a multiple of r. In this scheme groups of r first-layer hashes are hashed in the second layer. Given a message M=M₁M₂ . . . M_n, the scheme of the second embodiment generates the partial hash computations h₁, . . . , h_m, where $m=\lceil \frac{n}{r}\rceil \mathrm{as}\mathrm{follows}\text{:}$ $\begin{array}{cc}\begin{array}{c}{g}_1=H\left({\mathrm{IV}}_0,M_n\right)\\ g_2=H\left({\mathrm{IV}}_0,M_{n-1}\right)\\ ⋮\\ g_r=H\left({\mathrm{IV}}_0,M_{n-\left(r-1\right)}\right)\\ h_1=H\left({\mathrm{IV}}_0,g_1,\dots ,g_r\right)\\ g_{r+1}=H\left({\mathrm{IV}}_0,M_{n-r}\right)\\ g_{r+2}=H\left({\mathrm{IV}}_0,M_{n-\left(r+1\right)}\right)\\ ⋮\\ g_{2r}=H\left({\mathrm{IV}}_0,M_{n-\left(2r-1\right)}\right)\\ h_2=H\left(h_1,g_{r+1},\dots ,g_{2r}\right)\\ ⋮\\ g_{r\lfloor \frac{n}{r}\rfloor -r+1}=H\left({\mathrm{IV}}_0,M_{n-\left(r\lfloor \frac{n}{r}\rfloor -r\right)}\right)\\ g_{r\lfloor \frac{n}{r}\rfloor -r+2}=H\left({\mathrm{IV}}_0,M_{n-\left(r\lfloor \frac{n}{r}\rfloor -r+1\right)}\right)\\ ⋮\\ g_{r\lfloor \frac{n}{r}\rfloor }=H\left({\mathrm{IV}}_0,M_{n-\left(r\lfloor \frac{n}{r}\rfloor -1\right)}\right)\\ h_{m-1}=H\left(h_{m-2},g_{r\lfloor \frac{n}{r}\rfloor -r+1},\dots ,g_{r\lfloor \frac{n}{r}\rfloor }\right)\\ g_{r\lfloor \frac{n}{r}\rfloor +1}=H\left({\mathrm{IV}}_0,M_{n-\left(r\lfloor \frac{n}{r}\rfloor \right)}\right)\\ ⋮\\ g_n=H\left({\mathrm{IV}}_0,M_1\right)\\ h_m=H\left(h_{m-1},g_{r\lfloor \frac{n}{r}\rfloor +1},\dots ,g_n\right)\end{array}& \left(5\right)\end{array}$

[0088] Similarly to the scheme of the first embodiment, in the process of computing h₁, . . . , h_m, the scheme of the second embodiment computes auxiliary hash values g₁, . . . , g_n which are not sent. The initial sender transmits (M, σ_Sk(h_m)), and the value of IV₀ can be used as the IV for the computation of all the g_i values.

[0089] Alternatively, the sender may decide to transmit the hash value h_i along with every r-th message block (M₁,σ_Sk(h_m)),(M₂), . . . ,(M_r, h_m−1),(M_r+1), . . . , (M_2r,h_m−2), . . . (M_n,h₀).

2.2 Signature Update

[0090] Now, suppose an intermediate node wants to strip off n−k arbitrarily located message blocks. It generates a resulting “message” M′, identical to M but where n−k blocks have been removed. The receiver needs to be able to authenticate M′.

[0091] Given the received n-block message M, the intermediate node computes “new” blocks M′₁, . . . , M′_n. For each message block M_n−i+1 (starting from the end, i=1 to i=n), it computes the corresponding auxiliary and partial hashes
g_i=H(IV₀, M_n−i+1)  (6)

[0092] Depending on whether the block will be forwarded or dropped, the intermediate node computes
M′_n−i+1=M_n−i+1, if block M_i is forwarded, or g_i, if block M_i is dropped  (7)

[0093] The hash values h_m, . . . , h₁ are computed as in the signing operation. The intermediary finally transmits M₁′ . . . M_n′k,σ_Sk(h_m).

[0094] The above transmission requires buffering r packets to perform verification. In practice r will be quite small. For a SHA-1 based scheme r=3 and for an MD-5 based scheme, r=4.

[0095] Alternatively, the intermediary may transmit the hash values hi along with the “new” message blocks (M₁′,σ_Sk(h_m)),(M₂′), . . . ,(M_r′, h_m−1),(M_r+1′), . . . , (M_2r′,h_m−2), . . . , (M_n′,h₀).

2.3 Verification

[0096] The receiver can verify the signature by computing h_m from M′₁, . . . , M′_n as follows. First, for each message block M′_n−i+1 (starting from the end, i=1 to i=n), and depending on whether the received block is a “message block” or a “hash”, the receiver computes
g′_i=H(h_i−1,M′_n−i+1), if M′_n−i+1 is a “message block” M′_n−i+1, if M′_n−i+1 is a “hash”  (8)

[0097] Finally, the receiver computes h_m: $\begin{array}{cc}\begin{array}{c}{h}_1=H\left({\mathrm{IV}}_0,g_1^{\prime },\dots ,g_r^{\prime }\right)\\ h_2=H\left(h_1,g_{r+1}^{\prime },\dots ,g_{2r}^{\prime }\right)\\ ⋮\\ h_m=H\left(h_{m-1},g_{r\lfloor \frac{n}{r}\rfloor +1},\dots ,g_n\right)\end{array}& \left(9\right)\end{array}$

[0098] The receiver can then verify the signature on h_m as normal using the verification algorithm v.

[0099] To perform online verification, the receiver needs to be able to compute the intermediate hash h_i. To do so, the receiver needs to buffer r blocks so it can compute the appropriate g values. The online verification of this scheme is analogous to that of the first embodiment.

2.4 Security

[0100] Similarly to the first embodiment, so long as the signature scheme is not easily susceptible to forgery and the hash function is not easily susceptible to collisions, the scheme of the second embodiment is secure.

2.5 Performance

[0101] Similarly to the first embodiment, when the intermediary removes blocks, it only needs to compute the hash of the block being removed.

[0102] It takes less time for the subsequence scheme of the second embodiment to both compute and verify the signature compared to the subsequence scheme of the first embodiment, since only one second-layer hash is performed for every r first layer hashes. If r is chosen carefully (for example, setting r=3 for SHA-1 or r=4 for MD-5), then each second-layer hash only requires a single call to the compression function. So, in the second embodiment, only $\lceil \frac{n}{r}\rceil$
compression function calls are made in the second layer compared to the n calls in the first embodiment.

[0103] In addition to the advantages of the first embodiment, the receiver of the second embodiment can verify the authentication information after receiving every r blocks. In practice, r will be fairly small—on the order of 2 or 3, thus reducing the number of the second layer hashes.

3. Simulcast Authentication: the Multiplex Scheme

[0104] Now, assume the original sender S transmits k different streams M⁽¹⁾, M⁽²⁾, . . . , M^(k) simultaneously. Each stream consists of n blocks of length b, M^(j)=M₁^(j), . . . , M^n(j). The scheme of the third embodiment allows the intermediate node not only to select one stream and retransmit it in an authenticated fashion, but also to “switch” to some other stream adaptively (at any point during block transmission). Of course, the receiver should be able to authenticate the resulting stream.

3.1 Signing

[0105] FIG. 9 shows a basic linear simulcast authentication scheme according to one embodiment of the present invention. Given a message M, signature generation follows the same approach as in the first and second embodiments, i.e., reverse iterated hash, but computing partial hashes of every block in each stream.

[0106] Given messages M⁽¹⁾, M⁽²⁾, . . . , M^(k), where M^(j)=M₁^(j), M₂^(j), . . . , M_n^(j), the scheme of the third embodiment of the present invention generates the partial hash computations h₁, . . . , h_n as follows: $\begin{array}{cc}\begin{array}{c}{d}_1^{\left(1\right)}=H\left(M_n^{\left(1\right)}\right)\\ d_1^{\left(2\right)}=H\left(M_n^{\left(2\right)}\right)\\ ⋮\\ d_1^{\left(k\right)}=H\left(M_n^{\left(k\right)}\right)\\ h_1=H\left(h_0,d_1^{\left(1\right)},\dots ,d_1^{\left(k\right)}\right)\\ ⋮\\ d_n^{\left(1\right)}=H\left(M_1^{\left(1\right)}\right)\\ d_n^{\left(2\right)}=H\left(M_1^{\left(2\right)}\right)\\ ⋮\\ d_n^{\left(k\right)}=H\left(M_1^{\left(k\right)}\right)\\ h_n=H\left(h_{n-1},d_n^{\left(1\right)},\dots ,d_n^{\left(k\right)}\right)\end{array}& \left(10\right)\end{array}$  .
d_n⁽¹⁾=H(M₁⁽¹⁾)
d_n⁽²⁾=H(M₁⁽²⁾)
.
.
.
d_n^(k)=H(M₁^(k))
h_n=H(h_n−1,d_n⁽¹⁾, . . . ,d_n^(k))  (10)

[0107] The initial sender transmits σ_Sk(h_n) and then sends M⁽¹⁾, . . . , M^(k) simultaneously. In practice, the message blocks of the different streams will be interleaved in the transmission.

3.2 Signature Update

[0108] Suppose an intermediate node wants to select a possibly different stream (message) for each message block received. For instance, if each message encodes a video stream of different quality, the intermediate node may want to select a lower or higher quality depending on network congestion. It generates a “resulting message” M′, comprising “chunks” (consecutive message blocks) of the different streams. The intermediate node may pick a single stream (message) at each moment. It should be understood that the present invention allows for the possibility of layered streams. The receiver needs to be able to authenticate M′.

[0109] Given the received n-block messages M⁽¹⁾, . . . , M^(k), the intermediate node computes “new” blocks M′₁, . . . , M′_n. For each set of message blocks M_n−i+1⁽¹⁾, . . . , M_n−i+1^(k), (starting from the end, i=1 to i=n), it computes the partial hashes $\begin{array}{cc}\begin{array}{c}{d}_i^{\left(1\right)}=H\left(M_{n-i+1}^{\left(1\right)}\right)\\ d_i^{\left(2\right)}=H\left(M_{n-i+1}^{\left(2\right)}\right)\\ ⋮\\ d_i^{\left(k\right)}=H\left(M_{n-i+1}^{\left(k\right)}\right)\\ h_i=H\left(h_{i-1},d_i^{\left(1\right)},\dots ,d_i^{\left(k\right)}\right)\end{array}& \left(11\right)\end{array}$

[0110] Then if stream l is chosen, 1≦l≦k, it computes
M′_n−i+1=(d_i⁽¹⁾, . . . ,d_i^l−1),M_n−i+1^(l),d_i^(l+1), . . . ,d_i^(k)).  (12)

[0111] The intermediate node finally transmits M′₁ . . . M′_n,σ_Sk(h_n).

[0112] Alternatively, to enable on-line verification, the intermediate node transmits
(M′₁,h_n−1,σ_Sk(h_n)),(M′₂,h_n−2), . . . , (M′_n,h₀)  (13)
3.3 Verification

[0113] The receiver can verify the signature by computing h_n, from M′₁, . . . , M′_k and h₀=IV₀. For each message block M′_n−i+1 (starting from the end, i=1 to i=n) if M′_n−i+1 is of the form
M′_n−i+1=(d_i⁽¹⁾, . . . ,d_i^(l−1),M_n−i+1^(l),d_i^(l+1), . . . ,d_i^(k))

[0114] then, the receiver computes
d_i^(k)=H(M_n−i+1^(k))
h_i=H(h_i−1,d_i⁽¹⁾, . . . ,d_i^(l−1),d_i,d_i^(l+1), . . . ,d_i^(k))  (14)

[0115] The receiver can then verify the signature on h_n as normal using the verification algorithm v.

[0116] The alternative on-line verification procedure is straightforward. The receiver computes the partial hash h_n from (M′₁, h_n−1) using relation (14) and then it verifies the signature on the partial hash h_n. Afterwards, for i=2, . . . , n, it computes the partial hash h_i from (M′_i, h_n−i) using (14) and verifies the so computed hash matches the hash value received in iteration i−1.

3.4 Performance

[0117] In addition to the advantages of the scheme of the first embodiment, the hash step of the scheme of the third embodiment can be iterated using a compression function with either the linear chaining scheme or a Merkle scheme.

[0118] By using a Merkle tree-like construction to hash down each sequence of blocks M_i⁽¹⁾, . . . , M_i^(k), bandwidth can be saved at the cost of more intensive computation (by the intermediate node).

4. Tree Scheme for Subsequence Authentication

[0119] The fourth embodiment of the present invention is a scheme for authenticating subsequences using Merkle Trees. Like the linear subsequence authentication scheme, the tree-based scheme allows stream authentication even when arbitrary blocks from the message are removed. As long as the blocks sent by the intermediate node are a proper subsequence of the original message, the receiver can authenticate the stream. By exploiting certain aspects of the tree structure, the tree scheme is more efficient with respect to bandwidth than the linear scheme.

4.1 Signing

[0120] FIG. 10 illustrates a tree-based subsequence authentication scheme according to one embodiment of the present invention. Given a message M=M₁M₂ . . . M_n, the scheme of the fourth embodiment generates a Merkle tree shown in FIG. 6. If v denotes the root of the tree and x denotes the value associated with the root, then the initial sender transmits (M, σ_Sk(x)).

4.2 Signature Update

[0121] If an intermediary wants to strip off k arbitrarily located message blocks, the intermediary generates a resulting “message” M′, identical to M, but with k blocks removed. The receiver needs to be able to authenticate M′. Let d₁, . . . , d_k denote the indices of the blocks that will be dropped and let s₁, . . . , s_n−k denote the blocks that will stay. Given the received n-block message M, the intermediate node computes the corresponding authentication information as follows.

[0122] 1) For all blocks M_d1, . . . , M_dk that are to be dropped, the intermediary first determines the set of vertices corresponding to leaves l_d1, . . . l_dk in the Merkle tree associated with these blocks.

[0123] 2) If any pair of vertices are siblings in the Merkle tree, the intermediary replaces these two vertices both with their parent.

[0124] 3) The intermediary keeps repeating the above process until no two vertices in the set are siblings.

[0125] 4) The intermediary takes this set of vertices, and computes the Merkle tree values x₁, . . . , x_r associated with them. The intermediary can easily perform this step since the

[0126] fifth and sixth switches (405, 406) connected between the first terminal

[0127] The intermediate node finally transmits
M_s1 . . . M_sn−k,σ_Sk(x),x₁, . . . x_r  (15)

[0128] Similarly to other embodiments of the present invention, applying standard encoding to the block contents facilitates distinguishing between “message blocks” and “hashes”.

4.3 Verification

[0129] The receiver verifies the signature by computing the value of the root of the Merkle tree, using the following algorithm:

[0130] 1) For every actual message block M_si received, compute the value y_i=H(IV₀,M_si).

[0131] 2) Consider the set of all hashes y₁, . . . , y_n−k, x₁, . . . , x_r. Each of these corresponds to values of vertices in a Merkle tree.

[0132] 3) For each pair of values, if they correspond to vertices who are siblings, then replace the pair with their hash (which corresponds to the parent node).

[0133] 4) Repeat the above step until only one value remains—this value is the root.

[0134] If one has all the initial message blocks, then the above algorithm constitutes the standard algorithm for computing the root of a Merkle tree. Whenever the receiver receives some hashes x₁, . . . , x_r, these come from the intermediary running the same algorithm on the subset of missing blocks. Therefore, the intermediary and receiver have together run the algorithm on all n blocks which yield the value of the Merkle root. This is why the above computation yields the Merkle root.

[0135] With the value of the Merkle root, the receiver can verify the signature it receives.

4.4 Security

[0136] The Merkle hash construction is collision resistant so long as the underlying hash function H is collision resistant. In particular, if one finds a collision in the Merkle tree, then at some point there is a collision at an internal node, which means one can find a collision on the hash function H. If an adversary can come up with a non-subsequence forgery (that is, come up with a message/signature pair that is not obtained by merely taking a subsequence of the original message), then one can demonstrate either a collision in the hash function or a forgery on the underlying signature scheme. Therefore, as long as the signature scheme is not easily susceptible to forgery and the hash function is not easily susceptible to collisions, the scheme of the fourth embodiment is secure.

4.5 Performance

[0137] When the intermediary removes blocks, it needs to provide the receiver with a sufficient number of internal hashes to compute the Merkle root of the tree without those message blocks. The intermediary will require k hashes for each of the blocks to be dropped and then at most k−1 hashes when replacing pairs of hashes with a single hash (since a single hash results in replacing two values with a single one, thereby reducing the net number by one). The total computation is therefore at most 2k−1 hashes. The total hashes computed by a single common switch (the number of switching elements is 12) to be shared.

[0138] When the receiver receives the stream, it needs to compute the root. If it has all the message blocks, this would require 2n−1 hashes−n to initially hash each block, and then n−1 additional hashes when replacing pairs of hash values with a single hash (since a single function computation results in replacing two values with a single one, and at the end only one value is remaining). However, t of these hashes are computed by the intermediary. Therefore the receiver only has to compute 2n−1−t hashes.

[0139] The total work in this scheme between the intermediary and the receiver is at most 2n−1 hashes. In the previous linear schemes 2n hashes were required.

[0140] In terms of bandwidth, the tree based scheme may be much more efficient. Only r≦k hashes are finally sent. In the best case, if all k blocks to be dropped entirely constitute all leaves of a subtree in the Merkle tree, then only the single value corresponding to the root of this subtree is sent, that is r=1. In the worst case, if no pair of blocks are siblings, then the bandwidth requirements are the exact same as in the linear case, and k hash values need to be sent.

5. Tree Scheme for Simulcast Authentication

[0141] The fifth embodiment of the present invention is a tree-based scheme for authenticating multiple parallel streams in which one data block is selected from one stream at each step of the transmission. As in the linear multiplex setting of the third embodiment, it is assumed that the original sender S transmits k different streams M⁽¹⁾, M⁽²⁾, . . . , M^(k) simultaneously. Each stream consists of n blocks of length b, M^(j)=M_I^(j), . . . , M^n(j). This scheme allows the intermediate node not only to select one stream and retransmit it in an authenticated fashion, but also to “switch” to some other stream adaptively (at any point during block transmission). Of course, the receiver is able to authenticate the resulting stream. As in the tree-based scheme for subsequence authentication of the fourth embodiment, the scheme of the fifth embodiment exploits certain aspects of the tree structure, so as to be more efficient with respect to bandwidth than the analogous linear scheme. On the other hand, like the tree construct of the fourth embodiment, the scheme of the fifth embodiment does not readily lend itself to online verification. Instead, the receiver has to wait for all packets before it can verify. In practice, the delay can be reduced by splitting the stream into segments of reasonable size and authenticating each segment separately.

5.1 Signing

[0142] Given k different streams M⁽¹⁾, M⁽²⁾, . . . , M^(k), the signature generation of the scheme of the fifth embodiment works as follows.

[0143] 1) The signer first generates a separate Merkle tree for each stream. Let v⁽¹⁾, . . . , v^(k) denote the k roots of the tree, and let x⁽¹⁾, . . . , x^(k) denote the respective values associated with these roots.

[0144] 2) The signer then computes x=H(IV, x⁽¹⁾, . . . , x^(k)). Here the hash function H can be computed using a Merkle tree construction as well.

[0145] 3) Finally, the signer transmits (M, σ_Sk(x)).

5.2 Signature Update

[0146] Now, suppose an intermediate node wants to select a possibly different stream (message) for each message block received. For instance, if each message encodes a video stream of different quality, the intermediate node may want to select a lower or higher quality depending on network congestion. It generates a resulting “message” M′, comprising “chunks” (consecutive message blocks) of the different streams. The receiver needs to be able to authenticate M′.

[0147] If the receiver can accurately compute each of the x_i values, then it can verify the signature. Therefore, the intermediary simply has to provide the user with the information necessary to compute these values. By treating each Merkle tree separately, the intermediary can compute the set of required values as it did in the Merkle scheme of the fourth embodiment. The intermediary transmits these values to the receiver which can then compute the x_i values and in-turn verify the authentication information.

[0148] Specifically, for each i with 1≦i≦k, let ks(i) denote the number of blocks that will actually be sent from stream M⁽ⁱ⁾. For the stream M⁽ⁱ⁾, let s₁⁽ⁱ⁾, . . . , s_ks(i)⁽ⁱ⁾ denote the indices of the blocks that will be included. Let M′⁽ⁱ⁾ denote these blocks: $\begin{array}{cc}{M}^{\prime }\left(i\right)=M_{s_1\left(i\right)}^{\left(i\right)}\dots M_{s_{\mathrm{ks}\left(i\right)}^{\left(i\right)}}^{\left(i\right)}& \left(16\right)\end{array}$

[0149] As to the indices of blocks that are to be dropped, for each i with 1≦i≦k, let kd(i) denote the number of blocks that will actually be dropped from stream M⁽ⁱ⁾. For the stream M⁽ⁱ⁾, let d₁⁽ⁱ⁾, . . . , d_kd(i)⁽ⁱ⁾ denote the indices of the blocks that will be dropped.

[0150] As in the tree scheme of the fourth embodiment, for each stream M⁽ⁱ⁾ the intermediary computes the values necessary for the receiver to verify as follows:

[0151] 1) For all blocks $M_{d_1^{\left(i\right)}}^{\left(i\right)},\dots ,M_{d_k^{\left(i\right)}}^{\left(i\right)}$
that are to be dropped, the intermediary first determines the set of vertices corresponding to leaves $l_{d_1^{\left(i\right)}},\dots ,l_{d_{\mathrm{kd}\left(i\right)}^{\left(i\right)}}$
in the Merkle tree associated with these blocks.

[0152] 2) Now, if any pair of vertices are siblings in the Merkle tree, the intermediary replaces these two vertices both with their parent, i.e., the hash of concatenation of the values associated with the siblings.

[0153] 3) The intermediary keeps repeating the above process until no two vertices in the set are siblings.

[0154] 4) The intermediary takes this set of vertices, and computes the Merkle tree values X⁽ⁱ⁾=x₁⁽ⁱ⁾, . . . , x_r⁽ⁱ⁾ associated with them. The intermediary can easily perform this step since the cryptographic hash function is globally computable.

[0155] The intermediate node finally transmits the following information:
{M′⁽¹⁾, . . . , M′⁽ⁿ⁾}, σ_Sk (x), X⁽¹⁾, . . . , X^(k)  (17)

[0156] The stream is sent in the proper order, that is, blocks from each of the M′⁽ⁱ⁾ may be interleaved so that the receiver can view the stream. Some standard encoding is applied to the block contents so the receiver can distinguish between message blocks versus hash values.

5.3 Verification

[0157] The receiver verifies the signature by first computing the values of the roots of each of the Merkle trees—after that it hashes these values and verifies the signature. It achieves this goal using the following algorithm which is run for each i:

[0158] 1) First, for every actual message block M_sj⁽ⁱ⁾ received, the receiver computes the value y_j⁽ⁱ⁾=H(IV₀,M_sj⁽ⁱ⁾).

[0159] 2) Consider the set of all hashes computed above in the previous step as well the hash values contained in sets X⁽¹⁾, . . . , X^(k) received in the transmissions.

[0160] 3) For each pair of values, if the pair corresponds to vertices who are siblings, then replace the pair with their hash (which corresponds to the parent node in the Merkle tree).

[0161] 4) Repeat the above step until only one value remains—this value is the root x⁽ⁱ⁾.

[0162] If one has all the initial message blocks, then the above algorithm constitutes the standard algorithm for computing the root of a Merkle tree. Whenever the receiver receives some hashes x₁⁽ⁱ⁾, . . . , x_r⁽ⁱ⁾, these come from the intermediary running the same algorithm on the subset of missing blocks. Therefore, the intermediary and receiver have together run the algorithm on all n blocks which yield the value of the Merkle root. This is why the above computation yields the Merkle root.

[0163] With the values of the Merkle roots, x⁽¹⁾, . . . , x^(k), the receiver can compute x=H(IV, x⁽¹⁾, . . . , x^(k)) and verify the signature it receives.

[0164] FIG. 11 illustrates the signing and verification of the fifth embodiment of the invention, an example with four streams and four message blocks. As shown, each of the four streams M⁽¹⁾, M⁽²⁾, M⁽³⁾, M⁽⁴⁾ consists of four blocks. The black leaves denote the message blocks that are actually sent. The remaining ones are dropped. The shaded vertices represent the cover; that is, the values corresponding to these vertices are sent to the receiver. The roots of the four Merkle trees are x⁽¹⁾, x⁽²⁾, x⁽³⁾, and x⁽⁴⁾ respectively. The final root value x is computed by hashing the Merkle roots x⁽¹⁾, x⁽²⁾, x⁽³⁾, x⁽⁴⁾. This hash can be also be performed in a Merkle-like fashion. Finally, the value x is actually signed. In this scheme only six hash values are sent to the receiver. In the linear simulcast scheme, twelve hashes (three per each block transmitted) would have been transmitted. Thus, savings is achieved whenever dropped blocks are clustered. For example, in FIG. 11, all blocks in the stream M⁽⁴⁾ are dropped. As a result, one only needs to send the root x⁽⁴⁾ of the associated Merkle tree.

[0165] Also, because the Merkle roots are themselves hashed in a Merkle-like construction, there is room for further optimization. In particular, suppose that all blocks are dropped for two entire subtrees whose Merkle roots are siblings in the even larger tree. Then, instead of sending the two Merkle roots, their hash could be sent.

5.4 Security

[0166] Similarly to the fourth embodiment, the fifth embodiment is secure as long as the signature scheme is not easily susceptible to forgery, and the hash function is not easily susceptible to collisions. Thus the invention presented above is secure.

5.5 Performance

[0167] The performance of the fifth embodiment can be analyzed by extending the analysis for the tree-based subsequence scheme and the linear simulcast scheme.

[0168] In all embodiments above, a hash function with a specific payload size and a specific IV is used. The chaining constructions tend to take some existing output and use that as the IV of the next block. In a further embodiment, instead of loading the current output as an IV, the current output can be concatenated to the next payload.

[0169] The linear and tree schemes of the present invention can be combined to obtain hybrid solutions, giving rise to useful tradeoffs. In a further embodiment, a scheme starts by splitting each stream M⁽ⁱ⁾ into segments of length b blocks. Then, a tree scheme is applied on the first segment of all streams to compute the Merkle root x₁, then the root on the second segment, and so on, until all segments are processed. In this way, Merkle roots x₁, . . . , x_└n/b┘ are obtained. Instead of signing each one of these roots, as in the tree schemes described above, the roots are combined using the linear scheme. Hence, if the receiver can buffer b blocks, then verification can be done “on-line”. Moreover, the communication overhead is decreased compared to the plain linear scheme since for each segment of b blocks, the number of transmitted hashes may be much less than the number of dropped blocks (although equal on the worst case). A similar approach can be taken for subsequence authentication. This hybrid approach allows trading buffer space for communication overhead.

[0170] In a further embodiment, a linear scheme is applied to each stream, and then a Merkle tree is computed on the results.

[0171] Although the embodiments described above use binary Merkle trees, the constructions can be applied to general trees. It may be more advantageous to group certain blocks together if they have similar behavior; i.e., they either all will be dropped or all will be kept.

[0172] If there are correlations among blocks, then it makes sense to cluster these blocks together in the tree-based schemes. For example, if a group of blocks will either all be dropped or all be kept, it is advantageous to have these blocks constitute all the leaves of a subtree. Then, if the packets are dropped, only the root of the subtree must be sent.

[0173] In addition, the Merkle tree construction could be optimized. In one embodiment, if one of the streams will more likely be used than the others, it is advantageous to use a lopsided Merkle tree in which the priority stream is close to the root (e.g., perhaps right below it). In conjunction with the hybrid scheme mentioned previously, the streams are prioritized, so that the high priority streams are closer to the final value in the chain. This ordering particularly makes sense when layered streams are used. In such cases, the voltage difference between V(T1) and V(,, 2) to 2:1.

[0174] There are blocks that should never be dropped, such as, an I frame in an MPEG stream, or the base layer in a scalably coded stream. The signer can avoid directly computing the initial first-layer hash on a block that will not be dropped. In the linear schemes, there are two hash layers. If a block will not be dropped, then there is no need to compute the hash in the first layer; instead only the second layer needs to be computed.

[0175] The schemes of the present invention can be interpreted as having two phases. In the first phase, it finds a convenient way to hash each data block. In the second phase, it signs the hashes. The reason for doing so is that if a block is dropped, it is not necessary to retransmit it in its entirety. Instead, only the hash computed in the first phase is transmitted. This information is sufficient to allow the receiver to verify, since the signature can be viewed as being performed on the hashes. dividing the voltage difference between V(T1) and V(,, 2) to 2:1. In FIG. 6, the is, the sender drops particular blocks on purpose. Of course, in many practical applications, one may have to deal with uncontrolled loss situations. These situations may occur, for example, if the transport protocol is not reliable such as the case with UDP, or if the environment is subject to lossy behavior such as is the case with wireless networks. The present invention can be used to deal with the uncontrolled loss by replicating the hashes that would be sent if the packet were dropped.

[0176] By applying Forward Error Correction (FEC) techniques such as Erasure Codes to the hashes of the present invention, it is possible to deal with the uncontrolled loss situation without having to replicate. This approach might be especially useful in a multicast setting where different receivers have lost different packets but can be provided with identical error-correcting information. One consideration of this approach is that the receiver must perform a decoding step so may have to compromise the ability to verify authentication information in an online manner.

[0177] Moreover, schemes of the present invention involve an intermediary which can adaptively choose the amount of forward error correction to the authentication information (i.e., hash outputs). In other words, rather than having a source estimate how much loss will occur and include sufficient authentication forward error correction information to accommodate that, the source can choose not to include authentication forward error correction information at all, and instead allow an intermediary to include the authentication forward error correction information dynamically to further increase the probability that the stream can be authenticated.

[0178] The intermediary becomes an integral part of a scheme which considers both uncontrolled losses handled through forward error correction as well as adaptive and intelligent controlled losses. For example, in the Merkle tree constructions, it may suffice for the recipient to recover intermediate nodes (as opposed to just leaf nodes). In such a case, the intermediary can choose to supply forward error correction information to allow recovery of the (possibly interior) nodes necessary to authenticate, thus requiring possibly less forward error correction information.

[0179] If the intermediary is sending different versions of the same stream to multiple receivers, because, for example, each has a different resource constraint with respect to the quality they view, the intermediary can recycle the work effort. In particular, the between the terminals T1, T2 and adapted to receive as input data bit signals D1B, most one full set of first-layer hashes.

[0180] Along these lines, work can be recycled between the source and the intermediary. That is, the source can provide the intermediary with any necessary hash computations for assisting with authentication. Then, the intermediary is not required to perform any work of a cryptographic nature. Instead, it can choose which blocks to drop and select the corresponding authentication information to be transmitted.

[0181] Another application of the present invention is insertion and selection of advertisements in a stream. The intermediary or some other party provides advertisements or a hash of advertisements, for example hashed using a Merkle tree, to the source. The source then includes the Merkle hash in its stream as a placeholder, allowing the intermediary to choose which advertisement it would like to use. Of course, this concept is not necessarily limited to advertisers.

[0182] Although the focus of the present invention is on authenticating information, the above scheme can also be used in conjunction with an encryption scheme provided that the scheme is designed to permit the recipient to decrypt a given block without requiring the decryption of or presence of many other blocks. Two block cipher encryption modes facilitate this approach. One is counter-mode encryption and the other is electronic code book (ECB) encryption. Alternatively, it is possible to use a stream cipher, though a caveat is that the receiver may need to perform work that is proportional to the size of the original stream as opposed to the portion of it that he receives. One may be able to use chaining or feedback modes (cipher block chaining (CBC), output feed back (OFB), etc) provided that the receiver receives any intermediate information to decrypt. Such information may include intermediate IVs or actual ciphertext blocks. Yet another approach is to mix the modes, i.e., for large segments which will not be dropped, a chaining or feedback mode can be used; whereas for other blocks, a counter mode or ECB mode can be used. For example, in an MPEG stream, I-frames are never dropped intentionally, so they can be treated differently and encrypted using CBC mode. A similar remark applies to the base layer of any scalable coding scheme.

[0183] While the invention has been described in detail above with respect to various embodiments, the ordinarily skilled artisan will appreciate that variations of these embodiments are possible without departing from the scope and spirit of the invention. Therefore, the invention should be considered as limited only by the scope of the appended claims.