Error concealment in digital transmissions

[0002] CROSS REFERENCE TO RELATED APPLICATIONS

[0003] This is a continuation of Ser. No. 09/345,546 filed Jun. 29, 1999 now abandened which is a continuation of Ser. No. 08/778,949, filed Jan. 6, 1997, which issued as U.S. Pat. No. 6,006,173, which is a continuation of Ser. No. 08/648,484, filed May 15, 1996, now abandoned, which is a continuation of Ser. No. 07/962,216, filed Jan. 26, 1993, now abandoned which claims priority through and is a continuation of PCT/EP92/00754, filed Apr. 3, 1992, which claims priority through German Patent Application P4111131.1, filed Apr. 6, 1991.

[0004] The invention relates to a method of transmitting or storing, over an interference affected channel, digital audio signals, wherein transmission errors are detected at the receiving end and corrected if necessary or masked, with the masking being effected in that the interfered-with signal section is muted or replaced by a signal preceding the interfered-with signal section in the same channel or by a synchronous, not interfered-with signal section of an adjacent channel. Such a method is disclosed in German Patent DE 3,638,922.C2.

[0005] In digital audio signal transmissions and storage, the received and read-out audio signals, if they contain bit errors, may either be not decodable at all or not decodable in part after all error detection and error correction methods have been exhausted. In that case, it is the custom to switch to a decoder for muting over a broad band in such a way that for a certain time period the entire signal is set at 0. In the case of digital audio signals transmitted by radio, this case occurs relatively frequently at the fringes of the reception area which is extremely annoying particularly in connection with mobile reception. The same applies for audio signal storage if the tape material or audio heads are worn to a degree that exceeds a tolerance value.

[0006] To reduce the annoying effect of missing signals in radio transmissions, German Patent DE 3,638,922.C2 discloses a mutual offset in time between the left and right stereo channels and, if there is an uncorrectable signal interference, placing the complementary stereo information transmitted at an earlier or later point in time instead of the interfered-with original information. Although the complementary stereo information is not identical with the associated, interfered-with original information, but is connected with it only by way of left-right correlation, such a substitution is acceptable in any case for a short period of time since direction and distance perception as well as the perception of spatial relationships are subject to a certain inertia in the human ear. However, if such a masking technique is employed for a longer period of time, the stereophonic impression is lost, since the masking always covers the full bandwidth of the interfered-with signal and therefore also replaces spectral signal components that did not suffer interference.

[0007] In contrast thereto, it is the object of the invention to provide, in a method of the above-mentioned type, a subjectively better masking of errors which permits better reconstruction of the interfered-with original audio event and, particularly also during longer use, maintains the stereophonic impression.

[0008] The above and other objects are accomplished according to the invention by the provision of a method of transmitting or storing, over an interference affected channel, digital audio signals that have been subjected to data reduction to result in a reduced data, digital audio signal that is present in at least one of the time domain and spectral domain, depending on source coding, the method comprising the steps of: detecting an interfered-with signal section in the reduced data, digital audio signal at a receiving end; and masking the interfered-with reduced data, digital audio signal section by one of (a) muting only interfered-with spectral values or subbands, or groups of spectral values or subbands in the signal section and (b) replacing only interfered-with spectral values or subbands, or groups of spectral values or subbands on the signal section with a signal component from the same channel or an adjacent channel including at least one of time domain sampled values, spectral domain sampled values, scale factors, and control information, depending on the source coding of the reduced data, digital audio signal.

[0009] The invention is based on the consideration of intentionally muting, repeating, estimating or replacing for a certain period of time only those spectral components of a complete audio signal that have actually experienced interference. The invention takes advantage of the fact that reduced data, digital audio signals (i.e., digital audio signals subjected to known data reduction processes) are present in the time and spectral domains, depending on whether they are subdivided into subbands (subband coding) or into spectral values (transformation coding). The following masking strategies can be employed:

[0010] 1. Muting

[0011] With this masking strategy, only those spectral components of the interfered-with signal are muted which have actually been interfered with. Depending on the source coding method employed, these may be individual subbands or spectral values or groups of subbands or spectral values of a digital audio signal that has been subdivided into n subbands or spectral values, with n being a value equal to or greater than 1.

[0012] 2. Repeating

[0013] With this masking strategy, components of the same channel signal are employed as substitutes for the interfered-with signal components, which hereinafter will be called “repeating”. Depending on the source coding method employed, the components to be repeated may be individual subbands or spectral values or groups of subbands or spectral values of a digital audio signal that has been subdivided into n subbands or spectral values, with n again being a value equal to or greater than 1. Again depending on the source coding method employed, these components may be composed of synchronous or spectral sampled values as well as control informations or scale factors. The repetition may also be effected several times.

[0014] 3. Left-Right Substitution

[0015] In this case, synchronous components of the audio signal that are present Without interference or in processed form in the adjacent channel are employed as substitutes, which will hereinafter be called “left-right substitution”. The components to be replaced may be, as in the case of repeating and depending on the source coding method employed, individual subbands or spectral values or groups of subbands or spectral values of an audio signal that has been subdivided into n subbands or spectral values, with n again being a value equal to or greater than 1. Again depending on the source coding method employed, these components may be composed of time domain or spectral domain sampled values as well as of control information or scale factors.

[0016] 4. Estimating

[0017] With this masking strategy, not interfered-with or processed components of the same channel signal or of the signal from the adjacent channel are utilized by way of estimation (e.g. interpolation) to determine the components required to substitute interfered-with components. Due to the probability of combinations in the time domain or the spectral domain, a conclusion can be drawn from the use of non-interfered-with or processed spectrally or timely adjacent components of the same channel signal or of a signal from the adjacent channel as to the original content of interfered-with components that must be replaced. Depending on the source coding method employed, the components to be estimated may be individual subbands or spectral values or groups of subbands or spectral values of an audio signal that has been subdivided into n subbands or spectral values, with n here again being a value equal to or greater than 1. Again, depending on the source coding method employed, these components may be composed of time domain or spectral domain sampled values as well as of control informations or scale factors. Various methods for fading the signals in and/or out can be employed for the above-mentioned masking strategies. That means that the changeover from the not interfered-with component to the replaced or muted component and/or from the replaced or muted component to the not interfered-with component is made over a certain transition time and with a certain transition function so that sudden transitions are avoided.

[0018] All of the above-mentioned masking strategies can be combined with one another if required.

[0019] In connection with the described masking strategies, the invention takes advantage of certain characteristics of the human sense of hearing. In the case of muting, repeating or estimating, the auditory characteristics with respect to timely and/or simultaneous masking thresholds are utilized to the extent that these masking measures remain substantially inaudible or hidden as long as they do not exceed a certain time, spectrum and level range.

[0020] In the case of the left-right substitution, advantage is taken of the fact that direction and distance perception as well as the perception of spatial relationships are subject to a certain inertia. This can be utilized to the extent that short-term changes in the display of directions and distances of sound sources and of a spatial impression remain inaudible if they are shorter than a certain length of time and do not occur too frequently. If these two conditions can be met, the correct application of the described masking strategies results in a substantial improvement over conventional, known masking techniques.

[0021] The invention will be described in greater detail for one embodiment thereof that is illustrated in sole drawing FIG. 1 which is a block circuit diagram for the implementation of the method according to the invention in the playback channel of a stereo or multi-channel audio transmission or storage system.

[0022] In the illustrated exemplary case, it is assumed that a digital audio signal with reduced data is transmitted or stored whose source code includes three subbands. In reality the number of subbands is considerably higher and amounts to, for example, 32 subbands.

[0023] After the digital audio signal has been read out of a memory or received over any type of transmission path, the reduced data digital audio signal under consideration—divided into its three subbands—is present at the inputs 11, 12 and 13 of a channel decoder 10. Channel decoder 10 decodes the channel code of the subband signals at its input and—insofar as this is possible—performs an error correction according to the capabilities of the respective channel code. The channel decoded subband signals are fed via outputs 16, 17 and 18 of channel decoder 10 to a multiplexing device 20 which includes, for each one of the three subbands, a multiplexer 21, 22 and 23, respectively, that is symbolized as a selector switch. Multiplexers 21 to 23 are controlled by separate outputs 31, 32 and 33, respectively, of an analyzer 30 which analyzes the channel decoded subband signals at outputs 16, 17 and 18 as to how long the interference contained therein has lasted and what type of interference it is. Moreover, analyzer 30 is connected with an output 15 of channel decoder 10 so as to obtain information about errors that channel decoder 10 was unable to correct, in which subband signal, and at what points in time. This information is also fed to the analyzer of adjacent channel No. 2. At the same time, analyzer 30 receives the corresponding information from the output 15aof the channel decoder of adjacent channel No. 2.

[0024] According to the result of the subband specific analysis performed, the analyzer decides at which point in time and in which subband the respective multiplexer 21 to 23 is switched from the channel decoded subband signal at output 16, 17 or 18, respectively, to one of four alternatively available masking signals. The total of five different positions that can be selected for each multiplexer 21 to 23 are indicated in the drawing by the corresponding number of switch terminals on each multiplexer. The mentioned four alternatives correspond to the above-mentioned masking strategies “muting”, “repeating”, “left-right substitution” and “estimating”. This will be described in greater detail below.

[0025] To implement the mentioned masking strategies, a memory device 40 is provided which is controlled through the outputs 51, 52 and 53 of a memory control unit 50, specifically with respect to the respective subbands. The input of the memory device is connected with the outputs 24, 25 and 26 of multiplexers 21, 22 and 23 of its own channel and with outputs 16a, 17aand 18aof the channel decoder of the adjacent channel. Memory device 40 thus stores the processed subband signals of its own channel as well as the unprocessed subband signals of the adjacent channel, with the latter signals being employed only if they do not contain interference. The processed subband signals of the own channel are employed if the “repeating-muting” masking strategy is employed, while the unprocessed subband signals of the adjacent channel are employed in the “left-right substitution” strategy. For use of the “estimating” masking strategy, memory device 40 receives corresponding estimated values from the outputs 51 to 53 of memory control unit 50, with memory control unit 50 being connected with an output 34 of analyzer 30 through which it receives information about the subband in which the corresponding estimates are to be made at which times.

[0026] The outputs 41 to 49 of the memory device are subdivided into three groups each having three outputs, with each one of these groups being connected with an associated multiplexer 21 to 23. Each group of three outputs carries the informations for the respectively associated subband corresponding to the three mentioned masking strategies “repeating”, “left-right substitution” and “estimating”. To implement the fourth masking strategy “muting”, a switch contact of each multiplexer 21 to 23 is symbolically connected to ground.

[0027] It is understood that multiplexer device 20 is preferably realized by computer software, which is also applicable for the remaining function blocks of the illustrated block circuit diagram.

[0028] The subband signals at outputs 24, 25 and 26 of multiplexers 21, 22 and 23, respectively, are thus subband signals which either contain no interference or were subjected to error correction in the channel decoder or were processed by means of multiplexer device 20 according to one or several of the described masking strategies. The term “processed subband signal” is intended to indicate this fact. Multiplexer device 20 is followed by a source decoder 60 which receives the processed subband signals at outputs 24, 25 and 26. In source decoder 60, the processed subband signals are decoded to yield a digital audio signal which, after digital/analog conversion 4n a converter 70, can be played back, for example, by way of a channel loudspeaker 80.

[0029] It is understood that the block circuit diagram described for only one channel must be Provided for every other channel of a stereo or multi-channel playback system.

[0030] The alternatively possible connections in the block circuit diagrams for the individual channels have already been described.