Audio Frame Synchronization for Embedded Audio Demultiplexers

The present invention relates to signal synchronization, and more particularly to audio frame synchronization for embedded audio demultiplexers that detects the boundary between audio frames to synchronize the embedded audio with the video.

Multiple channels of digital audio may be multiplexed with a digital video signal during the inactive period between the end of active video for one line (EAV) and the start of active video for the next line (SAV) for component video, or during the horizontal and vertical intervals for composite video, so that the combined signal is transmitted over a single video cable. At the receive end of the video cable the video and individual audio signals are separated by a demultiplexer. A proposed Society of Motion Picture and Television Engineers (SMPTE) standard 259M, "Formatting AES/EBU Audio and AES/EBU Auxiliary Data into Digital Video Ancillary Data Space", describes the number of digital audio samples from each audio channel per video frame in order to maintain a desired audio sample rate at the receive end of the video cable. When the audio sample rate is not an integer multiple of the video frame rate, several video frames may elapse before an integer number of audio samples coincides with a video frame boundary. The number of video frames required to match an integer number of audio samples at a given sample rate is called an audio frame.

Demultiplexers currently have no way of determining the phase of the audio frame. Consequently the demultiplexers randomly select a phase before beginning to demultiplex the embedded audio signal. Different demultiplexers select different audio frame phases, resulting in differing audio phases at their outputs. The phase shift due to audio frame misalignment may exceed two audio samples between separate demultiplexers. For example with an audio sample rate of 48 kHz the time difference between two misaligned audio channels is 20.83 microseconds per sample of difference. This time difference is quite small and not likely to be heard directly when the error is due to unsynchronized audio frames with an error of one to two samples. However the problem becomes apparent when two or more misaligned channels are mixed together. Therefore if two channels are mixed that contain some audio that is common to both channels and some audio that is different in each channel, which would happen with a stereo microphone setup, the phase difference due to the audio frame misalignment causes distortion in those audio components that are common to both channels. The problem is more acute at higher frequencies where the misalignment represents a higher percentage of the frequency's period. A two sample shift between the channels causes frequencies near 12 kHz to cancel almost entirely. Thus the effects of mixing misaligned channels may range from small phase changes to complete cancellation of some audio frequencies.

What is desired is a demultiplexer that provides audio frame synchronization in a demultiplexer to align an audio frame representing embedded audio data with video.

Accordingly the present invention provides audio frame synchronization for embedded audio with video in demultiplexers. The audio data is separated from the video data in a serial digital video signal and is stored in a memory at video sample rates. The pattern of audio samples per video frame within an audio frame is detected to generate an audio frame signal that provides phasing information for the audio data. The audio data is read out from the memory and transmitted at audio sample rates. For a specific pattern, determined by the video and audio standards for the serial digital video signal, an audio frame boundary is determined by two consecutive video frames having the same number of audio samples. Therefore the pattern detection requires counting the number of audio samples per video frame, comparing the number of audio samples between consecutive video frames, and generating an audio frame signal when equality is detected.

The objects, advantages and other novel features of the present invention are apparent from the following detailed description when read in conjunction with the appended claims and attached drawing.

FIG. 1 is an illustration of the relationship between an audio frame and video frames including some timing signals according to the present invention.

FIG. 2 is a block diagram for a demultiplexer according to the present invention.

FIG. 3 is a block diagram for a specific implementation of a pattern detector for the demultiplexer according to the present invention.

FIG. 4 is a timing diagram to illustrate the operation of the demultiplexer according to the present invention.

The present invention is described with respect to an audio sample rate of 48 kHz and a video field rate of 59.94 Hz (525 line component or composite video). The audio frame in this case repeats every five video frames, i.e., there are a total of 8008 audio samples per every five video frames. Referring now to FIG. 1 the audio samples are broken into groups of either 1601 or 1602 samples per video frame, the actual average number of audio samples per video frame being 1601.6. The audio frame has the number of audio samples per video frame alternating between the two groups, starting and ending with a group of 1602 samples. Therefore to detect the boundary between audio frames the audio samples in each video frame are counted and compared with the result for the previous video frame. Since the start and end video frames within the audio frame have the same number of audio samples, when two consecutive video frames have the identical number of audio samples the audio frame boundary is detected.

More generically the number of video frames required per audio frame is a function of the specific video and audio standards, i.e., the respective sample rates. Depending upon the particular standards a pattern of audio samples per video frame within the audio frame is established. The number of audio samples per video field are counted and the particular established pattern is detected to detect the boundary of the audio frame. In the specific example referred to above the pattern detection is achieved by finding two video frames with the same number of audio samples.

Referring now to FIG. 2 a demultiplexer 10 is shown for component video. An input serial digital video signal having an embedded digital audio signal is input to a deserializer 12 that converts the serial signal into n-bit parallel data words and provides timing signals derived from the input serial digital video signal. An EAV detector and audio extractor circuit 14 detects the EAV words for each line of video data and strips the audio data words from the parallel data words during the interval between EAV and SAV. For a composite video demultiplexer the EAV detector and audio extractor circuit 14 becomes a sync detector and audio extractor circuit for identifying the horizontal and vertical sync intervals and extracting the digital audio data words from those intervals. The detector/extractor 14 generates an enable signal during the EAV-SAV interval, or during the horizontal or vertical sync interval. The digital signal with the audio data words stripped from it is the video signal. The audio data words from the detector/extractor circuit 14 are input to a first-in/first-out (FIFO) memory 16 during the EAV-SAV interval. The audio data words are clocked into the FIFO 16 at a video clock rate, such as 27 MHz. A microcontroller (uC) 18 takes the audio data words from the FIFO 16 together with the timing signals from the deserializer 12 and the enable signal from the detector/extractor 14, adds the proper audio format subframe header information, and serializes the audio data words. The serial audio data words are then passed to an audio transmitter 20 at audio sample rates, the transmitter encoding the data words according to the appropriate audio specification. The audio data words are clocked out at the audio sampling rate, such as 48 kHz, which is locked to the serial digital video input signal.

The uC 18 performs the audio synchronization as shown in FIG. 3. A video field rate signal FRS, which is one of the timing signals from the deserializer 12, is input to a first counter 22 that is a recirculating counter from 0 to 2N-1, where N is the number of video frames per audio frame, i.e., N=5 in the current example. An AND gate 24 is coupled to the outputs of the first counter 22 to provide an audio frame signal AF when the counter recirculates to zero count. AF is latched into a first flip-flop 26, the output of which provides AF as the beginning of the audio frame sequence. A video clock signal VCLK, which is the video input sample rate from the deserializer 12 timing signals, causes the first counter 22 to increment when enabled by FRS, with FRS being only one video sample in duration and one of the deserializer timing signals. VCLK also latches the output from the AND gate 24 into the first flip-flop 26 so that AF has a duration equal to one video field. The leading edge of AF delineates the audio frame boundary.

A sample counter signal SCNT, the enable signal from the detector/extractor 16, occurs during the interval between EAV and SAV as shown in FIG. 4, or during the horizontal and vertical intervals for composite video. SCNT enables a second recirculating counter 28 to clock the audio samples during the ancillary data interval at the VCLK rate. The second counter 28 is reset at the beginning of each video frame by a frame pulse signal FRM from the deserializer timing signals. FRM also enables a register 30 to store the least significant bits of the prior audio sample count from the second counter 28 at the end of the video frame. The outputs from the second counter 28 and the register 30 are input to a comparator circuit 32 that provides an equality output signal EQ, a logical "1", when the two quantities are equal. EQ is latched into a second flip-flop 34 by VCLK when enabled by FRM for one clock interval. The output of the second flip-flop 34 is an audio frame signal AFRM that indicates that two consecutive video fields have had an equal number of audio samples according to the present example, i.e., indicates that the audio sample pattern boundary for the video frames within an audio frame has been detected. A third flip-flop 36 and a second AND gate 38 act as a differentiator to provide a pulse output on the leading edge of AFRM. This pulse output is clocked into a fourth flip-flop 40 by VCLK to provide a reset signal FRMSYNC for the first counter 22 so that the output of the first counter is synchronized with the detected boundary of the audio frame to provide an accurate AF output signal.

Thus the present invention provides automatic audio frame synchronization for synchronizing embedded audio with video in a serial digital video demultiplexer by separating the audio from the video at video sample rates, detecting a boundary for an audio frame according to the audio samples per video frame within the audio frame, and outputting the audio samples at audio sample rates in the appropriate format synchronized with the video.