System and method for automated testing of digital television receivers

[0002] The present invention is related to that disclosed in U.S. Provisional Patent Application No. 60/176,073 entitled “Automated Test System for Digital TV Receivers” filed Jan. 13, 2000 (the specification of the present invention claims priority to this provisional application under 35 U.S.C. §119(e)(1)). The provisional patent application and the inventions disclosed therein are assigned to the assignee of the present invention and are incorporated herein by reference for all purposes as if fully set forth herein.

[0003] The present invention is directed, in general, to automated test systems and, more specifically, to a system for the automated testing of digital television receivers.

[0004] In recent years significant efforts have been made to develop objective, quantitative techniques for measuring the quality of visual images. The algorithms that have resulted from these efforts have been primarily used to test the performance of image processing equipment and to analyze faults within the equipment. The specific technique utilized for analyzing the image quality of a particular system depends not only upon the characteristics of the image but also upon the technology used to generate the image.

[0005] While objective and quantitative image measurement techniques were evolving, new types of technology were also being developed to transfer digital images from a transmitting station, such as a television station, to one or more remotely located receiving stations, such as digital television receivers. These developments eventually resulted in the Advanced Television Systems Committee (ATSC) adopting voluntary standards for advanced television technology. The ATSC standard was accepted by the Federal Communications Commission (FCC) in November 1995. The ATSC standard became the basis for the United States digital television standard that was formally accepted in December 1996. The ATSC standard incorporates the Moving Pictures Experts Group (MPEG) compression scheme for full motion video with audio. Most current digital high definition television (HDTV) systems utilize the MPEG2 standard. Newer HDTV systems will use the MPEG4 standard in the future. MPEG2 and MPEG4 offer a wide range of compression ratios with varying degrees of audio and video quality.

[0006] The first broadcast of HDTV in the United States utilizing the ATSC standard occurred in November 1998. It is presently planned that all television broadcasts within the United States will use digital HDTV television signals by the year 2006. The evolution of HDTV has resulted in development of several new technologies for the encoding, transmission, reception, decoding and display of digital video images.

[0007] However, present manufacturing, testing and diagnostic techniques are not designed to adequately and objectively assess the quality of digital video images. The relatively recent development of digital television technology has created a demand for better and more efficient digital video image testing and diagnostic equipment.

[0008] There is therefore a need for a system and method for testing digital television receivers. In particular, there is a need for a system and method for an automated test system for digital television receivers. More particularly, there is a need in the art for an improved system and method for an automated test system for digital television receivers that provides an objective quantitative analysis of the quality of digital video images.

[0009] To address the above-discussed deficiencies of the prior art, it is a primary object of the present invention to provide an improved system and method for the automated testing of digital television receivers.

[0010] The system and method of the present invention comprises a digital video signal unit capable of sending an encoded digital video test pattern to a digital television receiver. The digital video signal unit is capable of receiving from the digital television receiver a decoded reconstructed digital video signal representing the digital video test pattern as displayed by the digital receiver television receiver.

[0011] The automated test system also comprises a test processing unit that is capable of comparing the digital video test pattern with the reconstructed digital video signal to determine the video quality of operation of the digital television receiver. The system and method of the present invention compares the digital video test pattern and the reconstructed digital video signal by 1) comparing the amount of jitter, and 2) comparing the signal to noise ratio, and 3) computing percentage similarity between image blocks for the two video signals.

[0012] It is an object of the present invention to provide an automated test system for digital television receivers capable of sending a digital video test pattern to a digital television receiver and capable of receiving a reconstructed digital video signal representing the digital video test pattern as displayed by the digital television receiver.

[0013] It is also an object of the present invention to provide an automated test system for digital television receivers capable of comparing a digital video test pattern with a reconstructed digital video signal to determine the video quality of operation of a digital television receiver.

[0014] It is another object of the present invention to provide an automated test system for digital television receivers capable of comparing the amount of jitter in a reconstructed digital video signal with a digital video test pattern to determine the video quality of operation of a digital television receiver.

[0015] It is still another object of the present invention to provide an automated test system for digital television receivers capable of comparing the signal to noise ratio in a reconstructed digital video signal with a digital video test pattern to determine the video quality of operation of a digital television receiver.

[0016] It is an additional object of the present invention to provide an automated test system for digital television receivers capable of computing percentage similarity between an image block in a reconstructed digital video signal and an image block in a digital video test pattern to determine the video quality of operation of a digital television receiver.

[0017] The foregoing has outlined rather broadly the features and technical advantages of the present invention so that those skilled in the art may better understand the detailed description of the invention that follows. Additional features and advantages of the invention will be described hereinafter that form the subject of the claims of the invention. Those skilled in the art should appreciate that they may readily use the conception and the specific embodiment disclosed as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. Those skilled in the art should also realize that such equivalent constructions do not depart from the spirit and scope of the invention in its broadest form.

[0018] Before undertaking the DETAILED DESCRIPTION, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document: the terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation; the term “or,” is inclusive, meaning and/or; the phrases “associated with” and “associated therewith,” as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like; and the term “controller” means any device, system or part thereof that controls at least one operation, such a device may be implemented in hardware, firmware or software, or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior, as well as future uses of such defined words and phrases.

[0019] For a more complete understanding of the present invention, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, wherein like numbers designate like objects, and in which:

[0020] FIG. 1 illustrates a block diagram of an exemplary digital television receiver under test and an exemplary digital television receiver automated test system, according to one embodiment of the present invention;

[0021] FIG. 2 illustrates a block diagram of a digital television receiver under test and an exemplary digital video signal unit, according to one embodiment of the present invention;

[0022] FIG. 3 illustrates a block diagram of an exemplary memory unit, according to one embodiment of the present invention;

[0023] FIG. 4 illustrates a portion of an original digital video test pattern and a portion of a reconstructed digital video test pattern for illustrating an exemplary pixel shift analysis, according to one embodiment of the present invention; and

[0024] FIG. 5 illustrates a flow chart depicting the operation of an exemplary automated test system for digital television receivers, according to one embodiment of the present invention.

[0025] FIGS. 1 through 5, discussed below, and the various embodiments used to describe the principles of the present invention in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the invention. Those skilled in the art will understand that the principles of the present invention may be implemented in any suitably arranged configuration of an automated test system for digital television receivers.

[0026] FIG. 1 illustrates a block diagram of an exemplary digital television receiver 105 and a block diagram of an exemplary digital television receiver automated test system 120 (automated test system 120), according to one embodiment of the present invention. Digital television receiver 105 comprises digital video monitor 110 and digital video receiver 115. Digital television receiver 105 may sometimes be referred to as the Unit Under Test (UUT). As will be more fully explained, digital video receiver 115 receives digital video signals from automated test system 120. Digital video receiver 115 sends digital video signals to digital video monitor 110 and to automated test system 120. Digital video receiver 115 and automated test system 120 are also coupled by a first RS232 data communication line (RS232 COMM1). Digital television receiver 105 may also comprise additional circuits (not shown) for connecting to an external test system such as automated test system 120.

[0027] Automated test system 120 comprises test processing unit 125 and digital video signal unit 145. In one embodiment, test processing unit 125 comprises output devices 130, hard disk 135, and processor unit 140. As will be more fully explained, automated test system 120 also comprises software located within processor unit 140 or within hard disk 135 (where it is accessible by processor unit 140). Output devices 130 may include a high-quality laser printer that provides hard copy print-outs of the automated test results. Output devices 130 preferably include a printer that is capable of displaying and printing video test images. Output devices 130 may also include a video monitor or other similar computer display device. Hard disk 135 provides nonvolatile storage for software programs, test results, and other information related to the operation of automated test system 120.

[0028] Processor unit 140 is coupled to standard input devices such as a keyboard and a mouse (not shown). Processor unit 140 receives software program information from hard disk 135. Processor unit 140 contains memory unit 155 for storing data and software programs. Processor unit 140 also receives user input from the keyboard or mouse. Processor 140 operates with instructions from software programs (stored on hard disk 135 or in memory unit 155) and instructions from user input (from the keyboard or mouse). Processor unit 140 and the software in memory unit 155 (and on hard disk 135) together comprise a test processing unit that is capable of carrying out the present invention. As will be more fully explained, processor unit 140 generates video test patterns to analyze the video quality of operation of digital television receiver 105.

[0029] Digital video signal unit 145 operates under the direction of test processing unit 125 to create encoded video test patterns in the form of digitized video signals with red (R), green (G), and blue (B) components (RGB components) for transfer to digital television receiver 105. Digital video signal unit 145 also receives the resultant decoded RGB video signals from digital television receiver 105 and converts these signals and transfers them to test processing unit 125 as a reconstructed video test pattern. Digital video signal unit 145 and processor unit 140 are coupled by a second RS232 data communication line (RS232 COMM2) and by CPU bus 150. The operation of digital video signal unit 145 is described more fully below.

[0030] FIG. 2 illustrates a block diagram of an exemplary digital television receiver 105 and a block diagram of an exemplary digital video signal unit 145, according to one embodiment of the present invention. Digital video signal unit 145 comprises frame grabber 205, red-green-blue pixel (RGB pixel) to video signal generator 210, ATSC-MPEG encoder 215 (encoder 215), and video signal to red-green-blue pixel (RGB pixel) converter 220.

[0031] Frame grabber 205 receives a frame of the decoded video bit streams from digital television receiver 105 that represent the reconstructed video test pattern and transfers them to processor unit 140 (not shown in FIG. 2) through CPU bus 150. Frame grabber 205 also receives outgoing video bit streams from processor unit 140 through CPU bus 150. The outgoing video bit streams represent the original video test pattern. Frame grabber 205 transfers frames of the outgoing video bit streams to digital television receiver 105 through encoder 215. The outgoing video bit streams from processor unit 140 represent video test patterns sent to test the operational video quality of digital television receiver 105.

[0032] Frame grabber 205 uses well known techniques to detect a video bit stream that is associated with a test pattern video frame from processor unit 140. After frame grabber 205 detects a test pattern video frame from processor unit 140, frame grabber 205 isolates the red (R), green (G), and blue (B) pixel components of the detected test pattern video frame and provides the respective R, G, and B video data as input R, G, and B signals to RGB to video signal generator 210.

[0033] RGB to video signal generator 210 receives the test pattern R, G, and B signals from frame grabber 205 and uses well known techniques to convert these test pattern R, G, and B signals to analog signals for output to encoder 215.

[0034] Encoder 215 exchanges operational information with test processing unit 125 on second RS232 data communication line (RS232 COMM2). Encoder 215 operates under the direction of test processing unit 125. In one embodiment, encoder 215 may receive information from test processing unit 125 for compressing the video signal generated from the RGB to video signal generator 210. Encoder 215 may be controlled by test processing unit 125 on second RS232 data communication line.

[0035] Encoder 215 accepts analog video signals from RGB to video signal generator 210 and digitally compresses the signals according to a known compression algorithm such as ATSC-MPEG2. Encoder 215 then converts the compressed video frames to appropriate levels to simulate an incoming digital video signal of the type that digital television receiver 105 would normally receive from an antenna or cable (not shown). Encoder 215 then sends the resultant compressed video test pattern to digital video receiver 115.

[0036] As shown in FIG. 2, video signal to RGB converter 220 receives decoded video signals from digital television receiver 115. The decoded video signals are the same decoded video signals that digital television receiver 115 provides to digital video monitor 110. The decoded video signals represent the reconstructed video test pattern. In other words, video signal to RGB converter 220 receives video signals that reflect the video test pattern after it has been received and converted for display by digital television receiver 115. These signals will be referred to as the reconstructed video test pattern.

[0037] Video signal to RGB converter 220 uses well known techniques to convert the incoming reconstructed video signals to R, G, and B signals required for input to frame grabber 205. Frame grabber 205 accepts the R, G, and B signals that represent the reconstructed video signals. Frame grabber 205 formats the R, G, and B signals as video frames and transfers the video frames to test processing unit 125 on CPU bus 150. Test processing unit 125 then analyzes the video frames to compare the reconstructed video signals with the original video test pattern. In this manner, test processing unit 125 analyzes the operational video quality of digital television receiver 105.

[0038] FIG. 3 illustrates an exemplary memory unit 155 within processor unit 140, according to one embodiment of the present invention. Memory unit 155 stores data and programs associated with automated test system 120. Memory unit 155 may be a volatile memory device such as random access memory (RAM). Memory unit 155 contains software for control and monitor routines 310, software for pixel shift analysis 315, software for root mean square (RMS) error analysis 320, and software for block by block analysis 325. Control and monitor routines 310 direct the general interchange of communications on CPU bus 150, RS232 COMM1, and RS232 COMM2. Control and monitor routines 310 also coordinate the exchange of data between different locations in memory unit 155. Control and monitor routines 310 also provide the routines that prepare the test results of digital television receiver 105 for user display, printing, and storage as directed by the user and other program elements.

[0039] Pixel shift analysis 315 compares an digital video test pattern with the corresponding reconstructed digital video test pattern output by digital television receiver 105 to determine the image jitter that has been introduced by digital television receiver 105. Pixel shift analysis 315 accomplishes this comparison through the use of a pixel shift analysis algorithm (to be described more fully below). Pixel shift analysis 315 transfers the results of this comparison to control and monitor routines 310 as a quality measurement for the jitter associated with the reconstructed digital video test pattern displayed by digital television receiver 105.

[0040] RMS error analysis 320 determines the signal to noise ratio (SNR) of the original digital video test pattern and the signal to noise ratio of the reconstructed digital video test pattern as output by digital television receiver 105. RMS error analysis 320 calculates the root mean square error (RSME) between the two images and provides the result as a measurement of the signal to noise ratio (SNR) that is transferred to control and monitor routines 310. RMS error analysis 320 calculates the signal to noise ratio (SNR) as follows:

[0041] SNR=20 log₁₀(1/(RSME))  (1)

[0042] where RSME is the root-mean-square error:

[0043] RSME=(3^−½)(N⁻¹X_max⁻¹)[(_i=1Σ^N)(_j=1Σ^N)(X_ij−X_ij{circumflex over ( )})²]^½  (2)

[0044] and where N is the image dimension, X_ijis the amplitude of an arbitrary original input element, X_ij{circumflex over ( )} is the amplitude of the reconstructed input element, and X_maxis the maximum amplitude of the input element.

[0045] The root-mean-square error (RSME) between the original digital video test pattern and the reconstructed digital video test pattern for each color component and for each luminance component can be calculated in a similar manner after converting each pixel of the RGB color images into their corresponding components in the luminance-chrominance domain for the video signal.

[0046] Block by block analysis 325 compares the original digital video test pattern and the reconstructed digital video test pattern image using a minimum distance algorithm for image similarity. The minimum distance algorithm for image similarity computes the similarity between the two images as a function of the distance between the images with respect to the distance between the original video test pattern image block and its most distorted possible replica.

[0047] The similarity factor for a block of the original digital video test pattern and a block of the reconstructed digital video test pattern is computed as follows:

[0048] S=1−(d/d_max)  (3)

[0049] In Equation (3) the value d is the Euclidean distance between the original and the reconstructed video test pattern image blocks. The value of d is calculated by:

[0050] d=(Σ_iΣ_j(X_ij−X_ij{circumflex over ( )})²) ^½  (4)

[0051] where X_ijis the amplitude of an arbitrary input element of the input image, and X_ij{circumflex over ( )} is the amplitude of the reconstructed input element that is the counterpart of X_ij.

[0052] In equation (3) the value d_maxis the distance between the input image block and the most distant image block. The value of d_maxis calculated by:

[0053] d_max=(Σ_iΣ_j(X_ij−X_dij)²)^½  (5)

[0054] where X_ijis the amplitude of an arbitrary input element of the input image and X_dijis the amplitude of the reconstructed input element that is the most distant counterpart of X_ij.

[0055] As previously mentioned, pixel shift analysis 315 compares the original digital video test pattern with the reconstructed digital video test pattern to determine the amount of image jitter that has been introduced by digital television receiver 105. FIG. 4 illustrates a portion 405 of an original digital video test pattern and a portion 410 of the reconstructed digital video test pattern that corresponds to portion 405. The differences between portion 405 and portion 410 in FIG. 4 are exaggerated for clarity. The horizontal axis for comparison of the two portions, 405 and 410, is labeled X₁and the vertical axis is labeled X₂. In this embodiment, portion 405 is a frame that covers an M pixel by M pixel surface area. Portion 410 is a frame that covers an N pixel by N pixel surface area. The function t(X₁, X₂) represents the pixel pattern within portion 405. The function f(X₁, X₂) represents the pixel pattern within portion 410.

[0056] A pixel shift analysis algorithm is used to determine the amount of image jitter between portion 405 and portion 410. The algorithm calculates the sum of the differences of function t(X₁, X₂) and function f(X₁, X₂) of the two portions, 405 and 410, as follows:

[0057] S_ft(Y₁,Y₂)=Σ_X1Σ_X2|f(X₁,X₂)−t(X₁−Y₁, X₂−Y₂)|  (6)

[0058] where Y₁is the amount of shift along the x axis and where Y₂is the amount of shift along the y axis. When S_ft(Y₁,Y₂) equals zero, there is an exact match between the original digital video test pattern and the reconstructed digital video test pattern. That is, when S_ft(Y₁,Y₂) equals zero, there is zero jitter.

[0059] FIG. 5 illustrates a flow chart 500 depicting the operation of automated test system 120, according to one embodiment of the present invention. First, automated test system 120 selects an original digital video test pattern to analyze the operation of digital television receiver 105. Digital video signal unit 145 generates video signals representing the selected original digital video test pattern and transfers the resultant video signals to digital television receiver 105 (process step 505).

[0060] Digital video signal unit 145 receives video signals representing the reconstructed digital video test pattern from digital television receiver 105. Digital video signal unit 145 converts the video signals of the reconstructed digital video test pattern to RGB signals and transfers the resultant video frames to test processing unit 125 for further processing (process step 510).

[0061] Test processing unit 125 performs a pixel shift analysis on the reconstructed digital video test pattern, stores the results in memory unit 155 for later retrieval, and displays the results for further evaluation by the test technician (process step 515).

[0062] In a similar manner, test processing unit 125 then performs a root-mean-square (RMS) analysis of the original and reconstructed digital video test patterns, stores the results in memory unit 155 for later retrieval, and displays the results for further evaluation by the test technician (process step 520).

[0063] Then test processing unit 125 performs a block by block analysis of the original and reconstructed digital video test patterns, stores the results in memory unit 155 for later retrieval, and displays the results for further evaluation by the test technician (process step 525).

[0064] Test processing unit 125 then compiles the automated test results, stores the automated test results in memory unit 155 for later retrieval, and displays the compiled results for evaluation by the test technician (process step 530).

[0065] In an alternate embodiment, automated test system 120 may further automatically evaluate a digital television receiver 105 using any one of (or any combination of) the pixel shift analysis, the RMS analysis, and the block by block analysis according to difference limits that have been pre-selected as acceptable deviation limits for a particular digital television receiver 105 under test. Automated test system 120 may subsequently generate analysis reports that indicate the amount of deviation, if any, from the acceptable deviation limits.

[0066] In an alternate embodiment, automated test system 120 may further incorporate suggested corrective actions when digital television receiver 105 deviates beyond a particular range for one or more of the automated tests.

[0067] In another embodiment, automated test system 120 may have a multitude of digital video test patterns that are selected based upon the particular test being performed and a range of acceptable test results. For instance, automated test system 120 may use a single pre-determined initial digital video test pattern to acquire an initial performance analysis for each of the three characteristics being tested (jitter, signal to noise ratio, and block errors). Depending upon the performance of digital television receiver 105, automated test system 120 may then select one or more additional digital video test patterns for re-analysis according to whether the initial test results appear to be out of the desired range.

[0068] In another embodiment, automated test system 120 may have one or more digital video test patterns that are specifically dedicated to a particular automated test. That is, there may be one or more digital video test patterns that are used only with one type of automated test.

[0069] Although the present invention has been described in detail, those skilled in the art should understand that they can make various changes, substitutions and alterations herein without departing from the spirit and scope of the invention in its broadest form.