System and method for detecting picture freeze within a video signal

[0001] Applicants claim priority based on Provisional Application No. 60/276,347 filed Mar. 16, 2001 and entitled “Method And Circuit For Detecting Picture Freeze Within A Video Signal” which is incorporated herein by reference.

[0002] This invention relates to video signal analysis, and more particularly to a new and improved system and method for detecting picture freeze within a video signal.

[0003] Modern, multi-stream broadcast facilities have need for monitoring multiple and diverse equipment failures, including link failure modes, and for providing an effective alarm to indicate such failures. Of particular interest is providing a user alarm in response to detection of transmission and/or processing equipment failure or any other malfunction leading to video signal freeze.

[0004] Prior art approaches to the foregoing problem are deficient in several respects. Prior methods have looked for motion within the active picture in contrast to the method utilized by the present invention. Prior solutions in most, if not all, cases required field or frame store solutions which, in turn, require more hardware and therefore add considerable cost. Also, prior art solutions have not addressed the noise factor.

[0005] It is, therefore, a primary object of the present invention to provide a new and improved digital video analyzer that can detect a picture freeze in the video signal.

[0006] It is another object of the present invention to provide a filter system that can distinguish between a video signal with frozen picture and a video signal with a short sequence of motionless action.

[0007] It is a further object of the present invention to provide a filter system that can distinguish between a video signal with time varying noise added on top of a frozen picture and a video signal with minute amounts of motion.

[0008] Broadly, the present invention relates to a system and method for detecting significant motions within a video signal by applying the following steps:

[0009] 1. Comparing sums of luma samples in a certain number of relevant and matching horizontal lines between two frames of video.

[0010] 2. Comparing sums of luma samples in a certain number of relevant and matching vertical lines between two frames of video.

[0011] 3. Filtering the foregoing comparison results to reject short sequences of motionless video.

[0012] The foregoing algorithm relies on the observation that a picture with motion is characterized by either or all of: scene change, lighting change, or an object movement in horizontal or vertical or both dimensions. In all cases, even with the smallest moving object that is representable in the video signal, at least one of the sums of luma samples across vertical or horizontal lines will change from one frame to the next.

[0013] Preferably, the foregoing method restricts comparisons to frames with matching color frame sequence numbers, in order to minimize effect of imperfect luma separation in any video decoder feeding the present invention.

[0014] The foregoing and additional advantages and characterizing features of the present invention will become clearly apparent upon a reading of the ensuing detailed description together with the included drawings.

[0015] The following detailed description of the invention, when read in conjunction with the accompanying drawing, is in such full, clear, concise and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the invention.

[0019] FIG. 1 shows the overall organization of the video picture freeze detection system and method of the present invention. Picture freeze has been defined as no activity above a preset noise level in an active picture. A related condition, picture black, has been defined as no active picture above 7 IRE. A video interface 90 provides inputs to the system 100 in the form of digital video luma samples on line 106 and digital video clock and timing signals on line 107. The digital video luma samples are obtained from a luminance decoder in a manner well understood by those skilled in the art. By the term “luma” is meant the standard definition of “luma” meaning black/white as commonly understood by video engineers. During time segments of the video signal when no luma samples are available, null (zero) samples are applied to the input 106. The necessary timing signals are easily derived in a manner well known to those skilled in the art. In particular, the digital video timing signals 107 represent a standard derivation of timing signals from an input video source as commonly understood by video engineers. By way of example, in the illustrative system shown in FIG. 1, signal 106 is a 10 bit signal and signal 107 is a 5 bit signal.

[0020] The system separately detects motion across the vertical axis of the picture using Vertical Motion Detector, 101 (described in detail later on), and across the horizontal axis of the picture using Horizontal Motion Detector, 102 (also described in detail later on). The luma samples and the timing signals are fed to both detectors simultaneously. Any detection of motion will trigger a pulse at the outputs of the corresponding detector. Both outputs are logically OR'ed in gate 103 and the result is used to reset a counter 104. The resettable frozen frames counter 104 is a standard, commercially available component having a non-proprietary code for counter functionality. The counter 104 counts up with every end of frame pulse 110 which is one pulse at the end of each picture frame. At any time, its output, 112, reflects the number of motionless frames since the last motion detection. When output 112 reaches a user defined number represented by the signal on line 113, the comparator 105 will assert its output, 114, to indicate or signal a frozen picture condition. The user-controlled freeze time tolerance signal on line 113 is obtained externally from the system user's control designated 113a in FIG. 1, whereby the user provides an input to set a level for identifying a frozen picture state. The setting is required because some video content could appear to be frozen but in reality is not. The unit of measure is number of video frames, a term commonly understood by video engineers. The control 113a sets duration, in frames, of video activity under the picture noise level that is considered a fault. The fault definition for picture freeze duration is 6 to 902 frames (in 4 frame increments), 302 frames being a typical setting. By way of example, in the illustrative system of FIG. 1, signal 113 is an 11 bit signal. The output signal on line 114 is utilized to activate an alarm or to provide the appropriate visual indication on a monitor. For example, signal 114 is coupled by an interface 116 to an alarm 118 or a monitor 120. The user-controlled noise rejection threshold signal on line 115, obtained externally from the system user's control designated 115a, which will be described in further detail presently, distinguishes motion from background noise. By way of example, in the illustrative system of FIG. 1, signal 115 is a 20 bit signal.

[0021] FIG. 2 illustrates the vertical motion detector 101 in further detail. Before the beginning of every active line, the accumulator, 201, is reset by a beginning of active line pulse 208. Pulse 208 consists of one pulse directly preceding the first luma sample of any active line as commonly understood by video engineers. Subsequently, all the luma samples, 106, from one video line are then accumulated in 201 producing horizontal luma sums. In particular, in the horizontal line luma samples accumulator 201, an active line pulse 208 precedes an active line of video and this resets the accumulator 201. Video luma samples 106 from the new line are inputted to the accumulator 201. Luma samples from this line are added and the value is stored in a memory buffer for the line. The next line active pulse 208 sets the accumulator 201 to repeat luma sample accumulation from the next line. The value is stored in the memory buffer for that line. The process continues for the remaining active lines for the digital video luma samples, i.e. for the number of lines programmed into RAM 202. Upon completion the accumulator 201 resets, and the accumulated line luma values are stored in respective line luma memory buffers. The accumulator 201 repeats every 8 fields or 4 frames at a minimum.

[0022] At the completion of the active line, the accumulated output, 210, of the accumulator 201, is subtracted in component 203 from the similarly derived value stored N frames (where N is a programmable integer number, typically N=4) earlier in RAM 202. In particular, horizontal luma sample sum programmable delay RAM FIFO 202 is a standard component which stores luma samples horizontally along an active line in a field of video. The difference, 212, is then processed in 204 to obtain its absolute value, 213, which is then compared in 205 against a user-controlled noise threshold, 115. The user-controlled noise threshold signal on line 115 is obtained externally from the system user's control to set the approximate level of noise expected in the video signal feed. It is used by the freeze detect feature of the present invention to distinguish motion from background noise on top of a video feed. By way of comparison, a signal to noise ratio of 1 corresponds to digital freeze, i.e. no noise on top of a frozen picture, as compared to a 40 dB signal to noise ratio of 10. The fault definition for picture noise level includes 1 to 10 with 9 being preferred. Noise can give the impression that the video is non-frozen even though the content is truly frozen. The setting of the threshold on line 115 rejects or filters the noise to create a “clean” signal for freeze determination.

[0023] Component 204 is a signal processor block (software code within a field programmable gate array) that performs the arithmetic function of absolute value on the input data. Component 204 is a standard block that takes an unsigned value and generates a signed value. In particular, at component 203 two unsigned values are subtracted resulting in the unsigned value 212 which is the input to component 204. The output 213 has been converted to a signed value by component 204. The output, 215, of the comparison performed in component 205 is useful only at the end of the active line and thus is gated in 206 with timing signals 209 to create the vertical motion detect pulses, 108 shown in FIG. 1. Timing signal 209 is an end of active line pulse which consists of one pulse following the last luma sample of any active line as commonly understood by video engineers. Simultaneously, at the end of the active line, signal 209 saves the computed horizontal luma sum, 210, in RAM 202 which effectively acts as a delay FIFO with time delay set to 4 frames. By way of example, in the illustrative system of FIG. 1, output 210 of accumulator 201 is a 20 bit signal, input 212 to component 204 is a 21 bit signal and output 213 of component 204 is a 20 bit signal.

[0024] The horizontal motion detector 102 is shown in further detail in FIG. 3 During the first active line of any picture frame, an input signal named first-active-line 310, which is asserted during the first active line of the picture frame, causes the multiplexer 302 to shift entire line-worth of luma samples into the line delay FIFO 303. The ram used as 1 line delay FIFO 303 is a standard component which uses the same position luma sample over vertical lines in a field of video. During subsequent lines, each new luma sample on input 106 is added in 301 to the output, 309, of the memory cell in line delay FIFO 303. Thus all luma samples from one vertical line will be accumulated into one memory cell of 303 to produce vertical luma sample sums. During the last active line of picture frame, the output of mux 302 contains new vertical luma sums, 313. Simultaneously, another RAM FIFO, 304, presents on its output similarly derived vertical luma sample sums, 314, stored N frames earlier (where N is a programmable integer number, typically N=4). The vertical luma sample sum programmable delay RAM FIFO 304 is a standard component which stores luma samples vertically across the same sample on different lines in a field of video. Both the new, 313, and the old, 314, sums are subtracted from each other in 305 to produce the difference, 315. It is processed in 306 to obtain its absolute value, 316, which is then compared in 307 against the user-controlled noise threshold, 115, which is the same as threshold 115 in FIG. 2. Component 306 is a signal processor block (software code within a field programmable gate array) that performs the arithmetic function of mathematical absolute value on the input data and is similar to component 204 of FIG. 2. At component 305 two unsigned values are subtracted resulting in the unsigned value 315 which is input to component 306. The output 316 has been converted to a signed value by component 306. The output, 318, of the comparison is useful only during the last active line and thus is gated in 308 with timing signal 311, which is asserted during the last active line of the picture frame, to create horizontal motion detect pulses, 109 shown in FIG. 1. Simultaneously, during the last active line, signal 311 saves the computed vertical luma sums, 313, in 304 which effectively acts as a delay FIFO with time delay set to 4 frames.

[0025] When the picture noise level is increased via control 115a, it is preferred that the picture freeze duration be increased via control 113a as well. This is because the higher the picture noise level, the lower is equipment's motion sensitivity, thus long periods without significant on-screen movement are more likely to trigger a “false” freeze alarm. Setting up the optimum picture noise level and picture freeze duration parameters will depend on the amount of noise in the video path from the equipment with freeze-frame capability to the monitoring point. The maximum amount of time permissible between the moment of freeze and the alarm should be determined. Setting this time as high as tolerable has two benefits. It lowers the frequency of “false” freeze alarms generated when a perfectly valid content contains long motionless periods. It allows raising the picture noise level parameter, without increasing frequency of “false” freeze alarms.

[0026] Preferably, the picture noise level should be set after setting the picture freeze duration. If the video path is fully digital, then the picture noise level can be set depending on bit-error rate of the link as follows. For bit-error rates less than 1 in 10^E-12, the value should be set in the range of 1 to 5. For bit-error rates greater than 1 in 10^E-12, the value should be set in the range of 6 to 10. If the path is even partially analog and if the equipment can be placed farthest upstream in the video path to go to a “freeze frame” mode, then the picture noise level should be optimized by initiating the “freeze frame” mode in the upstream equipment and adjusting picture noise level as low as possible, without losing the freeze alarm. It should be noted that because of the random nature of noise, the freeze alarm might be intermittent at some picture noise level settings. The optimal picture noise level setting is obtained when the loss of freeze alarm occurs no more than once every 5 minutes.

[0027] If the path is even partially analog and the equipment farthest upstream in the video path cannot go to a “freeze frame” mode, then picture noise level should be optimized by adjusting it as high as possible. If one triggers false freeze alarms more often than acceptable, the picture noise level setting should be lowered.

[0028] The picture noise level and picture freeze duration controls 115a and 113a, respectively are able to detect short-term “digital” freezes such as MPEG or motion JPEG server artifacts. When devices have a significant problem with the content that they are de-compressing, they will typically start to produce a “blocky” effect. If the problem is severe enough, they will freeze a frame of video and play it out for a number of frames. With the picture noise level set to 1 (i.e. only detect exact, or nearly exact pictures) and the picture freeze duration set to minimum, the system can detect these quick “digital freezes”.

[0029] Advantageously, picture changes due to lighting variation or outright scene change are detected equally effectively by both horizontal and vertical motion detectors and thus do not necessitate any extra hardware or software.

[0030] Thus, the present invention provides a signal processor, for use with any user control and status reporting interface, which has one video input signal and which determines whether the content of the video data is non-moving, even in presence of time-varying noise in the video input. An algorithm is implemented in a manner that does not require video frame storage devices. The system of the present invention will utilize results from the signal processor to trigger in-video text alarms, external alarms, and/or interface to remote network based monitoring systems. The efficient algorithm of the present invention provides a low-cost implementation that can be utilized in a multi-stream, modern broadcast facility. This technology can be implemented in a variety of additional applications.

[0031] While an embodiment of the present invention has been described in detail, that is done for the purpose of illustration, not limitation.