System and Method for Displaying a Video Menu

The invention relates generally to interactive television systems, and more particularly to a low-bandwidth interactive video menu system.

Previous architectures for interactive television systems require high-bandwidth transmission facilities to enable high-resolution images to be displayed and quick response times to be achieved during interactive sessions. This bandwidth is costly in terms of available resources and network elements involved.

Industry efforts have been directed toward developing and improving technology for storing and transporting information using an analog coaxial cable channel. A typical coaxial channel has a bandwidth of about 6 MHz, which accommodates a bit rate of up to about 28 Mbps.

The Motion Picture Experts Group (MPEG), an International Standards Organization (ISO)-sanctioned group provides a standard for full motion digital video compression. The MPEG compression standard for full motion video takes advantage of the fact that full motion video is made up of many successive frames consisting of large areas that are not changed, such as blue sky background. MPEG performs "differencing", that is noting differences between consecutive frames. If two consecutive frames are identical, the second can be stored with the appropriate information.

In server applications it is desirable to convert input data to bit sequences insertable in a bit stream for transmission. For example, according to the MPEG standard, input video data is encoded as compressed bitstreams as specified in the MPEG standard specification: MPEG TEST MODEL 4, "Coded Representation of Picture and Audio Information", ISO-IEC/JTC1/SC29/EG11, CCITT SG XV, Working Party XV/1, Document AVC-445b, February 1993.

Video data can be streamed through a coaxial cable using the MPEG-2 compression standard at a bit rate in a range from 2.5 Mbps through 6 Mbps without noticeable degradation of video imagery, and without artifacts. It is difficult to create an interactive video menu session that creates the appearance of a dedicated session using lower bit rates than that.

Previous interactive video menu architectures require large pipes (i.e., large transmission bandwidth) and costly set-top boxes (STBs) associated with the television. A larger pipe (i.e., more bandwidth) is necessary if compression of the video is performed at a higher speed. Some architectures use locally generated graphics for the background, requiring costly and more complex STBs associated with the television. As a result, there is a continuing need for a low-cost, low-memory menuing system for use in interactive television, which is able to transmit still, moving, or picture-in-picture video images using MPEG-2 compression at below 1.5 Mbps to cost-effectively minimize bandwidth, while maintaining image quality and quick response times.

The aforementioned deficiencies are addressed and an advance in the art is provided by an interactive video menu system using an economical method for the storage, transmission, and reception of data necessary for interactive video sessions, which allows high-resolution images to be broadcast to end users along with all the control information necessary to display and navigate through these images. A simple interpretive script is run on the end user's device which traps the broadcast information and allows rapid response time and control of the interactive session without requiring expensive high-bandwidth facilities. Buffer equalized coding is used so that still, picture-in-a-still, and moving video images can be compressed at rates below those typically used with the MPEG-2 standard and continuously transmitted as video backgrounds in a seamless loop saving bandwidth without degradation in video quality. Data that instructs the end-user device how to display the video backgrounds and what foreground text to instantiate (i.e., superimpose) over the video backgrounds is broadcast simultaneously in the bit stream This creates the appearance of a dedicated connection to the server while minimizing bandwidth and equipment cost and complexity.

Other features and advantages of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the features of the invention.

FIG. 1 is a schematic representation of an interactive video menu system according to the principles of the invention;

FIGS. 2-5 show several clips displayed as video backgrounds according to the principles of the invention; and

FIGS. 6-8 show a series of displayed menus with foreground text instantiated over video backgrounds according to the principles of the inventions.

The interactive video menu system provides an economical method for the storage, transmission, and reception of data necessary for interactive video sessions, which allows high-resolution images to be broadcast to end users along with all the control information necessary to describe these images and navigate through these images. A simple interpretive script is run on the end user's device which traps the broadcast information and allows the user to receive similar response time and control of the interactive session without requiring expensive high-bandwidth storage, transmission and reception facilities.

Referring to FIG. 1, post-produced video images 10 which can be still images, still images with a moving picture-in-picture or moving pictures, are entered into a program encoder 12. The program encoder 12 then compresses the material at a very low bit rate in a range of less than 2.5 Mbps, for example 0.75 Mbps to reduce the amount of storage needed and the bandwidth necessary to transmit them in a broadcast environment through a broadband transmission medium.

In the preferred embodiment, the program encoder 12 is embodied as an AT&T Program Encoder. The encoder parameter settings consist of variable and constant parameters The parameters that stay constant are parameters that encoder developers recommend to stay at a fixed value for best results. The parameters that vary are parameters that most affect the system and when strategically varied can produce optimal video quality. A set of constant and variable parameter settings for the program encoder were determined during an experimental study which allowed a low bit rate to be used with little or no degradation in video quality. Many other settings can be used with the low bit rate enabled by the invention to provide similar video quality. Such constant and variable parameter settings for the program encoder are presented in TABLE I.

The compressed images from the program encoder 12 are then sent to a processing engine 14 where they are processed to attain "buffer equalized coding" in accordance with the principles of the invention. Buffer equalized coding is the subject of copending U.S. patent application Ser. No. 08/327,176 filed Oct. 21, 1994 entitled "METHOD FOR CONTROLLING A COMPRESSED DATA BUFFER", which is incorporated by reference as if fully set forth herein. U.S. patent application Ser. No. 08/327,176 relates to data buffers and, in particular, to controlling a data buffer for transmitting video data compressed according to MPEG standards That patent application describes in detail a system which includes an encoder for encoding input data as a first bit sequence. A processor includes the data buffer and is operatively associated with the encoder for detecting a condition of the first bit sequence having a size less than a predetermined size, and for converting the first bit sequence to a modified bit sequence, which has the predetermined size, in the bit stream That system includes a decoder for decoding the modified bit sequence. The processor generates the modified bit sequence by inserting at least one predetermined bit, such as a logical 0 bit, into the first bit sequence. In addition, the encoder and decoder include means for encoding and decoding, respectively, the input data according to Motion Picture Experts Group (MPEG) standards, with the decoder buffer having a predetermined fixed capacity.

Thus, the data buffer is controlled by receiving input data; encoding the input data using the encoder to generate a first bit sequence; and detecting a condition of the first bit sequence having a size less than a predetermined size. The first bit sequence having a size less than the predetermined size is converted to a modified bit sequence in the bitstream which has the predetermined size. The bitstream is received and the modified bit sequence is decoded. Buffer equalized coding facilitates editing. The set of modified bit sequences, assembled in an arbitrary order, are decodable by the decoder to generate a decoded output signal free of editing artifacts. Such concatenatable modified bit sequences provide seamless switching between bitstreams without gaps or loss of data caused by buffer overflows or underflows. As a results only a short "clip" (five seconds minimum) of the video image is needed for the next steps of the present invention

The compressed, buffer-equalized-coded clip 16 is then stored on a server 18. The server 18 then outputs the buffer-equalized-coded clip 16 in a continuous loop, which repeats, for example, every five seconds (i.e., replaying the clip over and over again) in the bit stream.

According to the principles of the invention, the short buffer-equalized clip is transported via a broadband transmission network 20 to a set-top box ("STB") 22 of an end-user device which is coupled to the broadband transmission network. It is contemplated that the set-top box can be any type of digital video player (DVP) digital video home terminal (DVHT), or equivalent. The STB 22 plays the clip continuously and seamlessly for an indefinite amount of time without having any decoder resets or video artifacts show on the display screen 24 of the end-user device. The inventive technique creates storage savings in the server 18 that is playing the clip 16. The server uses less memory to store the short clip. The clip 16 can be used for many purposes, for example, as a background image for on-screen display of text, graphics or other material that changes locally (e.g. a "menu selection" screen); and as a barker channel The buffer-equalized clip provides a low-bandwidth seamless loop for the set-top box 22 to display. Multiple clips can be sent simultaneously to give each menu page a different "look and feel" depending on what the user is doing.

The post-produced video images 10 are also fed to a script creation process 26. At the script creation process authors output (a) text based on the post-produced video material 10 to be displayed as foreground, along with (b) simple instructions for the STB on what to do with the text and how to respond to navigation commands (e.g., via handheld wireless remote control ("RC") 27 or keypad button pushes) from the user.

The foreground text and the instructions, forming the interpretive script, are then sent to the processing engine 14. The processing engine assembles the script into a small number of packets 28, which are then associated with the buffer-equalized (i.e., compressed and processed with buffer equalized coding) clip 16.

In an alternate embodiment, audio clips are used to provide help and instructions to the user in conjunction with the displayed background video screens and foreground text. The audio clips can be any type of music, sounds and/or vocal instructions.

The clips and scripts are both loaded into the server 18 in the form of MPEG-2 transport streams. A number of buffer equalized clips 16 and interpretive script packets 28 are broadcast continuously and simultaneously to all the set-top boxes 22, 22-1, . . . 22-N (where N is an integer) coupled to the broadband transmission network 20 throughout the network of end-user devices. There is generally one clip associated with one script packet for each menu or application page in the menu system. In some cases, however, multiple interpretive script packets 28 can be associated with the same single buffer-equalized clip 16. This saves bandwidth by reducing the number of clips 16 that need to be broadcast.

In the interactive video menu network according to the invention, all STBs 22, 22-1, . . . 22-N coupled to the broadband transmission medium 20 receive the same clips 16 and script packets 28. Each STB 22, 22-1, . . . 22-N operates to display the video clips 16 on the network of display screens 24, 24-1, . . . 24-N and, based on the text and instructions indicated by the interpretive script packets 28, add foreground text overlay to each display screen 24, 24-1 . . . 24-N to present on each display screen a video image combined with text that includes a menu page.

There are different methods of controlling the end-user device to support the session in accordance with the interactive video menu system invention. A first method involves loading the whole program into the end-user device. A second method involves sending queries upstream to the server requesting instructions and waiting for the responses indicating instructions A third method as taught herein involves constantly looking in the continuous bit stream for instructions contained in the scripts. According to the third method of the invention, some software resides in the end-user device for interpreting the script packets (foreground text and instructions) as instructions are received. This minimizes the memory required for the end-user device in balance with the number of upstream queries to the server for instructions so as to more efficiently use the limited upstream bandwidth. Retrieving instructions from the script packets in the continuous bit stream takes a little longer than if all the instructions were initially loaded in the end-user device. However, the delay is small, being in a range from about 0.1 ms to 1 ms, in terms of response time during the interactive session.

The user is able to give commands to the system during the interactive session using the remote control 27. Each STB decodes instructions in the interpretive script packets 28 to react to the user's navigation commands, which are based on the displayed menu page creating a virtual interactive session. In the virtual interactive session, each STB 22 reacts to the user's navigation commands with similar response time and control of the interactive session as if each STB had a dedicated session connected to the server 18. This virtual interactive session saves bandwidth and avoids expensive facilities by not requiring a dedicated connection from the server 18 to each STB 22, 22-1, . . . 22-N in the network and using less upstream transmission equipment and bandwidth because the number of upstream messages and queries sent back to the server 18 is reduced in this design.

Many of the services that require interface with an end-user employ still menus as the interface format. In the interactive video menu system taught herein, menu pages are created by authors at the script creation process 26. The authors define the specific actions that occur whenever a button (on the remote control 27 or STB 22) is pressed by the user in response to a displayed menu. The script creation process is a development station at which an author can, for example, use a computer to add foreground text to video backgrounds to see how the overlaid text looks on the background, and create the logic and/or code in the scripts. Examples of these actions are "HIGHLIGHT THE NEXT OPTION IN THE MENU", "DISPLAY THE NEXT MENU PAGE" and "SEND A MESSAGE UPSTREAM THAT THE USER WANTS TO BUY MOVIE X". These actions are translated into scripts (i.e. with coding) that the STB understands. These scripts are designed to work with the STB regardless of the video decoder used in the STB. The STB is capable of receiving those scripts and, in a run-time environment, executing the scripts. The script creation process 26 defines the behavior associated with each of the menus, including the navigational controls and specific "hot spots" of the menu on the display screen 24. The effect of the buttons on the end user's remote control 27 is determined according to these scripts.

The processing engine 14 encodes these video backgrounds and scripts into compressed MPEG-2 video bit streams with the accompanying scripts in the private data sections of the MPEG-2 bit streams or some other part of the stream. The MPEG-2 transport streams are stored in the server 18. In an exemplary embodiment of the interactive video menu system the number of menus available for delivery is thirty-six (36) menus per 6 MHz channel encoded at 0.75 mbps. The benefits of this implementation are minimal network traffic, predictable latencies, and minimal set-top box complexity.

The server 18 sends all the menu information on predetermined Program Identifications (PIDs) in a 6 MHz channel of the broadband transmission network. Whenever the menus need to be displayed, the STB 22 tunes to the 6 MHz channel and the appropriate PIDs. The first menu PIDs and tuning information are prestored in the STB or broadcast at a low speed rate through the signaling channel, which is at a different frequency from the MPEG-2 frequency, and are stored in the STB.

When the user enters a menus the STB 22 reads the private data (identified by PID number) associated with that menu and interprets the scripts 28 to respond to user actions The scripts 28 are interpreted and executed in a run-time environment in the STB 22. If the user makes menu selections that require a new menu to be displayed, the STB looks in the script of the current menu to determine where to look for the next menu. It then tunes to the next menu and reads the private data (identified by an appropriate PID number) that contains the scripts for that menu. At some point in time the user may make a selection, such as ordering an event (e.g., a movie) that requires the STB 22 to send a message or query upstream to the server 18. The command to send the message upstream and the format of the message, which conforms to predefined application programming interfaces (APIs) for the server and the STB, are also contained in the scripts.

When the user purchases an event such as a movie, the STB 22 sends a message upstream indicating the purchase request. The server 18 allocates bandwidth and gets authorization from the access control manager and then sends a confirmation message to the STB 22. The confirmation message can includes for examples information such as: movie title, time to showing, and prices The STB then tunes to another menu screen (as dictated by the script associated with the previous menu) that has blank fields which the STB fills up with bit-mapped characters that show, for example, the movie title, time to showing, etc. The purpose of the blank fields is so that the "look and feel" of the confirmation menu is similar to that of previous menus. The STB 22 is instructed concerning what characters to display and where.

The user interface is designed to be responsive to user commands while minimizing back-channel (i.e., upstream) communication with the server 18. One user interface with the STB for ordering services is via remote control 27 and on-screen (e.g., TV or monitor screen) display of text and/or images. Alternatively, the user may utilize a keypad on the STB to enter commands. The user is presented with menus which facilitate navigating through the labyrinth of options and making a selection using the remote control or keypad.

The following illustrates how an event, such as a movie, can be ordered in an exemplary interactive session between the interactive video menu system and the user. The user is able to give commands to the system either using the remote control 27 or with the keypad on the STB 22 in order to navigate through a series of menus for ordering the movies The scripts 28 direct the system to respond to the user commands and control the foreground text and the background video images displayed as menu pages on the display screen 24 during the interactive session. FIGS. 2-5 show four exemplary clips which can be displayed as video backgrounds at various points during the interactive session, and on which foreground text can be overlaid by the system according to the invention. FIG. 2 represents a clip A. FIG. 3 represents a clip B. FIG. 4 represents a clip C. FIG. 5 represents a clip D. FIGS. 6-8 each show a combined background video image and foreground text instantiated (i.e., superimposed) thereon produced on the display screen 24 for the user during this example of an interactive session.

The clips and scripts are continuously and simultaneously being bit-streamed through the broadband transmission network to the STB of the end-user device. The following represent five exemplary simplified pseudo-scripts used in the system:

Turning to FIGS. 2-8 in view of SCRIPTS 0-4, FIG. 6 shows what would initially be displayed on the display screen (e.g., a television screen) for the user to work with according to SCRIPT 0. Assume the user selects "1"from the menu page displayed in FIG. 6. Then the menu page displayed in FIG. 7 would be shown on the display screen 24 of the end-user device according to SCRIPT 1. Assume the user decides to select "1"from the menu page displayed in FIG. 7 and depresses the appropriate button on the remote control 27. Then the menu page displayed in FIG. 8 would be shown on the display screen 24 of the end-user device for the user according to SCRIPT 4. In FIG. 8, the foreground text "YOU HAVE PURCHASED" and "ENJOY THE MOVIE!" is instantiated (i.e., superimposed) over the background video image corresponding to clip A. The title of the selection, "TERMINATOR", is inserted in a field between the phrases "YOU HAVE PURCHASED" and "ENJOY THE MOVIE!" so as to also be instantiated over the background video image on the display screen for the user to see. Then the movie plays, and the user or subscriber can be billed accordingly.

The user has thus navigated through multiple screens without an excessive number of upstream messages to the server 18. This minimizes the required upstream bandwidth and equipment cost of the interactive video menu system.

The interactive video menu system is designed to support the user interface and also to minimize the costs associated with the STB. Menu delivery uses an interactive application in which video backgrounds audio, foreground text overlay data, and instructions are compressed and loaded onto the server. Upon loading this content, the server computes the necessary MPEG Program Identification (PID) mapping information and stores this information for broadcast into the broadband transmission network. The server continuously broadcasts the information through the broadband transmission network. The broadcast of the menuing system occupies one or more 6 MHz RF channels. The STB is provisioned with such RF channels identified as menuing channels such that the STB application can translate user navigational commands to menu selections and server specific messages.

While several particular forms of the invention have been illustrated and described, it will also be apparent that various modifications can be made without departing from the spirit and scope of the invention.