Interactive Video Playback System

U.S. Pat. No. 4,034,990, assigned to the assignee of the present application, relates to a technique for allowing interaction between symbols from signals recorded on a video medium (such as a video tape or disc) and regenerated by a video playback unit and symbols from a video game unit whereby the symbols from the video playback unit and the game unit are both dislayed on the screen of a cathode ray tube (CRT) display. The signals derived from the video playback unit are of such characteristic that data can be generated from them to identify, for example, where on the screen of the CRT the symbols from the video playback unit are, and, thus, permit the symbols from the game unit to interact with the symbols from the video playback unit.

In one embodiment described in the aforementioned patent, the video signals from the video playback unit which generate the symbols on the CRT display are coded by being generated at a maximum white level and decoded in the game unit by identifying signals of such maximum white level. The decoded signals can then be used to develop logic level signals which are time related to the symbol locations on the CRT screen and, thus, the logic signals can be used to indicate such things as coincidence between a signal generated from the video playback unit and signals generated by the game unit.

While this technique is adequate for many applications, it does have its drawbacks. The amount of data which can be encoded by such level identification is small.

Accordingly, it is an object of this invention to provide an improved interactive video playback unit system.

The present invention has wide application in various systems wherein symbols derived from a video tape or disc are displayed on the screen of a cathode ray tube display in conjunction with symbols from other apparatus and wherein interaction between the symbols from the two sources are to be provided. The principles of this invention are, thus, useful in video games, in various teaching and training applications, etc. The principles of the invention will be described as related to a simulated pinball game; however, the scope of the invention and the uses therefor are much greater and the pinball application is merely used for tutorial purposes.

In a simulated pinball game, a video playback unit can provide colorful background for the playing field surface as well as game components; such as bumpers, typically found on standard pinball machines. The video playback unit is used in conjunction with a microprocessor-controlled video game which provides other symbology as well as controls to be operated by the players. Microprocessor-controlled video games are well known in the art. In this embodiment the video game supplies symbology representing flippers, a ball, certain bumpers and numerical scoring presentations. During the process of the simulated pinball game the ball interacts with the bumpers, including those provided by the video playback unit.

The video disc or tape employed with the video playback unit has data bits embedded in the horizontal retrace interval portion of each line of each field of the recorded signal. Such digital data bits are required to identify the coordinates of one or more of the objects or symbols contained in the pictorial portion of the particular field. Thus the set of digital data, or instructions, embedded in one or more horizontal intervals is associated with the video signal on a field-by-field basis. Knowledge of this data is required by the game microprocessor to enable it to control interaction between an internally generated symbol and a video tape or disc pictorial symbol simultaneously shown on the screen.

The above-mentioned and other features and objects of this invention will become more apparent by reference to the following description taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a pictorial view illustrating the major components of an interactive video playback system;

FIG. 2 is a diagram of the screen of the display of the system of FIG. 1;

FIG. 3 is a block diagram of the system of FIG. 1;

FIG. 4 is a block diagram of the video tape recorder interface employed in the system of FIG. 3;

FIG. 5 is a block diagram of the microprocessor employed in the system of FIG. 3;

FIG. 6 is a block diagram of the TV interface employed in the system of FIG. 3;

FIG. 7 is a block diagram of the TVI controller employed in the TV interface of FIG. 6; and

FIGS. 8A-8E is a flow chart for the pinball game described herein.

Referring now to FIG. 1 of the drawings, there is illustrated thereby a pictorial view of an interactive video playback system configured according to the principles of the invention. The invention has broad application in the fields of games, training and the like; however, the principles thereof will be described in conjunction with a pinball type game and the particular elements pertinent to that game are described for the system.

The system includes a microprocessor-controlled video game unit or game generator 10 which includes two flipper buttons 12 (left and right) to control flippers in a manner simulating flippers in a conventional pinball game. Microprocessor-controlled game unit 10 further includes a ball launch switch 14 which launches a ball to the pinball game simulating the inputting of a ball onto the surface of a conventional electromechanical pinball type game. The microprocessor-controlled game unit also includes a system reset switch 16, a timing reset switch 18 and a game reset switch 20.

The system also includes a video tape recorder 22; however, a disc playback unit or any other suitable medium can be substituted therefor. Alternatively, the information normally supplied from the video tape recorder may be received from a television broadcast or cablecast system. The output from the tape recorder 22 is coupled by a cable 24 to the microprocessor-controlled game unit 10. A cable 26 couples the game unit 10 to a display 28 which is preferably a conventional broadcast television receiver or monitor.

Shown on the screen 30 of the display 28 would be the various symbols representing the elements of a pinball game and a colorful background. FIG. 2 illustrates the symbols displayed on screen 30 for the embodiment described herein, including a ball symbol 32 which is launched on to the playing field by actuating the ball launch control 14 such that the ball enters the screen moving to the top thereof in the manner of a conventional pinball game. The other symbology on the screen 30 includes bumpers 34, 36, 38, 40, 42, 44, 46, 48, 50, and 52. Bumper 34 has the appearance of a numeral (in the figure the number 0). However, during the playing of a game the number displayed changes. Each time the bumper 34 is hit by ball 32 it changes by increasing in value by 1 until the number 9 is displayed. Thus, the first time bumper 34 is struck by ball 32 it will change from zero to a one, the next time it is hit it will change from 1 to 2, etc. Bumpers 34, 36, 38, 40, 42, 46, 48, 50 and 52 are stationary bumpers while bumper 44 oscillates as indicated by the arrows 54.

Also displayed on screen 30 are a pair of flippers 56, 58 which move as shown by the arrows 60, 62. The flippers are actuated by depressing the switches 12.

The other symbology displayed on screen 30 are alphanumerics including: the score 64 of the present game, the score 66 achieved during the last game played and the number of balls 68 remaining to be played during a game. The wall or boundaries 70 of the simulated pinball table field are also displayed.

In this embodiment all of the symbology shown on the screen 30 is generated by the game generator 10 except for the symbols representing bumpers 44, 46, 48 and 50 and the boundaries or walls 70 of the playing field; however, this is exemplary only and other symbols now supplied by the game generator could be supplied by the video tape recorder and vice versa.

During the playing of a game, actuating the ball launch mechanism 14 causes a ball 32 to be launched to the top of the playing area. As the ball moves through the playing area and collides with a bumper, scoring is achieved. If the ball hits bumper 34, the player scores ten times the value of the bumper and the bumper is then incremented by one. Hitting bumpers 36, 38, 40, 42 or 52 scores ten points and results in a wipe-out of the part of the bumper that was hit. Hitting any of bumpers 44, 46, 48 or 50 also scores ten points.

The flippers 56, 58, when actuated by flipper controls 12 are used to keep the ball in play, that is, to "hit" the ball up when it nears the bottom of the playing field.

The ball 32 also rebounds off certain of the bumpers when coincident therewith and also rebounds from the walls 70 when coincident therewith.

FIG. 3 of the drawings illustrates a block diagram of the video playback system and includes video tape recorder 22, microprocessor-controlled video game unit 10 and display 28. The output from the video tape recorder 22 is supplied by cable 24 to the microprocessor-controlled video game unit 10. This output includes composite video and synchronization signals recorded on the video tape, and includes in the video or pictorial field of the tape video signals representing bumpers 44, 46, 48 and 50 and wall 70. These bumpers are recorded on the video tape by way of example, by pointing the camera at an entire (or portion of a) "real" pinball machine or portions of models thereof. The video field can also contain a suitable picture for background as in conventional pinball machines.

The information from the video tape playback unit also includes digital data. The digital data in this embodiment trails the horizontal synchronization signals such that data may be provided in many of the horizontal lines of the field. The data is located sufficiently close to the blanking signals such that unintended display will not occur on the screen 30. In a color system these digital data signals will generally follow the chroma burst. Alternatively, the digital data may be provided in the vertical interval; that is, following color burst in the first 20 to 25 horizontal lines during vertical interval which are not shown in the screen due to vertical blanking and receiver overscan.

While in the embodiment described herein the digital data is located close to the blanking signals so as to preclude unintended display on the screen, it is not necessary to do so. In fact if larger amounts of data were to be provided per field, it can be done by nesting or inserting "1"s and "0"s along one or more entire horizontal lines and even on all lines of an entire field. This can be done without affecting the display since it is under microprocessor control and the microprocessor can blank the display in each area data is located. For certain applications, a large amount of data would be dumped and the microprocessor could turn off the incoming video altogether for a short period sufficient to dump the data.

The digital data in each field relates to the images from the field to be displayed on the television receiver. In the embodiment illustrated, wherein the information from the video playback unit provides the bumpers 44, 46, 48 and 50 and the wall 70, the digital data in the horizontal interval will provide to the microprocessor information concerning the location of those particular bumpers and the wall so that any signals generated in the microprocessor-controlled video game unit can interact with the signals representing the bumpers and wall. For example, the microprocessor is continually apprised of where on the screen ball 32 is located, it being generated by the game unit 10. The microprocessor also is apprised of where the bumpers 44-50 and wall 70 are located from the digital data received from the video tape recorder. The microprocessor knows the location of the ball and the location of the bumper or wall. It, thus, may determine that the ball has in fact touched the bumper or wall, and can process instructions to move the ball accordingly. For example, the microprocessor might cause the bumper to hold the ball for some length of time and then cause the ball to rebound upward or in some other direction, in the manner of a regular electromechanical pinball game. An important feature of the invention is, thus, that the digital data contained on video tape in the horizontal and/or vertical interval relates to the video signal on the same field of the visual or pictorial portion of the television scan.

The entire signal from the video tape recorder, including the video signal representing objects for display on the screen 30 of display 28, the synchronization signals, and the digital data nested in the horizontal interval during and/or after vertical interval, is applied via line 24 to a video tape recorder (VTR) interface 72 which is illustrated in greater detail in FIG. 4.

The VTR interface monitors the signals emanating from the video tape recorder 22 and includes a VTR stripper 74, a horizontal sync phase-locked loop 76, a master oscillator 78 and a universal synchronous/asynchronous receiver transmitter (USART) 80.

In the specific embodiment of the invention described herein, one data bit has been stored on every other raster line of the video tape. This has been done due to speed limitations of the USART. However, new devices coming onto the market allow faster speeds which permit a data bit to be encoded onto each raster line.

Data is received from the video tape recorder 22 asynchronously. A data word is composed of a start bit, eight bits of data and one stop bit. Television sets have a horizontal raster line rate of 15.734 KHz. Since ten bit are required for each data word, the baud rate is 786 words per second. Upon receipt of a data word, the VTR interface signals the microprocessor of the event.

The VTR stripper 74 extracts sync and data signals from the signal from the video tape recorder, applying the data to the USART and the horizontal synchronous signals to the horizontal sync phase-locked loop 76 which outputs a signal to the master oscillator 78.

Referring again to FIG. 3, the game generator includes a microprocessor 84 and a control panel 86. The control panel in this embodiment includes the switches 12, 14, 16 and 20 of FIG. 1. The control panel 86 generates control data along line 88 which is applied to the microprocessor 84.

The microprocessor in this embodiment is an Intel 8085 and is shown in greater detail in FIG. 5. It comprises a central processing unit (CPU) 90, a programmable read-only memory 92, a random access memory (RAM) and an I/O port 96. This configuration is standard. The rules of the game are stored in PROM 92 and the RAM 94 is used as a scratch pad memory. I/O port 96 provides signals to the control panel and to a sound interface 100 (see FIG. 3).

The sound interface circuit 100 transforms the logic signals applied thereto from the microprocessor to audio frequency signals compatible with a loudspeaker 102 simulating the sounds of a "real" pinball game in a manner similar to standard microprocessor video game techniques.

The game generator also includes a TV interface 104 (shown in detail in FIG. 6) which interfaces the other components of the system with the cathode ray tube display 28 to generate a video display. Timing signals from the VTR interface are processed by a master timing circuit 106. A TVI controller 108 (shown in detail in FIG. 7 and to be described hereinafter) is used to act as an interface between the microprocessor and the RAM's of the TV interface. Thus, the TVI controller buffers the data bus of the microprocessor from that of the TVI. Because of this buffer, the TVI may access its own RAM while the microprocessor is also accessing its RAM. The TVI also includes a character RAM 110, CRT RAM 112, video coder 114, monitor interface 116 and a coincidence circuit 118. Dynamic RAM's are used in the CRT RAM to implement a RAM intensive display. The CRT RAM is configured so that it maps the desired CRT display. Characters to be displayed are loaded from the microprocessor into the character RAM 110 by the TVI controller 108. Thereafter, for each character to be displayed, the Cartesian coordinate of the character is loaded into the TVI controller by the microprocessor. The microprocessor then enables the transfer of the character from the character RAM to the CRT RAM. The TVI controller effects the transfer. The video coder 114 is used to latch the contents of the CRT RAM so that the signals sent to the monitor interface 116 will be free of discontinuities. These discontinuities exist because of the nature of the dynamic RAM's used in the CRT RAM. These signals are combined with synchronization signals and a chroma burst so that the signal sent to the television monitor will be of the proper form. Coincidence circuitry 118 is used to monitor the transfer of data from the character RAM to the CRT RAM. If a character is overwritten onto another, coincidence is detected.

The TVI controller is shown in detail in FIG. 7. The circuitry TX status 120 is an output port for the microprocessor. Signals emanate from the port and sequence the various functions of the TVI. A character address circuit 122 contains row and column coordinates of an element of a character that is being accessed in the character RAM. An address summer 126 performs the addition of the cartesian coordinates of the location of a character and the contents of the character address. The resultant address is the location to which a character element must be written. Dynamic RAM Timing circuitry 128 generates the addresses necessary for the mapping of the CRT RAM to the CRT.

Referring now to FIGS. 8A to 8E of the drawings, there is illustrated thereby a flow chart of a typical controlling program for the pinball game described in this application. This program is stored in the PROM 92 of microprocessor 84. FIG. 8A is a glossary to the various symbology used in the flow chart.

The program starts at step 200. The microprocessor first clears the CRT RAM 112 of the television interface at step 202. The CRT RAM corresponds to a map of the raster scan display; that is, each RAM location has a corresponding location on the television screen 30. The next step of the program, 204, initializes the microprocessor RAM 94 and initializes the character RAM 110 of the television interface. Initialization of the microprocessor RAM is the fetching of information from the microprocessor PROM 92 and entering into the scratch pad RAM 94. The various commands necessary to permit the CPU 90 to converse with the peripherals of the CPU are an example of this information. Initialization of the character RAM 110 of the TVI is performed to define the shape of the various characters to be displayed on the pinball field. The microprocessor then at step 206 goes into a wait mode.

It waits until it receives an interrupt 208. The interrupt 208 occurs at the beginning of the vertical interval of a scan. The microprocessor at this point prepares an area of its RAM 94 for the storage of data from USART 80 of the VTR interface. This data is the coordinate of the characters regenerated by the video tape recorder (the data bits located in the horizontal retract interval).

At step 210 the USART 80 is initialized. Initialization of the USART is to inform the USART of the format which will constitute a data word. The format used for the present embodiment has been described hereinbefore. At step 212 request is made to the TVI to erase the ball character from its old position. The first time around in this program this is a dummy operation since the ball is not yet displayed on the screen. At this step the flag DSPFLG was initially set to zero. Step 214 is a loop to ascertain whether DSPFLG equals 1. If it does not equal 1, the microprocessor stays in the loop. If it does equal 1, indicating that the TVI is finished erasing the ball from the old position, then step 216 is executed wherein the TVI is requested to write the ball to its new position in memory. The microprocessor then via the loop 218 determines if the writing of the ball to the new position has been completed. If not, the microprocessor stays in the loop. If it has been completed, and thus, DSPFLG equals 1, step 220 of the program is executed, followed by steps 222, 224 and 226. Steps 220, 222, 224, and 226 are similar to steps 212 and 216 previously described with respect to erasing a ball from an old position and writing of the ball to a new position. In steps 220, 222, 224, and 226, the microprocessor requests the TVI to erase the left flipper from the old position, write the left flipper to the new position, erase the right flipper from the old position, and write the right flipper to the new position. The loops 228, 230, 232 and 234 are used for the ascertation as to whether or not the erasing and writing of the flippers has been accomplished prior to going onto the next step of the flow.

While the ball and flippers were being repositioned, the USART 80 has been receiving data bits from the video tape recorder. After the USART has received a data word (8 bits for this embodiment), it initiates a type B interrupt shown in the flow chart at various positions 236. These interrupts cause the CPU to cease whatever it was doing and be ready to store the contents of the USART (the received data word) in the RAM 94 of the CPU. The interrupt 236 is an asynchronous interrupt; it can occur at any time.

The next step of the flow 238 determines if the gate flag (GFLAG) (listed in the flow chart key) has been set to zero. When the RAM was originally initialized, the GFLAG was ONE. If the gate flag has been set to 1, the microprocessor requests the TVI to write in the bumpers 36, 38, 40, 42, and 50 at step 240.

At step 242, the microprocessor reads the reset game switch 10 of game generator 10 and if at this time the player has manipulated the switch and requested a game reset, the microprocessor will be so informed. If the game switch actuation has been acknowledged, at step 244 of the flow, the microprocessor will reset the flags, preset the gate flag to GFLAG=1, update the last score at position 66 on the display of FIG. 2, reset the present score to zero at position 64 on the display, and preset the number of balls on the display at position 68 to five. The flow then proceeds through steps 246 through 264. These portions of the flow chart, as well as all the other steps previously described, are set forth in detail in the computer program which is set forth in the body of the specifications after this description of the flow charts.

If the ball is in the ball launcher, step 246 makes a determination if the ball has a change in velocity due to the release of the ball launcher switch 14 or due to curvature of the ball race 71. In step 248 the microprocessor looks at the flipper switches 12 and determines the proper position of the flippers due to actuation of the flipper switches. In step 250 an increment of velocity is added to the vertical component of the velocity of the call due to the ball preceding from the top of the pinball field towards the bottom thereof.

Step 252 requires a determination to ascertain if there is coincidence between a ball and any of the bumpers, and if there is such coincidence, the microprocessor updates the velocity of the ball accordingly, changes the characteristics of the bumper such as incrementing bumper 34, or erasing part of the bumpers 36, 38, 40, 42, and 52, and, if necessary, updates the score due to the coincidence with the bumper.

Step 254 occasions the microprocessor to ascertain whether coincidence has occurred between the ball and a flipper and updates the ball velocity and direction accordingly. In step 256 the microprocessor determines coincidence between the ball and a wall 70 and again updates ball velocity accordingly.

Step 258 of the program is to limit the maximum speed of the ball to take some energy out of the system which in a real pinball game is accomplished by certain inelastic bumpers which slows the ball down.

Step 260 of the program is to execute ball movement after the ball velocities have been set in accordance with the steps above. The microprocessor multiplies the velocity by a time increment and establishes a new ball position.

In step 262 execution of the score and sound modules occurs. After a coincidence a flag is set so that later on in the sound module 100, that flag requests a sound to be emitted and the sound module accordingly operates speaker 102.

In step 264 the microprocessor looks at the USART RAM buffer, looks at the information stored there and processes it. This information pertains to the location and the characteristics of bumpers which were regenerated by the video tape recorder. After this step, the program then reverts to step 206 for a repeat of the flow described above. The computer program for accomplishing the aforementioned flow is set forth hereinafter.

This program is set forth in assembly code and to enter it into the microprocessor (in this embodiment an Intel 8085) a cross assembler is employed to convert from the assembly language to the machine language code. It is not necessary to go through this program in detail since along with the mnemonics describing the operand and data fields there are various comments to the program user indicating the function of the program instructions. For example, at step 03100 (page 1) it is easy to see that the start note is equivalent to the start step 200 of the flow chart of FIG. 8. Likewise at step 04800 (page 1) of the computer program there are clear instructions that this portion of the program clears the CRT RAM as in step 202 of the flow diagram.

Referring further along in the computer program at step 20900 (page 3) the instruction indicated by GFLAG1 is the instruction to accomplish the step 238 of the flow chart. Proceeding further into the computer program, it is apparent that in step 09900 (page 4) labeled "gravity" through step 11500 comprise the routine for accomplishing the execution of the gravity module or step 250 of the flow chart. Step 11800 (page 4) is labeled ball-flipper coincidence and the steps that follow through step 28700 clearly comprise the routine to ascertain ball-flipper coincidence as per the flow chart step 254.

After step 50300 (page 6) which is the end of step 264 of the flow chart, there are indicated a number of codes. These codes consist of data tables providing the CPU with information such as flipper origin, alphanumeric form factors, flipper outlines and the like. These are clearly indicated in the comments set forth along with the code.

The previous description of the apparatus, flow charts and software has illustrated a typical pinball game. However, the invention goes beyond that of a mere pinball game and illustrates techniques for interactive games, training, teaching, etc., in which physical objects as, for example, bumpers, walls, etc., which are derived from a video tape or disc and endowed with any number of characteristics through software.

This is illustrated by a training application for training military personnel in the use of a light antitank weapon. In this application, recorded on the video tape recorder are a background and a moving tank. The object of the training exercise is to teach military personnel how to shoot at the tank. The output from the tape recorder displays on a television receiver the background and moving tank.

The system also includes a simulated weapon having a photocell therein, which the trainee aims at the screen in order to shoot at the tank. When the trainee presses the trigger of the simulated weapon, a light spot appears on the television screen in the location where the trainee should have been pointing to hit the tank, e.g., on the tank turret. If he was in fact aiming his weapon correctly, the photocell in the weapon will produce an output responsive to the light spot. The interactive feature lies in that the video tape recorder has embedded in the horizontal retrace interval data pertaining to the tank location such that when the trainee operates his weapon, he causes the data in the horizontal interval to generate via circuitry a signal to occasion the light spot to be illuminated on the screen. Like the pinball application the data in the horizontal retract interval pertain to the information in the pictorial portion of the television scan on a field-by-field basis.

While the particular game has been described in conjunction with a video source, namely a video tape recorder or disc, any video source will suffice. For example, cable television operations are constantly looking for new uses for the cable, clearly one channel could be devoted to games and could supply the same information over the cable which is supplied by the video tape recorder 22. In broadcast environments the information now supplied by video tape recorder 22 could be supplied over the air either as presently constituted from broadcast stations or via satellite as contemplated for the near future. In either the cable or broadcast situations the cable or broadcast situation would generate the required video signals from a recording or in real time from a camera, etc. Accordingly, it is to be understood that the embodiment shown is illustrative only and that many variations and modifications may be made without departing from the principles of the invention herein disclosed and defined by the appended claims. ##SPC1## ##SPC2## ##SPC3## ##SPC4## ##SPC5## ##SPC6## ##SPC7## ##SPC8## ##SPC9##