Graphics Display System Using Frame Buffers

This invention relates to a novel method and apparatus for producing a video display of highly complex and visually pleasing graphics, particularly but not exclusively including the production of animation, where, through the use of frame buffers, continuity of display is ensured, the handling of additional amounts of data in real time is accomplished, and a considerable amount of updating of stored data during normal display can be accomplished.

Over the past two decades, computers have been pervasive in penetrating many areas of industry, entertainment, defense and art. This increased use and acceptance of computers has generated a need for them to produce accurate and versatile results while at the same time being easy to operate by non-skilled users and involving the use of relatively simplified computer hardware. In no specific area has this need been more apparent than in connection with the production of computer graphics. In the past, images produced by reasonably priced computer systems were in general too crude to be useful in realistic imaging applications and current systems provide only limited animation capability.

A graphics display system which to a large extent satisfied that need is disclosed in U.S. Pat. No. 4,700,181 entitled "Graphics Display System" and assigned to the assignee of this application and herein incorporated by reference. The general history of the art is set forth in that application under the heading "Background of the Invention". The system described in that application represented a significant advance over the prior art, particularly in the ability to select from vast amounts of data the information necessary to produce display images of desired precision and complexity, all within real time, but, as with any system, there was an upper limit on the amount of data which could be thus handled in real time.

The system disclosed in U.S. Pat. No. 4,760,390 entitled "Improved Graphics Display System", also assigned to the assignee of this application and also herein incorporated by reference, resulted from the strong incentive to produce a system which could further push forward the technology for displaying quality images, expanding the amount of data which could be manipulated in real time to produce such images, and further facilitating the ability to animate such images, all while still utilizing commercially practical computer hardware.

The desirability of being able to handle ever more data in real time was always before us. Moreover, it was recognized that in many instances it was not the total amount of data to be handled in real time that was controlling, but that rather the need to handle increasing data detail concentrated in selected portions of the display screen was often a determining factor. Being able to display a limited number of highly precise and complex images in different portions of the screen, thus requiring the application of much data to those limited screen portions, while other portions of the screen could have little or only rudimentary data applied thereto, was often far more important than the total amount of data that had to be handled for the entire screen area. Dealing with the display on a line-by-line basis, as was disclosed in the aforementioned U.S. Patents, did not prove to be optimum in solving this particular problem.

In addition, in most display systems the stored data must from time to time, and sometimes quite frequently, be revised or updated. This can be done by providing the new or revised data from an external fixed data source such as tapes, cassettes, or optical or magnetic discs, or that new data may be provided on a "live" basis, as from a video camera or by a "painting" operation. The system of U.S. Pat. No. 4,760,390 had some capability for permitting stored data revision while display was going on, but that capability was limited by the time required to construct the images in the construction/display buffers. Only the excess of the total available construction time over the construction time actually needed could be used for memory updating. If more updating time than that was required, only a partial image construction was possible, and that is obviously undesirable.

The enhanced capability of the system of the present invention significantly expands the potentialities of a graphic system, including the systems of the aforementioned patents, particularly in terms of manipulation of data in order to create a scene and change it, all within the time constraint of a full motion video display system such as a CRT monitor or a TV set.

As used in this specification and in the claims, "real time" refers to the time required by a full motion video display system, such as one meeting standards of the National Television Standards Committee, to provide commercially acceptable representations of motion. Typical of such display systems are CRT monitors, TV receivers and the like. The system of the present invention produces instant interactive images within the time required to scan a frame of the display system, and can sustain those images indefinitely. Thus, designed with speed in mind, this system is capable of producing life-like animation for domestic television sets or monitors. This animation can be made up of entirely computer generated shapes or pictures scanned into the host computer with a video camera, tape, disc or other graphic system. In either case, the resolution provided is far better than what current low cost computer video systems provide.

It is a prime object of the present invention to provide a graphics display system in which an increased amount of data can be handled in real time, particularly with respect to selected portions of the display where greater display detail or animation is desired.

It is another prime object of the present invention to provide a display system in which display continuity is assured even though extensive stored data revision is taking place.

It is a further prime object of the present invention to organize the display system so that a very substantial amount of stored data revision can take place without interfering with the continuity of changeable or animated displays, even to the extent of enabling the new stored data to be introduced from a "live" source such as a TV camera without interrupting the continuity of the dynamic display.

The objects set forth are accomplished through the use in particular ways of frame buffers, either in place of or in addition to the line buffers specifically disclosed in the aforementioned U.S. Patents, for construction and display purposes. (By "line buffers" is meant a buffer with a capacity corresponding to one or more lines, but fewer lines than an entire frame.) In perhaps the simplest embodiment, a full frame buffer is employed, fed by the line buffers and retaining within itself the data for the last-constructed scene produced by those line buffers. Should those line buffers temporarily become inoperative because memory is being updated and therefore is not available for access by the buffers, the frame buffer will produce a scene on the display screen. That scene will be static, not dynamic, but that is much more acceptable than if the screen were to go blank.

In more sophisticated embodiments, the line buffers of our previous applications are supplanted by frame buffers. This has several advantages. One is that it eliminates the necessity for "painting" on a line-by-line basis, which is essentially uneconomic from a time viewpoint, and permits the painter to operate object by object, a much more efficient use of the "painter's" time, thus increasing the amount of data that can be handled in real time. Another is that particular objects can be displayed with considerably more detail than was possible with our previous systems. Yet another is that the frame buffers can also provide for a static display if access to the memories is cut off, as when the latter are being recharged with data.

Separate frame buffers can be provided for the odd and even lines respectively of the display, those separate frame buffers alternately functioning as construction buffers or display buffers analogously to the alternate construction/display functioning of the line buffers of the aforementioned U.S. Patents. An even more advantageous arrangement is to utilize separate full frame buffers for alternate construction and display modes. This arrangement has the advantage that since each of these buffers is constructed in one frame time and will then provide display data for two frame times, one for the odd lines and the other for the even lines, the nondisplaying memory unit can be made available to an external data source for updating or revision during one of those frame times. Hence, continuous variable and animated displays can be produced while at the same time allocating half of the total time to the external data source, which in that case can be a "live" source, such as a TV camera.

To the accomplishment of the above, and to such other objects as may hereinafter appear, the present invention relates to a system (method and apparatus) for forming a graphics display, as defined in the appended claims, and as described in this specification, taken together with the accompanying drawings, in which:

FIG. 1 is a block diagram of a first embodiment of the present invention;

FIG. 2 is a block diagram of a second embodiment of the present invention;

FIG. 3 is a block diagram of a third embodiment of the present invention; and

FIG. 4 is a time chart showing the performance of various functions by the embodiment of FIG. 3.

All of the systems here disclosed are similar to those of U.S. Pat. No. 4,760,390 in utilizing a pattern memory A, a system memory B, a third memory component C, a system processor D, a graphics painter E, buffers F used for construction and display, a color map G, and a digital to analog converter H, from which a composite video signal I goes to the display instrumentality. The interaction between those elements is set forth in considerable detail in the aforementioned U.S. Pat. No. 4,760,390, here incorporated by reference. Briefly, the pattern memory A receives and stores data, usually in the form of a bit map, defining the appearance of those object elements which, it is expected, will be displayed over a period of time, although in most instances not all of those elements will be displayed at any given moment. The system memory B may contain program instructions and will also contain data, preferably in the form of linked lists of the type generally described in U.S. Pat. No. 4,700,181, identifying various components of a desired display and containing display instructions relative thereto. This data stored in the system memory B relates to all portions of the display which are to be formed throughout the period of operation of the system, and is not limited to the data needed for a display at any particular moment. A data processing unit, generally designated D and here called the "system processor", functions before a display run is commenced to deposit the appropriate data in the pattern memory A and the system memory B, obtaining that data from some external source J, and the system processor D is also used to revise or update the data in pattern memory A and/or system memory B in accordance with instructions and data either internally stored or received from the external data source J while the system is operating to produce displays. The system processor D further loads color information into the color map memory G.

When the system produces a display from the data stored in memories A, B and G, the system processor D reads from the linked lists of system memory B that identification and display instruction data relevant to creating a display at a particular moment, and it deposits that data into the third memory component C. That which is deposited will hereafter be termed "the compiled list", which may well be in the form of a sequential list, and hence the memory component C is termed "the compiled list memory C". The compiled list represents the object element identification and relevant display instructions for a particular instantaneous display, this being usually only a small proportion of the corresponding data stored in memories A and B.

A separate data processing unit generally designated E, and hereinafter termed the "graphics painter", addresses the identification and instruction data stored at any given moment in the compiled list C and, for each object element identified in the compiled list C, reads from the pattern memory A the data defining the appearance of that object element and then, in accordance with the display instructions for that object element stored in the compiled list C, the graphics painter E produces display data which it feeds to the two alternately acting display/construction buffers generally designated F, which as here specifically disclosed, correspond to the alternately acting buffers 16 and 18 of U.S. Pat. No. 4,700,181 and the alternately acting buffers F-I and F-II of U.S. Pat. No. 4,760,390.

As in the aforementioned patents, the output from the construction/display buffers F goes to the color map G, into which appropriate data had previously been stored by the system processor D, and from there the display data goes to digital to analog converter H, from which a composite video signal I goes to the display instrumentality, all in well known manner, and for example as described more in detail in U.S. Pat. No. 4,700,181.

Two compiled lists C-I and C-II may be provided, each of which may contain the appropriate identification and display instruction data for a given line, group of lines, or frame. When one of the compiled lists C-I or C-II is being accessed by the graphics painter E in order to produce display data for one line or frame, the list for the next line or frame is being constructed by the system processor D in the other compiled list C-II or C-I.

In the above-described system, a display can be produced only so long as the line buffers F have access, via the graphics painter E, to the pattern memory A, since it is only with such access that the display lines can be constructed in the buffers F. Whenever the pattern memory A has to be available to other sources, formation of a display must cease. This greatly restricts the capability of the system to function while at the same time permitting updating of the pattern memory A. Yet it is often necessary to revise or update the contents of pattern memory A (and also system memory B) to an extent such as to be impossible to accomplish within real time constraints. When that occurs, the display screen goes blank. That may occur for only a fraction of a second or for many seconds, depending upon the extent to which the contents of pattern memory A are changed, but such blanking of the display is undesirable in any event.

The embodiment of FIG. 1 avoids this disadvantage by interposing between the construction/display line buffers F and the color map G a frame buffer K. The lines constructed by the line buffers F are in turn constructed in the frame buffer K, where they remain until modified, and the line-by-line data is fed from frame buffer K to the composite video signal I in any appropriate known manner. Thus, if for any reason construction of new lines in the line buffers F is interrupted--for example, because the pattern memory A is being updated and hence access thereto by the graphics painter E has been interrupted--the frame buffer K ensures that a scene is still displayed. That scene is static because no changes are being made in any of the lines stored in the frame buffer K, but a static scene is infinitely preferable to no scene at all.

A second limitation in the functioning of the systems of our two prior applications is that construction is accomplished on a line-by-line basis. As a result the total number of objects that can be supported by the system is limited by the "highest band width" line. In addition, the entire object list in system memory B is traversed for each line to be constructed although a good portion of that list will be irrelevant to the particular line under construction. This results in a significant waste of operating time. Both of these factors significantly adversely affect the amount of data that can be handled by the system in real time and tend to limit the degree of detail with which particular objects can be displayed.

To overcome this disadvantage, the embodiment of FIG. 2 substitutes frame buffers F' for the line buffers F of the embodiment of FIG. 1. In that embodiment, each of the frame buffers F' will accommodate half of the total frame, the buffer F'-I being adapted to contain the odd lines of the frame and the buffer F'-II being adapted to contain the even lines of the frame (it is conventional, in making up a display on a screen, to first sequentially display the odd lines of the frame and then to sequentially display the even lines thereof).

This use of frame buffers F' rather than line buffers F results in a significant overall improvement in the operation of the system. In the first place, since the display no longer need be constructed on a line-by-line basis, each object can be constructed in a given frame buffer F' in its entirety. Hence, the "highest band width" line no longer presents a limitation, and the construction time of the various objects to be represented in a display can be averaged over a given frame time, thus enabling certain objects to be displayed in greater detail than had been previously possible, and permitting an increase of perhaps a full order of magnitude in the number of objects that can be displayed. In addition, because the display can now be constructed in the buffers F' on an object-by-object basis, considerable time can be saved in accessing relevant portions of the pattern memory A.

Moreover, because the two buffers F'-I and F'-II contain data for a complete display, if access to the pattern memory A by the graphics painter E is interrupted, as described above, a static display can be produced by the buffers F'-I and F'-II. Hence, there need be no blanking of the display screen even when extensive updating of pattern memory A from the external source J takes place.

The broken lines in FIG. 2 show an optional modification of the system there disclosed. Instead of having the frame buffers F'-I and F'-II feed directly to the color map G, they can be caused to feed to an extra full frame buffer K', which in turn feeds the color map G. This extra full frame buffer K' is not needed to prevent blanking of the display when the pattern memory A is not available; the frame buffers F'-I and F'-II accomplish that, as has been described. The function of the extra frame buffer K' is to reduce the time that the display must be static by making the buffers F'-I and F'-II available to be used for construction while the buffer K' ensures continuity of static display, and also enables the buffers F' to be accessed by a third source, such as a TV camera, while continuity of the static display is ensured.

The embodiment of FIG. 3 is similar to the embodiment of FIG. 2 except that the construction/display buffers F" are each full frame buffers having the capacity to store data for both the odd lines and the even lines of the display. With this arrangement, considerably greater leeway in updating or changing pattern memory A from an external data source J is possible, as is indicated by the timing chart of FIG. 4. That chart discloses four time slots, each typically representing the time required to display either the odd or even lines of a display, two such time slots of 1/60th of a second each making up the conventional frame time of 1/30th of a second. A first frame is displayed in time slots 1 and 2; a second frame is displayed in time slots 3 and 4, and so on.

In the first time slot, the painter E constructs a frame in buffer F"-I, an earlier frame having been previously constructed in buffer F"-II. In that same time slot, the even lines of the display are produced from the appropriate data in buffer F"-II.

In the second time slot, the odd lines of the display are produced from buffer F"-II, but since the next frame has already been constructed in buffer F"-I, nothing more need be done with respect to that buffer. Hence, the time of that second time slot is now available for the external data source J to update the data in pattern memory A via the system processor D. The graphics painter E is prevented from having access to the pattern memory A during that second time slot, but it does not need that access.

In the third time slot, the graphics painter E is again given access to the pattern memory A and constructs the next frame in frame buffer F"-II, and in the same time slot, the even lines of the display are produced from the appropriate data in buffer F"-I.

In the fourth time slot, the odd lines of the display are produced from the appropriate data in buffer F"-I, but because buffer F"-II has already been completely constructed, the time of this time slot is available for updating of pattern memory A.

Alternatively, the system of FIG. 3 could be employed in conjunction with a "live" data source such as a TV camera, to construct one frame buffer F"-I from data provided by the graphics painter E and construct the other frame buffer F"-II from data provided by the "live" data source.

The embodiment of FIG. 3 could, if desired, be provided with the extra full frame buffer K' of FIG. 2 to produce the results previously set forth above.

As will be appreciated from the above explanation, the data handling capacities and flexibility of use of the systems of the two aforementioned patents, and particularly the system of U.S. Pat. No. 4,760,390, have been significantly enhanced by the system modifications here disclosed. It will be apparent that many variations may be made in the embodiments here set forth, all within the spirit of the invention as defined in the following claims.