System and method for processing real-time video streams

[0001] 1. Field of the Invention

[0002] The present invention relates to a system and method for data processing, and in particular to a system and method for processing real-time video streams.

[0003] 2. Description of the Related Art

[0004] One method of achieving the serial effect required by the persistence of vision effect deriving animation playback comprises alternating data between the front and back buffers in a display buffer.

[0005] A front buffer is visible, and displays video data on a display device, while a back buffer is invisible, and stores blocks in memory. Video data stored therein is automatically flipped between the front buffer and back buffer, and transferred from back buffer to front buffer.

[0006] Another method of achieving the effect known as above is block blitting, is regarded as more effective and is in common use.

[0007] 3D graphic design simulates objects by utilizing complex mechanisms or models to mirror behaviors of real objects in a 3D animation scene for projection and display on a two-dimensional (2D) monitor.

[0008] Objects are modeled with dots, lines, and surfaces, using 3D coordinates (X, Y, Z), which are enlarged, minimized, moved, and deformed, by linear algebra. FIGS. 1A1E are schematic diagrams showing the conventional method for generating 3D animation. Vertex data of a 3D object, including a coordinate (X, Y, Z), colors (RGBA), textures, and a coordinate (U, V), are transformed to coordinates (X_s, Y_s, Z_s) on the screen. Lighting effects, such as R_sG_sB_sA_s colors, are transformed by view port clipping, and scaling, as shown in FIG. 1A, wherein X_s and Y_s refer to screen coordinates and Zs refers to depth of field. A screen coordinate (X_p, Y_p, Z_p) of each pixel is generated according to the screen coordinate (X_s, Y_s, Z_s), as shown in FIG. 1B, a color R_pG_pB_pA_p of each pixel is generated according to the lighted color R_sG_sB_sA_s, as shown in FIG. 1C. Next, a color R_tG_tB_tA_t on textures is obtained with respect to the coordinate (U, V) by texture mapping and an object color R_oG_oB_oA_o is then generated by a blending the colors R_pG_pB_pA_p and R_tG_tB_tA_t, as shown in FIG. 1D. Next, a desired color R_fG_fB_fA_f of the 3D object, as shown in FIG. 1E, is generated by blending the colors R_oG_oB_oA_o and R_dG_dB_dA_d stored in the current display buffer, and the pixel of a coordinate (X_p, Y_p) is drawn with the color R_fG_fB_fA_f. The steps above are repeated until the entire 3D animation scene is generated.

[0009] Current 3D animation games are displayed in full screen on monitors to enhance game effects. Video applications, however, cannot be simultaneously displayed in a 3D animation scene. A conventional method to solve the problem is to connect an external video mixer to mix video signals and input signals from the computer system. This method, however, is incapable of processing video streams from the Internet.

[0010] Accordingly, an object of the present invention is to provide a method for processing real-time video streams, displaying video streams played by a video player application in a 3D animation scene using a method for 3D texture mapping with a shared memory.

[0011] According to the object described above, the present invention provides a method for processing real-time video streams. First, addresses of a shared memory shared by a 3D animation program and a video player application are defined. Video data is obtained through the video player application and is then copied to the shared memory. The computer system determines whether the 3D animation program is being executed while the video player application is executed, and, if so, the video data retrieved from a video source is copied to the shared memory. Next, a driver of the 3D animation program determines whether the 3D animation program has drawn a refreshed picture to a back buffer, and, if so, the video data stored in the shared memory is loaded to a video memory. Finally, the video data is blended with the refreshed picture stored in the back buffer, in which the refreshed picture and the video data is displayed on a display device.

[0012] The present invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:

[0013] FIG. 1A1E are schematic diagrams showing the conventional method for generating 3D animation;

[0014] FIG. 2 is a schematic diagram showing the shared memory for data communication between a 3D animation program and a video player application according to the present invention;

[0015] FIG. 3 is schematic diagram showing the architecture of the system for processing real-time video streams according to the present invention;

[0016] FIG. 4 is a flowchart showing the detailed steps of the method for retrieving video data from a video source through a video player application according to the present invention; and

[0017] FIG. 5 is a flowchart showing the detailed steps of the method for processing real-time video streams according to the present invention.

[0018] The present invention discloses a method and system for processing real-time video streams. A 3D animation program calls an application programming interfaces (API) to display pictures on a monitor when finished drawing a 3D animation scene, combining and drawing pictures retrieved from video streams to the 3D animation scene by a 3D texture mapping method. A video player application receives refreshed pictures from the video streams repeatedly to be displayed while the 3D animation program is executed.

[0019] FIG. 2 is a schematic diagram showing the shared memory for data communication between a 3D animation program and a video player application according to the present invention. Both the 3D animation program and video player application are independent applications that cannot share resources, they must communicate video data through a shared memory to be displayed in a 3D animation scene with a video window object.

[0020] FIG. 3 is schematic diagram showing the architecture of the system for processing real-time video streams according to the present invention. The architecture comprises a video player module 1, having a video capture unit 100, and a 3D animation module 2, having a video processing unit 200, a video drawing unit 300, and a video display unit 400.

[0021] Video capturing unit 100 retrieves a video picture 15 from a video source 10, automatically copying video picture 15 to a shared memory 150 while a 3D animation program is executed, such that the combined image data fills the entire display monitor, and informs a computer system to refresh video data. Video processing unit 200 loads video picture 15 from shared memory 150 to video memory 250, in which video picture 15 is represented as texture. Video drawing unit 300 creates a video window object 330 for displaying video picture 15, drawing the texture (video picture 15) and video window object 330 to a back buffer 350 using a 3D texture mapping method. Finally, video picture 15 is displayed with video window object 330 through video displaying unit 400 while APIs of the computer system are called.

[0022] FIG. 4 is a flowchart showing the detailed steps of the method for retrieving video pictures from a video source through a video player application according to the present invention.

[0023] In step S11, addresses of a shared memory are defined for access by a video player application.

[0024] In step S12, the video player application determines whether a new video picture has been created, and, if so, the process goes to step S13, and, if not, the process goes to step S15.

[0025] In step S13, a computer system determines whether a 3D animation program is executed while the new video picture is produced, and, if so, the process goes to step S14, and, if not, the process goes to step S15.

[0026] In step S14, the video picture is copied to the shared memory while the 3D animation program is executed, and a video refreshing flag is then set to determine whether there are video pictures in the shared memory. If so, the value of the video refreshing flag is initialized to one, and, if not, initialized to zero.

[0027] In step S15, the computer system determines whether the video player application has stopped. The video player application is determined as stopped if no video picture has been retrieved from the video source, or the process goes to step S12 to continue the video picture generating procedure.

[0028] FIG. 5 is a flowchart showing the detailed steps of the method for processing real-time video streams according to the present invention.

[0029] In step S21, vertex data of a video window is defined to determine the size and location of the video window in a 3D animation scene. It should be noted that the step S21 is optional in that values of the size and location can be predetermined or fixed so that vertex data need not to be defined.

[0030] In step S22, addresses of a shared memory are defined to be accessed by a 3D animation program.

[0031] In step S23, the 3D animation program determines whether a refreshed picture has been drawn to a back buffer. A first driver and a sub-program thereof used to draw the refreshed picture are obtained by a computer system. The first driver is loaded to a memory of the computer system through a second driver to obtain an entry point for the sub-program. The second driver is registered in the computer system for replacing the first driver, determining whether the refreshed picture has been drawn, and, if so, the process goes to step S24, and, if not, the process goes to step S27.

[0032] In step S24, the 3D animation program determines whether the shared memory stores video pictures according to a video refreshing flag, and, if so, the process goes to step S251, and, if not, the process goes to step S252.

[0033] In step S251, the video window is drawn to the back buffer if no video picture is stored in the shared memory.

[0034] In step S252, video pictures are loaded from the shared memory to a video memory of a 3D graphic card, and the video window and the video picture are added to the refreshed picture, thus initializing the value of the video refreshing flag to 0.

[0035] In step S26, the refreshed picture with the video window and the video picture stored in the back buffer are displayed in the 3D animation scene by executing flipping operations.

[0036] In step S27, the computer system determines whether the 3D animation program has stopped. If so, the program is determined as stopped, and, if not, the process goes to step S23 to continue the process.

[0037] The method of the present invention displays pictures, stored in a back buffer, played by a video player application in a 3D animation scene through a shared memory and a 3D texture mapping method while a 3D animation program is executed, such that the combined image fills the entire display monitor.

[0038] While the invention has been described by way of example and in terms of the preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art). Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.