User Interface for a Video Conferencing System

The present invention relates to the field of telecommunications. More particularly, the present invention relates to video conferencing.

Video conferencing has become increasingly popular as the availability of conferencing equipment has grown and the cost of carrying the audiovisual data has dropped. Despite its relative wide availability, however, conferencing equipment remains, relative to a telephone, difficult to operate for the casual user. This relative complexity continues to be a deterrent to even wider use of video conferencing.

The present invention provides method and apparatus for displaying video in a dual monitor video conferencing system. According to one example embodiment, remote end video is displayed in a first monitor during all modes of operation, while near end video and shared applications are displayed in a second monitor.

According to another example embodiment, remote end video is displayed in a first monitor and the near end video is displayed in a second monitor, and in another mode of operation, near end and far end video are displayed in the second monitor, and shared applications in the first monitor.

According to still another example embodiment, near end, remote end, and control options are displayed in a first monitor during all modes of operation, while shared applications are displayed in the second monitor.

FIG. 1 illustrates in block diagram form an example embodiment of a computer system adapted for video conferencing according the present invention.

FIGS. 2 and 3 illustrate an example embodiment of state diagrams for the video conferencing interface according to the present invention.

FIGS. 4-12 illustrate an example embodiment of the user interface according to the present invention.

FIGS. 13-20 illustrate another example embodiment of the user interface according to the present invention.

FIGS. 21-27 illustrate still another example embodiment of the user interface according to the present invention.

FIGS. 28-33 illustrate yet another example embodiment of the user interface according to the present invention.

In the following detailed description of the preferred embodiments, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration specific embodiments in which the invention may be practiced. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.

Referring now to FIG. 1, there is illustrated in block diagram form a computer system of one embodiment of the present invention. The computer system comprises bus 100, keyboard interface 101, external memory 102, mass storage device 103 and processor 104. Bus 100 can be a single bus or a combination of multiple buses, and provides communication links between components in the computer system. Keyboard controller 101 can be a dedicated device or can reside in another device such as a bus controller or other controller. Keyboard controller 101 allows coupling of a keyboard to the computer system and transmits signals from a keyboard to the computer system. External memory 102 can comprise a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, or other memory devices. External memory 102 stores information from mass storage device 103 and processor 104 for use by processor 104. Mass storage device 103 can be a hard disk drive, a floppy disk drive, a CD-ROM device, or a flash memory device. Mass storage device 104 provides information to external memory 102. Processor 104 can be a microprocessor and is capable of decoding and executing a computer program such as an application program or operating system. An audio input/output device 105 is also provided, and includes a microphone 106 and speaker 110 to receive sound in the environment of the system and convert it to a digital form that can be processed by the system, and in particular processor 104. In addition, the computer system includes a video input device 107, and includes a video camera 108 that is positioned to view a visual field proximate the computer system. Video input device outputs a digital video signal that can be processed by processor 104, as described below. Also included is a video interface 112 for driving monitor 113 and, optionally, a second monitor 114.

The present invention provides a user interface for a video conferencing system, and in particular a video conferencing system configured on a host system, such as a personal computer or workstation platform as described above, which may, for example, take the form of an IBM-compatible computer executing the Microsoft.RTM., Windows.RTM. 95, Windows.RTM. 98, or Windows.RTM. NT operating system. According to one example embodiment, the video conferencing system takes the form of a Microsoft.RTM., Windows.RTM. application executing on a platform such as that of FIG. 1 configured with the Windows.RTM. operating system.

Referring now to FIG. 2, there is illustrated a state diagram showing the various states of the video conferencing system according to the present invention. The Ready view 202 is the initial state of the system. If a "pre-load document" mode is selected (through a user interface mode select control, such as a "button"), the system performs any pre-load operations necessary 204 and waits for an incoming video call 206, or, if the Video Call mode is selected, the system gets a video number 208 from the user. If the call is placed, video call progress is displayed 210. Views 206 and 210 are have the same appearance as the Ready view 202, with a status message indicating the sub-state. If the call is hung up, the system returns to the Ready view.

If a call is connected, the system changes from the Ready view 202, and in particular the Call Progress view 210, to the Video Only view 212. This view and the Full-screen Video view 214 can be toggled between. The Video with Data not Shared view 216 can be entered from the Video Only view 212. From this view 216, the Data view 218 is entered if a user chooses to show or share a document. The Video with Data view 220 is entered from the Data view 218 if the user chooses the "show video" mode. Similarly, the Data view 218 is entered from the Video view with Data 220 if a user chooses the "show data" mode. Similarly, the Full-screen Video view 214 and Data view 218 can be toggled between by selecting the "show document" and "show video" options displayed in each respective view. All views return to the Ready view upon hanging up of the connection.

A Telephone view 222 is provided to handle voice only calls. A state diagram for telephone calls is shown in FIG. 3. If the "telephone call" button is selected while in the Ready view 202, the system retrieves a phone number to be dialed 224. If the call is placed, a voice call progress view is displayed, as a variant of the Ready view. If the call is successful, the Telephone view 222 is displayed. Hanging up at any point returns the system to the Ready view 202.

As will illustrated below, the above-described system modes provide six distinct views: Ready, Video, Full-screen Video, Floating Video, Data and Telephone Call view. According to one embodiment of the invention, these views are managed automatically for a user based on the current state of the conference. In general, these modes of operation provide for simultaneously displaying local and far end video, and sharing data through an application. These views are described below with respect to the position and size of the local video image, local video image, data tools, and application window. Illustrated below are various different embodiments of the user interface according to the present invention.

Referring now to FIGS. 4-12, there is illustrated a first embodiment of the invention. In this embodiment, and the others described herein, the system runs on an operating system such as Microsoft.RTM., Windows.RTM., or the Apple Macintosh.RTM., Finder.RTM.. As illustrated in FIG. 4, the user interface for the system is displayed in a main window 300 which has two principal areas where video and/or data applications can be viewed, namely a small video area or window 230 and a large video/application area 234. The size of the larger area 234 may be proportioned based on the video resolution available on the host system.

As illustrated in FIG. 5, the Ready view includes a local video image 302 (the video feed from the local site) sized and positioned to fill the large video/application area, within the application window 300. The remote video image 312 (the video feed from the remote site) is hidden from view. A status message 304 indicates that the system is ready to dial. Four control options, in the form of mouse activated buttons, are prominently presented for the user's choice: video options 305, video call 306, preload documents 308 and telephone call 310, allowing the system to be switched between states. Video options may include, in one example embodiment, controls allowing the selection of video display modes like full screen video. However, the mechanism used to provide video options is not important to the invention. The video call button 306 initiates a video call sequence (FIG. 2; 202, 208, 210). The preload documents button 308 initiates a document loading sequence (FIG. 2; 204). The telephone call button 310 initiates a telephone call sequence (see FIG. 3). Also shown in FIG. 5 is the operating system menu bar 309, in this case the Taskbar for the Windows.RTM. operating system, which includes a "start" button.

Referring now to FIG. 6, there is illustrated the Video Only view according to this embodiment of the invention. The local video image 302 is moved and sized to fill the area of the small video window. The remote video image 312 is sized and displayed in the large video/application area. Five control options, in the form of buttons, are prominently presented for the user's choice: video options 305, hang up 314, volume 316, mute 318 and camera 320. The hang up button 314, volume button 316 and mute button 318 provide for, respectively, hanging up a call, controlling volume and muting. The camera button 320 provides camera control functions to control for example, aiming or zooming the local or remote camera.

Referring now to FIG. 7 there is illustrated the Full-screen Video view. This view is only displayed by direct user action, which, for example, may require the user to click on a menu item in the video options 305 when in a video conference. The user is permitted to switch between the Video Only view and the Full-screen Video, or Data and Full-screen Video view. In the Full-screen Video view, the local video image 302 window is moved and sized to fill the small video window. The remote video image 312 is sized and positioned to fill the entire screen display. In this case, only video is available to the user, and the Windows.RTM. menu bar 309 is hidden from view.

Referring now to FIG. 8, there is illustrated the Video with Data Not Shared view. A list of documents to share 321 is displayed, wherein one of the listed documents can be highlighted with a mouse pointer, and then chosen using the share button 322. In one example embodiment, a data tools button 324 is also displayed, providing, for example, access to data tool options such as a pointer that may be used to point at data in a shared application.

Referring now to FIGS. 9 there is illustrated the Data view according to this embodiment of the invention, entered by selecting the share button 322. In the display of FIG. 9, the main window of the video conferencing application 300 is reshaped into an "L" configuration. The large video/application area (FIG. 4, 324) is filled with a data sharing application 326, for example, a "whiteboard" application, a word processing application, or a spreadsheet application. Detection of data sharing may be ascertained, for example, using the functionality of Microsoft's NetMeeting system. Within the L-shaped main window, the local video image 302 is hidden from view. The remote video image 312 is moved and sized to fill the small video window. Control options are the same as are provided in the Video with Data Not Shared view, with the addition of a show video button 328.

Choosing the show video option toggles the system to the Video with Data view shown in FIG. 10, which is similar to that of FIG. 9, with the show video button 328 toggled to a show documents button 330. Choosing this button toggles the system back to the interface of FIG. 9.

Referring now to FIG. 11, there is illustrated the Floating Video view. This view is invoked when the system detects that an application 326 has been launched full screen on top of the system's main window 300. In this view, the main window 300 is hidden from view. A new window is created, sized and positioned over the small video window area. This new window contains a new remote video image 312 window. In one example embodiment, this transition appears seamless enough to a user so that it appears as if the main window has shrunk to become just the new local video image window. The small video window is set to topmost in the Windows.RTM. operating system (meaning that it cannot be covered by another window), and can be moved around on the screen by the user. Video options 305 are available from this window.

Referring now to FIG. 12, there is illustrated the Telephone Call view. This view is displayed when a voice-only telephone call is established. In this view, there is no video so the local and remote video image 312 windows are hidden from view. A telephone-like keypad 332 is positioned over the small display area and a graphic of a phone 334 is drawn in the large display area.

Referring now to FIGS. 13-20, there is illustrated another embodiment of the invention adapted for platforms with dual monitors. According to this embodiment, the near end video and the shared applications are displayed on the main, or primary monitor display 340. The far end video is displayed in the second monitor display 342.

Referring now to FIG. 13, there is illustrated the Ready view according to this embodiment of the invention. This view is similar to the view illustrated in FIG. 5, with the second monitor display 342 blank. Referring now to FIG. 14, there is illustrated the Video Only view according to this embodiment of the invention. The local video image 302 remains displayed in the large video/application area. The remote video image 312 is displayed full screen on the second monitor display 342. Referring now to FIG. 15, there is illustrated the Full-screen Video Only view. In this view, the local video image 302 window is sized and positioned to fill the entire screen display of the primary monitor display 340, while the remote video image 312 continues to be displayed full screen in the second monitor display 342. In display 340, the Windows.RTM. menu bar 309 is hidden from view. The control buttons video options 305, hang up 314, volume 316, mute 318 and camera 320 are displayed on top of the full screen video. Referring now to FIG. 16, there is illustrated the Video with Data Not Shared view. This view is the same as that of FIG. 14, with the addition of the document sharing options (321, 322, 324). Referring now to FIG. 17, there is illustrated the Data view according to this embodiment of the invention. In the interface of FIG. 17, the primary monitor display 340 presents the same display as illustrated in FIG. 9, except that the local video image 302 is shown in the small display area, with the local video image in the second monitor display 342. Choosing the show video 328 option toggles the system to the view shown in FIG. 18, which is similar to that of FIG. 16, with the show video button 328 changed to a show documents button 330. Choosing this button toggles the system back to the interface of FIG. 17. Referring now to FIG. 19, there is illustrated the Floating Video Only view. This view is invoked when the system detects that application 326 has been launched full screen on top of the system's main window 300. In this view, the main window 300 is hidden from view. A new window is created, sized and positioned initially over the application 326, and contains a new remote video image 312 window. Referring now to FIG. 20, there is illustrated the Telephone Call view, which is similar to the embodiment of FIG. 12.

Referring now to FIGS. 21-26, there is illustrated another embodiment of the invention adapted for platforms with dual monitors. According to this embodiment, the near end and far end video is presented in separate monitors when data is not shared. Shared applications are displayed in a separate monitor, and local 302 and remote video image 312 is combined.

Referring now to FIG. 21, there is illustrated the Ready view according to this embodiment of the invention. This view is similar to the view illustrated in FIG. 5, with the second monitor display 342 blank. Referring now to FIG. 22, there is illustrated the Video Only view according to this embodiment of the invention. The local video image 302 remains displayed in the large video/application area. Referring now to FIG. 23 there is illustrated the Full-screen Video Only view, which is the same as the interface described with respect to FIG. 15. Referring now to FIG. 24, there is illustrated the Video with Data Not Shared view, which is the same as that of FIG. 16. Referring now to FIG. 25, there is illustrated the Data view according to this embodiment of the invention. In this view, the primary monitor 340 has the same user interface as is illustrated in FIG. 8, with the shared application 326 shown in the second monitor display 342. Referring now to FIG. 26, there is illustrated the Data view with Full-screen Video and a shared application. In this view, the local video image 302 is sized for the small video window (which may be moved by the user), while the remote video image 312 is displayed full screen on the primary monitor display 340. The shared application continues to be displayed full screen in the second monitor display 342. In the case of the primary monitor display 340, the Windows.RTM. Taskbar is hidden from view. The control buttons video options 305, hang up 314, volume 316, mute 318 and camera 320 are displayed on top of the full screen video. Referring now to FIG. 27, there is illustrated the Telephone Call view, which is similar to the view illustrated with respect to FIG. 20.

Referring now to FIGS. 28-33, there is illustrated yet another embodiment of the invention adapted for platforms with dual monitors. According to this embodiment, the primary monitor display 340 has standard PC or workstation functionality and the shared applications. The second monitor display 342 operates like the first described embodiment above (FIGS. 4-12) except that the shared applications appear in the first monitor display 340.

Referring now to FIG. 28, there is illustrated the Ready view according to this embodiment of the invention. This view is similar to the view illustrated in FIG. 5, with the Ready view displayed on the second monitor display 342, and the primary monitor display 340 unused by the video conferencing system. Referring now to FIG. 29, there is illustrated the Video Only view according to this embodiment of the invention. This view is the same as that of FIG. 6, displayed in the second monitor 342, and the primary monitor 340 unused. Referring now to FIG. 30 there is illustrated the Full-screen Video Only view. This view is the same as the view described with respect to FIG. 7, except it is displayed in the second monitor display 342. Referring now to FIG. 31, there is illustrated the Video Only view with document sharing options displayed in monitor display 342. Referring now to FIGS. 32, there is illustrated the Data view according to this embodiment of the invention. The second monitor display 342 remains in the same state as described above with respect to FIG. 31. However, a shared application 326 is now displayed on the primary monitor display 340. This application may be maximized to full screen. Referring now to FIG. 33, there is illustrated the Data view with Full-screen Video and a shared application. This view is the same as FIG. 32, except that the second monitor 342 includes a full screen video of the remote video image 312. Referring now to FIG. 34, there is illustrated the Telephone Call view. This view is similar to the view of FIG. 27.

In all the embodiments described above, when one or both of the video windows is hidden from view, the video stream associated with that window may be, if desired, paused to conserve CPU cycles. The video stream is also paused when the video windows are moved and/or sized allowing these actions to happen much faster than is possible if the video is playing.

Thus, the embodiments described above may reduce the technical challenges and time a user faces in having to manage their video and data application windows while also trying to hold an effective video conference. As illustrated, the system manages video conference related windows throughout the conference and outside the conference, based on simple menu choices made by the user. Although the invention has been described with respect to specific embodiments, it shall be understood that these embodiments are exemplary only, and that it is contemplated that the described methods and apparatus of the invention can be varied widely while still maintaining the advantages of the invention. Thus, the disclosure should not be taken as limiting in any way the scope of the invention. In addition, as used herein, the term "unit" shall refer to a digital device that may take the form of a hardwired circuit, software executing on a processor, or a combination of both.