US 6,819,752 B2Grant

Intelligent switching system for voice and data

Issue Date:2004-11-16

•29 Claims

•13 Drawing Sheets

Abstract

A teleconferencing system for voice and data provides interconnections among user sites via a central station. User stations at user sites each alternate operation between a data mode connecting a user computer and modem to a user telephone communication path and a voice mode connecting a telephony circuit to the communication path. The teleconferencing system is adapted for conducting a voice conference over standard telephone lines while allowing simultaneous viewing of data objects such as slides, graphs, or text. A host computer connected to the central station serves as a central repository for storage and retrieval of data objects for use in teleconferences.

Metadata

Inventors

Keith Raniere
Thomas A. Delaney
Steven Danzig
Saul Miodownik

Application Information

Application Number:US 10/000,634

Filing Date:2001-10-31

Priority Date:1995-02-16

Art Unit:7

Classifications

IPC:

H04M 356H04M 3523H04N 715

Field of Search:

348 1408348 141370260370261370262370263370264370265370266370267370270375222379 9001379 9301379 9314379 9326379 9328379 9305379 9309379 9337379 932137920201379203013792040137920501379206013792650137926601379309379900379902

Patent Drawings (13 sheets)

Description

[0002] This application is a continuation of Ser. No. 09/567,854, filed on May 9, 2000, now U.S. Pat. No. 6,373,936; which is a continuation of Ser. No. 09/203,110, filed on Nov. 30, 1998, now U.S. Pat. No. 6,061,440; which is a continuation of Ser. No. 08/390,396, filed Feb. 16, 1995, now U.S. Pat. No. 5,844,979. The entire teachings of these applications are incorporated herein by reference.

Background of the Invention

[0003] During a voice teleconference, it is often desirable for two or more conference participants to be able to both view and discuss common data objects such as a set of images. For example, one participant may desire to conduct a slide presentation concurrent with a voice conversation among the participants. Systems which enable a person to receive and view image data over a telephone line concurrent with voice conversation are known. One such system interrupts the telephone conversation momentarily to allow visual data to be transmitted over the telephone line. Following the transmission of visual data, the voice communication can be automatically resumed. Other known systems for sharing voice and image data generally require relatively complex equipment, specially enhanced modems, or dedicated high speed digital lines such as Integrated Services Digital Network (ISDN). Still other systems require multiple telephone lines to handle both data object manipulation and voice. Generally, the known desktop teleconferencing systems are for point-to-point communications rather than multipoint, multiparty communications.

Summary of the Invention

[0004] It would be advantageous to provide a teleconferencing system which avoids having to interrupt voice communication in order to transmit data objects such as images meant to be viewed and discussed concurrently. It would also be advantageous to be able to provide such a teleconferencing capability without requiring complex devices or dedicated lines. The above and other advantages are achieved by the present invention. In general, the teleconferencing provided by the present invention allows multiple users to conduct a voice conference over standard telephone lines while simultaneously viewing shared slides, graphs, text, or other data objects.

[0005] The improved teleconferencing system stores a common set of data objects, such as slides, in a digital computer associated with each participant prior to the start of a voice conference. Once a voice conference is underway, a lead speaker can generate audible signals (e.g., Dual Tone Multi-Frequency (DTMF) tones) to which the computers respond to coordinate simultaneous display of the data objects on each participant's computer screen. Since the data objects are stored prior to the conference and the audible signals are sent with voice communications signals, the voice conversation can flow naturally and uninterrupted by data transmissions.

[0006] To facilitate storage and dissemination of the common data objects, a novel arrangement is employed whereby a host computer, preferably configured as a bulletin board system (BBS), serves as a central repository for collecting subsets of data objects from conference participants and combining the subsets into a common set prior to a conference. Participating speakers each initially connect to the host computer and transmit a subset of data objects to the host computer. Participants (speakers and conference attendees) in turn are able to retrieve the common set of data objects from the BBS host computer before entering into a voice conference.

[0007] The host computer connects to a central station having a crosspoint switching matrix for interconnecting participants, or users, into multipoint voice conferences. The participants connect to the central station over standard telephone lines. An inexpensive user station controlled by the participant's digital computer facilitates selection between voice and data modes of access to the central station and the host computer. A participant can generate and receive audible signals via the user station to control conference and switching actions in relation to the user station and the central station. For example, a participant can generate an audible signal which switches the participant's user station from a voice mode to a data mode and concurrently reconfigures the central station to connect the participant either to the BBS host computer or to another participant for data mode operation.

[0008] Accordingly, a voice/data teleconferencing system interconnects a plurality of user sites and a central site over a plurality of communication paths, such as telephone lines. Each user site may include a user digital computer connectable to a user modem for transmitting and receiving data signals. A host digital computer at the central site is connectable to a plurality of host modems for transmitting and receiving data signals to and from the user sites.

[0009] A user station located at each user site includes a data port connectable to a user modem, a telephony circuit capable of transmitting and receiving voice communications signals, and a mode selector for alternately connecting the data port and the telephony circuit to a communication path. The mode selector has a data mode position in which the data port is connected to the communication path and a voice mode position in which the telephony circuit is connected to the communication path and the data port is connected to a user modem loopback impedance. The mode selector is responsive to a selector control signal to switch from data mode to voice mode, and the selector defaults to the data mode in the absence of the selector control signal. Thus, in the data mode, data signals can pass between the data port and the communication path, and in the voice mode, voice communications signals can pass between the telephony circuit and the communication path while the user modem is held “off-hook” by the loopback impedance. The user digital computer is programmed to control the user station.

[0010] The user station further includes a tone generator and tone detector for respectively transmitting and receiving audible conferencing signals with the voice communications signals over the communication path.

[0011] A central station at the central site includes a plurality of line ports and a switching matrix for interconnecting the user stations. The switching matrix comprises a plurality of crosspoint switching elements, each switching element actuable to connect one line port to another line port in response to matrix control signals. The central station further includes a plurality of tone generators and tone detectors coupled to respective line ports for transmitting and receiving audible conferencing signals to and from user stations.

[0012] The central station further includes a plurality of computer ports connectable to respective host modems. The central station includes central selector means for alternately connecting a line port and a host modem loopback impedance to a respective computer port. The central selector means has a pass-through position in which the line port is connected to the computer port and a loopback position in which the host modem loopback impedance is connected to the computer port. The central selector means responds to central selector control signals to switch from a pass-through position to a loopback position and defaults to the pass-through position in the absence of central selector control signals, such that in the pass-through position, data signals can pass between a line port and a respective computer port.

[0013] The central station further includes a controller for controlling conference and switching actions in the central station.

[0014] According to another aspect of the invention, each user site can record audible conferencing signals and voice communications signals in a voice conference for playing back at a subsequent time in conjunction with simultaneous display of the data objects on the user computer screen.

[0015] The above and other features of the invention including various novel details of construction and combinations of parts will now be more particularly described with reference to the accompanying drawings and pointed out in the claims. It will be understood that the particular teleconferencing system embodying the invention is shown by way of illustration and not as a limitation of the invention. The principles and features of this invention may be employed in varied and numerous embodiments without departing from the scope of the invention.

Brief Description of the Drawings

[0016] FIG. 1 is a block diagram of the teleconferencing system of the present invention.

[0017] FIG. 2 is a schematic circuit diagram of the user station shown in FIG. 1.

[0018] FIG. 3 is a schematic block diagram of the central station shown in FIG. 1.

[0019] FIG. 4 is a circuit diagram of the switching arrangement for connecting a line port to the switching matrix shown in FIG. 3.

[0020] FIG. 5 is a representation diagram of the reception area view of the user interface displayed on computer screen CS1 shown in FIG. 1.

[0021] FIG. 6 is a representation diagram of the conference room view of the user interface displayed on computer screen CS1 shown in FIG. 1.

[0022] FIG. 7 is a representation diagram of the overhead view of the user interface displayed on computer screen CS1 shown in FIG. 1.

[0023] FIG. 8 is a representation of the format used for audible conference control signals.

[0024] FIG. 9 is a flow chart of the user station activation procedure.

[0025] FIG. 10 is a flow chart of the procedure for switching to voice mode during a conference.

[0026] FIG. 11 is a flow chart of the procedure for controlling simultaneous viewing of slides in a conference.

[0027] FIG. 12 is a flow chart of the procedure for conducting a whiteboard session in a conference.

[0028] FIG. 13 is a flow chart of the procedure for leaving a conference.

[0029] FIG. 14 is a block diagram of a networking arrangement of teleconferencing systems of the present invention.

Detailed Description of the Preferred Embodiment

[0030] In general, the teleconferencing provided by the present invention allows multiple users to conduct a voice conference while simultaneously viewing distributed slides, graphs, text, or other data objects. Shown in FIG. 1 is a block diagram of a preferred embodiment showing a plurality of user sites (SS1-SSN) connected to a central station SW over a plurality of user telephone lines (SL1-SLN). The central station SW provides connections between user digital computers (SC1-SCN) and a host computer HC preferably configured as a BBS server. The central station SW also provides interconnections among user sites (SS1-SSN) to provide conferencing services. Each user site includes a user station (SU1-SUN) for selecting between voice and data modes of access to the central station SW and the host computer HC.

[0031] Operation of the preferred embodiment of the present invention in a BBS environment will now be described at a high-level. A user at a user site SS1 initiates a data mode connection between the user computer SC1 and the host computer HC. Having established a data mode connection, the user enters into a typical data mode session with the host computer HC. One option presented to the user by the host computer HC is to enter a conference.

[0032] There are two general conferencing modes: voice mode and data mode. In voice mode conferencing, two or more users are interconnected via the central station SW. Users in a voice conference transmit and receive voice communications signals and audible conference control signals. The audible conference control signals, which can be DTMF tones, for example, are described in detail herein. A user interface and conference control functions are also described further below.

[0033] The voice conferencing mode will now be described in further detail for the preferred embodiment. A key aspect of the present invention is the ability to conduct a voice conference among multiple users while simultaneously viewing shared data objects, such as slides. This aspect is achieved by having the individual users initially perform file transfers between their respective user computers (SC1-SCN) and the host computer HC in order to collect and disseminate the data objects required for a conference prior to the start of the voice mode portion of the conference. Thus, the host computer HC serves as a repository and distribution point for the data objects used in a conference. Further, conducting file transfers of the data objects prior to the voice mode portion of the conference eliminates interrupting the voice mode for downloads of data objects. For the initial file transfer of data objects, the respective user stations are placed in data mode and the central station SW is configured to pass-through data to the host computer HC.

[0034] “Data objects” can be any of several information types represented in any of a multiplicity of data formats. Data objects include, but are not limited to the following:


Object Type	Description	File Format

image	Joint Photographic	.JPG
	Experts Group (JPEG)
image	graphics interchange	.GIF
	format
image	fractal image	.FTC
	compression
video	full multimedia	.MPG
	(MPEG)
text	standard ASCII text	.TXT
rich text	Microsoft rich text	.RTF
	format
CAD	computer aided	many
	design
draw	drawing programs	.BMP & others
spreadsheet	spreadsheet programs	.XLS, .WKl &
		others
word processing	word processing	.DOC & others
slides	slide presentation	.PPT & others

[0035] The audible conference control signalling of the present invention can be used to emulate the application linking capabilities of Dynamic Data Exchange (DDE) and Object Linking and Embedding (OLE) for data object sharing within compatible applications at each user site during voice conferencing.

[0036] Once the multiple users enter the voice mode, a virtual visual conference is established simultaneously with the voice conference whereby display of the data objects on each user computer can be coordinated. The data objects display coordination is accomplished by having a designated speaker select a current data object to be displayed, the selection of which causes the speaker's associated user station to generate an audible conference control signal. The conference control signal is transmitted in-band with voice communications signals to the other users in the conference. The user station of each of the other users in the conference detects the conference control signal which then causes the locally stored copy of the data object selected by the speaker to be displayed.

[0037] In an alternative embodiment, the equipment at each user site is adapted for point-to-point conferencing whereby two user sites can be connected through the public-switched telephone network without passing through the central station SW. In this alternative embodiment, the two users are able to conduct a voice conference while simultaneously viewing shared data objects. This aspect is achieved by having one of the two users initially perform a file transfer with the other user so that each user has a copy of the data objects required for a conference prior to the start of the voice mode portion of the conference. Again, since the file transfers of the data objects occur prior to the voice mode portion of the conference, interruption of the voice mode to download data objects is avoided. When the two users in the point-to-point conference enter the voice mode, a virtual visual conference is established simultaneously with the voice conference by coordinating the display of the data objects using the same in-band conference control signals as described for the multipoint conferencing.

[0038] There are three forms of data mode conferencing in which users may transmit and receive data signals. In a data collaboration mode, two or more users are able to collaborate to perform a whiteboard function. The users in the data collaboration mode are connected through the central station SW to the host computer HC, each user having an individual data session. In a private data file transfer mode, two users are interconnected through the central station SW to provide for private file transfers between the two users without involving the host computer HC. There is also a private data broadcast mode in which two or more users are interconnected via the central station SW to allow a user to broadcast a file to the other users. In the private data broadcast mode, there is no handshaking between user modems.

[0039] The data collaboration mode for the preferred embodiment will now be described in more detail. As noted above, users in the data collaboration mode are passed through the central station SW to the host computer HC and conduct individual data sessions with the host computer HC. However, if the users seeking to take part in the data collaboration mode are currently in a voice mode conference, the users must be switched into data mode before data collaboration can occur. This is accomplished by the user computer associated with the currently designated lead speaker sending a DTMF conference control signal via its respective user station to the other users in the conference to switch to data mode after a designated time, e.g., ten seconds. The user stations of the other users in the conference detect the conference control signal and are switched from voice mode to data mode at the designated time.

[0040] Once the data collaboration mode is entered and the whiteboard session is activated in the host computer HC, the designated lead speaker may freely draw upon a virtual whiteboard area using an input device such as a mouse connected to the user computer of the lead speaker. The lead speaker's annotations to the whiteboard area are received by the host computer HC and then broadcast as annotation commands to the other users in the conference. User annotations may be overlaid upon data objects such as slides. The lead speaker may elect to allow inputs from other conferees, in which case the annotation input from each additional annotator is assigned a specific color and broadcast to the users in the conference, including the lead speaker. When the whiteboard annotation session is over, the lead speaker sends either a conference control signal to deactivate the whiteboard session while keeping the conference in data mode or a signal to return the conference to voice mode.

[0041] In the preferred embodiment, a protocol known as Remote Image Protocol (RIP) is used to control manipulation of the data objects which are stored at each of the user computers (SC1-SCN) in a conference. A family of RIP commands coded as DTMF conference control signals may be transmitted, for example, by the lead speaker to other conference attendees during a voice mode conference to coordinate display of selected data objects at each of the attendees respective user computer (SC1-SCN). In a data mode scenario, such as in a whiteboard session, the annotation commands may be RIP commands coded as a series of escape (ESC) sequences. A description of the RIP protocol can be found in “RIP Aint User's Guide,” Telegrafix Communications Inc. (September, 1993, Version 1.54), the contents of which are incorporated herein by reference.

[0042] Referring again to FIG. 1 to provide more details, the user lines (SL1-SLN) connect to the central station SW via a telephone network NW and terminate on a plurality of line ports (LP1-LPN). The central station SW includes a plurality of computer ports (CP1-CPN) which are coupled to host computer HC via a plurality of host modems (HM1-HMN). A plurality of through ports (TP1-TPN) on the central station SW are available for accessing other services and external networks described further herein. The central station SW provides connections between line ports (LP1-LPN) and computer ports (CP1-CPN) and between line ports (LP1-LPN) and through ports (TP1-TPN). A switch matrix SMX in the central station SW provides interconnections among multiple line ports (LP1-LPN) to form conferences. Operation of the switch matrix SMX will be described further below. The central station SW includes a controller CL for controlling conference and switching actions in the central station SW. The controller CL communicates with the host computer HC through control/data lines HCL. The central station SW also includes a revenue generation module RG which records billing information for all conference and switching actions provided through the central station SW.

[0043] At user site SS1, the user line SL1 connects to a user station SU1. The user station SU1 includes a microphone interface M1 and speaker interface S1 for telephone access, an external interface EXT1, and an auxiliary port AUX1 coupled to an auxiliary device AD1. The user station SU1 includes a data port ML1 which connects to a user computer SC1 via a user modem SM1. The user station SU1 is controlled through control lines CL1 by the user computer SC1 running a TSR (terminate and stay resident) program (not shown).

[0044] The auxiliary device AD1 connected to the auxiliary port AUX1 can be, for example, a typical audio recording machine. Such an audio recording machine can be used to record the voice communications signals and DTMF conference control signals received over user line SL1 during the voice mode of a conference. This recording arrangement would allow a user to replay and review a complete conference presentation in conjunction with the locally stored data objects from the recorded conference.

[0045] The user computer SC1 can be any standard personal computer or workstation having a display screen CS1 and associated memory and peripheral circuitry. The host computer HC is any standard personal computer workstation having a memory MEM and associated peripheral circuitry. The host computer HC is preferably configured as, but not limited to, an electronic bulletin board system. The host computer HC can also be multiple workstations connected via a local area network or other bus arrangement.

[0046] Each of the host modems (HM1-HMN) and the user modems (SM1-SMN) can be any standard modem which is capable of modulating digital signals into quasi-analog signals for transmission and demodulating quasi-analog signals into digital signals.

[0047] Shown in FIG. 2 is a schematic circuit diagram of user station SU1. The paths SL1aand SL1bof the user line SL1 from the network NW terminate at TIP and RING terminals of the user station SU1. A mode selector 29 comprising switches 20, 22, 24, 26, and 28 provides for selection between voice and data modes of operation. Data port paths ML1a, ML1bconnect to user line paths SL1a, SL1bthrough switches 22, 24. A loopback resistor RL for holding the connection to the user modem SM1 is provided across data port paths ML1aand ML1bthrough switch 20. The resistor RL is preferably 600 ohms to provide an expected termination towards the modem SM1. Switches 26, 28 provide connection between user line paths SL1a, SL1band telephony paths 72a, 72b. A voice interface circuit 40 is coupled to telephony paths 72a, 72bto provide telephone voice access to the user line SL1. The voice interface circuit 40 provides auxiliary ports AUX1a, AUX1bfor connecting auxiliary devices such as the voice recording machine noted above. Path 41bof the voice interface 40 connects to telephony path 72bthrough a switch 30 and an opto-isolator 32 connected in series. Operation of switch 30 is enabled through control line 18. The opto-isolator 32 detects off-hook, on-hook condition of the telephony circuit and passes detection information on control line 34.

[0048] A DTMF encoder 42 and a DTMF decoder 46 respectively transmit and receive the audible conference control signals. The DTMF encoder 42 receives tone control information on control line 38 and passes tone outputs on line 44 to voice interface 40. The DTMF decoder 46 is inductively coupled across telephony paths 72a, 72bvia transformer T1. The DTMF decoder 46 signals tone detection on line 58 and passes decoded tones on line 60.

[0049] A caller identification detector 48 is coupled to the telephony paths 72aand 72bto provide standard caller identification information on control line 62. A serial EEPROM 50 stores and maintains security information regarding the user station SU1. Information passes from the EEPROM 50 over path 64.

[0050] The switches (20-30) preferably are solid state switching devices, such as the OptoMOS® Solid State Switch LCA110 manufactured by CP Clare Corporation. The DTMF decoder 46 may be, for example, a MC145436 Dual Tone Multiple Frequency Receiver manufactured by Motorola. The DTMF encoder 42 may be, for example, a National Semiconductor TP5088 DTMF Generator. The voice interface 40 may be implemented, for example, with a Motorola MC34114 Telephone Speech Network with Dialer Interface. The caller identification detector 48 may be any one of several calling line identification devices such as a Motorola MC145447 Calling Line Identification Receiver.

[0051] Control and data signals pass between the user computer SC1 and a user station SU1 over control lines CL1 which connect through port connector 74. A manual switch 56 allows a control input 66 to be switched between lines 68 and 70. When manual switch 56 is in the position connecting control line 66 to control line 68, the control logic circuitry 52 in response to control signals from the user computer SC1 controls the user station SU1 for voice and data operation. When the manual switch 56 is in the position which connects control line 66 to line 70, the user station SU1 is in a pass-through mode whereby signals on external interface EXT1 may pass through connector 76 through to port connector 74. The pass-through mode allows the port connector 74 to function as a regular LPT parallel port so that a peripheral such as a printer or a satellite downlink receiver may be connected to the user computer.

[0052] In the default condition, the mode selector 29 operates in the data mode position. A low signal on control line 12 causes switches 20, 26, and 28 of selector 29 to open, thereby disconnecting the telephony paths 72a, 72bfrom the user line paths SL1a, SL1band disconnecting the load resistor RL across the data port paths ML1a, ML1b. The low signal on control line 12 is inverted by an inverter 14 to provide a high signal on control line 16. The high signal on control line 16 causes switches 22, 24 to operate, thereby connecting the data port paths ML1a, ML1bto the user loop paths SL1a, SL1b. Thus, the user modem SM1 (FIG. 1) is connected to the user line SL1 for data mode operation.

[0053] A high signal on control line 12 causes the selector 29 to switch to the voice mode., The high signal on control line 12 causes switches 20, 26 and 28 to operate. The operation of switches 26, 28 connects user line paths SL1a, SL1bto telephony paths 72a, 72b. To operate in the voice mode, control line 18 must also be high to operate switch 30. Thus, telephony access is afforded through the voice interface 40 coupled to the telephony paths 72a, 72b.

[0054] The operation of switch 20 due to a high signal on control line 12 completes the path between data port path ML1band load resistor RL. The high signal on control line 12 is inverted by inverter 14 to produce a low signal on control line 16. A low signal on control line 16 causes switches 22, 24 to open, thereby disconnecting the data port paths ML1a, ML1bfrom the user line paths SL1a, SL1b. Thus, the user modem SM1 (FIG. 1) is disconnected from the user line SL1 and is placed in a loopback state. The load resistor RL across the data port ML1 causes the data port ML1 to appear to be in an off-hook state to a user modem SM1 having on-hook/off-hook detection. In addition, in order to keep the user modem SM1 from timing out due to loss of received carrier, the user computer SC1 sends to the user modem SM1 a disable carrier timeout command, e.g., the AT command ATS10=255.

[0055] During the voice mode, an off-hook condition is detected by opto-isolator 32 when current flows through the LED portion of opto-isolator 32 which is coupled to telephony path 72bthrough switch 30. The photo-darlington portion of the opto-isolator 32 pulls the output line 34 low when current is detected.

[0056] Shown in FIG. 3 is a schematic block diagram of the central station SW. The switch matrix SMX comprises a plurality of horizontal and vertical rails (L1-LN) interconnected through a number of crosspoint switches. The minimum number of crosspoint switches required for interconnection is determined according to the integer value N*(N−1)/2, where N is the number of line ports. In a preferred embodiment, the number of line ports is 16; however, this is noted by way of illustration and not as a limitation of the invention. Each of the line ports (LP1-LPN) is connected to the switch matrix SMX through a switching arrangement as shown in FIG. 4. The TIP (LP1a) and RING (LP1b) of a line port LP1 from the network NW (FIG. 1) are connected through a pair of switches (108a, 108b) and across an isolation transformer T2 which couples AC signals between the line port LP1 and rail L1 of the switch matrix SMX. The rail L1 connects to other rails (L2-LN) through the crosspoint switches (S12-S1N).

[0057] Referring again to FIG. 3, the computer ports (CP1-CPN) have paths (CP1a, CP1b-CPNa, CPNb) which connect to respective line ports (LP1-LPN) through switches (102-1-102-N). A resistor R1 and switch 104-1 in series across computer port paths (CP1a, CP1b) provides a loopback termination to hold a connection to the computer modem HM1 in the same manner as described above for the user modems. This loopback termination is repeated on each of the other computer ports (CP2-CPN).

[0058] A plurality of DTMF encoder/decoder circuits (TG1-TGN) are coupled to respective rails (L1-LN) for transmitting and receiving audible conference control signals to and from respective user stations (SU1-SUN) (FIG. 1). The through ports (TP1-TPN) are connected to respective line ports (LP1-LPN) through switches (106-1-106-N). All of the switches and DTMF encoder/decoder circuits in the central station SW operate in response to control signals from the controller CL. The switches preferably are solid state switching devices, such as the OptoMOS® Solid State Switch LCA110 manufactured by CP Clare Corporation.

[0059] The central station includes a subconferencing circuit SCF which passes attenuated voice and DTMF signals from a main conference to a subconference. The subconference comprises a subset of the main conference attendees who may drop into the subconference to conduct a private conversation while still receiving the main conference voice and DTMF signals. In the subconference circuit SCF, the through ports (TP1-TPN) are connected to a multiplexer MX. The through port associated with the main conference is selected via the multiplexer MX and fed into amplifier AP which attenuates the main conference signal by an appropriate amount, preferably 20 to 30%. The attenuated output is switched via a demultiplexer DX to the through port associated with the subconference to be mixed with the subconference voice signals. In subconferencing operation, the appropriate switches (106-1 to 106-N) are operated to connect the main and subconference through ports to the appropriate rails (L1-LN).

[0060] Virtual Conference User Interface

[0061] The user interface of the conferencing system of the present invention will now be described. The virtual visual conferencing aspect supports functions that one would find in a real world conference, including: reception area, information folders, conference room, overhead projector and screen, speakers, and attendees.

[0062] Initially a user establishes a data mode session with the host computer HC and selects a conference room option from a conference menu provided by the host computer HC. The user's screen then displays a representation of a conference reception area as shown in FIG. 5. A conference reception area screen 200 includes several functional areas or icons which are activated via a mouse or keyboard selection. A receptionist icon 202 provides a help function for new users of the conference interface and a reservation function for reserving conference rooms for later conferences. In addition, the receptionist icon 202 provides an announcement/messaging function for announcing arrivals and departures during conferences and for informing attendees of incoming messages. A photocopier icon 204 provides a file transfer function whereby conference presenters upload files containing data objects which are then made available for conference attendees to download through activation of a file folder icon 206. The downloaded data objects, such as slides, are then available for simultaneous viewing during subsequent voice conferences.

[0063] A payphone icon 208 provides a telephone function for allowing a user to switch to voice mode and dial an outside line either via a through port (TP1-TPN) or via an unused line port (LP1-LPN) on the central station SW. A table icon 210 provides a periodicals function whereby a conference attendee waiting for a conference to begin can access an information source, such as an electronic newspaper. A conference room doors icon 212 provides access to a conference room area represented by a conference room screen 220 as shown in FIG. 6. Other functional icons may include a conference schedule icon to provide conference information such as topic, speakers, room number, scheduled date and time, and meeting description.

[0064] Referring again to FIG. 5, to reserve a specific conference, the user selects the receptionist icon 202 via a mouse or keyboard input device. The user then enters relevant information regarding the conference, such as conference date and time, topics, approved speakers, and attendees. The user then receives a conference room number which serves to confirm the reservation.

[0065] Between the time the reservation has been made and some period before the conference is due to begin, e.g., twenty minutes, all conference presenters, including the lead speaker, may submit any slides, graphs, prepared text, or other data objects to be used in the scheduled conference. To submit such data objects, a presenter at the conference reception area screen 200 selects the photocopier icon 204. The presenter enters appropriate conference identification information and then performs a data mode file transfer from the respective user computer SC1 to the host computer HC. The data objects are then combined with any other data objects submitted for the same conference and the combined data objects are represented by the folder icon 206 in the reception area screen 200. Each modification made by a presenter to previously submitted material is indicated by a revision letter on the folder icon 206. When the lead speaker and all other approved presenters have “signed off” on their respective contributions, the folder icon is marked “FINAL”, which signifies to conference attendees that they are retrieving the correct and complete version of the data objects for the conference.

[0066] At conference time, the lead speaker selects the conference room doors icon 212 on the reception area screen 200 and is then presented with the conference room screen 220 (FIG. 6). The lead speaker and the attendees are each presented with conference information on their associated computer screens (CS1-CSN), including a visual depiction of the conference room and function buttons for activating certain overhead and voice/data functions. In the visual depiction of the conference room, the lead speaker and attendees are represented at individual chairs 222 positioned around a conference table 224. The visual depiction of the conference room will automatically adjust to accommodate as many attendees as are present. An overhead projector screen area 226 is zoomed in when the presentation begins to display the stored data objects in a sequence selected under the control of the lead speaker. A conference room doors icon 228 provides access to the reception area screen 200 and allows a conference attendee to leave the conference. A telephone icon 230 provides a telephone function for allowing voice conference attendees to “patch-in” an outside line either via a through port (TP1-TPN) or via an unused line port (LP1-LPN) on the central station SW.

[0067] When the lead speaker elects to begin a presentation, an overhead view screen 240 is displayed on computer screen CS1 as shown in FIG. 7. The overhead view screen 240 includes a conference display area S and a conference control area CA. In the conference display area S, locally stored data objects such as slides are presented. The conference control area CA includes software-controlled conference function buttons, some of which are available only to the lead speaker. The conference function buttons can include, for example, sequential slide movement buttons A and B (forward and reverse respectively), random slide access buttons (1−N), refresh slide button G, go digital directive button C, go voice directive button D, leave conference button F, mute control button I, dumb control button J, lead speaker selection button H, and whiteboard annotation button E. Other conference functions can include gain control, whisper sub-conferencing, and a kick out function. Examples of these conference functions are described in more detail herein.

[0068] Conference Control Signals and Functions

[0069] The conference control signals corresponding to conference commands comprise a series of opcodes implemented by DTMF tones in a format shown in FIG. 8. The opcode format comprises a preamble field 252, an opcode class field 254, an opcode number field 256, a routing address field 258, and a user address field 260. The preamble field 252 consists of a single DTMF hexD tone. The opcode class field 254 comprises a single DTMF tone to indicate up to 15 classes of opcodes. The opcode number field 256 comprises two DTMF tones to indicate up to 225 opcodes per class. The routing address field 258 and the user address field 260 are optional, depending upon the opcode used. The routing address field 258 can be from zero to eight tones in length and is used to indicate a particular central station in a network of central stations of the present invention as described further below. The user address field 260 can be from zero to eight tones in length and is used to indicate a particular line port of the central station indicated in the routing address field 258.

[0070] The opcode tone duration is defined on a conference by conference basis. Initially, the system attempts a tone duration of 40 ms and increases the duration in increments of 10 ms, up to a duration of 100 ms, until a duration is reached that satisfies all conference participants.

[0071] The conference control opcodes are illustrated by, but not limited to, the protocol records in the following table:


Command Name	Class	Origin	Destination	Opcode Format

use_payphone	telephony	user	Host computer and central
			station
hang_up	telephony	user	Central station	D001
digital_to_voice	mode	user	Host computer and Central
			station
end_conference	mode	speaker	Central station and user	D101
			stations
go_digital_self	mode	user n	Central station	D111n
go_digital_all	mode	speaker	Central station and user	D110
			stations
go_digital n	mode	speaker	User station n	D112n
kick_out n	mode	speaker	User station n	D121n
leave_conference_self	mode	user n	Central station	D131n
mute_user n	environment	speaker	User station n	D211n
dumb_user n	environment	speaker	User station n	D221n
hear_user n	environment	speaker	User station n	D222n
speak_user n	environment	speaker	User station n	D212n
show_slide x	environment	speaker	User stations	D231x
private_show_slide xn	environment	speaker	User station n	D232xn
refresh	environment	speaker	User stations	D240
turn_floor n	environment	speaker	User station of new speaker n	D251n
whisper n	environment	user	Central station and user	D261n
			station n
return_whisper n	environment	user n	Central station	D262n
whiteboard	environment	speaker	Central station and user	D270
			station(s)
turn_floor external rn	routing	speaker	User station n on central	D311rrn
			station r
go_digital rn	routing	speaker	User station n on central	D321rrn
			station r
kick_out rn	routing	speaker	User station n on central	D331rrn
			station r
mute_user rn	routing	speaker	User station n on central	D341rrn
			station r
dumb_user rn	routing	speaker	User station n on central	D351rrn
			station r
hear_user rn	routing	speaker	User station n on central	D352rrn
			station r
speak_user rn	routing	speaker	User station n on central	D342rrn
			station r
whisper rn	routing	user	User station n on central	D371rrn
			station r

[0072] Control Procedures

[0073] The procedure to activate an individual user station SU1 (FIG. 1) is illustrated by the flow chart shown in FIG. 9. The procedure begins at step 300 when the TSR program is loaded into the user computer. The TSR polls the parallel ports of the user computer at step 302, looking for valid information from EEPROM 50 (FIG. 2). If the TSR does not recognize a valid user station at step 304 based upon the information stored in the EEPROM 50, the TSR aborts at step 306. If a valid user station is recognized at 304, the TSR enables the user station for data mode operation at step 308 by causing control line 12 (FIG. 2) to go low. Once the user station is in the data mode, the user can then dial the host computer at step 310. Having connected to the host computer, the user at step 312 selects an application to run on the host computer, e.g., conferencing and voice services.

[0074] In general, the TSR monitors the user modem COM port for data signals which are represented by simple ASCII character strings. The TSR will filter this data from the user and act upon it immediately. The TSR also monitors the user parallel port for signals from the user station.

[0075] A description of several of the conference control procedures possible within a conference will now be provided. The procedure for placing all the conference attendees into voice mode is illustrated by FIG. 10. Starting at step 320, the users are connected in a conference and are in data mode. The lead speaker may select a go_voice icon at step 322 to initiate the procedure. At step 324, the host computer broadcasts the digital go_voice command to all the conference attendees. The TSR at each attendee user computer filters the command at 326 and instructs the user station to switch to voice mode at step 328. At the same step, the host computer communicates with the controller CL of the central station SW (FIG. 1) to interconnect the conference attendees in voice mode.

[0076] The procedure for coordinating the simultaneous viewing of data objects, or slides, is illustrated in FIG. 11. The procedure begins at step 340 with the users connected in a conference. The lead speaker selects the slide icon at step 342. The TSR determines whether the conference is in data or voice mode at step 344. If the conference is in data mode, the slide command from the lead speaker is broadcast to all conference attendees by the host computer at step 346. If the conference is in voice mode, the speaker's TSR filters the slide command at step 348 and converts the command to a show_slide_n opcode at step 350. The TSR instructs the user station to send the opcode via the DTMF encoder 42 (FIG. 2) at step 352. The opcode tone sequence is broadcast to attendees through the central station SW (FIGS. 1, 3) at step 354. At each attendee user station, the DTMF decoder 46 (FIG. 2) detects the opcode tone sequence at step 356. At step 358, the TSR of each attendee instructs the user computer to display slide n. Thus, although data mode actions between users and the host computer follow a client/server model, in voice mode, many peer-to-peer events take place between users for efficiency and simplicity.

[0077] The procedure for conducting a whiteboard annotation session is illustrated in the flow chart of FIG. 12. Beginning at step 370 with the users connected in a conference, the speaker may select the whiteboard icon at step 372. The TSR determines whether the conference is in voice or data mode at step 374. If the conference is in data mode, the host computer broadcasts the speaker's annotation commands to all attendees at step 376. If the conference is in voice mode, the conference must be switched over to data mode before annotation can begin. The TSR of the speaker filters the whiteboard command at step 378 and converts the command to a whiteboard opcode at step 380. At step 382 the TSR instructs the user station to send the opcode via the DTMF encoder. The opcode tone sequence is broadcast to the conference attendees through the central station SW (FIG. 3) at step 384. The opcode tone sequence is detected at each user station at step 386. The TSR of each attendee instructs the respective user station to switch to data mode at step 388. At this point, the central station SW also switches to data mode. Annotations from the speaker are then broadcast to the other attendees via the host computer at step 376. In a case where one or more attendees are also given the opportunity to annotate, the annotations of these attendees are broadcast in the same manner.

[0078] The procedure for allowing an attendee to leave a conference is illustrated in the flow chart of FIG. 13. Beginning at step 400 with the users connected in a conference, an attendee may select the leave_conference icon at step 402. The attendee's TSR determines at step 404 whether the conference is in voice or data mode. If the conference is in data mode, the attendee is returned to the reception area screen 200 (FIG. 5) at step 406. If the conference is in voice mode, the attendee's TSR filters the leave_conference command at step 408 and converts the command to a leave_conference_self opcode at step 410. The TSR instructs the user station to send the opcode via the DTMF encoder 42 (FIG. 2) at step 412. The opcode tone sequence is detected by a DTMF decoder TGi (FIG. 3) at the central station SW at step 414. From step 418, the attendee's TSR switches the user station to data mode while simultaneously the attendee's line port on the central station SW is switched to data mode. At the data mode, the attendee is returned to the reception area screen at step 406.

[0079] The teleconferencing system of the present invention can also be configured in a network arrangement as shown in FIG. 14. Two central stations SWa, SWb are shown connected through a frame relay network FR. Two central stations are shown for purposes of example and not as a limitation, i.e., any number of central stations may be interconnected through network FR. The frame relay network FR includes connections to Internet network IN and a Ku-band uplink server UL. The uplink server UL in turn connects via a satellite network KN to a Ku-band receiver KR located at user site SSN and connected to user station SUN.

[0080] The network arrangement provides for sharing of communications resources among cooperating BBS systems. Thus, for example, a conference can be conducted among user sites connected across such a network arrangement. A second example is where a first BBS system, such as host computer HCa, has several voice recognition servers. Users associated with a second BBS system, host computer HCb, can then be routed via the frame relay network FR to central station SWa and host computer HCa in order to access a voice recognition application.

[0081] A third example is where the uplink server UL connected to the frame relay network FR performs high-speed information downloading to receiver KR. Many telecommunications service providers want to be able to deliver bi-directional data at high bandwidths for multimedia applications. However, in most applications it is more important to deliver large amounts of data to the end user (downloaded), while very little data needs to be sent back to central file servers from end users (uploaded). With the satellite networking arrangement of the present invention shown in FIG. 14, users may receive high-speed file downloads or real-motion video via satellite rather than over narrow bandwidth telephone lines. In operation, the user at user site SSN may request a large file download from the local BBS, host computer HCa. Host computer HCa queues up the file from its internal file storage, or from the Internet IN. The requested file is then routed over the frame relay network FR to the uplink server UL with the user address information. The file is transmitted over the Ku band satellite network KN with user addressing and encryption and received by receiver KR. The received file is then routed through user station SUN to the local digital computer at user site SSN. Billing information for the file transfer is forwarded along with file transfer confirmation to the host computer HCa after a checksum verification indicates that the file was received error-free.

[0082] A fourth example combines satellite technology for real-time multimedia information delivery with user/host computer communications to form a complete multimedia system. Referring again to FIG. 14, a user at user site SSN may request to see, for example, a one minute product presentation. One minute of compressed video occupies approximately 12 Mb. The host computer HCa transmits 2 Mb frames every ten seconds through the frame relay network FR and the satellite network KN to the user receiver KR. Each new frame can be loaded directly to the on-board memory of the user digital computer at user site SSN for audio/video playback. Real-time user feedback is communicated between the user and the host computer HCa via the modem connection.

[0083] Equivalents

[0084] While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

What is claimed is:

1. A method of conferencing, comprising the, steps of:

retrieving by at least two storage devices, a set of data objects from a host computer;

processing the set of data objects in each storage device, each storage device located at a respective user site and having a storage system for data object storage, wherein the retrieving step and the processing step each occur prior to a conference;

operationally attaching a display system to at least one storage device;

operationally connecting the user site into the conference;

selecting a data object from the set of data objects and transmitting a control signal to indicate the selected data object; and

receiving the control signal and displaying the selected data object on the display system in response to the control signal.

2. The method of claim 1 wherein the set of data objects comprise image data.

3. The method of claim 1 further comprising the step of, at a respective user site, transmitting and receiving a signal to and from other user sites.

4. The method of claim 3 further comprising the steps, at a user site, of: recording the signals to produce recorded signals;

playing the recorded signals; and

displaying selected data objects on a display system in response to the recorded signals.

5. Apparatus for conferencing, comprising:

a host computer;

at least two storage devices, each said storage device storing a set of data objects retrieved from the host computer for a conference, each said storage device located at a respective user site, and each said storage device having a storage system for data object storage;

a display system, operationally attached to at least one storage device;

a network for interconnecting the user site into the conference; wherein the user site includes:

(i) a system for selecting a data object from the set of data objects;

(ii) a control signal generator for transmitting a control signal indicating the selected data object to other user sites;

(iii) a receiver for receiving the control signal, the selected data object being displayed on the display system in response to the control signal.

6. The apparatus of claim 5 wherein the user site further comprises a circuit for transmitting and receiving communications signals to and from the other user sites.

7. The apparatus of claim 5 wherein the data objects comprise image data.

8. A method of conferencing over an Internet network, comprising the steps of:

retrieving by at least two storage devices, a set of data objects from a host computer;

processing the set of data objects in each storage device, each storage device located at a respective user site and having a storage system for data object storage, wherein the processing step occurs prior to a conference over the Internet network;

operationally attaching a display system to at least one storage device;

operationally connecting the user site into the conference;

selecting a data object from the set of data objects and transmitting a conference control signal to other user sites to indicate the selected data object; and

receiving the conference control signal and displaying the selected data object on the display system in response to the conference control signal.

9. The method of claim 8 wherein the data objects comprise image data.

10. The method of claim 8 further comprising the step of at the user site, transmitting and receiving voice communications signals to and from the other user sites.

11. The method of claim 1, operationally connecting the user site into the conference via a wide area network, wherein the wide area network comprises an Internet network.

12. The method of claim 1, operationally connecting the user site into the conference via a wide area network, wherein the wide area network comprises a satellite network.

13. The method of claim 1, wherein the data objects comprise executable objects.

14. The method of claim 1, wherein the data objects comprise video objects.

15. The method of claim 1, wherein the data objects comprise multimedia objects.

16. The method of claim 1, wherein the data objects comprise image objects.

17. The method of claim 1, wherein the receiving step further comprises;

storing the received control signals;

subsequently retrieving the stored control signals; and

displaying the selected data objects in response to the retrieved control signals.

18. The method of claim 17, wherein the retrieving and displaying steps occur non-simultaneously at each user site.

19. The apparatus of claim 5, wherein the network comprises an Internet network.

20. The apparatus of claim 5, wherein the network comprises a satellite network.

21. The method of claim 3, wherein the signal is a voice communications signal.

22. The method of claim 1, wherein the set of data objects includes common data objects and private data objects.

23. The apparatus of claim 5, wherein the set of data objects includes common data objects and private data objects.

24. The method of claim 8, wherein the set of data objects includes common data objects and private data objects.

25. A method of conferencing, comprising the steps of:

processing a set of data objects in a storage device, the storage device located at a respective user site and having a storage system for data object storage, wherein the processing step occurs prior to a conference;

operationally attaching a display system to at least one storage device;

operationally connecting the user site into the conference;

selecting a data object from the set of data objects and transmitting a control signal in-band with voice communications signals to indicate the selected data object; and

receiving the control signal and displaying the selected data object on the display system in response to the control signal.

26. The method of claim 25 wherein the data objects comprise image data.

27. The method of claim 25 further comprising the step of, at a respective user site, transmitting and receiving a signal to and from other user sites.

28. The method of claim 27 further comprising the steps, at a user site, of: recording the signals to produce recorded signals;

playing the recorded signals; and

displaying selected data objects on a display system in response to the recorded signals.

29. Apparatus for conferencing, comprising:

at least one storage device, said storage device storing a set of data objects for a conference and located at a user site, and having a storage system for data object storage;

a display system, operationally attached to at least one storage device;

a network for interconnecting the user site into the conference; wherein the user site includes:

(i) a system for selecting a data object from the set of data objects;

(ii) a control signal generator for transmitting a control signal indicating the selected data object to other user sites, wherein the control signal is adapted to be transmitted in-band with communications signals;

(iii) a receiver for receiving the control signal, the selected data object being displayed on the display system in response to the control signal.

Patent Citations (25)

Patent	Date	Inventor	Cited By
US4291200(A)	1981-09-01	Smith	Applicant
US4524244(A)	1985-06-01	Faggin et al.	Applicant
US4546212(A)	1985-10-01	Crowder, Sr.	Applicant
US4578537(A)	1986-03-01	Faggin et al.	Applicant
US4656654(A)	1987-04-01	Dumas	Examiner
US4719617(A)	1988-01-01	Yanosy, Jr. et al.	Applicant
US4845704(A)	1989-07-01	Georgiou et al.	Applicant
US4953159(A)	1990-08-01	Hayden et al.	Applicant
US4987586(A)	1991-01-01	Gross et al.	Applicant
US5065425(A)	1991-11-01	Lecomte et al.	Applicant
US5136586(A)	1992-08-01	Greenblatt	Applicant
US5142565(A)	1992-08-01	Ruddle	Applicant
US5151972(A)	1992-09-01	Lorenz et al.	Applicant
US5239580(A)	1993-08-01	Bruno et al.	Applicant
US5365577(A)	1994-11-01	Davis et al.	Applicant
US5440624(A)	1995-08-01	Schoof, II	Applicant
US5444476(A)	1995-08-01	Conway	Applicant
US5483588(A)	1996-01-01	Eaton et al.	Applicant
US5625679(A)	1997-04-01	Gutzmer	Applicant
US5844979(A)	1998-12-01	Raniere et al.	Applicant
US6061440(A)	2000-05-01	Delaney et al.	Applicant
US6373936(B1)	2002-04-01	Raniere et al.	Examiner
DE4229359	1994-01-01		Applicant
FR41902	1981-12-01		Applicant
WO94/26056	1994-11-01		Applicant

Non-Patent Literature (10)

Reinhardt, A., “Doing It All on One Line”, Byte, pp. 145-148 Jan. Jan. 1995.Applicant
“VoiceView®: A New Protocol for Integrated Voice & Data Applications,” brochure from Radish Communications System, Inc., pp. 1-11, 1994.Applicant
“Intel Technology Briefing—The Communicating PC Another Communications Breakthrough,” sales brochure from Intel Corp. pp. 1-4, 1994.Applicant
Stockbridge, C., “The Evolution of a Multimode Multilocation Audioconferencing System,” Electro Conf. Record, Section 29/3, Vol 4, Apr. 1979.Applicant
Higginbotham, A.F. et al., “Teleconference Systems,” IBM Technical Disclosure Bulletin, vol. 22, No. 9, pp. 3923-3925, Feb. 1980.Applicant
Tanigawa, H. et al., “Multipoint Communication Control for Document-Oriented Teleconferencing,” 1988, Inter'l. Zurich Seminar on Digital Communications—Session A3, pp. 29-35, Mar. 1988.Applicant
Kazuhisa Watanabe, et al., “Audio and Visually Augmented Teleconferencing,” Proceedings of the IEEE, vol. 73, No. 4, pp. 656-670, Apr. 1985.Applicant
Pate, L.R., “Trends in Multimedia Applications and the Network Models to Support Them,” GLOBECOM 90 IEEE Global Telecommunications Conference & Exhibition—Session 308.6, vol. 1, pp. 317-321, Dec. 1990.Applicant
Horn, D.N., and Sharma, A., “A Versatile Audio Bridge for Multimedia Conferencing,” Supercomm/ICC 94, pp. 1754-1762, May 1994.Applicant
Nakamura, K., et al., “Personal Multimedia Teleconferencing Terminal,” IEEE International Conference on Communications—Session 211.2.1, vol. 1, pp. 123-127, Apr. 1990.Applicant

Abstract