Apparatus and method for providing multimedia conferencing services with selective information services

[0002] This non-provisional application claims the benefit of U.S. provisional application No. 60/112,740 entitled “Multimedia Conferencing Services and Collaboration Services Integrated with Information Services” filed on Dec. 16, 1998 and is a continuation application of application Ser. No. 09/370,827 filed Aug. 9, 1999 U.S. Pat. No. 6,600,725. The Applicant of the provisional application, parent and this continuation application is Radhika R. Roy.

[0003] 1. Field of Invention

[0004] The present invention is directed to an apparatus and method for providing multimedia conferencing services with selective information services.

[0005] 2. Description of Related Art

[0006] Multimedia conferencing over communication networks to a plurality of parties is generally known in the art. Multimedia conferencing in packet-switched networks such as asynchronous transfer mode (ATM), frame relay and the like is well known, as is multimedia conferencing in circuit-switched networks such as ISDN, PSTN and the like. However, in a typical multi-party multimedia conferencing service, the same multimedia signal, including the video, audio and data portions, is provided to all parties of the multimedia conference. There is no ability to selectively provide multiple parties to a multimedia conference with information services during the multimedia conference. Thus, new technology is needed to provide a means by which each party to a multi-party multimedia conference may selectively receive information services during the multimedia conference.

[0007] The invention provides an apparatus and method that provides multimedia conferencing service to multiple parties where each party can request information services during the conference. Different information services may be provided to each party based on the preferences of the respective party. Using the invention, when an operator of a source user device enters the destination address for another user device with which communication is sought, a multimedia bridge receives the signals from the source user device. The signals include information of the identity of the source user device and the destination user devices.

[0008] Based on the information received, the multimedia bridge sends start-up requests to the selected destination user devices to be involved in the multimedia conference. Once the start-up requests are accepted, the multimedia bridge enables the multimedia conferencing service to provide communication between the user devices.

[0009] Once the multimedia bridge enables the multimedia conference, the multimedia bridge determines which user devices request information services. The multimedia bridge then provides continuous or non-continuous information services to the user devices requesting information services.

[0010] These and other features and advantages will be described in or are apparent from the following detailed description of the preferred embodiments.

[0011] The preferred embodiments of the invention will be described with reference to the following figures, wherein like numerals designate like elements, and wherein:

[0012] FIG. 1 is an exemplary block diagram of a multimedia conferencing system in which the invention operates;

[0013] FIG. 2 is a block diagram of the multimedia bridge of FIG. 1;

[0014] FIG. 3 is a exemplary data structure of the database of FIG. 1;

[0015] FIG. 4 is a flowchart of the operation of the invention according to an exemplary embodiment; and

[0016] FIG. 5 is a flowchart of an exemplary operation for selectively providing information services to user devices of the invention.

[0017] FIG. 1 is an exemplary block diagram of a system in which the invention operates. The system includes a plurality of user devices 100, 106, 108 and 112 in communication with at least one network 120 through Local Access Providers (LAPs) 102, 104 and 110. The user devices 100, 106, 108, and 112 communicate with one another through the network 120. The network 120 is further in communication with a multimedia bridge 114.

[0018] The multimedia bridge 114 is in communication with a database 116 and/or any number of information services providers 122, 124 and 126. The database 116 is any type of database capable of providing information services to the multimedia bridge 114. For example, the database 116 may be a device for storing information to be repeatedly accessed by multiple user devices, such as business advertisements and the like. The information services providers 122, 124 and 126 may provide real-time information at the request of the user devices 100, 106, 108, and 112, such as stock updates, sports scores, traffic reports, weather reports, and the like. The information services providers 122, 124 and 126 may be directly accessible to the multimedia bridge 114 as shown in FIG. 1, or accessible by the multimedia bridge 114 through the network 120.

[0019] The multimedia bridge 114 modifies the multimedia signals to be sent to the user devices 100, 106, 108 and 112 to include information services received from the database 116 and/or the information services providers 122, 124 and 126, as requested by the respective user devices 100, 106, 108 and 112. The multimedia bridge 114 sends the modified multimedia signal, including the information services, to the user devices 100, 106, 108, 112 via the network 120.

[0020] In multimedia conferencing, many parties or users participate in the multimedia conference call using their respective user devices 100, 106, 108 and 112. A multimedia conference call may last from a few minutes to many hours. During the multimedia conference, one of the users may be speaking while the other users may be listening. The users who are in a listening mode might choose to see some other information while listening to the speaker.

[0021] According to this invention, each user may select any information service that the user prefers for viewing while she/he is in a listening mode during the multimedia conference. The information service selected may be, for example, stock service provided by stock service provider 112, sports service provided by sports service provider 124, weather service provided by weather service provider 126, or other like services. Audio, video, and/or data is bridged by the multimedia bridge 114. All audio signals go to the multimedia bridge 114 for bridging, and the multimedia bridge 114 may detect which user is speaking through activity detection of the audio signal. Thus, the multimedia bridge 114 may determine which of the user devices 100, 106, 108, and 112 participating in the multimedia conference are in a listening mode and which user devices 100, 106, 108, and 112 are in a speaking mode.

[0022] In addition, each user device 100, 106, 108, and 112 provides the multimedia bridge 114 with their respective user's preferences for information service for use while the user is in a listening mode. This information about the information services selected by the users may be stored, for example, in a service profile of the database 116 or in a memory of the multimedia bridge 114.

[0023] Once the user devices 100, 106, 108, and 112 provide the users' selections for information services, the multimedia bridge 114 will instruct the corresponding information service entity (e.g., sports service provider 124, stock service provider 122, or weather service provider 126) to provide the desired service to each user in accordance with their selected preferences. Each information service provider 122, 124 and 126 is provided with necessary information to send data to the respective user devices 100, 106, 108, and 112 by the multimedia bridge 114.

[0024] These information services can be as simple as one or more displays that are produced in the user devices 100, 106, 108, and 112 based on information sent by the service provider. Thus, the displays may also be provided to the user devices 100, 106, 108, and 112 even when the user of the user device 100, 106, 108, and 112 is a speaker. For example, the time for switching between the listening mode to the speaking mode of a user in the multimedia conferencing can always be unpredictable and may vary from milliseconds to few minutes or even more. The delay for sending the information from the information service providers 122, 124 and 126 may be larger than the time for switching between modes. If a user becomes active after a few 10 s of milliseconds, the user may still receive the information from the information service provider for a few milliseconds even if the user changes between the listening mode to the speaking mode.

[0025] A user can send an indication to the controller 201 (shown in FIG. 2) to discontinue the service at any time. The multimedia bridge provides specific instructions to the information service providers 122, 124 and 126 to disable the service to the user device 100, 106, 108, and 112. The information sent by the information server 100, 106, 108, and 112 may consist of audio information, video information, and/or data. If a user becomes a speaker after remaining silent while receiving the information from information servers 122, 124 and/or 126, and if the information from the information servers 122, 124 and/or 126 is still being sent while the user is speaking, the user may disable the data channel of the information server.

[0026] If, for example, the operator of user device 100 wishes to communicate with destination user devices 106, 108, 112 in a multimedia conference, the operator need only activate user device 100 and send start-up signals to the multimedia bridge 114. The start-up signals may include identification information regarding the destination addresses, for example the telephone number, Universal Resource Locator (URL), TCP/IP address, and the like, of the user devices 106, 108, 112.

[0027] The multimedia bridge 114 then sends a start-up request to the destination user devices 106, 108, 112. The destination user devices 106, 108, 112 respond by sending a start-up signal to the multimedia bridge 114. The multimedia bridge 114 then establishes a communication connection between the user devices 100, 106, 108 and 112.

[0028] The user devices 100, 106, 108 and 112 may be any devices that allow for the transmission of signals over communications links. For example, the user devices 100, 106, 108 and 112 may be any devices capable of sending and receiving multimedia conferencing signals such as computers, video telephones, video conference apparatus, and the like.

[0029] The user devices 100, 106, 108 and 112 are in communication with the network 120 through LAPs 102, 104 and 110 over communications links. These communications links may be any type of connection that allows the transmission of information. Some examples include conventional telephone lines, fiber optic lines, direct serial connection, cellular telephone connections, satellite communication links and the like.

[0030] The user devices 100, 106, 108 and 112 send communication signals to the multimedia bridge 114 over the communications links through LAPs 102, 104 and 110. The LAPs 102, 104 and 110 are devices that provide an interface to the network 120. The LAPs 102, 104 and 110 may include Local Exchange Carriers (LECs), Internet Access Providers (IAPs), satellite base stations, and the like.

[0031] The communication signals are received by the multimedia bridge 114 and are routed in the network 120 to the destination user device. The routing may be performed by link (e.g. ATM) or network (e.g., IP) entities, for example, which are generally known in the art.

[0032] The network 120 may be a single network or a plurality of networks of the same or different types. For example, the network 120 may include a local telephone network in connection with a long distance network. Further, the network 120 may be a data network or a telecommunications network in connection with a data network.

[0033] The network 120 is further in communication with at least one multimedia bridge 114. The multimedia bridge 114 controls the communication between a source user device (i.e., the user device initiating the communication) and destination user devices as described below.

[0034] The operator of a user device, for example user device 100, sends start-up signals to the multimedia bridge 114, including the identity of the user device 100 and of the user devices with which communication is sought, for example, user devices 106, 108, 112. The multimedia bridge 114 receives the signals from the user device 100 and sends start-up requests to the necessary user devices 106, 108, 112 involved in the multimedia conference. If the user devices 106, 108, 112 accept the start-up request, the user devices 106, 108, 112 send start-up signals back to the multimedia bridge 114.

[0035] The multimedia bridge 114 then establishes the multimedia conference between the user devices 100, 106, 108, 112. A multimedia conferencing can be established in many ways. Some examples can be seen in U.S. patent application Ser. Nos. 10/153,689, 10/154,975 and 10/132,573 which are hereby incorporated by reference in their entirety. For purposes of the following description, it is assumed that all multimedia conference calls, whether point-to-point or point-to-multipoint, are routed through the multimedia bridge 114.

[0036] For example, if a user of the user device 100 decides to set up a conference call with user devices 106, 108, and 112, the user device 100 sends start-up signals, i.e., a multimedia conference request, to multimedia bridge 114 with the required multimedia conference information. The multimedia conference information may include the addresses of user devices 100, 106, 108, and 112 and corresponding network layer performance related parameters for the user devices 100, 106, 108, and 112. The multimedia bridge 114 then retrieves the profile information from the database 116 for each user device 100, 106, 108, and 112.

[0037] The multimedia bridge 114 then examines whether the request is valid and/or has proper authorization. If the multimedia request of the user device 100 is not valid and/or not authorized, the multimedia bridge 114 sends a rejection message to the user device 100. If the request is valid, the multimedia bridge 114 will send an invitation, i.e., start-up signals, to the user devices 106, 108, and 112.

[0038] The user devices 106, 108, and 112 examine all criteria for joining the multimedia conference call including the performance parameters. Each user device 106, 108, and 112 then communicates, i.e., sends start-up reply signals, to the multimedia bridge 114 and identifies the desired criteria, including the performance parameters, for their joining the multimedia conference.

[0039] The multimedia bridge 114 examines the start-up reply of each user device 106, 108, and 112, and sets the desired performance criteria and information services of each party to be in accordance with the information provided in the start-up replies from the user devices 106, 108, and 112. The multimedia bridge 114 then prepares itself to bridge this multimedia conference call in accordance with the performance criteria agreed upon by each user device 100, 106, 108, and 112. It should be noted that all of the user devices 100, 106, 108, and 112 may not have the same performance parameters and information services. For example, user 100 may subscribe to sports services provided by sports service provider 124, user device 106 may subscribe to stock services provided by stock service provider 122, and user device 112 may subscribe to weather services provided by weather service provider 126.

[0040] If needed, the multimedia bridge 114 may also store detailed information of the multimedia conference call in database 116 or in an associated memory for later use, such as for billing purposes. The multimedia bridge 114 may then send a reply back to each of the user devices 100, 106, 108, and 112 confirming that the multimedia conference call has been accepted. After obtaining the confirmation of the acceptance of the multimedia conference call, the user devices 100, 106, 108, and 112 may start to communicate with one another using audio, video, and/or data as desired. The multimedia conference continues until an end condition occurs.

[0041] It may happen that some of the user devices 106, 108, and 112 may not be willing to join the multimedia conference call. The respective user devices 106, 108, and 112 that declined to join the multimedia conference will not be connected by the multimedia bridge 114, and information to this effect may be transmitted to the user device 100.

[0042] Using the above architecture, the multimedia bridge 114 can selectively provide information services to user devices 100, 106, 108 and 112. Each user device 100, 106, 108 and 112 can independently choose to receive information services during a multimedia conference. The information services may be provided in a continuous manner or a non-continuous manner, for example, an update provided every five minutes.

[0043] Although the above description is directed to providing information services when a user device is in a listening mode, the information services may also be provided at all times during the multimedia conference. In this case, detection of which user device is the speaker is not necessary.

[0044] Further, the information services may be provided to the user devices 100, 106, 108 and 112 in any multimedia form, such as text, audio, video, graphics, or any combination of these. For example, the information services may be superimposed over a portion of a video display of the user device 100, 106, 108 or 112 (e.g., scrolling text across the top or bottom of the display). Alternatively, the information services may be provided in a banner separate from the multimedia conference video (e.g., a banner positioned at the top or bottom of the display). As another alternative, the multimedia video and the information services may be provided in a split screen format or a picture-in-picture format, with the information services provided as text, video, graphics, audio, or a combination of any of the these forms. The format in which the information services are provided may be determined by the service provider, the business providing the advertisements, hardware and software limitations, and/or user preferences.

[0045] As discussed above, the information services may be information stored in the database 116, such as a business advertisement, or real-time information provided by the service providers 122, 124, 126, such as stock updates, sports scores, traffic updates, weather updates, and the like. In the case of advertising information, the advertiser may agree to pay for all or a portion of the multimedia conference fees for each user device that requests that advertiser's information. On the other hand, in the case of real-time information, the user device may owe additional service fees to the entity providing the real-time information service. The entity may be the same entity that is providing the multimedia conference, or it may be an independent entity.

[0046] FIG. 2 is a block diagram of the multimedia bridge 114 of FIG. 1. The multimedia bridge includes a controller 201, a network interface 202, a database interface 203, media bridge 210, a bus 204, a memory 205, and service provider interfaces 206, 207 and 208. The controller 201 controls the operation of the multimedia bridge 114 and directs communication between itself, the memory 205, the media bridge 210 and the interfaces 202, 203, 206, 207 and 208. The memory 205 stores information regarding each user device's preferences for receiving information services during the multimedia conference. The media bridge 210 bridges the various media types (audio, video and data) among the user devices 100, 106, 108 and 112. The controller 201, the memory 205, the media bridge 210 and interfaces 202, 203, 206, 207 and 208 communicate with one another over the bus 204.

[0047] The network interface 202 provides a communication pathway between the controller 201 and the network 120. The database interface 203 provides a communication pathway between the controller 201 and the database 116. The service provider interfaces 206, 207, 208 provide communication pathways between the controller 201 and the real-time information services providers 122, 124, 126.

[0048] The operation of the multimedia bridge 114 will now be explained with reference to an exemplary operation of the invention. In this example, it is assumed that three user devices 100, 106 and 112 are involved in the multi-party multimedia conference. However, the example operation is also applicable when there are any number of user devices involved in a multimedia conference.

[0049] When an operator of the user device 100 wishes to communicate with the operator of the user devices 106, 112, for example, the operator of the user device 100 transmits a start-up message to the multimedia bridge 114 containing identification information identifying the source user device 100 and the destination user devices 106, 112. The controller 201 of the multimedia bridge 114 receives the start-up signal through the network interface 202.

[0050] The controller 201 then sends a start-up request to the user devices 106 and 112. The start-up request may be any graphic, video or audio prompt, for example, a pop-up menu, a pull-down menu, an audio request, and the like.

[0051] Once the user devices 106 and 112 accept the request, the controller 201 can commence the multimedia conference. The controller 201 can then prompt the user devices to choose whether to receive information services. Alternatively, the user devices 106 and 112 may include requests for information services when requesting or accepting the start-up signals.

[0052] The controller 201 then determines which user devices request to receive information services, whether the information services are to be provided continuously or non-continuously, and what information services each user device wants to receive. The controller 201 stores these user requests in the memory 205. Thus, the controller 201 can provide information services to the requesting user devices throughout the multimedia conference.

[0053] Once the multimedia conference is commenced, the controller 201 monitors the multimedia conference to determine when to provide information services to those user devices that only requested information services when they are not speaking. The controller 201 may detect which user device is speaking by comparing the decibel levels of the incoming audio streams from each user device 100, 106 and 112, or in any other manner generally known in the art.

[0054] The media bridge 210 can detect the speech, voice or audio activity whether a user is speaker or not. For example, if a user speaks, his/her audio signal energy will increase and the audio energy will decrease or remain at zero if the user does not speak. In this way, the media bridge 210 is able to detect whether a user is speaking or not. The media bridge 210 communicates this information to the controller 201 which may take necessary actions with respect to providing the appropriate information services.

[0055] In addition, the controller 201 monitors the multimedia conference to determine whether any user device wishes to change its preferences, such as switching from non-continuous information services to continuous, no information services to continuous, etc. A change in user preferences can be effectuated through the use of pull-down menus, pop-up menus, commands entered through a user interface, audio commands, and the like.

[0056] FIG. 3 is an example of a data structure of the database 116 according to the first embodiment. As shown in FIG. 3, the data structure contains a listing of advertisers and information services 300, what portion or amount an advertiser pays for the user device's multimedia conference costs 301, and what fee is associated with each information service 302. Thus, the multimedia bridge 114 may access the database 116 and provide the user devices 100, 106, 108, 112 with a menu of options for listening to advertisements or viewing information services. The menu may also specify what amount the advertisers will pay for the user device's multimedia conference costs and what cost is associated with each information service. For example, when a user device requests stock prices, the user device will be charged at the rate of $4.99/hour. On the other hand, when a user device elects to receive advertising information from AT&T, AT&T pays for 50% of the cost of the user device's multimedia conference call.

[0057] As a practical example of the operation of the multimedia bridge 114, consider Alan, Bob and Cindy, wishing to communicate with each other in a multimedia conference. Alan initiates the communication by turning on the user device 100 and sending a start-up request to the multimedia bridge 114 requesting a multimedia conference with Bob and Cindy, and identifying Bob's and Cindy's user devices 106, 112. The multimedia bridge 114 receives the signals from Alan's user device and sends start-up requests to Bob's and Cindy's user devices.

[0058] The multimedia bridge 114 then sends a start-up request to Bob's and Cindy's user devices 106, 112. Bob's and Cindy's user devices 106, 112 send signals to the multimedia bridge 114 accepting the start-up request. Bob's acceptance signal also includes information requesting continuous weather information services. The multimedia bridge 114 completes the communication connection and initiates the multimedia conference. The multimedia bridge 114 also prompts Alan's and Cindy's user devices for information services. Alan chooses to receive advertising information from Lucent Technologies while he is not speaking. Cindy chooses not to receive information services during the multimedia conference.

[0059] The multimedia bridge 114 continues the multimedia conferences while providing Bob with continuous weather information and providing Alan with Lucent Technologies advertisements when he is not speaking. For example, the weather information is provided to Bob in a split screen format such that Bob may continuously view The Weather Channel and control the volume of the audio. Alan receives the Lucent Technologies advertisements in a picture-in-picture format so that when Alan is not speaking he receives video and audio of the advertisements, but when he is the speaker, the advertisements are paused and the volume is muted.

[0060] During the multimedia conference, Cindy decides that she would like to receive stock updates while she is not speaking. Thus, Cindy's user device sends a signal to the multimedia bridge 114 requesting the information services. The change in user preferences can be effectuated though the use of pull-down menus, pop-up menus, commands entered through a user interface, audio commands, and the like.

[0061] While monitoring for changes in user preferences, the multimedia bridge 114 detects Cindy's request and begins providing her with real-time stock updates when she is not speaking. For example, the real-time stock updates are provided to Cindy in a banner format, rather than being superimposed over the multimedia video signals.

[0062] The multimedia bridge 114 also monitors for changes in the speaking user device so that non-continuous information services can be provided to those parties who do not want information services on their monitor while they are speaking. The controller 201 may detect which user device is speaking by comparing the decibel levels of the incoming audio streams from each user device 100, 106, 108, 112, or in any other manner generally known in the art, as discussed above.

[0063] FIG. 4 is a flowchart of an exemplary operation of the multimedia bridge 114 according to a first exemplary embodiment. In step 405, the controller 201 receives a start-up signal through the network interface 202 from a source user device 100, for example, and goes to step 410. The start-up signal includes identifying information that identifies the source and destination user devices 100, 106 and 112, for example. In step 410, the controller 201 sends start-up requests to the other user devices that are to be involved in the multimedia conference—for example, destination user devices 106 and 112—and continues to step 415.

[0064] In step 415, the controller 201 receives start-up signals from the destination user devices 106, 112 and establishes a multimedia conference among the source and destination user devices 100, 106, and 112, for example, and continues to step 420. In step 420, the controller 201 determines which user devices, if any, request information services to be provided during the multimedia conference and goes to step 425. As described above, this can be accomplished with any graphical, video or audio input, such as pull-down menus, pop-up menus, commands entered through a user interface, audio commands and the like.

[0065] In step 425, if any user devices request information services, the controller 201 continues to step 430; otherwise, the controller 201 jumps to step 435. In step 430, the controller 201 selectively provides information services to the user devices that requested information services. An exemplary operation of the controller 201 in selectively providing information services is described in more detail below with respect to FIG. 5. The controller 201 then continues to step 435.

[0066] In step 435, the controller 201 monitors for a change in user preferences (i.e., whether a party wants to begin or cease receiving information services) and continues to step 440. In step 440, if the controller 201 detects a change in the preferences of one or more user devices, the controller 201 returns to step 420; otherwise the controller 201 continues to step 445.

[0067] In step 445, the controller 201 monitors for termination of the multimedia conference and continues to step 450. In step 450, if the multimedia conference is still active, the controller 201 returns to step 430; otherwise the controller 201 ends the operation. The end condition may be, for example, the disconnection of some or all of the user devices 100, 106 and 112, and the like.

[0068] FIG. 5 is a flowchart of an exemplary operation for selectively providing information services to a user device performed by the multimedia bridge 114 according to a first exemplary embodiment. The operation shown in FIG. 5 is executed for each user device determined to be requesting information services in step 420 of the above-described FIG. 4.

[0069] In step 505, the controller 201 determines whether the user device wants to continuously receive information services or only receive information services while not speaking (i.e., discontinuously receive information services). The controller then continues to step 510. In step 510, if the user device wants to continuously receive information services, the controller 201 jumps to step 525; otherwise, control continues to step 515.

[0070] In step 515, the controller 201 determines whether the user device is the speaker and continues to step 520. In step 520, if the user device is the speaker, the controller 201 ends the operation for selectively providing information services to the user device and returns to step 435 in the exemplary operation of the multimedia bridge 114 discussed with respect to FIG. 4. Otherwise, the controller 201 continues to step 525.

[0071] In step 525, the controller 201 provides information services to the user device. The controller 201 then ends the operation for selectively providing information services to the user device and returns to step 435 in the exemplary operation of the multimedia bridge 114 discussed with respect to FIG. 4.

[0072] The method and apparatus of the invention provide the ability for each operator of a user device participating in a multimedia conference to choose whether he/she wishes to receive information services during the multimedia conference. In addition, each user device may choose a different information service and/or a different format for viewing the information service.

[0073] As shown in FIG. 2, the method of this invention is preferably implemented on a programmed processor. However, the multimedia bridge 114 can also be implemented on a general purpose or special purpose computer, a programmed microprocessor or microcontroller and peripheral integrated circuit elements, an ASIC or other integrated circuit, a hardware electronic or logic circuit such as a discrete element circuit, a programmable logic device such as a PLD, PLA, FPGA or PAL, or the like. In general, any device capable of implementing the flowchart shown in FIG. 4 can be used to implement the multimedia bridge 114 functions of this invention.

[0074] While this invention has been described with specific embodiments thereof, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art. Accordingly, the preferred embodiments of the invention as set forth herein are intended to be illustrative, not limiting. Various changes may be made without departing from the spirit and scope of the invention.