Who said that? teleconference speaker identification apparatus and method

[0001] 1. Field of the Invention

[0002] The invention relates to a teleconference system that identifies remote speaking participants.

[0003] 2. Description of the Related Art

[0004] Teleconference systems bring multiple parties together from remote locations. Ideally, teleconference systems allow participants to communicate with each other as if they were seated in the same room. A teleconference system includes at least two stations set up in remote rooms interconnected by a transmission system such as a global network or a telephone system.

[0005] Participants at each of the stations communicate with each other through audio equipment. Audio equipment for each station typically includes one or more microphones, speakers, and the like. The microphones pick up participants' voices in the first station and generate an audio signal that is transmitted via the transmission system to the second, remote, station. The speakers reproduce and amplify the audio signal transmitted from the first to the second station.

[0006] Teleconference systems have audio drawbacks. Often, participants have trouble distinguishing unfamiliar voices of remote participants. A common experience is an off line canvassing of others in a first station—who said that?—after hearing an unfamiliar voice from a remote participant in a second station.

[0007] Managed teleconference calls such as a corporate earnings call typically have an operator that identifies participants before they speak. Such an arrangement ensures smooth transitions between participants. Managed teleconference calls, however, are expensive and onerous for typical ad hoc meetings with remote participants.

[0008] Accordingly, a need remains for an improved telephone conference system that identifies remote speaking participants.

[0016] FIG. 1 is a diagram of a teleconference system 100. Referring to FIG. 1, the system 100 includes a plurality of stations 1, 2, . . . , N. Each station is geographically distant or distinct from other stations. For example, station 1 might be located in an office building in San Jose, Calif.; station 2 might be located in another office building in San Jose; and station N might be located in Portland, Oreg. Stations 1, 2, . . . , N might be for example conventional analog or digital conference phones modified as described below. The design and operation of conventional conference phones are well known and will not be discussed in further detail. A person of reasonable skill in the art should understand that stations 1, 2, . . . , N might have other embodiments that come within the scope of the present invention.

[0017] A person of reasonable skill in the art should recognize that any number of stations in any number of locations comes within the scope of the invention. A person of reasonable skill in the art should recognize that stations located in geographically distant locations refers to stations located different parts of the same building, different buildings, cities, states, countries, and the like.

[0018] A number of conference participants associate with a corresponding station through corresponding connections. For example, participants 1₁, 1₂, and 1₃ associate with station 1 (e.g., in a first building in San Jose) through connections 1₁₁, 1₂₂, and 1₃₃, respectively. Participants 2₁ and 2₂ associate with station 2 (e.g., in a second building in San Jose) through connections 2₁₁ and 2₂₂, respectively. And participants N₁, N₂, and N₃ associate with station N (e.g., in Portland) through connections N₁₁, N₂₂, and N₃₃, respectively. A person of reasonable skill in the art should recognize that any number of participants might associate with any given station. A person of reasonable skill in the art should recognize that the connections might be hardwired or virtual. An example of a hardwired connection is where participant 1₁ connects to station 1 through earphones. An example of a virtual connection is where participant 2₁ connects to station 2 via sound waves picked up by one of more microphones mounted on station 2. A person of reasonable skill in the art should recognize other possibilities of hardwired or virtual connections exist that come within the scope of the present invention.

[0019] Stations 1, 2, . . . , N determine the particular position of its associated participants when each participant speaks. For example, station 1 determines that participants 1₁, 1₂, and 1₃ are located at positions 1_A, 1_B, and 1_C, respectively. Station 2 determines that participants 2₁ and 2₂ are located at positions 2₁₁ and 2₂₂, respectively. And station N determines that participants N₁, N₂, and N₃ are located at positions N_A, N_B, and N_C, respectively.

[0020] Stations 1, 2, . . . , N determine the position of the current speaking participant using well known technology, including a radial coordinate (r, theta) system. In one embodiment, stations 1, 2, . . . , N determine the current speaking participant's position or angle relative to the station using a radial array of directional microphones positioned within the corresponding stations 1, 2, . . . , N. In another embodiment, stations 1, 2, . . . , N might localize multiple sound sources using two microphones as described by Chen Liu and others in a paper titled “Localization Of Multiple Sound Sources With Two Microphones,” published in 2000 in The Journal of the Acoustical Society of America, volume 108, number 4, pages 1888-1905. In yet another embodiment, stations 1, 2, . . . , N determine the current speaking participant's location using a microphone array as described by Rabinkin and others in a paper titled “A DSP Implementation of Source Location Using Microphone Arrays,” published April 1996 in The Journal of the Acoustical Society of America, volume 99, number 4, part 2, page 2503.

[0021] The stations 1, 2, . . . , N include a request means 105 that a participant activates to transmit a request 108 to a central server 114 via the network 104. The request 108 asks the server 114 to identify the participant last to speak. The request means 105 might be a dedicated button mounted on the station that a participant pushes to transmit the request 108 to the server 114. The request means 105 might alternatively be a sequence of preprogrammed button presses on a conventional conference phone. A person of reasonable skill in the art should recognize any number of request means 105 that might be used to enable a participant to send a request 108 to the server 114.

[0022] The stations transmit their corresponding participants' speech through a network 104 such that participants located in geographically distant locations (e.g., San Jose to Portland) communicate with each other. The network 104 might be a digital network such as an Internet Protocol (IP) network that digitally transports information packets from a destination to a source. In one embodiment, the network 104 carries the audio portion of the call via, e.g., Voice Over Internet Protocol (VoIP). Or the network 104 might be an analog network such as a telephone network, e.g., a Private Branch Exchange (PBX) or a public telephone network that transports analog signals from a destination to a source.

[0023] The teleconference system 100 includes an audio server 106 and the central server 114. A person of reasonable skill in the art should recognize that the central server 114 might be part of or completely separate from the audio server 106. The audio server 106 initializes, maintains, and controls the conference call between the stations 1, 2, . . . , N through the network 104. The design and operation of the audio server 106 is well known and will not be discussed in further detail.

[0024] FIG. 2 is a block diagram of the central server 114 shown in FIG. 1. Referring to FIG. 2, the central server 214 receives a request 208 to identify the participant last to speak. The central server 214 includes an initialization module 222 and an identification module 224. The initialization module 222 creates a table 216 (116 in FIG. 1) that the identification module 224 uses to respond to the request 208. During conference call initialization, the initialization module 222 builds the table 216 by adding a recorded voice segment 220 of each call participant at each of the stations 1, 2, . . . , N to the table 216.

[0025] FIG. 3 is a block diagram of the initialization module 222 shown in FIG. 2. Referring to FIGS. 1-3, the initialization module 322 initiates a roll call process for each connected station. The initialization module 322 includes station identification receiving module 332 for receiving a station identification 340 from each station included in the conference call. A prompt tone module 330 sequentially sends a start recording prompt 338 (e.g., a tone, explicit recording instruction, light, and the like) to each of the stations 1, 2, . . . , N included in the conference call. Responsive to the start recording prompt 338, each station, e.g., station 1, sends a speaker prompt (not shown separately) to each of its associated participants, e.g., 1₁, 1₂, and 1₃, through a corresponding connection, e.g., 1₁₁, 1₂₂, and 1₃₃, respectively. Each participant, e.g., participants 1₁, 1₂, and 1₃, records a voice segment 320. A recorder 336 records the voice segment 320.

[0026] Almost simultaneously with each participant recording its voice segment 320, the station, e.g., station 1, determines the position 342, e.g., 1_A, 1_B, and 1_C, of each of its associated participants, e.g., 1₁, 1₂, and 1₃, respectively, using any of the methods described above. A person of reasonable skill in the art should recognize that the station or the central server should have a reasonable amount of audio storage (not shown separately) to buffer the voice segment 320 from a speaker until the station is able to determine the position of the participant. Once the station determines a position, it provides its participants' positions 342 to a position-receiving module 334 included in the initialization module 322.

[0027] FIG. 5 is an embodiment of the table 316 shown in FIG. 3. Referring to FIG. 5, a table 516 includes a plurality of columns and rows indicating the number of stations and associated participants to the conference call. The table 516 includes a column 550 identifying the station, a column 552 identifying the participant position, and a column 554 containing the recorded voice segment 308 (FIG. 3) of the corresponding participant. For the example conference call shown in FIG. 1, station 1 includes participants 1₁, 1₂, and 1₃ further identified by their recorded voice messages contained in column 554. Participant 1, recorded “Bob.” Participant 12 recorded “Hi, I'm Jane.” And participant 13 recorded “Um, Joe.”

[0028] Once all the participants at a prompted station have recorded their corresponding voice segments, the prompt tone module 330 sends a prompt to a next station, e.g., station 2. Station 2 determines the position 2_A and 2_B of each of its participants 2₁ and 2₂, respectively. Each participant provides it voice segment 320 to the recorder 336. Participant 2₁ records “John” and participant 22 records “Marie.” Other participants at other stations recorded voice segments as shown in table 516. The initialization module 322 detects conference entry and conference exit tones that collide with a participant speaking. When this occurs, the initialization module 322 scrubs the recording just made and re-prompts the participant. The initialization module 322 terminates the roll call process responsive to a signal from the station, e.g., a key press such as #. A person of reasonable skill in the art should recognize that roll call termination might be implicit, e.g., by the detected silence after a speaker prompt (prompting a participant) or by the detected silence after a start recording prompt 338 (prompting a station).

[0029] Referring to FIG. 1, the central server 114 receives and records an ongoing stream 112 of station identification (340 in FIG. 3) and participant position (342 in FIG. 3) information from the stations connected to the call. Each station contributes such an item to the server whenever it believes that it has an active, speaking, participant. At any point, a participant (e.g., participant 2₁) makes a request 108 to the central server 114 to identify the participant who last spoke (e.g., participant N₁). The request 108 is made using a request means 105 included in the station. For example, participant 2₁ presses a “who said that?” button 105 on station 2.

[0030] Referring to FIGS. 2 and 4, an identification module 424 in the central server 214 receives the request 408. Since the table 416 is continuously updated with the position of the participant last to speak by a continuous stream of updated information, e.g., stream 112, look up module 446 looks up the identity of the participant last to speak in the table 416 responsive to the request 408. The transmitting module 444 transmits and plays the recorded voice segment 420 of the participant last to speak to the enquiring participant as indicated by the table 416. In the example shown in FIG. 1, the central server 114 responds to the request 108 from participant 22 with the recorded voice segment of participant N₁, that is, the central server 114 responds with “Jimmy.”

[0031] In one embodiment, the central server 114 provides the enquiring participant with a list of recent stations and positions of participants who recently spoke through the network 104. The enquiring participant can then select (e.g., by using a web browser or other such tool) the participant's position and the server 114 transmits and plays back the voice segment associated with the selected participant. By doing so, the enquiring participant is not necessarily limited to only enquiring about a last speaker but has a list of recent speakers from which to chose. Such an interface can also allow the enquiring participant to set up filters. For example, the enquiring participant can provide nicknames for particular station/position pairs or filter those out, such that frequent or familiar speaking participants are ignored in the list and the unfamiliar ones more readily identified.

[0032] One embodiment of the system 100 involves ordinary telephones. A particular touch-tone or preprogrammed key presses can be designated as the request means 105. The server 114 would treat the phone as a single party conference station. The telephone itself will not send the relevant information to the central server 114, nor will it determine a participant's position. The audio server 106 might be able to perform these functions for it. In this situation, the audio server 106 transmits station identification and a value (e.g., 0) indicating the direction of the participant at that station to the server 114. The server 114 treats this single person station in the same way as any other type of station, particularly during initialization and operation.

[0033] FIGS. 6A-B is a flow diagram of a teleconferencing method. Referring to FIGS. 6A-B, a central server initializes a table at 602. The central server initiates the roll call process by identifying a first station at 604 and 606. The server prompts the first station at 608 and locates the position of a first participant at 610 and 612. The server prompts the first participant to record a voice segment 614. The server builds the table by adding the position and recorded voice segment of the participant at 616. The server determines whether it has identified all of the participants at the first station. If not, the server locates the next participant at 612 and 634. The server continues this process until all of the participants at the first station have been located and identified. The server determines whether all of the stations have been processed. If not, the server identifies the next station 606 and 636. The server continues this process until the roll call is complete (or terminated) and all of the participants at each of the stations have been located and identified. As explained earlier, a key press can terminate the roll call process at any station. The server receives a request to identify a participant last to speak at 622. Since the server receives a continuous stream of position/station identification information relating to the current speaker, it can identify the position of the last speaking participant (624) and look up the voice segment associated with the participant at that position/station (626). At 628, the server transmits and plays back the recorded voice segment of the participant associated with the identified position. The server processes all such requests (630) until the conference call terminates or no more requests are received.

[0034] Having illustrated and described the principles of our invention, it should be readily apparent to those skilled in the art that the invention can be modified in arrangement and detail without departing from such principles. I claim all modifications coming within the spirit and scope of the accompanying claims.