Identifying a location of a voice-input device

[0001] Homes are becoming more wired and connected with the proliferation of computing devices such as desktops, tablets, entertainment systems, and portable communication devices. As computing devices evolve, many different ways have been introduced to allow users to interact with these devices, such as through mechanical means (e.g., keyboards, mice, etc.), touch screens, motion, and gesture. Another way to interact with computing devices is through speech.

[0008] This disclosure describes, in part, techniques for identifying a location of a voice-controlled device within an environment. After identifying a location of the device, the device may receive a voice command from a user within the environment and may determine a response to the command based in part on the location, may determine how to output a response based in part on the location or may determine how to interact with the user based in part on the location.

[0009] As described below, the location of a device may be determined in based on implicit and/or explicit data captured from an environment in which the device resides. For instance, the voice-controlled device may determine its location based on sound captured by the device. In one example, a microphone of the device may capture sound and generate an audio signal based on the sound. The device may then analyze the audio signal for identifying background noise proximate to the device. For instance, if the device determines that the background noise includes a television or stereo, the device may determine that it resides within a living or entertainment room. If, however, the device determines that the background noise includes sounds associated with cooking (e.g., a sizzle of a frying pan, a mixer, etc.), then the device may determine that it resides within a kitchen.

[0010] In another example, the device may include one or more sensors in addition to the microphone for the purpose of determining its location. For instance, the device may include a light sensor, a temperature sensor, a camera or the like. The device may then gather data via the sensors and determine its location based on this data. For instance, the device may determine an amount or direction or color temperature of light in the room and/or a temperature of the room and may compare this data to known characteristics of rooms within the environment. In addition or in the alternative, the device may capture images (e.g., static images or video) of the room in which the device resides and may determine its location based on analyzing these images.

[0011] In still another example, the device may analyze previous interactions between the device and users within the environment when determining its location. For instance, the device may determine a father of the house most often issues commands to the device and the device may accordingly determine that the location is likely an office of the father. Or, the device may determine that a daughter in the family often communicates with the device and often requests recipe information and, based in part on this information, the device may determine that it is located in the kitchen. In still other examples, the device may simply query the user as to its location and receive a response (e.g., audibly) indicating which the room in which the device resides.

[0012] After identifying its location, the device may alter its responses to received voice commands based at least in part on the location. For instance, if a device determines that it is located in an entertainment room, then the device may be more likely to issue video search results in response to receiving a query at the device. If, however, the device determines that it is located in the kitchen, then the device may be more likely to issue search results that are cooking-focused.

[0013] In another example, the device may determine how to output a response to a voice command based on the device's location. For instance, if a user requests that the device play a particular song, then the device may play the song on its own speakers if no other speakers reside within the room of the device. If, however, a home theater system is located within a room that includes a home theater system, then the device may utilize the home theater system to play the song. In the latter instances, the device may couple (e.g., wirelessly) with the home theater system and may be configured to issue commands and, hence, control the output of the home theater system.

[0014] In still another example, the device may determine how to interact with a user based in part on the location of the device. For instance, the device may engage a user in a longer or different question-and-answer dialog in one room as compared to another room. Of course, while a few examples have been provided, the device may perform or alter any action at least partly based on its determined location within an environment. Furthermore, the devices and techniques described above may be implemented in a variety of different architectures and contexts. One non-limiting and illustrative implementation is described below.

Illustrative Environment

[0015] FIG. 1 shows an illustrative voice interaction computing architecture 100 set in an environment 102 that includes multiple rooms. The architecture 100 also includes a respective electronic voice-controlled device 104(1), 104(2), . . . , 104(N) (referred to as “voice-controlled device 104”) located in several of the rooms. Users within the environment may interact (e.g., via voice commands) with the devices 104(1)-(N) in the environment. Each voice-controlled device 104 may be positioned on a table, a ceiling, a wall, in a lamp, beneath a table, under a chair or the like. Further, more than one device 104 may be positioned in a single room, or one device may be used to accommodate user interactions from more than one room.

[0016] As described in further detail below, each device 104 may use implicit and/or explicit data for the purpose of determining its respective location within the environment 102 (e.g., a room in which it resides). For instance, FIG. 1 illustrates that a room in which the device 104(1) resides includes a television 106 that emits audio associated with a television show or movie. The device 104(1) may capture this sound, determine that the device 104(1) resides in a room that includes a television and use this information for helping determine which room of the multiple rooms the device 104(1) resides in.

[0017] Generally, a voice-controlled device 104 has a microphone unit comprising at least one microphone 108. The device 104 also includes at least one speaker 110 to facilitate audio interactions with users within the environment, or may provide audio for output on speaker(s) of another device. In some instances, the voice-controlled device 104 is implemented without a haptic input component (e.g., keyboard, keypad, touch screen, joystick, control buttons, etc.) or a display. In certain implementations, a limited set of one or more haptic input components may be employed (e.g., a dedicated button to initiate a configuration, power on/off, etc.). Nonetheless, the primary and potentially only mode of user interaction with the electronic device 104 may be through voice input and audible output. One example implementation of the voice-controlled device 104 is provided below in more detail with reference to FIG. 2.

[0018] The microphone 108 of the voice-controlled device 104 detects audio from the environment 102, such as sounds uttered from a user. As illustrated, the voice-controlled device 104 includes a processor 112 and memory 114, which stores or otherwise has access to a speech-recognition engine 116. As used herein, a processor may include multiple processors and/or one or more processors having multiple cores. The speech-recognition engine 116 performs speech recognition on audio captured by the microphone, such as utterances spoken by the user. The voice-controlled device 104 may perform certain actions in response to recognizing different speech from the user. The user may speak predefined commands (e.g., “Awake”; “Sleep”), or may use a more casual conversation style when interacting with the device 104 (e.g., “I'd like to go to a movie. Please tell me what's playing at the local cinema.”).

[0019] In some instances, the voice-controlled device 104 may operate in conjunction with or may otherwise utilize computing resources 118 that are remote from the environment 102. For instance, the voice-controlled device 104 may couple to the remote computing resources 118 over a network 120. As illustrated, the remote computing resources 118 may be implemented as one or more servers 122(1), 122(2), . . . , 122(P) and may, in some instances form a portion of a network-accessible computing platform implemented as a computing infrastructure of processors, storage, software, data access, and so forth that is maintained and accessible via a network such as the Internet. The remote computing resources 118 do not require end-user knowledge of the physical location and configuration of the system that delivers the services. Common expressions associated for these remote computing devices 118 include “on-demand computing”, “software as a service (SaaS)”, “platform computing”, “network-accessible platform”, “cloud services”, “data centers”, and so forth.

[0020] The servers 122(1)-(P) include a processor 124 and memory 126, which may store or otherwise have access to some or all of the components described with reference to the memory 114 of the voice-controlled device 104. For instance, the memory 126 may have access to and utilize the speech-recognition engine 116 for receiving audio signals from the device 104, recognizing speech and, potentially, causing performance of an action in response. In some examples, the voice-controlled device 104 may upload audio data to the remote computing resources 118 for processing, given that the remote computing resources 118 may have a computational capacity that far exceeds the computational capacity of the voice-controlled device 104. Therefore, the voice-controlled device 104 may utilize the speech-recognition engine 116 at the remote computing resources 118 for performing relatively complex analysis on audio captured from the environment 102.

[0021] Regardless of whether the speech recognition occurs locally or remotely from the environment 102, the voice-controlled device 104 may receive vocal input from a user and the device 104 and/or the resources 118 may perform speech recognition to interpret a user's operational request or command. The requests may be for essentially any type of operation, such as database inquiries, requesting and consuming entertainment (e.g., gaming, finding and playing music, movies or other content, etc.), personal information management (e.g., calendaring, note taking, etc.), online shopping, financial transactions, and so forth.

[0022] The voice-controlled device 104 may communicatively couple to the network 120 via wired technologies (e.g., wires, USB, fiber optic cable, etc.), wireless technologies (e.g., RF, cellular, satellite, Bluetooth, etc.), or other connection technologies. The network 120 is representative of any type of communication network, including data and/or voice network, and may be implemented using wired infrastructure (e.g., cable, CAT5, fiber optic cable, etc.), a wireless infrastructure (e.g., RF, cellular, microwave, satellite, Bluetooth, etc.), and/or other connection technologies.

[0023] As illustrated, the memory 114 of the voice-controlled device 104 also stores or otherwise has access to a location determination engine 128, a response engine 130, and one or more room profiles 132. The location determination engine 128 functions to determine a physical location of a respective voice-controlled device 104 within the environment 102. For instance, the engine 128 may determine a particular room in which the device 104 resides, a location within a room in which the device 104 resides, a floor of a building in which a device resides, a building in which a device resides, or any other type of location.

[0024] The location determination engine 128 may determine a location of the device 104 based on any combination of implicit and/or explicit data. For instance, a user may explicitly specify to the device (e.g., audibly, via a display, etc.) a location at which the device 104 resides. Additionally or alternatively, the engine 128 may capture data from the environment proximate to the device 104 and may compare this data to one or more of the room profiles 132. Based on this comparison (and/or based on explicit data), the device 104 may then determine its location within the environment. An example room profile 132(1), for instance, shows that a living room within the environment includes background noise from a television, that the users “Dad” and “Grace” are often present within the environment and may often communicate with this particular device, that the living room has a particular light level (or light-level range), that the living room has a particular temperature (or temperature range), that certain devices are located within the living room and, hence, are proximate to this particular device, that the living room has a particular set of echo characteristics and/or the like.

[0025] The room profiles 132 may be based on one or both of information that has been captured by the device 104 within the environment or information about rooms or locations generally. For instance, one room profile may include an indication that the living room within the particular environment 102 includes the television 106, with the voice-controlled device 104 having learned that information from capturing sound within the environment. A room profile for a kitchen, meanwhile, could include an indication that a device within the kitchen will frequently identify certain cooking sounds (represented by a certain frequency, amplitude, etc.). As such, the device 104 can both be pre-configured with the room profiles 132 and/or may build or customize the room profiles for a particular environment 102.

[0026] In one example, the engine 128 may receive audio signals from the microphone 108 and may determine a location of the device 104 based at least in part in on the analysis of the audio signals. For instance, the engine 128 may identify background noise from the audio signals. The background noise may represent sound from a television 106, sound from a stereo system, users talking, cooking sounds, typing sounds, wind or other weather-related sounds or the like. In some instances, the engine 128 may compare characteristics of the audio signals, such as frequency, volume, waveform, etc., to known audio signals to identify these noises. That is, the engine 128 may compare an audio signal to a datastore of labeled audio signals to identify the sound (e.g., a cooking sound, the wind, etc.).

[0027] In addition, the engine 128 may analyze the audio signals to identify (e.g., via voice-recognition techniques) which users are proximate to the device. The engine 128 may also analyze the audio signal to identify echo characteristics of the room in which the device resides. In another example, the speaker 110 of the device 104 may output audio, which may be captured by the microphone 108 of the device 104. The engine 128 may then analyze an audio signal generated by the microphone 108 to determine the echo characteristics of the room (e.g., an amount of reflected sound indicated in a generated audio signal).

[0028] Furthermore, in some instances the device 104 may include one or more additional sensors, such as a light sensor, a temperature sensor, a camera and/or the like. The location determination engine 128 may receive data captured by the sensors and may determine a location of the device 104 based at least in part on this data. For instance, the engine 128 may receive an indication of an amount or color temperature of light within the room from the light sensor, may receive a temperature from the temperature range and/or may receive one or more images of the room from the camera. The engine 128 may then compare this information to one or more room profiles 132 to determine the room in which the device 104 resides.

[0029] In still other instances, the device 104 may include one or more wireless interfaces (discussed below with reference to FIG. 2). The wireless interface of the device may operate to identify one or more other devices having wireless capabilities proximate to the device 104. For instance, the device 104 may identify (e.g., via Bluetooth, WiFi, etc.) other devices that are near the device 104, such as appliances, electronic devices, etc. If the device 104 determines that kitchen appliances (e.g., having Bluetooth capabilities) are proximate to the device, then the device 104 may determine that it is likely located in the kitchen. If, however, the device 104 determines that a home theater system is proximate to the device, then the device 104 may determine that it is likely located in the living room or an entertainment room.

[0030] After receiving and analyzing data from the microphone 108 and the one or more other sensors 134, the engine 128 may compare this data to one or more room profiles 132. The engine 128 may then determine a best match between the analyzed data and the room profiles to determine the likely location of the device 104.

[0031] After receiving an indication of a designated location of the device 104, the response engine 130 may determine content to output in response to receiving a voice command, with the content being based at least in part on the designated location of the device. For instance, if the user provides a voice command to “search for fried green tomatoes”, then the device 104 may determine content to output to the user based in part on the location of the device 104. For instance, if the device is located in the kitchen, the device 104 may output (e.g., audibly, visually, etc.) instructions for how to cook fried green tomatoes. If, however, the device 104 is located in an entertainment room, then the device 104 may return search results associated with the movie “Fried Green Tomatoes”.

[0032] In addition or in the alternative, the response engine 130 may determine how to output content based in part on the designated location of the device. For instance, if the user requests to play a certain song, then the response engine 130 may determine which set of speakers to utilize based on this location. If the device resides in a room without speakers that the device 104 controls, then the device 104 outputs the song via the speaker 110. If, however, the device resides in a room that includes a home theater system that is controllable by the device, then the device 104 may request that the home theater system output the song.

[0033] In still another example, the device 104 may determine how to interact with a user making a request based in part on the location of the device 104. For instance, if the user searches for “fried green tomatoes” in the entertainment room, the device 104 may return search results immediately. If, however, the user requests this information from a home office, the device 104 may query the user as to whether the user desires search results regarding how to cook fried green tomatoes or search results associated with the movie “Fried Green Tomatoes”.

Illustrative Voice-Controlled Device

[0034] FIG. 2 shows selected functional components of the voice-controlled device 104 in more detail. Generally, the voice-controlled device 104 may be implemented as a standalone device that is relatively simple in terms of functional capabilities with limited input/output components, memory and processing capabilities. For instance, the voice-controlled device 104 does not have a keyboard, keypad, or other form of mechanical input in some implementations, nor does it have a display or touch screen to facilitate visual presentation and user touch input. Instead, the device 104 may be implemented with the ability to receive and output audio, a network interface (wireless or wire-based), power, and limited processing/memory capabilities.

[0035] In the illustrated implementation, the voice-controlled device 104 includes the processor 112 and memory 114. The memory 114 may include computer-readable storage media (“CRSM”), which may be any available physical media accessible by the processor 112 to execute instructions stored on the memory. In one basic implementation, CRSM may include random access memory (“RAM”) and Flash memory. In other implementations, CRSM may include, but is not limited to, read-only memory (“ROM”), electrically erasable programmable read-only memory (“EEPROM”), or any other medium which can be used to store the desired information and which can be accessed by the processor 112.

[0036] The voice-controlled device 104 includes a microphone unit that includes one or more microphones 108 to receive audio input, such as user voice input, and one or more speakers 110 to output audio sounds. Audio signals generated by the microphone unit may be generated by a single microphone or an array of microphones. In some instances, the audio signals may also be combined, processed, filtered and/or the like prior to being analyzed in the manners described above and below.

[0037] One or more codecs 202 are coupled to the microphone 108 and speaker 110 to encode and/or decode the audio signals. The codec may convert audio data between analog and digital formats. A user may interact with the device 104 by speaking to it, and the microphone 108 captures the user speech. The codec 202 encodes the user speech and transfers that audio data to other components. The device 104 can communicate back to the user by emitting audible statements through the speaker 110 and, potentially, illuminating one or lights (e.g., LEDs) of the device 104 while outputting sound through the speaker 110. In this manner, the user interacts with the voice-controlled device simply through speech, without use of a keyboard or display common to other types of devices.

[0038] In the illustrated example, the voice-controlled device 104 includes one or more wireless interfaces 204 coupled to one or more antennas 206 to facilitate a wireless connection to a network. The wireless interface 204 may implement one or more of various wireless technologies, such as wifi, Bluetooth, RF, and so on.

[0039] One or more device interfaces 208 may further be provided as part of the device 104 to facilitate a wired connection to a network, or a plug-in network device that communicates with other wireless networks. The interfaces 208 may include USB or other forms of wired connections, such as a broadband connection. A power unit 210 is further provided to distribute power to the various components on the device 104.

[0040] The voice-controlled device 104 is designed to support audio interactions with the user, in the form of receiving voice commands (e.g., words, phrase, sentences, etc.) from the user and outputting audible feedback to the user. Accordingly, in the illustrated implementation, there are no haptic input devices, such as navigation buttons, keypads, joysticks, keyboards, touch screens, and the like. Further there is no display for text or graphical output. In one implementation, the voice-controlled device 104 may include non-input control mechanisms, such as basic volume control button(s) for increasing/decreasing volume, as well as power, reset and mute buttons. There may also be a simple light element (e.g., LED) to indicate a state such as, for example, when power is on. But, otherwise, the device 104 does not use or need to use any input devices or displays in some instances.

[0041] Several modules such as instruction, datastores, and so forth may be stored within the memory 114 and configured to execute on the processor 112. An operating system module 212 is configured to manage hardware and services (e.g., wireless unit, USB, Codec) within and coupled to the device 104 for the benefit of other modules.

[0042] In addition, the memory 114 may include the speech-recognition engine 116, the location determination engine 128, the response engine 130 and the room profiles 132, discussed above. Also as discussed above, some or all of these engines, data stores, and components may reside additionally or alternatively at the remote computing resources 118.

Illustrative Processes

[0043] FIG. 3 depicts a flow diagram of an example process 300 of the voice-controlled device 104 analyzing a first audio signal to determine the device's location and performing an operation based on a subsequently received voice command from a user based on the location of the device. While this figure describes the device 104 performing this process, in other instances one or more other entities may additionally or alternatively perform one or more of these actions.

[0044] The process 300 (as well as each process described herein) is illustrated as a logical flow graph, each operation of which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types.

[0045] The computer-readable media may include non-transitory computer-readable storage media, which may include hard drives, floppy diskettes, optical disks, CD-ROMs, DVDs, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flash memory, magnetic or optical cards, solid-state memory devices, or other types of storage media suitable for storing electronic instructions. In addition, in some embodiments the computer-readable media may include a transitory computer-readable signal (in compressed or uncompressed form). Examples of computer-readable signals, whether modulated using a carrier or not, include, but are not limited to, signals that a computer system hosting or running a computer program can be configured to access, including signals downloaded through the Internet or other networks. Finally, the order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the process.

[0046] At 302, the device 104 receives a first audio signal generated by a microphone of the device 104, with the signal being based on a sound captured by the microphone. At 304, the device 104 may then analyze the first audio signal to identify a location of the device. For instance, the device 104 may identify background noise from a television in the audio signal and may identify a room in which the device resides based on this information. That is, the device may map the identified background noise to the identified room based on a profile of the room or similarly situated rooms (e.g., living rooms of other users, etc.). The device 104 may additionally determine this location using data captured by one or more other sensors.

[0047] At 306, the device receives a second audio signal generated by the microphone. The second audio signal may be discrete from the first audio signal or may form a different portion of a same audio signal as the first audio signal. In either instance, the device may generate the first audio signal close in time or a large amount of prior to generating the second audio signal. In this example, the second audio signal includes a voice command from a user to “Play the Rolling Stones”.

[0048] At 308, the device 104 performs speech recognition on the second audio signal to identify the command and identifies and performs an operation based on the voice command and based at least in part on the identified location. This operation may include one or more of sub-operations 308(1), 308(2), and 308(3). At 308(1), the device determines which operation of multiple operations to perform based at least in part on the location of the device. At 308(2), the device 104 determines how to perform the operation based at least in part on the location. At 308(3), the device determines how to interact with the user based at least in part on the location of the device. In the illustrated example, the device 104(1) determines to output a song by the Rolling Stones on a home stereo system that is determined to be in the same room as the device 104.

[0049] FIG. 4 depicts a flow diagram of another example process 400. At 402, the process 400 determines a physical location of a device within an environment. This may be based on implicit data, explicit data or a combination thereof. At 404, the process 400 receives, via a microphone, a request from a user within the environment. At 406, the process 400 determines a response to provide to the request based at least in part on the request and on the location of the device. At 408, the process 400 outputs the response via a speaker.

[0050] FIG. 5 depicts a flow diagram of an example process 500. At 502, the process 500 captures, at a device, sound from an environment that includes multiple rooms. At 504, the process 500 analyzes an audio signal generated from the sound. At 506, the process 500 determines which room of multiple rooms the device resides in. At 508, the process 500 receives a voice command from a user within the environment. At 510, the process 500 identifies an operation to perform based on the voice command and based on the determined location of the device. Finally, at 512, the process 500 performs the operation.

[0051] Although the subject matter has been described in language specific to structural features, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features described. Rather, the specific features are disclosed as illustrative forms of implementing the claims.