Dialog interface system

[0002] 1. Field of the Invention

[0003] The present invention relates to an information processing apparatus furnished with a user interface adaptable, illustratively, to a computer or an electrical device, or to any variety of industrial machinery. The present invention relates more particularly to a dialog interface system that allows a human and an inanimate entity (hereinafter a “thing” or “things”), as well as a thing and a fellow thing, to converse by way of speech.

[0004] 2. Description of the Related Art

[0005] A remote control interface utilizing a remote control is pervasive in today's field of electrical appliances, of which, it may be said, the television and the video recorder/player are representative. Although the remote control may be suitable to tasks involving simple functions like changing television channels or adjusting television volume, it is unfit for more complex tasks. In fact, it cannot be said that even engineers, not to mention novices and the elderly, have completely mastered the use of electrical appliances comprising various and advanced functions.

[0006] In the computer field, in which more sophisticated tasks are required, graphical user interfaces (hereinafter “GUI”) utilizing windows and a mouse are widely employed. The specifications differ for each product, however, and there remain many inconveniences for those who use these products.

[0007] Further, because the GUI itself represents a task with which novices and the elderly have little experience, there are for them many difficulties. Also, because the GUI by its very nature requires a screen, it is not well-suited to many types of electrical appliances and industrial machinery that do not require a display.

[0008] For these reasons, speech recognition and synthesis technologies, which are natural to humans and require no display, recently have received attention as a next-generation user interface for these systems. These technologies are already employed presently in, illustratively, car navigation systems and some computer systems.

[0009] In research regarding interfaces, moreover, many have recently taken notice of the multi-modal interface (see, for example, R. A. Bolt, “The Integrated Multi-modal Interface”, (invited paper) IEICE Transactions on Information Systems, vol. J70-D, no. 11, pp. 2017-2025, November 1987; Katashi Nagao, “Multimodal Human-Computer Interaction: Agent Oriented and Real-World-Oriented”, (in Japanese), Journal of the Society of Instrument and Control Engineers, vol. 35, no. 1, pp. 65-70, January 1996; Tsuneo Nitta, “From GUI Multi-modal UI (MUI)*, (in Japanese), Journal of Information Processing Society of Japan, vol. 36, no. 11, pp. 1039-1046, November 1995.)

[0010] This research seeks to facilitate dialog with, for example, a computer by using not only GUI-like visualization, but also speech or gestures and other illustrative multi-modal aspects.

[0011] On the other hand, computers are connected to a variety of things found in the environments in which humans live. Research is being conducted regarding ubiquitous computing, which seeks to assist, among other things, human actions (see, for example, M. Weiser, “Some Computer Science Issues in Ubiquitous Computing”, Communications of the ACM, vol. 36, no. 7, pp. 74-85, July, 1993; Katashi Nagao, “Real-World-Oriented Human-Computer Interaction: A Survey”, (in Japanese), Systems, Control and Information, vol. 40, no. 9, pp. 385-392, September, 1996).

[0012] This would entail, by way of example, computers becoming ubiquitous on, illustratively, library shelves, thus endeavoring to support searching books by way of communication with a mobile computer or according to speech.

[0013] In respect of voice research, moreover, research relating to the interaction between computers and humans has been undertaken with great fervor in recent years (see, for example, R. Cole, L. Hirschman, L. Atlas, M. Beckman, A. Biermann, M. Bush, M. Clements, J. Cohen, O. Garcia, B. Hanson, H. Hermansky, S. Levinson, K. McKeown, N. Morgan, D. G. Novick, M. Ostendorf, S. Oviatt, P. Price, H. Silverman, J. Spitz, A. Waibel, C. Weinstein, S. Zahorian, and V. Zue, “The Challenge of Spoken Language Systems: Research Directions for the Nineties”, IEEE Transactions on Speech and Audio Processing, vol. 3, no. 1, pp. 1-21, January 1995; Katunobu Itou, “Speech Dialog System”, (in Japanese), The Institute of Electronics, Information and Communication Engineers, Technical Report, vol. 92, no. 127, Voices, no. SP92-38, pp. 23-30, July, 1992; Yoichi Takebayashi, “Human-Computer Dialogue using Multimedia Understanding and Synthesis Functions”, (in Japanese), The Institute of Electronics, Information and Communication Engineers, Technical Report, vol. 92, no. SP92-37, pp. 15-22, July, 1992).

[0014] Systems that utilize the above-described conventional speech recognition and synthesis technologies generally presume communication between only one human and one thing.

[0015] It is predicted that the number of systems with speech recognition and synthesis functions will continue to increase. When these systems are plural and are mixed with humans, moreover, it is predicted that a plurality of systems will execute in error a command issued to one system, and that a response issued to a human from one system will be mistakenly interpreted by a different system as a command issued to the latter system.

[0016] The present invention was conceived in consideration of the circumstances described above. The present invention aims to provide an interface system that is at once responsive to humans and practical and, further, that facilitates dialog by speech not only among humans, but also among things, free of mistaken operations, and without requiring a display.

[0017] The present invention proposes the concept of a “things' dialog interface” (hereinafter “TDI”) as one means of solving the problems inherent in this type of interface and its attendant networking and advancing positively the evolution of this kind of interface. TDI is an interface that facilitates conversation by speech among humans and things, as well as among fellow things, by adding not only to computers, but also to various things (all things, including, but not limited to, electrical devices, industrial machinery, and like objects), speech recognition and synthesis functions.

[0018] As above explained, speech recognition and synthesis technologies are already presently used in car navigation systems and in some computers. However, these applications generally assume dialog between only one human and one thing.

[0019] On the other hand, the TDI herein proposed is decidedly different, in that it makes possible dialog not only as among humans and things, but also among fellow things.

[0020] As represented in FIG. 1, a dialog in a conventional system is accomplished between a human and a thing on a one-to-one basis. For that reason, it is unnecessary to clarify, with respect to a particular dialog, for whom, or for what, that particular communication is intended. By contrast, in the TDI system, it is possible to undertake communication among a plurality of humans, among a plurality of things, or among a plurality of humans and things, as is shown in FIG. 2. Consequently, the TDI system must indicate with respect to whom a particular communication is initiated. To achieve this purpose, data (for example, in-to-whom) indicating by whom a dialog is initiated and data (for example, out-to-whom) indicating to whom a communication is directed are used.

[0021] In particular respect to the TDI system, it is possible, as is shown in FIG. 3, to designate as communication-related items data such as a specific human, a specific thing, “anyone”, or “everyone”, or any combination of these illustrative data items.

[0022] So that other things do not initiate a dialog while someone or something is presently engaged in a dialog, an out-wait-signal, for example, is used in the TDI system. In the conventional system, on the other hand, a function utilizing an out-wait-signal is not employed, because there is no interference with the dialog between one human and one thing.

[0023] Furthermore, because the TDI system is furnished with a function employing an out-wait-signal, it is possible to delay synchronously the execution of innate operations with communications that are similarly delayed. For that reason, a function utilizing, illustratively, an out-synchronize-signal, also, can be added. The conventional system is not equipped with this function employing an out-synchronize-signal.

[0024] By raising in one stretch the computer interface to a level that is natural to humans, like speech, the so called GUI war among competing companies becomes irrelevant. Because speech has long been used as a means of human communication, moreover, it is likely that the issue of standardization will become passe. Finally, that a screen is unnecessary, also, is a significant feature of the TDI system.

[0025] In many electrical devices and industrial machines, a large display like that used in a computer is not required. The TDI system is well-suited to this aspect, as well. Also, because a display is unnecessary, the TDI system is amenable to miniaturization.

[0026] Furthermore, because the TDI system requires no manual inputs, it is convenient for people occupied with tasks and for the physically infirm.

[0027] As to networking issues, speech is promising in interface networks within homes, for example, because speech is wireless and capable of simple broadcast. By taking advantage of existing telephone networks, moreover, a remote interface, too, is feasible.

[0028] Radio wave and infrared ray technologies are deemed promising in respect of radio, or wireless, networking. It is to be noted, however, that these technologies are suited to data networking. It seems plausible, then, to assert that speech is suitable for interface networking, particularly in view of the fact that a human can easily be interposed in interface networking.

[0029] TDI makes possible mutual communication between various electrical devices, industrial machinery, and the users thereof, and facilitates the communication, adjustment, and execution of information and tasks. It is likely that, in the future, the availability of electrical devices and industrial machinery will be limited to intelligent products equipped with learning and adaptive functions, in addition to speech recognition and synthesis functions. The possibility of employing TDI to other attributes, such as volume, communication velocity, quality, and rhythm, is also conceivable.

[0030] It is thus believed that TDI will be effective as a next generation, post-GUI interface.

[0031] An objective of the TDI system is to provide an interface that is at once easy for humans and practical. Because TDI, too, takes speech as its medium, it can be included as a part of the above-described multi-modal interface. Similarly, inasmuch as TDI adds a speech interface to a variety of things, it can also be included as a part of the above described ubiquitous computing.

[0032] As discussed above, research relating to the interaction between humans and computers has been carried out actively in recent years. It appears, however, that attention is not yet being paid to dialog between things, like that contemplated in the TDI system. One significant advantage to dialog by speech among things is that a human can simultaneously understand that dialog and, where necessary, make revisions and corrections. Thus, in the TDI system, a human can easily be interposed in the communication among things. This point, too, compels the conclusion that the TDI system is amenable to humans.

[0033] The dialog interface system contemplated by the present invention comprises, a speech recognition unit for converting input speech to an input semantic representation, a dialog management unit for outputting an output semantic representation that corresponds to the input semantic representation, based on the input semantic representation obtained by the speech recognition unit, and a speech synthesis unit for receiving the output semantic representation from the dialog management unit, converting the output semantic representation to output speech in which a specific dialog target is designated, and outputting the output speech.

[0034] The system can further comprise an innate operation execution unit for receiving an innate operation command from the dialog management unit and executing a function corresponding to the innate operation command. In this case, the dialog management unit outputs, to the innate operation execution unit, the innate operation command that corresponds to the input semantic representation, based on the input semantic representation obtained by the speech recognition unit.

[0035] The dialog management unit can identify the origin of the input speech, based on the input semantic representation received from the speech recognition unit and can identify the origin of the output speech, based on the input semantic representation received from the speech recognition unit, and can output the output semantic representation and the innate operation command after consideration of the identified origin.

[0036] The speech recognition unit can output a delay command that delays the output of the output speech to the speech synthesis unit, during the time that input speech is being inputted. Also, the dialog management unit can comprise a dialog rules storage unit for storing an aggregate of dialog rules for the input semantic representation and the output semantic representation. Additionally, the dialog management unit outputs at least one of the output semantic representation corresponding to the input semantic representation inputted from the speech recognition unit, based on the dialog rules stored in the dialog rules storage unit, or the innate operation command. In this case, the dialog management unit has functions for making deletions, changes, and additions with respect to the dialog rules stored in the dialog rules storage unit, and dismisses one of the input semantic representations, when one of the successive input semantic representations and the output semantic representations corresponding to the other input semantic representations are identical.

[0037] Similarly, the speech synthesis unit and the innate operation execution unit can synchronize the output of the output speech and the execution of the innate operation by way of a synchronization notification signal.

[0038] The dialog interface apparatus contemplated by the present invention comprises, a speech recognition unit for converting input speech to an input semantic representation, a dialog management unit for identifying an origin of the input speech, based on the input semantic representation obtained by the speech recognition unit, and outputting a corresponding innate operation command based on the identified origin and the input semantic representation, and an innate operation execution unit for executing a function corresponding to the innate operation command.

[0039] Similarly, another dialog interface apparatus contemplated by the present invention comprises, a dialog management unit for outputting an output semantic representation and data specifying the dialog target that should recognize the output semantic representation, and a speech synthesis unit for converting the output semantic representation and the dialog target to expressive output speech, based on the data received from the dialog management unit, and for outputting the output speech.

[0040] The method of the present invention is a method of processing based on a dialog, using a dialog management apparatus, and comprises the steps of: converting input speech to an input semantic representation, generating an output semantic representation corresponding to the input semantic representation, based on the input semantic representation, converting the output semantic representation to output speech, for which specific dialog target is designated, and outputting the output speech.

[0041] This method may also be capable of comprising the steps of, generating an innate operation command corresponding to an input semantic representation, based on the input semantic representation, and receiving the innate operation command and executing an operation corresponding to the innate operation command. Further, this method may comprise the steps of, identifying an origin of the input speech, based on the input semantic representation, and outputting the output semantic representation upon consideration of the identified origin.

[0042] Likewise, this method may comprise the steps of, identifying the origin of the input speech, based on the input semantic representation, and generating the innate operation command upon consideration of the identified origin. Further, this method may comprise the step of outputting a delay command that delays the output of the output speech, while input speech is being inputted.

[0043] This method may also comprise the step of outputting, based on the stored dialog rules, at least one of the output semantic representation corresponding to the input semantic representation or an innate operation command. Furthermore, this method may comprise the step of dismissing one of the input semantic representations, when one of the successive input semantic representations and the output semantic representations corresponding to the other input semantic representations are identical. Also, this system may comprise the step of synchronizing the output of the output speech and the execution of the innate operation, by way of a synchronization notification signal.

[0044] Another method contemplated by the present invention comprises the steps of, converting the input speech to an input semantic representation, identifying the origin of the input speech, based on the input semantic representation, outputting a corresponding innate operation command based on the input semantic representation, and executing an operation corresponding to the innate operation command.

[0045] Yet another method contemplated by the present invention comprises the steps of, outputting the output semantic representation and the data specifying the dialog target that is to recognize the output semantic representation; and converting the output semantic representation and the dialog target to expressive output speech and outputting that output speech.

[0046] The storage medium contemplated by the present invention is a computer-readable storage medium containing a program for causing a computer to execute a processing method, based on a dialog. Here, the “processing methods” can correspond to the method contemplated by the present invention. The storage medium of the present invention can be realized as an internal computer storage medium comprising, by way of illustration, ROM, RAM, or a hard drive, or as an external storage medium, of which CD-ROM, MD, and a floppy disk are representative.

[0047] FIG. 1 discloses the operation of a conventional interface system.

[0048] FIG. 2 discloses the operation of the TDI system.

[0049] FIG. 3 discloses the operating features of the TDI system.

[0050] FIG. 4 discloses the configuration of the interface system according to the present invention

[0051] FIG. 5 discloses the functional structure of an illustrative embodiment of the TDI system according to the present invention.

[0052] FIG. 6 discloses the configuration of the TDI system according to the present invention.

[0053] FIG. 7 discloses a hardware configuration for the TDI system according to the present invention.

[0054] FIG. 8 discloses the details of the hardware configuration for the TDI system according to present invention.

[0055] FIG. 9 discloses the integrated hardware configuration for the TDI system according to the present invention.

[0056] FIG. 10 discloses the speech recognition processing order according to the speech recognition system.

[0057] FIG. 11 is a flowchart depicting the first process of the speech recognition system.

[0058] FIG. 12 is a flowchart depicting the second process of the speech recognition system.

[0059] FIG. 13 discloses the dialog rules stored in the dialog rules database.

[0060] FIG. 14 discloses the processing order of the dialog management system.

[0061] FIG. 15 is a flowchart depicting the first process of the dialog management system.

[0062] FIG. 16 is a flowchart depicting the second process of the dialog management system.

[0063] FIG. 17 is a flowchart depicting the third process of the dialog management system.

[0064] FIG. 18 discloses the speech synthesis processing order according to the speech synthesis system.

[0065] FIG. 19 is a flowchart depicting the first process of the speech synthesis system.

[0066] FIG. 20 is a flowchart depicting the second process of the speech synthesis system.

[0067] FIG. 21 discloses the processing order of the innate operation system.

[0068] FIG. 22 is a flowchart depicting the processes of the innate operation system.

[0069] FIG. 23 shows an illustrative application of the present invention to an entrance light.

[0070] FIG. 24 shows the dialog rules relating to the switching on of the entrance light.

[0071] FIG. 25 shows an illustrative application of the present invention to clocks.

[0072] FIG. 26 shows the dialog rules relating to the clocks.

[0073] FIG. 27 shows an illustrative application of the present invention to a bath.

[0074] FIG. 28 shows the dialog rules relating to the bath.

[0075] FIG. 29 shows an illustrative application of the present invention to an informator, a video recorder, and a telephone.

[0076] FIG. 30 shows the dialog rules relating to the informator, video recorder, and telephone.

[0077] FIG. 31 shows the dialog rules relating to the informator, video recorder, and telephone.

[0078] FIG. 32 shows an illustrative application of the present invention to a closet.

[0079] FIG. 33 shows the dialog rules relating to the closet.

[0080] FIG. 34 shows an illustrative application of the present invention to cleaning robots.

[0081] FIG. 35 shows the dialog rules relating to the cleaning robots.

[0082] FIG. 36 shows an illustrative application of the present invention to a guide-robot and a navigator.

[0083] FIG. 37 shows the dialog rules relating to the guide-robot and the navigator.

[0084] FIG. 38 shows the dialog rules relating to the guide-robot and the navigator.

[0085] The present invention provides a practical system using the above-described TDI to enable interaction by speech not only between a human and a thing, but also among fellow things.

[0086] FIG. 4 is the principle configuration of the present invention. In this figure, a speech recognition system 1 converts input speech to an input semantic representation. A dialog management system 2 outputs an output semantic representation corresponding to an input semantic representation received from the speech recognition system 1 and an innate operation command, based on the dialog rules. A speech synthesis system 3 receives the output semantic representation from the dialog management system 2, converts that output semantic representation to output speech, and outputs that output speech. An innate operation system 4 receives the innate operation command from the dialog management system 2 and executes that operation.

[0087] In the present invention, the output speech responsive to the input speech is produced based on the dialog rules, and a dialog is accomplished between a human and a thing, as well as among fellow things. A prescribed process responsive to the content of the dialog, or a prescribed operation, is executed.

[0088] The simultaneous initiation of dialog can be avoided in the present system, by establishing in the speech recognition system 1 a function that sends out a delay notification to the speech synthesis system 3 when speech is being inputted.

[0089] Similarly, it is possible to make the system autonomous, by establishing in the dialog management system 2 functions for changing, adding, and deleting dialog rules pursuant to commands from other humans or things, or from the system itself.

[0090] Further, it is possible to avoid identical conversations, by establishing in the dialog management system 3 a function that dismisses one of two input semantic representations, when one of two successive input semantic representations and the output semantic representation corresponding to the other input semantic representation are identical.

[0091] Similarly, it is possible to carry out synchronously the production of speech and the execution of an innate operation, by establishing in the speech synthesis system 3 and the innate operation system 4 a function that mutually outputs a synchronization notification.

[0092] The preferred embodiment of the present invention is hereunder explained. Here, any one thing in which TDI is employed (for example, computers, robots, industrial machinery, electrical appliances, daily goods, and the like, hereinafter, to make contrast with “a human” are referred to as “things”.) is referred to as a TDI system, in order to draw a distinction between such things and humans.

[0093] [The Functional Configuration and Processing Order of the TDI System]

[0094] FIG. 5 discloses the functional configuration of the TDI system of this illustrative embodiment. This figure represents an instance in which the TDI is employed in one thing.

[0095] As is shown in this figure, the TDI system is made up of three systems, namely, a speech recognition system 1, a dialog management system 2, and a speech synthesis system 3. Further, the TDI system also comprises, as necessity demands, an innate operation system 4 for executing an operation, or operations, peculiar to that thing. This innate operation system 4 may include, illustratively, a recording function and a playback function, in the case of a video recorder.

[0096] FIG. 6 discloses a summary of the hardware configuration for the TDI system.

[0097] As is shown in this figure, a speech recognition apparatus 11, a dialog management apparatus 12, a speech synthesis apparatus 13, and an innate operation apparatus 14 (corresponding respectively to the speech recognition system 1, the dialog management system 2, the speech synthesis system 3, and the innate operation system 4), which collectively constitute the TDI system, are coupled by a bus 15, to which has been added a bus controller 16 to control the bus 15. Further, the speech recognition apparatus 11, the dialog management apparatus 12, the speech synthesis apparatus 13, and the innate operation apparatus 14 are configured respectively according to hardware like that which is shown for illustrative purposes in FIG. 7. Specifically, each of the preceding systems comprises an input/output apparatus 21, an operation apparatus 22, a storage apparatus 23, and an interface 25. These are linked by means of an internal bus 26. To these, also, is added a bus controller 24 to control the internal bus 26.

[0098] The input/output apparatus 21 comprises a microphone, a speaker, and a signal processing apparatus. The operation apparatus 22 is, illustratively, a CPU, and may comprise a bus controller 24. The storage apparatus 23 is, illustratively, a semiconductor memory apparatus or a magnetic disk apparatus. The interface 25 comprises a buffer, which is required when joining the buses.

[0099] Specifically, the TDI system hardware configuration shown in FIG. 6 can be rewritten, as is shown in FIG. 8. Realistically, it is possible to integrate several of the same hardware parts into a single element, according to performance and price. The hardware configuration for the most highly integrated embodiment of the TDI system is as depicted in FIG. 9.

[0100] The operation of the speech recognition system 1, the dialog management system 2, the speech synthesis system 3, and the innate operation system 4 are hereunder explained.

[0101] (1-1) The Speech Recognition System

[0102] The speech recognition system 1 receives input speech from a distinct entity consisting of a human or a TDI system and converts it, through a number of the following processes, to an input semantic representation, and, ultimately, transmits that input semantic representation to the dialog management system.

[0103] The input speech is stored in the buffer and, thereafter, is converted to an input sequence of phonemes, an input sequence of characters, and an input semantic representation, in first-in-first-out (FIFO) order.

[0104] Here, a phoneme is the basic unit of speech. Several inputted speech is broken down into time series of phonemes. The sequence of phonemes is further converted to a character sequence, that character sequence is structurally analyzed, and then converted to a formal semantic representation. The input semantic representations are ultimately transmitted in order to the dialog management system. The conversion from speech to semantic representation is the same as the conventional technique (see, for example, Sadaoki Furui “Speech Recognition”, (in Japanese), The Journal of the Institute of Electronics, Information and Communication Engineers, vol. 78, no. 11, pp. 1114-1118, November, 1995).

[0105] While the speech recognition system 1 is receiving the input speech, a delay notification is sent out from the speech recognition system 1 to the speech synthesis system 3. The speech synthesis system 3 brings about a delay in the transmission of the output speech by issuing the command “delay output during input speech”. This is done to ensure, at a minimum, that one thing does not attempt to initiate a dialog while someone or some other thing is already in the process of having a dialog.

[0106] FIG. 10 shows the processing order for the speech recognition system 1. FIGS. 11 and 12 depict the flow for that processing.

[0107] In FIGS. 10 through 12, the in-voice_u, in-phonemes_u, in-character_u, and in-speech_urepresent respectively the u^thinput speech, the input sequence of phonemes, the input sequence of characters, and the input semantic representation. The sr-in-buffer_topand sr-in-buffer_bottomrepresent the top and the bottom, respectively, of the input buffer, and the sr-out-buffer_topand sr-out-buffer_bottomrepresent respectively the top and the bottom of the output buffer. The input buffer and the output buffer can be combined in one buffer. The processes a and b shown in FIG. 10 can be carried out in parallel.

[0108] FIG. 11 is a flowchart depicting the speech recognition processing that is represented in the process a of FIG. 10.

[0109] In process a (sr-a), while speech is being inputted from a human or from a thing (step S11:T), the speech recognition system 1 continues to output to the speech synthesis system 3 an out-wait signal (step S12) and executes a “push-in” process (step S13) that pushes the in-voice_uto the bottom of the input buffer for the speech recognition system 1.

[0110] FIG. 12 is a flowchart depicting the speech recognition processing shown in the process b in FIG. 10.

[0111] When the in-voice_uis at the top of the input buffer (step S21:T), process b (sr-b) outputs the input speech (in-voice_u) out from the input buffer (step S22), converts the input speech to an input sequence of phonemes (in-phonemes_u) (step S23), and further converts the input sequence of phonemes to an input sequence of characters (in-character_u) (step S24). The input sequence of characters (in-character_u) is further converted to an input semantic representation (in-speech_u) (step S25) and pushed in to the bottom of the output buffer (step S26). The preceding processes are repeated during the interval in which input speech is being inputted.

[0112] (1-2) The Dialog Management System

[0113] The dialog management system 2 receives the input semantic representation from the speech recognition system 1, searches in the dialog database for the appropriate rules respecting the input semantic representation, outputs to the speech synthesis system 3 the output semantic representation based on the input semantic representation, and issues to the innate operation system 4 an order for an innate operation. As is hereinbelow explained, the dialog management system 2 comprises a function for updating and storing date, time, and like environmental variables as internal information. Further, the dialog management system 2 comprises a function for changing, adding, or deleting dialog rule according to a command issued by another entity, by a human or a thing, or by the dialog management system 2 itself.

[0114] In this illustrative embodiment, all conversational semantic representations are described as follows.

[0115] speech=[(from-whom), to-whom, message]

[0116] Here, “from-whom”, “to-whom”, and “message” mean respectively: by whom or by what the conversation is initiated; to whom or to what the conversation is directed; and to what kind of content the conversation pertains.

[0117] “From-whom” is enclosed in parentheses because sometimes the speaker does not positively communicate that it is the speaker who, or which, initiates the dialog. For this reason, it is assumed for purposes of this explanation, that, if necessary, the TDI comprises a function that is able to identify from speech quality, for example, who or what is the initiator of that conversation (see “Speech Recognition”, cited above).

[0118] On the other hand, when “to-whom” is not positively specified in the dialog, this explanation assumes the “to-whom” content to be “anyone”. It is possible also to make the “to-whom” content “everyone”, in which case, the speaker must positively make this designation.

[0119] According to the above explanation, the i^thinput semantic representation is in the discussion below described as follows.

[0120] in-speech_i=[(in-from-whom_i), in-to-whom_i, in-message_i]

[0121] Likewise, the j^thoutput semantic representation is described as follows.

[0122] out-speech_j=[(out-from-whom_j), out-to-whom_j, out-message_j]

[0123] Next, the specification defines the dialog rules. In this illustrative embodiment, the k^thdialog rule in the dialog rules database is described as shown in FIG. 13.

[0124] The “name-of-thing” in FIG. 13 is the name of the thing in which the TDI is employed. The symbol “→” represents a logical deduction. The dialog rules database is defined as the set of dialog rules shown in FIG. 13.

[0125] Specifically, as is shown in FIG. 13, the set of dialog rules defining the output semantic representation (out-speech_k) and the innate operation (innate-operation_k), with respect to an input semantic representation (in-speech_k), is stored in the dialog rules database.

[0126] FIG. 14 discloses the processing order of the dialog management system 2, and FIGS. 15 through 17 show the processing flow therefor.

[0127] In FIGS. 14 through 17, the symbols “”, “”, and “|” represent respectively (AND), (OR), and (NOR). The processes 1, 2, and 3 in FIGS. 15 through 17 are carried out in sequence.

[0128] FIGS. 15, 16, and 17 are flowcharts that depict respectively the dialog management system disclosed in the processing orders 1, 2, and 3, shown in FIG. 14.

[0129] As shown in FIG. 15, when an input semantic representation (in-speech_i) exists in the output buffer of the speech recognition system 1 (step S31:T), the input semantic representation (in-speech_i) is output from the top of the output buffer (step S32) in processing order 1 (dm-1).

[0130] Next, in processing order 2 (dm-2), also shown in FIG. 16, the dialog rules (dialog-rule_k) are searched (step S33), and the output semantic representation (out-speech_i) corresponding to the input semantic representation (in-speech_i), is determined based on the dialog rules (step S34). Further, an innate operation (innate-operation_i) is established (step S35).

[0131] Here, the “in-to-whom” can be any one of “name-of-thing” (i.e., the thing in which the TDI is employed), “anyone”, “everyone”, or “not-specified”. Similarly, the “out-from-whom_i” (the thing or human initiating the speech) is the “name-of-thing” (i.e., the thing in which the TDI is employed), and the “out-to-whom_i” (the counterpart with respect to whom or to which the dialog is initiated) is taken to be “out-to-whom_k”, where out-to-whom_kis specified, and is taken to be in-from-whom_i, where out-to-whom_kis not specified.

[0132] Next, in processing order 3 (dm-3), shown in FIG. 17, the out-speech_iis sent out to the bottom of the speech synthesis system 3 (step S37), when the next input semantic representation (in-speech_i+1) is not in the top of the output buffer for the speech recognition system 1, or when the in-to-whom_iis neither anyone nor not specified and the in-message_i+1is not equal to the candidate out-message_ifor the output semantic representation (step S36:T). Additionally, an innate-operation_iis sent out to the bottom of the buffer for the innate operation system 3 (step S38).

[0133] In step S36, the candidate message_ifor the i^thoutput semantic representation is compared with the next output semantic representation, namely, the i+1_thoutput semantic representation, which is recognized in the speech recognition system 1, and, if both of the semantic representations are the same, the input semantic representation in-message_i+1is dismissed. Thus, the duplication by a single thing or by a plurality of things of a conversation with the same content is avoided.

[0134] (1-3) The Speech Synthesis System

[0135] The speech synthesis system 3 possesses the converse functions of the speech recognition system 1. The speech synthesis system 3 receives the output semantic representation from the dialog management system 2, through several processes converts the output semantic representation to output speech, and, ultimately, transmits that output speech to one or plural humans and/or things furnished with the TDI.

[0136] The output semantic representation is converted to an output sequence of characters, an output sequence of phonemes, and output speech and is stored in the buffer. The output speech ultimately obtained is transmitted, in FIFO order, to one or plural humans and/or things furnished with a TDI. Furthermore, the conversion from the semantic representation to the speech is accomplished according to the conventional method (see, for example, Kazuo Nakata, “Speech Synthesis and Coding”, (in Japanese), The Journal of the Institute of Electronics, Information and Communication Engineers, vol. 78, no. 11, pp. 1119-1124, November, 1195).

[0137] FIG. 18 discloses the processing order of the speech synthesis system 3. FIGS. 19 and 20 show the processing therefor.

[0138] The “out-speech_v”, “out-characters_v”, “out-phonemes_v” and “out-voice_v” in FIGS. 18, 19, and 20 represent respectively the v^thoutput semantic representation, the output sequence of characters, the output sequence of phonemes, and the output speech. The “ss-in-buffer_top” and “ss-in-buffer_bottom” represent respectively the top and the bottom of the input buffer for the speech synthesis system 3, and the “ss-out-buffer_top” and “ss-out-buffer_bottom” represent respectively the top and the bottom of the output buffer. The input buffer and the output buffer can be combined in one buffer.

[0139] The out-voice_qis in an output standby state while the out-wait-signal (waiting signal) is being received from the speech synthesis system 2. Similarly, the output of the out-voice_qis synchronized with the execution of the innate-operation_qof the innate operation system 4. The process sequence a and the process sequence b, both shown in FIG. 18, can be carried out in parallel.

[0140] FIG. 19 is a flowchart depicting the speech synthesis processes shown in the processing order a of FIG. 18.

[0141] In processing order a (ss-a), if there is an output semantic representation (out-voice_v) in the top of the buffer for the speech synthesis system 3 (step S41:T), that output semantic representation (out-speech_v) is popped out from the buffer (step S42) and converted to a sequence of output characters (out-characters_v) (step S43). The sequence of output characters (out-character_v) is then converted to a string of output phonemes (out-phonemes_v) (step S44). The string of output phonemes (out-phonemes_v) is converted to output speech (out-voice_v) (step S45) and pushed into the bottom of the buffer (step S46). The process then returns to step S41.

[0142] FIG. 20 is a flowchart that represents the speech synthesis system processing disclosed in the processing order b and the processing order c of FIG. 18.

[0143] In processing order b (ss-b), if there is output speech (out-voice_q) in the top of the buffer for the speech synthesis system 3, and if a waiting signal (out-wait-signal) has not been received from the speech recognition system 1 (step S47:T), the output speech (out-voice_q) is popped out from the buffer (step S48).

[0144] In processing order c (ss-c), the system continues to send a synchronization signal (out-synchronize-signal) to the innate operation system 4 (step S49), and when the synchronization signal is received from the innate operation system 4 (step S50:T), the output voice (out-voice_q) is outputted to a human or to a thing (step S51).

[0145] (1-4) The Innate Operation System

[0146] The innate operation system 4 receives the innate operation (specifically, the command therefor) from the dialog management system 2 and executes that innate operation. Here, an “innate operation” would, in the illustrative instance of a video tape recorder, correspond to the functions of record and playback.

[0147] The processing order for the innate operation system 4 is disclosed in FIG. 21 and can be described as is shown in FIG. 21.

[0148] Here, the “innate-operation_q” expresses the q^thinnate operation. The “io-out-buffer_top” and “io-out-buffer_bottom” represent respectively the top and the bottom of the output buffer for the innate operation system 4. The execution of the innate operation_qis synchronized with the output of the out-voice_q.

[0149] FIG. 22 is a flowchart describing the processes of the innate operation system 4 shown in the processing order b and the processing order c of FIG. 21.

[0150] In processing order b (io-b), first, if there is an innate operation (innate-operation_q) in the buffer for the innate operation system 4 (step S62:T), that innate-operation_qis popped out from the buffer (step S63). If a synchronization signal (out-synchronize-signal) has in processing order c (io-c) been received from the speech synthesis system 3 (step S64:T), the synchronization signal is sent to the speech synthesis system 3 (step S65), and the innate operation (innate-operation_q) is executed (step S66). These processes are carried out repetitively.

[0151] [The Features of the TDI System]

[0152] Next, the disclosure explains the features of the TDI system contemplated by the present invention.

[0153] (2-1) The Specification and Non-specification of a Dialog Target

[0154] In the TDI system, it is possible to undertake both dialog with a specified target and dialog without a specified target.

[0155] An example of the former would be “reserve recording of program ABC!”, in the case of a video recorder. Illustrative of the latter, on the other hand, would be “(someone), tell me the time!”. In this case, those things possessed of compliant dialog rules would become response candidates. According to the “redundant conversation avoidance ” feature described above in (1-2), however, only those things with the fastest reactions would reply.

[0156] A dialog in which no target is specified is one type of a so called “broadcast.” This kind of dialog takes advantage of the fact that speech has a spatial expanse.

[0157] (2-2) The Avoidance of Simultaneous Dialog Initiation

[0158] The TDI system of the present invention is arranged to ensure at a minimum that, while someone or something is in the process of initiating a dialog, no other thing will initiate a dialog. As explained above in (1-1) and (1-3), this is accomplished by means of the buffer function in both the speech synthesis system and the innate operation system, and by having the speech recognition system command the speech synthesis system to delay output during any interval in which the speech recognition system is receiving input speech. The speech synthesis system is thus set to delay the transmission of the output speech.

[0159] (2-3) The Avoidance of Redundant Dialog

[0160] The TDI system of the present invention is set to prevent the duplication of conversations featuring the same content by a single thing or a plurality of things. This is accomplished as follows. As explained above in (1-2), the i^thoutput semantic representation candidate message_iis compared with the next input semantic representation derived in the speech recognition system, namely, the i+1^thinput semantic representation “in-message_i+1”, and, if it is the same, it is dismissed.

[0161] (2-4) The Connection with Other Interfaces and Innate Operations

[0162] The role of the dialog management system is not limited to the transmission of an established output semantic representation to the speech synthesis system and the initiation of a dialog according to speech. The dialog management system also transmits an established innate operation to the innate operation system, indicates the substance of the response, and executes the operations peculiar to a given thing. Specifically, the TDI system is responsible for the verbal aspects of distinct interfaces. Non-verbal aspects like visual aspects, for instance, or operations that are innate to a given thing, or both, are coordinated and realized in the innate operation system.

[0163] (2-5) The Initiative of a Thing

[0164] In the TDI system of the present invention, it is possible to issue to a thing itself a question or a command in the form of a response from a distinct human or a distinct thing. Specifically, it is possible for a certain thing to issue to a human and/or other things a question or a command. Thus, a thing furnished with the TDI system possesses initiative. “Initiative”, as herein used, means that a thing approaches others by its own intension.

[0165] (2-6) The Autonomy of a Thing

[0166] In the TDI system of the present invention, it is possible to direct, in the form of a command from either a human or a thing itself, or from a distinct human or a distinct thing, the addition of new dialog rules, the modification of existing dialog rules, or the deletion of obsolete dialog rules. Thus, a thing furnished with the TDI system possesses autonomy. Here “autonomy” means that a thing is autonomous, with respect to the outside world, while undertaking change within itself.

[0167] (2-7) The Simplicity of a Thing

[0168] Even where a plurality of unanticipated speech commands are inputted into the TDI system belonging to a certain thing, or where speech command intended for a distinct thing is inputted into the TDI system of that certain thing, the system simply renders no reaction, without producing an error. Thus, in the sense that the system reacts only to anticipated dialog rules, the system is simple.

[0169] (2-8) The Decentralization of the System

[0170] The groups of things in which the TDI system is employed are entirely decentralized. For this reason, the system is extremely flexible with respect to an increase or a decrease in participating humans or things. Although master/servant relationships are not present among the things, it is nevertheless feasible and simple to set the dialog rules for each thing so that a response is given only with respect to a command from a specific human or a specific thing.

[0171] (3) An Illustrative Application

[0172] Below, the disclosure illustrates particularized applications of the TDI system contemplated by the present invention. In order to simplify this explanation, without forfeiting generality, the from-whom items for both the in-speech and out-speech are omitted.

[0173] (3-1) The Turning On of an Entrance Light

[0174] The disclosure considers an entrance light that, as is depicted in FIG. 23, gives answers as follows.

[0175] Entrance Light

[0176] Human: Entrance-light, turn-on!=Entrance, turn-on-light!

[0177] Human: Everyone, turn-on-light!=All lights, turn-on!

[0178] Human: Everyone, I'm home.

[0179] The dialog rules for this entrance light can be described as is shown in FIG. 24.

[0180] Specifically, it is possible to cause the entrance light to turn on in response to speech, by storing in the dialog rules database the dialog rules defining a peculiar function (innate-operation: in this case, “turn-on-light” is the function that turns on the light) with respect to the input semantic representation (in-speech) as is shown in FIG. 24.

[0181] Further, the dialog target may be either specified, as in the above illustration, or not specified (see (2-1) above).

[0182] (3-2) A Clock

[0183] The disclosure next considers a clock that, as is shown in FIG. 25, gives answers as follows.

[0184] Clock

[0185] Human: What-time-is-it-now?

[0186] Clock-1: (Sir,) 12:34 a.m.

[0187] (Clock-2): (Sir,) 12:34 a.m.)

[0188] Clock-3: (Sir,) 12:43 a.m.

[0189] Human: Clock-3, Synchronize-to Clock-1

[0190] Clock-3: Clock-1, teach-time!

[0191] Clock-1: Clock-3, Change-time-to 12:36!

[0192] Clock-3: (Sir,) thank you.

[0193] In the above conversation, a human asks a clock for the time, and Clock-1 and Clock-3 inform the human of the time by way of speech (because the response that Clock-2 would have given is the same as that given by Clock-1, Clock-2 does not produce speech). The human commands Clock-3 to adjust its time. Clock-3 asks Clock-1 for the time and coordinates its time with that of Clock-1.

[0194] The dialog rules for this clock can be described as is shown in FIG. 26.

[0195] It is possible to realize a “speech response” with respect to a question for the present time, as well as a “time inquiry” function and a “time coordination” function in response to an instruction for time coordination, by storing in the dialog rules database, as dialog rules, an output semantic representation (out-speech) with respect to an input semantic representation (in-speech), as well as an innate operation.

[0196] Here, “$Time” and “$clock”, to which the dollar-sign is affixed represent variables. “$Time” is an environmental variable that represents the present time. The value of the time is updated automatically and stored as an internal information (see FIG. 2 and accompanying text, above).

[0197] In this case, because the time provided by Clock-2 is the same as that provided by Clock-1, care must be taken to ensure that the speech of Clock-2 is dismissed. Thus, it is possible in the case of this clock to avoid a redundant conversation (see (2-3)). (3-3) The Filling of a Bath

[0198] The disclosure now considers a bath that, as is shown in FIG. 27, gives answers as follows.

[0199] Bath

[0200] Human: Bath, Prepare!

[0201] Human:=Bath, Pour-hot-water!

[0202] The dialog rules for this bath can be described as is shown in FIG. 28.

[0203] It is possible to realize a “buzzer sounding” function and a “water pouring” function with respect to an instruction for the filling of a bath, by storing in the dialog rules database as the dialog rules, a peculiar function (innate-operation) with respect to the input semantic representation (in-speech) as is shown in FIG. 28.

[0204] Here, “beep-buzzer ( )” and “pour-hot-water ( )” represent functions of innate operations. In the case of this bath, a buzzer is sounded (another interface) and the bath is filled with hot water (an innate operation) according to the commands “prepare !” and “pour-hot-water!”. Thus, it is possible in the case of this bath to co-operate with another interface and an innate operation (see (2-4)).

[0205] (3-4) An Informator, Video Recorder, and Telephone

[0206] The disclosure next considers an informator, video recorder, and telephone that, as is shown in FIG. 29, give answers as follows. Here, an informator means an information terminal, like an electronic secretary.

[0207] Informator

[0208] Video Recorder

[0209] Telephone

[0210] Human: Informator, search “tonight's television-programs!”

[0211] Informator: (Sir,) “SMILE & SMILE from 10 p.m. to 11 p.m. on channel 8.”

[0212] Human: Informator, record on video “tonight's SMILE & SMILE”!

[0213] Informator: (Sir,) sure. Video recorder, record on video “SMILE & SMILE from 10 p.m. to 11 p.m. today on channel 8”!

[0214] Video: (Sir,) sure.

[0215] Human: Informator, search “Venus concert Tokyo”!

[0216] Informator: (Sir,) Venus concert Tokyo on Sunday, January 1, starting at 6:00 p.m., at Tokyo-Big-Stadium.

[0217] Human: Informator, Call “Venus concert Tokyo counter”

[0218] Informator: (Sir,) sure. Phone, dial 01-2345-6789!

[0219] Phone: (Informator,) sure.

[0220] (Operator): Hello, this is XYZ Ticket Service. Can I help you?

[0221] Human: Yes, please. I would like to purchase tickets to the Venus concert on January 1.

[0222] (Operator): OK. Which would you like, regular seats or special ones?

[0223] Human: Special, please.

[0224] (Operator): How many seats would you like?

[0225] Human: Two, please.

[0226] (Operator): OK. Would you tell me your name, phone number, and credit card number, please?

[0227] Human: My name is Taro Yamada. My phone number is 98-7654-3210, and the credit card number is . . . just a moment please.

[0228] Human: Informator, search “my visa card number”!

[0229] Informator: (Sir,) 11335577.

[0230] Human: Sorry, my card number is 11335577.

[0231] (Operator): OK. We will send two tickets within one week. Thank you very much for your order. Enjoy the concert!

[0232] Human: Thank you.

[0233] The preceding conversation illustrates an instance in which a “human” directs an “informator” to search for a television program and, after instructing a “video recorder” to record the program, directs the “informator” to search for a concert and to identify the appropriate ticket counter. The “informator” then directs a “telephone” to contact the ticket counter based on the immediately preceding instruction. Half of the foregoing conversation consists principally of telephone responses between the ticket counter operator and the “human”. (Except that, in the middle of the conversation, the “human” asks the “informator” for a credit card number.)

[0234] The dialog rules for the informator, video recorder, and telephone can be described as is shown in FIGS. 30 and 31.

[0235] By saving in the informator dialog rules database, as dialog rules, an output semantic representation (out-speech) with respect to an input semantic representation (in-speech) and an innate operation (innate-operation), as is disclosed in FIG. 30, it is possible to issue instructions for: a search operation with respect to, illustratively, television programs, concerts, and telephone numbers; a search result report; a storage operation with respect to the search result; a response to a video recording instruction for a television program; the recording of a television program by a video recorder; a search for a telephone number with respect to a dialing instruction; and the dialing of a telephone.

[0236] Similarly, by storing in the video dialog rules database, as dialog rules, an output semantic representation (out-speech) with respect to an input semantic representation (in-speech) and an innate operation (innate-operation), as is shown in FIG. 31, it is possible to perform a recording operation for a television program in response to a recording instruction.

[0237] Likewise, by storing in the telephone dialog rules database, as dialog rules, an output semantic representation (out-speech), with respect to an input semantic representation (in-speech), and an innate operation (innate-operation), as is shown in FIG. 31, it is possible to perform a dialing operation in response to a dialing instruction.

[0238] Here, items to which the dollar sign is attached as a prefix, such as “$key” and “$content”, represent variables. Items enclosed in quotation marks, such as “phone-no.”, represent constants.

[0239] The informator possesses initiative (see (2-5)), in the sense that it is able to command a video recorder to undertake recording.

[0240] (3-5) A Closet

[0241] The disclosure next considers a closet that, as is shown in FIG. 32, gives the following answers.

[0242] Closet

[0243] Human: Closet-1, keep “black-coat”!

[0244] Closet-1: (Sir,) sure.

[0245] Human: Closet-2, keep “ski-wear”!

[0246] Closet-2: (Sir,) sure.

[0247] Human: (Anyone) where -is “black-coat”?

[0248] Closet-1: (Sir,) Closet-1 keeps “black-coat”.

[0249] The dialog rules for this closet can be described as is shown in FIG. 33.

[0250] Here, the “add-dialog-rule [. . . ]” and “delete-dialog-rule [. . . ]” in the above dialog rules correspond respectively to high-level rules for adding and deleting dialog rules.

[0251] By storing in the dialog rules database, as dialog rules, an output semantic representation (out-speech) responsive to an input semantic representation (in-speech) and the high-level rules mentioned above, a dialog rule corresponding to a voiced-description, by a human, of an article of clothing put into the closet is added to the dialog rules database, and, when the human asks the closet where the article of clothing is, the closet is able to respond to the human.

[0252] Thus, in the sense that the closet is able by itself to add and to delete dialog rules, the closet possesses autonomy (see (2-6)).

[0253] (3-6) A Cleaning Robot

[0254] The disclosure next considers a cleaning robot that, as is shown in FIG. 34, answers as follows. The cleaning robots depicted in FIG. 34 consist of one chief robot, which apportions the work, and any number of cleaning robots, which actually do the cleaning.

[0255] Cleaning-Robot

[0256] Human: Chief-Robot, set-schedule-as 10:00-11:00 a.m., floors 15-18!Chief-Robot, set schedule-as 11:00 a.m.-12:00 p.m., floors 1-4!

[0257] Chief-Robot: Cleaning-Robots-1&2, set-schedule-as 10:00-11:00 a.m., floors 15-16!

[0258] Cleaning-Robot-1: (sir,) sure.

[0259] Cleaning-Robot-2: (sir,) sure.

[0260] Chief-Robot: Cleaning-Robots-1&2, set-schedule-as 11:00 a.m.-12:00 p.m., floors 1-2!

[0261] Cleaning-Robot-1: (sir,) sure.

[0262] Cleaning-Robot-2: (sir,) sure.

[0263] Chief-Robot: Cleaning-Robots-3&4, set-schedule-as 10:00-11:00 a.m., floors 17-18!

[0264] Cleaning-Robot-3: (sir,) sure.

[0265] Cleaning-Robot-4: (sir,) sure.

[0266] Chief-Robot: Cleaning-Robots-3&4, set-schedule-as 11:00 a.m.-12:00 p.m., floors 3-4!

[0267] Cleaning-Robot-3: (sir,) sure.

[0268] Cleaning-Robot-4: (sir,) sure.

[0269] The preceding conversation illustrates an instance in which a human orders a chief robot to clean floors 15-18 from between 10:00 and 11:00 a.m., and floors 1-4 from between 11:00 a.m. and 12:00 p.m. Receiving these orders, the chief robot allocates the work among the cleaning robots 1-4.

[0270] The dialog rules for these cleaning robots can be described as is shown in FIG. 35.

[0271] By storing in the dialog rules database for the chief robot, as dialog rules, an output semantic representation (out-speech) responsive to an input semantic representation (in-speech) and an innate operation (innate-operation), as is shown in FIG. 35, the chief robot is able to allocate work, in response to a cleaning command issued by a human to the chief robot, and to order the cleaning robots to perform the work.

[0272] Similarly, by storing in the dialog rules database for the cleaning robots, as dialog rules, an output semantic representation (out-speech) responsive to an input semantic representation (in-speech) and an innate operation (innate-operation), as is shown in FIG. 35, it is possible to cause the cleaning robots to perform work according to a command concerning the allocation of work by the chief robot.

[0273] (3-7) A Guide-Robot and a Navigator

[0274] The disclosure next considers a guide-robot and a navigator that, as is shown in FIG. 36, give the following answers. Here, a “guide robot” refers to a robot situated in a visitors bureau, for example, that provides information concerning, illustratively, scenic spots and places of historical interest. The term “navigator”, as herein used, is a type of small informator furnished with a GPS (global positioning system), or like system, and, in particular, has functions for providing information concerning routes and specific locations. In this case, the navigator is in the possession of a human, as is shown in FIG. 36.

[0275] Guide-Robot

[0276] Navigator

[0277] Human: Guide-Robot, teach “silent-temple”!

[0278] Guide-Robot: (Sir,) how-about “Ginkakuji-Temple”?

[0279] Human: Navigator, get-data-for “Ginkakuji-Temple”!

[0280] Navigator: (Sir,) sure. (Anyone,) set-address-to 111.222.333.444! (Sir,) sure. (Anyone,) send-data-for “Ginkakuji-Temple”!

[0281] Guide-Robot: (Sir,) receive-data-for “Ginkakuji-Temple”!

[0282] Navigator: Thank you.

[0283] Guide-Robot: (Sir,) you are welcome.

[0284] Human: Thank you.

[0285] Guide-Robot: (Sir,) you are welcome.

[0286] The preceding conversation illustrates an instance in which: a human addresses to a guide-robot an inquiry concerning “silent-temple”; “Ginkakuji-Temple” is recommended by the guide-robot; the human requests a navigator to collect data on the “Ginkakuji-Temple”; and the navigator relays its own address (111. 222. 333. 444) to and receives the data for the “Ginkakuji-Temple” from the guide-robot.

[0287] The dialog rules for the guide-robot can be described as is shown in FIGS. 37 and 38.

[0288] By storing in the dialog rules database for the guide-robot, as dialog rules, an output semantic representation (out-speech) with respect to an input semantic representation (in-speech) and an innate operation (innate-operation), as is shown in FIG. 37, it is possible to search for a site responsive to a question concerning a tourist site and to answer that question. It is further possible to store the address of, illustratively, the navigator, and to transmit the data for the tourist site relative to that address.

[0289] Similarly, by storing in the dialog rules database for the navigator, as dialog rules, an output semantic representation (out-speech) responsive to an input semantic representation (in-speech) and an innate operation (innate-operation) it is possible to relay one's address and forward data in response to an instruction for data collection and to cause the navigator to receive that data, as is shown in FIG. 38.

[0290] As explained above, it is possible to obtain the following results in the present invention.

[0291] (1) The TDI provides a practical interface that is amenable to humans. More specifically, humans and things, as well as fellow things, can interact by way of natural speech. Several advantages to the use of speech in interfaces have been identified over the conventional interface.

[0292] First, speech is natural to humans. That a display is not required, moreover, is also a significant advantage.

[0293] This fact is also well-suited to the compactness of the system. Further, because the system requires no manual inputs, the system is convenient for people who use the system while working or for those who are physically infirm. By using existing telephone networks, moreover, it is possible also to interact remotely.

[0294] The second advantage of speech, as described above, is that, because speech has long been used by humans as a means of communication, standardization issues are, ultimately, of no consequence.

[0295] The third advantage of speech relates to the issue of networking. Because speech is wireless and can be broadcast very simply, they are effective in interface networking within a home, for example.

[0296] (2) In addition to the advantages relating to speech, the TDI system has the following advantages. First, a human can simultaneously understand the interaction between things and, as required, make amendments or changes. Further, that things can be endowed with initiative and autonomy, and that things can be made intelligent, are additional advantages. Ultimately, these advantages contribute to the TDI system's providing an interface that is amenable to humans.