Integrated Voice Controlled Report Generating and Communicating System

The present invention relates to a reporting system and more particularly to a reporting system which employs a speech recognition means to enable a user to generate a complete report text and transmit that text to a selected recipient under voice control.

Particularly in certain medical situations, the creation and delivery of technical reports has presented difficulty both in terms of the time and expense involved and in the accuracy, completeness, and timeliness of the reports. Even if the report is generated in a prompt and accurate manner, delivery of the report to the desired recipient has often been a stumbling block, even when conventional facsimile machines were available to transmit a hard copy from the author to the recipient. It is, for example, highly desirable that the determinations made by a radiologist examining x-rays be made promptly available to the referring physician.

Among the several objects of the present invention may be noted the provision of a novel system for creating and delivering reports; the provision of such a system which permits reports to be generated quickly and accurately; the provision of such a system which allows reports to be delivered to a selected recipient promptly after the creation of the report; the provision of such a system which does not require the manual entering of substantial quantities of text; the provision of such a system which is highly reliable and which is of relatively simple and inexpensive construction. Other objects and features are in part apparent and in part will be pointed out hereinafter.

In a reporting system in accordance with the present invention, a speech recognition means provides a plurality of vocabulary action tables (VATs) which relate trained utterances to corresponding program input data elements. A command interpreter responsive to a first one of the VATs selectively initiates creation of a report text and the storing, printing and transmitting of report texts. A stored report form provides selectable sections of text and selectable text insertions, respective ones of the VATs providing keyword selection of text sections and of text insertions. The system also includes means for converting text to image data and for modulating an audio band signal with the image data for facsimile transmission. A further VAT relates trained utterances to stored telephone numbers, and the command interpreter is operative, in response to a respective spoken command, to cause the sending of a selected telephone number to a telephone dialing means and to cause the sending of generated report text to the converting and modulating means.

FIG. 1 is a block diagram illustrating the interconnection of the various elements of a reporting system in accordance with the present invention; and

FIG. 2 is a flowchart illustrating the operation of the reporting system.

Corresponding reference characters indicate corresponding parts throughout the several views of the drawings.

Referring now to FIG. 1, an audio signal obtained from a microphone 11 is provided to so-called front end processing circuitry 13 which performs an initial spectral analysis and encoding of the raw audio signal. The circuitry 13 generates, for each spoken word, a succession of encoded data frames which are provided to a general purpose programmed computer, designated by reference character 15. Various types of encoding are known in the art.

The computer 15 may, for example, be one of the well known types which utilizes the Intel 80386 processor and operates under the MS-DOS operating system. Though shown physically separated, the encoding circuitry 13 may in practice be implemented on a card included in the computer which the processor accesses through a bus structure. As is understood, such computers normally include a so-called hard disk rotating memory for storing the large quantities of data which are typically associated with speech recognition systems.

In particular, the computer 15 stores a large number of digitally encoded speech reference samples, each of which comprises a series of data frames derived from a training session during which the user speaks words or phrases to be included within the speech recognition system's vocabulary. To increase the likelihood of recognition, multiple representative reference samples may be provided for a single vocabulary word. A video terminal 17 and a keyboard 19 are provided for user interaction with the computer. Also conventionally, there is associated with the computer a printer 21. Printer 21 is preferably of a type which can print graphics characters as well as alphanumeric characters.

In order to constrain the number of possible vocabulary choices which need to be examined at various points in the process of generating a report and also to allow different translations at different times, the samples are preferably organized in vocabulary action tables (VATs) corresponding to the possible courses of action which may be taken at various junctures in the running of the report generation system. Each vocabulary action table relates trained utterances or reference samples to corresponding program input data elements, i.e. data strings which can control or act as responses to the computer system just as data strings entered from the keyboard 19 may provide such input elements.

Depending upon the state of the computer program, the program input elements generated in response to spoken utterances can be used for various purposes. As is conventional, the report generating program will include some form of command interpreter for selectively initiating various operations in the embodiment illustrated such operations include the initial creation of a report text and storing, printing and transmitting of such report texts. The command interpreter is responsive to a respective VAT as well as to input from the keyboard 19.

Generation of a report is facilitated by utilizing a stored report form which provides a large number of selectable sections of text and context based sets of words or texts that can be inserted in blanks left in the form. Selection of words to fill in blanks in the text is made by the speech recognition system utilizing a corresponding VAT. Appropriate sections of text are likewise selected by voice command, again using data from a respective VAT which is activated when the selection is applicable.

The setting up of standard report forms and the defining of suitable VATs for controlling the selections is preferably performed using the knowledge base editor available from Kurzweil Applied Intelligence of Waltham, Mass.. Particular medical applications are marketed by Kurzweil Applied Intelligence under the trademark "VOICEMED" and Kurzweil also markets front end signal processing such as that indicated by reference character 13 in a card form which is readily installable in IBM PC compatible computers of the type described hereinbefore using the Intel 80386 processor.

The report text generated by the VOICEMED systems are in ASCII text form. To facilitate the transmission of the reports generated to standard facsimile machines which are in wide spread use, the system of FIG. 1 incorporates circuitry and software for converting the text to a corresponding image, e.g. using character generating read only memories (ROMS) such as are commonly used in printers and video displays as well. Image data is provided to a modem 41 which is operated under the control of the computer 15. The modem 41 modulates an audio signal in correspondence with the image data and implements facsimile transmission of the data over telephone lines as indicated by reference character 43.

Circuitry suitable for performing the text to image data and for implementing the mode functions is manufactured and sold by the Quadram Corporation of Norcross, Ga. under the model designation JT FAX 9600. This circuitry is also provided on a circuit card suitable for inclusion in an IBM-PC compatible computer of the type described previously, although it is shown as physically separate in the diagram of FIG. 1. The Quadram product also incorporates circuitry for dialing telephone numbers provided to it by the computer processor and for signalling to the processor when a connection is established.

In addition to the VATs associated with the text selection and blank filling in, the system of the present invention also incorporates a VAT which relates trained utterances to stored telephone numbers. Thus, by speaking a name or other designator associated with a desired recipient, a user of the system can select that person or place to receive a copy of the report. The command interpreter VAT likewise includes entries which cause the selected telephone number to be applied to the modem and, upon establishment of the desired telephone connection, the initiation of transmission of the report text to the converting circuitry 27 and thence to the modem 41.

From the foregoing, it can be seen that a user can generate and transmit a technical report as essentially a single operation without requiring extensive entry of text or commands through a keyboard and without the usual step of printing out the report and then manually transferring it to a conventional fax machine for transmission to the desired recipient.

A typical sequential set of operations is illustrated in FIG. 2. Utilizing the COMMAND VAT, the user can initiate generation of a report and once the standard report form is loaded, the user can select appropriate text sections under the using words or phrases stored in the SELECTION VAT. Similarly, in each text section, terms or phrases for filling in blanks can be selected using respective TERM VATs. When the report is in a form suiting the user, he can designate that the report generation is complete by speaking in an appropriate COMMAND utterance. If desired, a hard copy of the report can be generated locally at the printer 21 and, at essentially the same time, the user can request that the text of the report be sent by facsimile and by voice command can select a recipient from entries in the corresponding RECIPIENT VAT. Upon being provided with a telephone number, the modem 41 preferably initiates dialing on a repeating basis at timed intervals until a connection is made at which point it signals back to the computer that the connection is established and the computer, in turn, initiates transferring of the report text in ASCII form to the converting circuitry 27 which in turn provides image data to the modem for facsimile transmission. While the conversion of ASCII text to image data is preferably performed just prior to transmission, it should be understood that image data could be generated earlier, e.g. as part of the report generation process. Further, other image data such as charts, graphs, x-rays, CAT scans or other digital images could be merged with the associated text for transmittal as an integrated document.

Accompanying this specification is an appendix which includes a listing of computer program components, written in the "C" programming language, which implement the report transmission functions and which can be incorporated in the Kurzweil VOICEME system described previously.

In view of the foregoing it may be seen that several objects of the present invention are achieved and other advantageous results have been attained.

As various changes could be made in the above constructions without departing from the scope of the invention, it should be understood that all matte contained in the above description or shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.