System for embedding programming language content in voiceXML

[0001] This application is related to and incorporates by reference herein in its entirety the commonly owned and concurrently filed patent application: U.S. patent application Ser. No. 09/732,812 entitled “OPEN ARCHITECTURE FOR A VOICE USER INTERFACE” by Mitsura Oshima.

[0002] Interactive voice response services are an increasingly common part of everyday life. Services of this type are used to provide everything from banking and credit card information to interactive driving systems. Interactive voice response services are also an increasingly popular way to access the World Wide Web. This is true in spite of the growing popularity of personal data assistants and web enabled cellular telephones. The nearly ubiquitous availability of telephones, and the ability to use voice in non-traditional environments (such as when driving) ensures that the popularity and diversity of interactive voice response services will continue to grow.

[0003] Voice extensible Markup Language (VoiceXML) is a response to the increasing use of interactive voice response services. VoiceXML is a language for scripting interactive voice response services.

[0004] As an example, consider the following VoiceXML fragment:

[0005]

	<vxml>
	<form>
	<block>
	<prompt>
	Hello, World!
	</prompt>
	</block>
	</form>
	</vxml>

[0006] When processed by a VoiceXML interpreter, the prompt portion of the script plays text-to-speech (TTS) “Hello, World!”

[0007] VoiceXML fragments of this type have proven to be a flexible mechanism for accomplishing many tasks. At the same time, there are important cases where VoiceXML is lacking in required flexibility. Consider, for example, the case of an interactive voice response email service. Designers of this type of service might wish to generate a prompt that welcomes each user by name and tells them how many emails they have received since their last visit (e.g., “Hello Mr. Smith, you have ten new emails.”). Unfortunately, this type of prompt requires dynamic generation—it includes fields that must be changed to match each user and each number of new emails.

[0008] In fact, the non-dynamic nature of VoiceXML contributes to a range of implementation difficulties. These difficulties become more severe (in most cases) in more complex VoiceXML applications. As a result, there is a need for systems that include dynamic content in VoiceXML and similar languages. This need is particularly important for complex interactive voice response services.

[0009] An embodiment of the present invention provides a method for providing an interactive voice response service. The method uses a VoiceXML interpreter in cooperation with a voice/audio application. The voice/audio application uses scripts coded in VoiceXML with embedded Java Server Pages (JSP). The use of VoiceXML along with JSP allows the present invention to provide an implementation that minimizes interaction between the voice/audio application and the VoiceXML server.

[0010] In one embodiment, the invention relates to a method for providing an interactive voice response service includes the steps of: creating a pool of audio prompts, subdividing the pool into segments, generating code to randomly select prompts from one of the segments, and sending the generated code to a Voice XML interpreter.

[0011] The foregoing has outlined rather broadly the objects, features, and technical advantages of the present invention so that the detailed description of the invention that follows may be better understood.

[0015] The present invention may be better understood, and its numerous objects, features, and advantages made apparent to those skilled in the art by referencing the accompanying drawings. The use of the same reference symbols in different drawings indicates similar or identical items.

[0016] The preferred embodiments of the present invention and their advantages are best understood by referring to FIGS. 1 through 4 of the drawings. Like numerals are used for like and corresponding parts of the various drawings.

Environment

[0017] In FIG. 1, a computer network 100 is shown as a representative environment for an embodiment of the present invention. Computer network 100 is intended to be representative of the complete spectrum of computer network types including Internet and Internet-like networks. Computer network 100 includes a number of computer systems, of which computer system 102a through 102f are representative. Computer systems 102 are intended to be representative of the wide range of large and small computer and computer-like devices that are used in computer networks of all types. Computer systems 102 are specifically intended to include non-traditional computing devices such as personal digital assistants and web-enabled cellular telephones.

[0018] FIG. 2 shows a representative implementation for computer systems 102. Structurally, each computer system 102 includes a processor, or processors 200, and a memory 202. Processor 200 can be selected from a wide range of commercially available or custom types. An input device 204 and an output device 206 are connected to processor 200 and memory 202. Input device 204 and output device 206 represent all types of I/O devices such as disk drives, keyboards, modems, network adapters, printers and displays. Each computer system 102 may also includes a disk drive 210 of any suitable disk drive type (equivalently, disk drive 210 may be any non-volatile mass storage system such as “flash” memory).

[0019] Computer network 100 also includes a telephony subsystem 104. Telephony subsystem 104 may be implemented as a separate system (as shown in FIG. 1) or, more typically, as a card included in one of computer systems 102. In either case, telephony subsystem 104 is connected to receive commands and data from computer system 102e. Telephony subsystem 104 is also connected to a telephone network 106. Telephone network 106 is connected, in turn to a telephone 108.

[0020] Computer network includes a series of software processes. As shown in FIG. 3, these include media telephony speech (MTS) 302, VoiceXML interpreter 304 and one or more voice/audio applications 306.

[0021] MTS 302 is hosted by computer system 102e and interacts with telephony subsystem 104. MTS 302 provides one or more high-level interfaces to telephony subsystem 104, such as telephony, text-to-speech and speech recognition and prompt playing. Software processes in computer network 100 typically interact with telephony subsystem 104 through MTS 302.

[0022] VoiceXML interpreter 304 and voice/audio application 306 are hosted (together or separately) by one or more computer systems 102 in computer network 100. VoiceXML interpreter 304 sends requests to voice/audio application 306. Voice/audio application 306 sends VoiceXML responses to VoiceXML interpreter 304. VoiceXML interpreter 304 interprets the VoiceXML responses and forwards the interpreted results to MTS 302.

[0023] The combination of MTS 302, VoiceXML interpreter 304 and voice/audio application 306 provides an interactive voice interface for voice/audio application 306. Users access MTS 302 through telephone network 106 and telephony system 104. This access is bi-directional—users can both issue verbal/audio requests and receive verbal/audio responses. MTS passes user requests to VoiceXML interpreter 304 where they are translated into HTTP requests and passed to voice/audio application 306. Voice/audio application 306 responds to the HTTP requests with VoiceXML responses. Voice/audio application 306 sends these responses to VoiceXML interpreter 304 where they are translated into calls to MTS 302. MTS 302 passes these responses, in turn to telephony system 104 telephone network 106 and eventually, the user.

[0024] In general, it should be appreciated that the specific network of FIG. 1, computer system of FIG. 2 and software components of FIG. 3 are all intended to describe a representative environment for a VoiceXML interpreter. Other, equally suitable environments may include these or other components. The specific functions of each component may also vary between different environments. Additional details of the representative environment of FIGS. 1 through 3 are described in a copending US Application entitled “Open Architecture for a Voice User Interface.” That disclosure is incorporated in this document by reference.

System for Embedding Programming Language Content in VoiceXML

[0025] Voice/audio application 306 uses a series of enhanced VoiceXML scripts. The enhanced VoiceXML scripts may include embedded elements of a high-level language. Voice/audio application 306 processes each script before sending it to VoiceXML interpreter 304 or in response to requests from VoiceXML interpreter 304. Voice/audio application 306 executes each high-level language (HLL) element as it is encountered during this processing.

[0026] As an example, consider the following VoiceXML fragment:

[0027]

<vxml>
	<form>
	<block> You have email from <%=msg.getFrom( )>%,
received at <%=msg.getDate( )%>. Subject is
<%=msg.getSubject( )%>
	</block>
	</form>
</vxml>

[0028] During processing, Voice/audio application 306 scans for the tokens “<%=” and “%>”. Voice/audio application 306 treats text enclosed by these tokens as EL expressions. Voice/audio application 306 evaluates (interprets) each HLL expression as it is encountered. Voice/audio application 306 replaces each HLL expression with its evaluated value.

[0029] After executing and replacing each HLL expression, the preceding fragment would have a form similar to:

[0030]

<vxml>
	<form>
	<block> You have email from Joe Smith, received at 4 am.
Subject is airplane delay
	</block>
	</form>
</vxml>

[0031] After processing, voice/audio application 306 sends the resulting script to VoiceXML interpreter 304. VoiceXML interpreter then interprets the updated VoiceXML fragment.

[0032] The process of token recognition, HLL expression execution and HLL expression replacement can be adapted to a range of different languages. The examples used in this description focus on the use of the Java programming language and the use of Java Server Pages (JSP) in particular. Java and JSP have the advantage of being widely adopted within the Internet programming environment.

[0033] The definition for Java Server Pages includes a range of tokens and directives. These include:

[0034]

<%= Expression %>	Voice/audio application 306 evaluates ex-
	pression and replaces it with the resulting
	value.
<% code %>	Voice/audio application 306 inserts code
	into the service method.
<%! Code %>	Voice/audio application 306 inserts code
	into the body of servlet class, outside of
	service method.
<%@ page att=“val” %>	General setup directions for Voice/audio
	application 306.
<%@ include file=“url” %>	Voice/audio application 306 includes the
	contents of the file identified by URL.
<%-- comment text --%>	Voice/audio application 306 ignores text
	between <%-- and --%>.
<jsp:useBean att=val />	Voice/audio application 306 finds or
	builds a Java Bean.
<jsp:setProperty att=“val” />	Voice/audio application 306 sets bean
	properties.
<jsp:getProperty	Voice/audio application 306 finds and out-
name=“propertyName”	puts bean properties.
value=“val” />
<jsp:forward page=“URL” />	Voice/audio application 306 forwards re-
	quest to designated page.
<jsp:plugin attribute=“val” />	Voice/audio application 306 generates
	code to request that an applet be run using
	a Java plugin.

[0035] Voice/audio application 306 can be configured to support all or a subset of these directives. The addition of Java Server Pages to VoiceXML provides a flexible framework for constructing voice applications. The efficiency of applications created with this framework can be greatly enhanced by creating VoiceXML scripts in a way that reduced interaction between VoiceXML interpreter 304 and voice/audio application 306. Each of the following examples has been selected with this consideration in mind. Code for the following examples is attached as Appendix A.

[0036] <jsp:useBean id=“random” class=“com.genmagic.util.RandomPrompt”>

• • [0037] <jsp:setProperty name=“random” property=“addPrompt” value=“hello_—1.wav”/>
  • [0038] <jsp:setProperty name=“random” property=“addPrompt” value=“hello_—2.wav”/>
  • [0039] <jsp:setProperty name=“random” property=“addPrompt” value=!“hello_—3.wav”/>
  • [0040] . . .
  • [0041] <jsp:setProperty name=“random” property=“addPrompt” value=“hello_—29.wav”/>
  • [0042] <jsp:setProperty name=“random” property=“addPrompt” value=“hello_—30.wav”

[0043] </jsp:useBean>

[0044] Execution of this fragment causes voice/audio application 306 to create a pool of audio files. In this case, thirty files are created but the same methods could be used with any number of files. This Java implementation for RandomPrompt subdivides the pool of files into segments. For this particular example, it may be assumed that six segments of five files are created.

[0045] VoiceXML scripts can generate code to select prompts from the prompt pool by including the expression:

[0046] <jsp:getProperty name=“random” property=“prompt”/>

[0047] During interpretation, Voice/audio application 306 replaces the getProperty directive with VoiceXML of the form:

[0048] <var name=“tmp” expr=“Math.random( )*4”/>

[0049] <if cond=“1.0>=tmp”>

• • [0050] <audio src=“builtin:hello_—5.wav”/>

[0051] <elseif cond=“2.0>=tmp”/>

• • [0052] <audio src=“builtin:hello_—6.wav”/>

[0053] <elseif cond=“3.0>=tmp”/>

• • [0054] <audio src=“builtin:hello_—7.wav”/>

[0055] <elseif cond=“4.0>=tmp”/>

• • [0056] <audio src=“builtin:hello_—8.wav”/>

[0057] <elseif cond=“5.0>=tmp”/>

• • [0058] <audio src=“builtin:hello_—9.wav”/>

[0059] </if>

[0060] Execution of this code causes VoiceXML interpreter 304 to randomly select an audio prompt from a segment within the pool of audio prompts. This segment of code can be executed multiple times with the prompt chosen varying between the five prompts included in the segment. Each re-execution is accomplished without further interaction between VoiceXML interpreter 304 and voice/audio application 306.

[0061] <jsp:useBeanidshuffle” class=“ShufflePrompt”>

• • [0062] <jsp:setProperty name=“shuffle” property=“addPrompt” value=“hello_—1.wav”/>
  • [0063] <jsp:setProperty name=“shuffle” property=“addPrompt” value=“hello_—2.wav”/>
  • [0064] <jsp:setProperty name=“shuffle” property=“addPrompt” value=“hello 3.wav”/>
  • [0065] <jsp:setProperty name**“shuffle” property=“addPrompt” value=“hello_—29.wav”/>
  • [0066] <jsp:setProperty name=“shuffle” property=“addPrompt” value=“hello_—30.wav”/>

[0067] <jsp:useBean>

[0068] Execution of this fragment caused Voice/audio application 306 to create a pool of audio files. In this case, thirty files are created but the same methods could be used with any number o files. This Java implementation for ShufflePrompt subdivided the pool of files into segments. For this particular example, if may be assumed that six segments of five files are created; Each segment includes a shuffled sequence-of audio prompts. Thus, one segment might include audio prompts 2, 24, 21, 7 and 10. Each segment is created without duplicate entries.

[0069] VoiceXML scripts can generate code to select prompts from the prompt pool by including the expression:

[0070] <jsp:getProperty name=shuffle” property=“prompt”/>

[0071] During interpretation, voice/audio application 306 replaces the getProperty directive with VoiceXML of the form:

[0072] <var name=“tmp” expr=“tmp+1”/>

[0073] <if cond=“tmp==6”>

• • [0074] <assign name=“tmp” expr=“1”/>

[0075] </if>

[0076] <if cond=“1==tmp”>

• • [0077] <audio src=“builtin:hello_—29.wav”/>

[0078] <elseif cond=“2 tmp”/>

• • [0079] <audio src=“builtin:hello_—3.wav”/>

[0080] <elseif cond=“3==tmp”/>

• • [0081] <audio src=“builtin:hello_—15.wav”/>

[0082] <elseif cond=“4==tmp”/>

• • [0083] <audio src=“builtin:hello_—11.wav”/>

[0084] <elseif cond=“5==tmp”/>

• • [0085] <audio src=“builtin:hello_—7.wav”/>

[0086] </if>

[0087] The expanded code implicitly selects one of the segments from the pool of audio prompts. The first execution of this code by VoiceXML interpreter 304 causes the first prompt in the selected segment to be played. Subsequent executions cause the remaining prompts to be played in order. Each re-execution is accomplished without further interaction between VoiceXML interpreter 304 and voice/audio application 306.

[0088] <field name=“next”>

[0089] <field>

• • [0090] <% if (mail.isLast( )) { %>
    • [0091] <prompt> That was the last one </prompt>
  • [0092] <% } else { %>
    • [0093] <goto next=“email.jsp#event=nextMail”/>
  • [0094] <% } %>

[0095] </field>

[0096] This fragment is intended to be used to help an application (and user) navigate within a variable length list of audio prompts. Lists of this type are used in applications where the number of prompts is not fixed. An example might be a voice email application that uses different prompts for different items in a user's inbox. Since the number of inbox items changes, the list of prompts has no fixed length.

[0097] The VoiceXML fragment is configured so that VoiceXML interpreter 304 firsts tests to determine if the end of the list has been reached (i.e., there is no next item. Only if this condition is false, does the Voice XML interpreter 304 retrieve the next item from voice/audio application 306. This avoids interaction between VoiceML interpreter 304 and voice/audio application 306 if there no next item.

[0098] This example applies to user interfaces that support multiple domains. User interfaces of this type are intended to allow users to perform multiple unrelated tasks, such as a reading email, listening to news and performing stock transactions. This type of application works best where the user is able to move randomly between domains (i.e., there is no preset order required).

[0099] In applications of this type, VoiceXML scripts do not know the identity of the next domain to be visited. This information can be generated by adding the expression:

[0100] <goto next=“domainjsp#event=nextDomain”/>

[0101] The following VoiceXML fragment can be used to generate different prompts for multiple domain user interfaces. The fragment causes VoiceXML interpreter to generate an initial prompt for a user's first visit to a domain. A secondary prompt is used for each subsequent visit.

[0102] <% if (domain.isCurrentDomainVisited( )==false) { %>

[0103] <prompt>You have 3 new messages and old messages </prompt>

[0104] <% } else { %>

[0105] <prompt>You returned back to email. </prompt>

[0106] <% } %>

[0107] This example applies to systems that support multiple services (i.e., multiple voice/audio applications 306). Systems of this type work best if VoiceXML interpreter 304 performs initial routing for each incoming call. To perform this type of routing, a VoiceXML fragment of the following type may be used.

[0108] <% if (DNIS.startsWith(“800123”)) { %>

• • [0109] <goto next=“http://banking.genmagic.com/bankingService.jsp”/>

[0110] <% } else if (ANI.equals(“4087744485”)) { %>

• • [0111] <goto next=“http://my.genmagic.com/oshimajsp”/>

[0112] <% } %>

[0113] This fragment causes VoiceXML interpreter to examine information about incoming calls. This information may include Direct Number Identification (DNIS), Automatic Number Identification (ANI) and User to User Identification (UUI) information. In each case, the information can be used to direct incoming calls to one or more voice/audio application 306.

[0114] <jsp:useBean id=“router” class=“com.genmagic.util.Router”/>

[0115] <jsp:setProperty name=“router” property=“dnis” param=“dnis”/>

[0116] <jsp:setProperty name=“router” property=“ani” param=“ani”/>

[0117] <jsp:setProperty name=“router” property=“uui” param=“uui”/>

[0118] <% if (router.isServiceFor(“onstar”)) { %>

• • [0119] <goto next=“http://onstar.genmagic.com/loginjsp”/>

[0120] <% } else if (router.isServiceFor(“bank”)) { %>

• • [0121] <goto next=“http://banking.genmagic.con/bankingServicejsp”/>

[0122] <% } %>

[0123] While the invention has been described with respect to the embodiments and variations set forth above, these embodiments and variations are illustrative and the invention is not to be considered limited in scope to these embodiments and variations. Accordingly, various other embodiments and modifications and improvements not described herein may be within the spirit and scope of the present invention, as defined by the following claims.