User interface substitution

[0001] The invention relates to systems and methods of defining a user interface for computer program.

[0002] Traditionally, graphical user interfaces have been employed to facilitate interaction of a user with software in a computer system. For example, typical objects in a GUI are icons presented on a background or “desktop”, with the icons representing a set of functions, application programs, system resources, or the like. These icons may be selected to instruct the computer to perform particular functions.

[0003] GUI objects are generally created as part of a static application program construct that describes how a particular GUI object appears and what function is performed when that GUI is selected. Many visually-based application builder tools, such as Visual Basic, available from Microsoft Corporation of Redmond, Wash., have a graphical user interface that includes a “toolbox”, a “form window”, and a “property sheet,” among others. A toolbox typically contains icons that represent different classes of components, or software modules. During a UI (user interface) construction session, a form window is displayed for the user to compose an application UI. A component selected from the toolbox may be “dropped”, or otherwise placed, within the form window. Placing a component means placing an icon or other visual representation of the component, within the form window. A property sheet displays properties, or attributes, which relate to the selected component. The properties can include, but are not limited to, information relating to the size, color, and name of the selected component. Programmers then insert code for each object to handle events generated by a user during operation.

[0004] After an application has been developed, a user generally has a limited ability to customize the user interface by modifying properties associated with certain GUI objects. In general, a property sheet displays a fixed set of properties that can be modified to customize, or configure, a given component. Hence, a user can select and modify different properties in the property sheet in order to customize a component. Simple components typically have fewer associated properties than larger components, and can therefore be readily customized by making selections in a property sheet.

[0005] As computer usage becomes increasingly widespread, the desire for custom computer applications that are specific to the needs of a particular user is also increasing. The desire is driven in part by the need to customize the software to the user's needs rather than force the user to adjust to the software, which can be tedious, time-consuming and non-productive.

[0006] In one aspect, a method defines a user interface for a computer program after execution of the computer program has begun by: reading a function description of a first function to be provided by the user interface; reading an appearance description of a first appearance to be presented by the user interface; associating the function description and the appearance description on the fly at run time; and executing the user interface with the associated function and appearance.

[0007] Implementations of the invention may include one or more of the following. The function description or the appearance description can be replaced during program execution. The method can include reading a map defining multiple functions to be provided by the user interface including the first function; reading a fashion defining all appearances to be presented by the user interface including the first appearance; associating the map and the function on the fly at run time; and executing the user interface with the associated map and function. The map or fashion can be replaced during program execution. The map can specify that a subordinate part of the user interface is specified by a second map-fashion pair. Events can be received from the map component or the fashion component. Business logic associated with the respective component can be executed. The components can be stored in a database.

[0008] In a second aspect, a method defines a user interface for a computer program by: associating a map component and a fashion component on the fly at run time to generate the user interface; and executing the user interface with the associated function and appearance.

[0009] Implementation of the method can include one or more of the following. A resource bundle associated with the map component can be loaded. Sub-components of the user interface can be loaded and instantiated. The fashion component can be called to allocate a resource to each sub-component. Each sub-component can be instructed to present itself in the user-interface. Events from the map component or the fashion component can be received.

[0010] Advantages of the invention include one or more of the following. The map (controller) and the fashion (view) need not be static and each can be changed (or can change itself) independently of the other during execution of a program. By changing the map, the logic of the user interface can change while the same fashion is used to provide continuity of presentation. By changing the fashion without changing the map, the logic of the user interface can be presented with radically different appearances. Further, to provide flexibility, the map can specify that a subordinate part of a user interface is to be constructed using a second map-fashion pair.

[0011] The logic and appearance of a user interface can be built independently, making the UI presentation more accessible to non-programmers. The UI can be modified without re-compiling the application. Because of late binding, the details of the UI presentation are established at execution time for the application. Hence, user interface development can be decoupled from code development. Hence, a visual designer can design a user interface without incurring delays associated with involving a programmer.

[0012] The UI specification is declarative rather than procedural. The declarative aspect is advantageous in that abstract, high-level specifications can be synthesized without a detailed procedural description of the required action, such as calling the system to allocate a particular resource, to get a layout object, to modify the layout object, and to attach the modified layout object to another layout object.

[0013] The appearances can be presented using a graphical interface, or can be presented in a non-visual modality such as a voice user interface that can be used in speech driven computers, including telephones and voice activated computers. Moreover, much of the UI is exposed in a way that facilitates localization. Thus, it is easy to create a user interface for an international product in which customers in different see different text, images, and other user interface elements. Use of the invention allows a corporation or other group to implement a corporate appearance for its user interface across multiple products.

[0025] As shown in FIG. 1, a system 100 has a user interface (UI) object. The UI object 102 can have a graphical modality such as a two-dimensional or three-dimensional graphical user interface, or can have a different modality, such as a voice user interface commonly used in telephone response system.

[0026] The object 102 has a map component 104. The map component 104 contains instructions for handling and processing objects contained in the user interface. The map component 104 thus embodies logic associated with the user interface object 102. The map component 104 knows that certain UI resources are available to it and are defined in a resource bundle 106 with a list of UI objects and their characteristics. The resource bundle can be created by an application programmer or can be programmatically generated. The resource bundle contains, directly or by reference, the layout rule for each UI object and the requisite system call that displays and enable that UI object to generate an event based on a user input.

[0027] The UI objects in the resource bundle 106 can include, for example, a text-box that provides an area to enter or display text. The UI object can be a checkbox, which displays a True/False or Yes/No option. The UI objects can include combo-box that combines a text box with a list box and allows a user to type in a selection or select an item from a drop-down list. The UI objects can include a button that carries out a command or action when a user chooses it. Yet another exemplary UI object can be a horizontal scrollbar and vertical scrollbar, which allow a user to add scroll bars to controls that do not automatically provide them.

[0028] Resource bundles contain locale-specific objects. When the application software needs a locale-specific resource, a “String” for example, the program can load it from the resource bundle that is appropriate for the current user's locale. In this way, program code can be largely independent of the user's locale by isolating most, if not all, of the locale-specific information in resource bundles. Each related subclass of the resource bundle contains the same items, but the items have been translated for the locale represented by that resource bundle subclass. For example, both MyResources (English version) and MyResources_de (German version) may have a String that is used on a button for confirming operations. In MyResources the String may contain “OK” and in MyResources_de it may contain “Gut.”

[0029] When the program needs a locale-specific object, it loads the resource bundle class using a “getBundle( )” method. The resource bundle lookup searches for classes with various suffixes on the basis of (1) the desired locale and (2) the default locale (baseclass); the search order is from lower-level (more specific) to parent-level (less specific). The result of the lookup is a class, but that class may be backed by a property file on disk. If a lookup fails, getBundle( ) throws a missing resource exception. Such an exception will generally cause the map component 104 to rely on a default locale.

[0030] The map component 104 contains logic to handle one or more events generated by the UI objects. An event has a life cycle that begins with the action or condition that initiates the event and ends with the final response by the event handler. Typically, a user action or condition associated with the event occurs, and an event object is updated to reflect the conditions of the event. The event fires to notify the logic in the map component 104. The map component logic executes an event handler associated with the event is called. The event handler carries out its actions and returns. The event is provided to the next element in the hierarchy, and the event handler for that element is called. For events that bubble, if there is no event handler bound to the source element generating the event, the event handler for the next element up the element's hierarchy is called.

[0031] The user interface object 102 also contains a fashion component 108. The fashion component 108 embodies the presentation, including the layout of the object 102. The fashion component is a collection whose elements represent each loaded form in an application. The fashion component 108 also derives its knowledge from a resource bundle 110. In one embodiment, the resource bundle 106 is the same as the resource bundle 110.

[0032] The user interface object 102 also can include additional user interfaces 110, 112 and 114. The UIs 110, 112, and 114 provide programmatical support for the user to navigate to another presentation. In one embodiment, the UI 110, 112 or 114 can be a button which, when clicked, moves users backwards or forwards in the application.

[0033] During initialization, the application associated with the user interface object 102 loads the map component 104 and the fashion component 108 before creating a presentation to the user. The map component 104 and the fashion component 108 need not be static. The map component 104 and the fashion component 108 can be replaced during execution. By changing the map component 104, the logic of the user interface can be altered while the same fashion component 108 can still be used to provide a continuity of presentation. Correspondingly, the fashion component 108 can be changed during execution to allow the logic of the user interface to be presented with a radically different appearance or even in some non-visual modality, such as the voice user interface.

[0034] A variety of data components can be identified with one or more map components and one or more fashion components. The map component as well as the fashion component can change under the control of their code or can be changed by other map and fashion components. Thus, the user interface object 102 of FIG. 1 allows for an arbitrary composition. Two parts of the application user interface can be built independently using different map components or fashion component pairs. Alternatively, one map component can specify that a subordinate part of the user interface be constructed by a second map component 104 and a second fashion component 108 pair as needed.

[0035] FIG. 2 illustrates an exemplary user interface. The user interface of FIG. 2 includes a top horizontal portion 120 that is positioned above a left lower portion 122 and right lower position 124. Further, each of the interface portions 120, 122 and 124 in turn can reference one or more secondary user interfaces.

[0036] FIG. 3 shows an exemplary structure associated with the user interface embodiment of FIG. 2. A user interface object 128 includes a map component 130 and a fashion component 132. The map component 130 and fashion component 132 each references one or more resource bundles that specify the UI objects such as buttons and text-boxes, for example. Moreover, the user interface 128 references a top user interface object 134, a right user interface object 136 and a left user interface object 138. During execution, the logic associated with the map component 130 finds the required subcomponents. In this case, the logic finds three objects 134, 136 and 138. The logic instantiates each object using the name of the map and the fashion. The user can navigate down the user interface hierarchy for subsequent display screens by selecting references to dependent user interface objects using various UI objects.

[0037] The exemplary structure of FIG. 3 also shows details associated with the left user interface object 138. Similar to the parent user interface object 128, the left user interface object 138 can designate a plurality of additional user interface objects 140, 142, 144, and 146. Although FIG. 3 only shows UI objects associated with the left user interface objects 138, it is to be understood that similar objects exist for the top and right user interface objects 134 and 136. Generally, the UI is developed through a process of recursive refinement until primitive UI components (a button or a text label, for instance) are specified. The resulting structure is a rooted, directed graph. Also, the same map or fashion may appear at different levels of the UI composition. More particularly, for two component UI objects (each of class UI) with map and fashion objects, the two maps (fashions) are different objects but may be the same class. For example, in the actual application, certain classes for simple navigation and for providing a particular look and feel for conventional UI widgets are used repeatedly.

[0038] Turning now to FIG. 4, a process associated with loading and using the user interface is shown. First, the map component and a fashion component are loaded (step 202). Next, a user interface is generated based on the map component and the fashion component loaded in step 202. A user interface or presentation is then presented to the user (step 204). Next, the application determines whether a new presentation is required (step 206). If so, the map and fashion components are updated (step 208) and the process 200 loops back to step 204 to present the user interface to the user. Alternatively, if a new presentation is not required, the process 200 executes the logic of the application (step 210).

[0039] After executing the business logic, the process 200 determines whether additional presentations are required (step 212). If so, the process 200 loops back to step 208 to reference new map and fashion components (step 208). Alternatively, if a new presentation is not required in step 212, the process 200 determines whether the user wishes to exit the application (step 214). If not, the process loops back to step 210 to continue executing the application. From step 214, if the user wishes to exit, the process 200 exits.

[0040] FIG. 5 shows a process 230 for initializing a user interface. First, the process loads one or more resource bundles associated with the map component or with a fashion component. Next, each subcomponent of the user interface is located (step 234). The located subcomponents are then instantiated (step 236). Next, the process 230 calls the fashion component to allocate resources to the subcomponents (step 238). Finally, each component is then instructed to present itself on the screen as a user interface element or object (step 240).

[0041] FIG. 6 illustrates the operation of the map component 104. In the process 250, the map component 104 instructs the fashion component 108 to present itself (step 252). Next, the map component 104 listens for events generated during the presentation or by the fashion component 108 (step 254). The logic of the application is then executed (step 256).

[0042] FIG. 7A shows pseudo-code associated with an exemplary user interface “next.” The “next” user interface includes a plurality of maps: Map1-Map3. Map 1 is associated with fashion 1 which defines a user interface region called top. Map 2 is associated with a fashion component 2 and is associated with a right region. Finally, Map 3 is associated with fashion component 3 and is used to define a left region.

[0043] Pseudo-code associated with the user interface of FIG. 2 is shown in FIG. 7B. In this figure, the layout of the user interface is defined as having a top region such as the region 120 of FIG. 2 as a vertical region. The vertical region is further specified as having a right region such as the region 124 of FIG. 2 and a left region such as the region 122 of FIG. 2, both of which are laid horizontally adjacent each other under the top region.

[0044] FIG. 8 is a diagram 300 illustrating the operation of one implementation embodying the map and fashion components. First, a user interface is created (step 301). The user interface includes a map component 304, a fashion component 306 and one or more properties files 308. Before the user interface is executed, a binding process 302 combines the map, fashion and property files into an executable ready form. During operation, the process 300 uses the fashion components to interpret the property files (step 310). Additionally, the nap components can be used to interpret various selection events generated by the user (step 312). Further, the map components can be used to interpret the property files (step 316).

[0045] During operation, the application can query a database, which can be resident with a host application or can be accessed remotely. In response, the database can provide information indicating the availability of different map components and fashion components. In one implementation, the identification of a server that will handle the query is provided in the query. Based on the response, an initial selection of the map component 104 and the fashion component 108 is made based on the query response. The selection can also be based on locale, which customizes the selection based on the geographic location of the user.

[0046] Based on the response to the query, an initial choice is made according to the environment variables and established preferences. A initial user interface is then presented to the user. The user can make certain selections using the UI defined in the map component or fashion component, and the presented interface can then be altered accordingly. Additionally, program logic embedded in the map component 104 can dictate the change of the map component 104 or the fashion component 108, respectively.

[0047] FIG. 9 shows a process 320 that is executed in one embodiment of the invention is shown. First, an application program is initiated (step 322). The execution of the program generates one or more queries directed at a database enquiring as to the availability of maps, fashions and locale (step 324). Next, a responsive database generates various environment variables and preferences, which are sent back to the application program (step 326). Next, the application program selects a map component (step 328). The application program then selects and loads a fashion component (step 330). Finally, a locale is selected and loaded (step 332). Next the user interface is created (step 324) and presented to the user (step 336). During this presentation, the fashion component uses the locale to localize data presented to the user (step 338). Further, the program and/or business logic information contained in the map component is used to either change the map component itself or the fashion component (step 340). From step 336, the user can choose another map, fashion or locale (step 342).

[0048] FIG. 10 shows apparatus associated with the process of FIG. 9. First, a remote database containing map components, fashion components and locale information is connected to a network 410. In addition to the map, fashion and locale information in the database 402, a property file can be contained in a second database 404 that is connected to the network 410.

[0049] On the other side of the network, the request is handled by application software, operating system interfaces and device drivers 422. The application, operating system interfaces and device drivers receive user instructions, as captured by an input/output hardware interface 424. The hardware 424 in turn drives a display device 426 and receives inputs from a user input device 428. Based on the map component, fashion component and locale information, an initial user interface is generated. The user interface can then be presented based on the user input as provided to the user input device 428. For each presentation, the map and the fashion can be changed independently of the other during execution of a program.

[0050] As shown above, the logic and appearance of a user interface can be built independently. Hence, an application visual designer can create the user interface without incurring delays associated with involving a programmer. Moreover, the UI specification is declarative rather than procedural. Thus, high-level specifications can be synthesized without a detailed procedural description of the required action. The appearances can be presented using a graphical interface, or can be presented in a non-visual modality such as a voice user interface that can be used in speech driven computers, including telephones and voice activated computers. Moreover, much of the UI is exposed in a way that facilitates localization by presenting localized text, images, and other user interface elements.

[0051] The techniques described here may be implemented in hardware or software, or a combination of the two. Preferably, the techniques are implemented in computer programs executing on programmable computers that each includes a processor, a storage medium readable by the processor (including volatile and nonvolatile memory and/or storage elements), and suitable input and output devices. Program code is applied to data entered using an input device to perform the functions described and to generate output information. The output information is applied to one or more output devices.

[0052] FIG. 11 illustrates one such computer system 600, including a CPU 610, a RAM 620, and an I/O controller 630 coupled by a CPU bus 640. The I/O controller 630 is also coupled by an I/O bus 650 to input devices such as a keyboard 660 and a-mouse 670, and output devices such as a monitor 680. Variations are within the scope of the following claims. For example, instead of using a mouse as the input devices, a pressure-sensitive pen or tablet may be used to generate the cursor position information.

[0053] Moreover, each program is preferably implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Each such computer program is preferably stored on a storage medium or device (e.g., CD-ROM, hard disk or magnetic diskette) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform the procedures described. The system also may be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner. While the invention has been shown and described with reference to an embodiment thereof, those skilled in the art will understand that the above and other changes in form and detail may be made without departing from the spirit and scope of the following claims.