Apparatuses, methods, computer programming, and propagated signals for modeling motion in computer applications

[0002] This application is a continuation-in-part of and claims priority under 35 U.S.C. §119(e) of the co-pending U.S. provisional application Ser. No. 60/129,165 filed by Grinstein et al. on Apr. 14, 1999 and entitled “Apparatuses And Methods For Modeling Motion In Computer Applications” (hereinafter “The Provisional Application”) The Provisional Application is also hereby incorporated by reference.

[0003] 1. RELATED APPLICATIONS . . .

[0004] 2. FIELD OF THE INVENTION . . .

[0005] 3. BACKGROUND OF THE INVENTION . . .

[0006] 4. SUMMARY OF THE INVENTION . . .

[0007] 5. DESCRIPTION OF THE DRAWINGS . . .

[0008] 6. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS . . .

[0009] 6.1 OVERVIEW . . .

[0010] 6.1.1 The OpenMotion System . . .

[0011] 6.1.2 The API And Its Relation To Application, Modeling, and Rendering Programs . . .

[0012] 6.1.3 The Run Time Engine . . .

[0013] 6.1.4 The Motion Translator, the OpenMotion Format, ClipMotions and the CMF . . .

[0014] 6.1.5 Motion-Enabled Objects . . .

[0015] 6.1.6 Encryption . . .

[0016] 6.1.7 Motion Editor/Viewer . . .

[0017] 6.1.8 Description Of The Types Of Motions Supported By The System . . .

[0018] 6.2 OPENMOTION API PROGRAMMER's GUIDE . . .

[0019] 6.2.1 Scope . . .

[0020] 6.2.2 Overview . . .

[0021] 6.2.2.1 Namespace . . .

[0022] 6.2.2.2 Update loop . . .

[0023] 6.2.2.2.1 Loop in application . . .

[0024] 6.2.2.2.2 Loop in graphic system . . .

[0025] 6.2.3 Principal types . . .

[0026] 6.2.4 Expressions . . .

[0027] 6.2.4.1 Values . . .

[0028] 6.2.4.2 Notation . . .

[0029] 6.2.4.3 Predefined values . . .

[0030] 6.2.4.4 Vars . . .

[0031] 6.2.4.5 Callbacks . . .

[0032] 6.2.4.6 Temporal predicates . . .

[0033] 6.2.5 Motions . . .

[0034] 6.2.5.1 Parameters . . .

[0035] 6.2.5.1.1 Initial conditions . . .

[0036] 6.2.5.1.2 Behavior . . .

[0037] 6.2.5.2 Attributes . . .

[0038] 6.2.5.2.1 Current Attributes Of Motion . . .

[0039] 6.2.5.2.2 Predict . . .

[0040] 6.2.5.2.3 Boundary . . .

[0041] 6.2.5.3 Complex Motions . . .

[0042] 6.2.5.3.1 Hierarchy . . .

[0043] 6.2.5.3.2 Blending . . .

[0044] 6.2.6 Behaviors . . .

[0045] 6.2.6.1 Interactive controllers . . .

[0046] 6.2.6.2 Constant controllers . . .

[0047] 6.2.6.3 Reset parameter . . .

[0048] 6.2.6.4 States and transitions . . .

[0049] 6.2.6.5 Assignments and callbacks . . .

[0050] 6.2.6.6 Attributes and predicates . . .

[0051] 6.2.7 Boundaries . . .

[0052] 6.2.7.1 Primitive shapes . . .

[0053] 6.2.7.2 Attributes . . .

[0054] 6.2.7.3 Composite boundaries . . .

[0055] 6.2.7.4 Expressions . . .

[0056] 6.2.7.5 Predicates . . .

[0057] 6.2.7.6 Behaviors . . .

[0058] 6.2.8 Examples . . .

[0059] 6.2.8.1 Swing . . .

[0060] 6.2.8.2 Two balls bouncing in box . . .

[0061] 6.2.8.3 Motion Derivatives . . .

[0062] 6.2.8.4 Motion Blending . . .

[0063] 6.2.8.5 Boundary Bounce . . .

[0064] 6.3 OPENMOTION FORMAT . . .

[0065] 6.3.1 Notation . . .

[0066] 6.3.2 Syntax Specification . . .

[0067] 6.3.3 Examples . . .

[0068] 6.3.3.1 The Simplest Motion . . .

[0069] 6.3.3.2 Velocity Controller Restricted To Plane . . .

[0070] 6.3.3.3 Flag waving (hierarchical motion) . . .

[0071] 6.3.3.4 One ball bouncing in box . . .

[0072] 6.3.3.5 Two balls bouncing in box . . .

[0073] 6.3.3.6 Harmonic (spring) motion . . .

[0074] 6.3.3.7 Circular motion . . .

[0075] 6.3.3.8 Orbital motion . . .

[0076] 6.4 OPENMOTION LIBRARY: MOTIONS AND INTERFACES FOR ACTIONS . . .

[0077] 6.4.1 Overview . . .

[0078] 6.4.2 Foundation Motions for Action Definition . . .

[0079] 6.4.2.1 Atomic Motions . . .

[0080] 6.4.2.2 Composite Motions . . .

[0081] 6.4.2.3 Reactive Motions . . .

[0082] 6.4.2.4 Deformations . . .

[0083] 6.4.2.5 New Motions . . .

[0084] 6.4.2.6 Saved Motions . . .

[0085] 6.4.2.7 Serial Motions . . .

[0086] 6.4.2.8 eMotions . . .

[0087] 6.4.2.9 Special or Behavioral Motions . . .

[0088] 6.4.2.10 Controlled Motions . . .

[0089] 6.4.3 Dialog Boxes . . .

[0090] 6.4.3.1 Atomic Motions . . .

[0091] 6.4.3.1.1 Basic Parameters . . .

[0092] 6.4.3.1.2 Advanced Parameters . . .

[0093] 6.4.3.1.3 Temporal Parameters . . .

[0094] 6.4.3.1.4 Specialized Parameters . . .

[0095] 6.4.3.2 Composite Motions . . .

[0096] 6.4.3.3 New Motions. . . .

[0097] 6.4.3.4 Wind Controller . . .

[0098] 6.4.3.5 Sway Controller . . .

[0099] 6.4.3.6 Other Controllers . . .

[0100] 7. MOJO EDITOR . . .

[0101] 8. 3D MODELS AND MOTION E-COMMERCE SITE . . .

[0102] 9. 3D NETWORKED GAME . . .

[0103] 10. ADDITIONAL INTERACTIVE CAPABILITIES OF TREE VIEW . . .

[0104] 11. THE INVENTION IS NOT LIMITED TO THE EMBODIMENTS DESCRIBED ABOVE . . .

[0105] 12. WHAT WE CLAIM IS: . . .

[0106] The invention relates to the modeling of motion in computer applications.

[0107] 3D graphics is a significant technology. Originally it was largely the domain of specialized applications, primarily in technical and scientific fields, such as computer aided design, manufacturing and engineering analysis (“CAD/CAM/CAE”), running on computer workstations and high-end personal computers (“PCs”). However, as more powerful and faster microprocessors have become available at significantly lower cost, the number of computers and other electronic systems capable of taking advantage of 3D graphics has grown exponentially, to the extent that today 3D processing capabilities are available on most all new computers, and also on specialized 3D graphics entertainment systems that cost less than $500.

[0108] As a consequence, the demand for 3D graphics is growing not only in CAD/CAM/CAE and the technical and scientific fields, but also in important new markets such as digital media content creation for games, educational software, entertainment software, graphic arts, and corporate desktop.

[0109] In response to the growing interest and demand for 3D graphics capabilities, the industry has seen significant advances in software enabling technologies. In basic terms, a 3D graphics development system can be seen as requiring four primary functional capabilities: a) a 3D modeling to create 3D objects or scenes; b) a 3D rendering to adequately display the 3D objects or scenes with as much realism as possible; c) a 3D motion generation and control that includes spatial manipulation of 3D objects and navigation within 3D scenes; and d) a graphical user interface to allow programmers and users to readily access and use the 3D functional capabilities.

[0110] In the last few decades, progress in 3D modeling has been evident. In the 1970s modeling applications generated objects using triangles and quadrilaterals and in the 1980s modelers were able to generate objects using triangle meshes and elementary splines, such as Bezier splines. In the 1990s 3D development applications have generated 3D objects and scenes using more complex splines such as Non-Uniform Rational B-splines (“NURBS”) and parametric solid geometry.

[0111] These advancements in 3D modeling have not only provided the ability to create richer more realistic 3D models and scenes, but have also made the 3D model building task easier and faster. For example, in parametric solid modeling, applications provide the ability to generate models that can be automatically sized and changed in a few lines of code or through pull-down menus while the model retains its fundamental attributes, and in 3D surface applications the technology exists to program sophisticated organic shapes in a few lines of code or through pre-defined pull-down menus. As a consequence, 3D development applications incorporating these advanced 3D modeling technologies have emerged as significant productivity tools for developers.

[0112] Advancement in 3D rendering technology is evident as well. In the 1970s renderers displayed 3D graphics models and scenes using mainly flat shaded triangles (“Gouraud shading”). In the 1980s this advanced to include the use of multiple light sources and complex lighting models (“Phong shading”). In the 1990s came the emergence of renderers that offer complex lighting model capabilities such as radiosity, as well as enhanced rendering attributes that made 3D scenes more realistic, such as the ability to create and display transparency, smoke and fog characteristics. The 1990s also saw the emergence of faster and more sophisticated texture mapping technology, including real-time 3D painting capabilities.

[0113] Again, these advances in rendering technology, along with advances in microprocessor capabilities, has resulted in the emergence of 3D rendering systems that allow easier, faster and better programming in 3D. It has also included the emergence of open standard application program interfaces (“APIs) for rendering, such as OpenGL from Silicon Graphics and more recently Direct3D from Microsoft. These open standard rendering APIs have encouraged the establishment of more 3D development environments capable of delivering 3D programs with richer 3D capabilities as well as portability and extendibility.

[0114] The adoption of a graphical user interface (“GUI”) as a means of making a computer easier to use has evolved dramatically over the past two decades, from the somewhat limited text-based key-stroke commands of the 1960s, 70s, and early 80s, to the current prolific use of mouse and keyboard-operated GUIs of today. The Apple Macintosh and Microsoft Windows operating systems are both examples of this trend. Throughout the 1980s and continuing into the 1990s there have been significant advancements in GUI capabilities, from better input device functionality to standardization of functions, establishing separation from application code, incorporating multi-processing with specific event handling and callbacks to applications. The advancements in GUI technologies has meant programmers are able to program specific GUI requirements easier, faster, and better with fewer lines of code.

[0115] Despite all of these advances, 3D graphics has not reached its full potential. Without the ability manipulate 3D objects intuitively, or to navigate through 3D scenes interactively, the objects or scenes become no more than still shots, or sequences of still shots, and manipulation and navigation is severely limited. Today most 3D applications are limited to a small set of predefined motions or animations. The reason is that motion-rich 3D environments are difficult to program.

[0116] Developers who want to add complex 3D motion and 3D interactive motion control capabilities to their applications generally need to “hard-code” these motions. That is, each motion is programmed for each object through the scene in a painstaking and time consuming manner. Additionally, programmers need to be conscious of the different types of input devices that will be used, and these have to be programmed into the application as well. The task can involve thousands of lines of code. As a consequence all this difficult work has inhibited the emergence of motion-rich 3D development environments in anything but very specialized packages.

[0117] To date, few GUIs have not been able to take advantage of 3D technology. The opportunity arises to introduce an advanced GUI that incorporates 3D functionality together with 2D technology seamlessly. Realization of this opportunity requires intuitive interaction and control of 3D objects.

[0118] The present invention relates to improvements in computer motion modeling. The invention has multiple aspects. Some embodiments of the invention, such as the OpenMotion™ System discussed below in the Detailed Description Of Preferred Embodiments, combine all of these aspects together into one software product . In other embodiments only one or a subset of these aspects may be used.

[0119] One aspect of the invention is its abstraction of motion from media. That is, we deal with motion as a concept unto itself, not necessarily intertwined with a graphic object, nor a side-effect of physical simulation. Motions may be combined using operators to achieve nesting, blending, and weighting. By means of this abstraction, one of our motions can be applied to a large number of different computer represented objects, or even combined together to make new motions. This enables our technology to be applied to a wide range of applications.

[0120] Another aspect of the invention features an application program interface (“API”) for motion specification and interactive control, as well as a corresponding runtime engine to execute the motions and to respond to interactions defined by use of the API. This is a 3D API for generating and invoking enhanced 3D motion and 3D interactive motion control capabilities independently of, but complementary to, popular 3D modeling applications and rendering APIs. These motions can be edited and re-used.

[0121] Yet another aspect of this invention features a motion software system which has an application program interface (API) and file interchange format which are designed to look alike and be equivalent in expressive power. Both source code and file format are seen as two representations of the same interface. The interface may be bound to new source languages as opportunities arise. This includes an efficient storage and retrieval mechanism for 3D motion models. In some embodiments of the invention the API is the OpenMotion™ API and the file format is the OpenMotion™ File Format, both of which are discussed below in the Detailed Description Of Preferred Embodiments.

[0122] Still another aspect of the invention is the ability to store motions and object models in a particular file format. In many embodiments of this aspect of the invention the motion software system comes with a library of such motions, and preferably also a library of object models which are built and stored in a particular way which enables them. to work with the Open Motion API. These models are called motion-enabled models. When such a model is combined with a motion defined by the system it is called Motion Art. In some embodiments of this aspect of the invention, an encryption mechanism controls the use and proliferation of such motions and models. In some embodiments, the file format used to store such motions is the OpenMotion™ File format discussed below in the Detailed Description and the file format used to store motion enabled models is an extension of the OpenMotion™ format. Motions stored in the OpenMotion™ File format are called ClipMotions™.

[0123] Still another aspect of this invention is its architectural model, which allows motion to be modeled and parameterized in a flexible way. This architectural model consists of a particular set of functional units that may be configured by the user in a multitude of ways to create virtually any desired motion. The units are Motion, Behavior, Boundary, and algebraic expressions. A Motion is the computational structure which defines the spatial properties of an object, such as position, orientation, and size. A Behavior is a common attribute of a motion which controls the changes in the spatial properties of a motion. A Boundary is a surface or volume with which one or more motions can interact, such as by bouncing off or being limited to stay on or within. Algebraic expressions enable motions, behaviors, or boundaries to be combined and they enable the attributes of motions, behaviors, and boundaries to be dynamically interrelated. In many preferred embodiments of the invention the algebraic expressions are retained expressions, which appear to be normal algebraic operators to the user, but which are specially defined operators that cause information defined by the many simultaneous equations, which can be represented by such algebraic expressions, to be propagated in an efficient manner similar to that in which updates propagate through a spreadsheet.

[0124] In some embodiment of the invention, motions, behaviors, and boundaries, are each modeled as classes in an object oriented language such as C++. This enables new instances of a particular motion, behavior, or boundary to be created easily, and new types of motions, behaviors, or boundaries to be created easily through inheritance. Since behaviors are attributes of motions, once a given class of behavior has been defined, it can be used to control different motions. Furthermore, in many embodiments of the invention, a behavior can be defined as a sequence of other behaviors, and behaviors can contain conditional statements which can evoke other behaviors, meaning that it is very easy to create complex behaviors out of simple ones.

[0125] Another aspect of the invention is the processing method for managing the update loop. The dispatch algorithm has to coordinate a number of independently-moving motions, the behaviors of which may be triggered by spatial predicates (collision detection, unscheduled events), temporal predicates (scheduled events), as well as user-defined Boolean conditions. The algorithm must synchronize and dispatch updates at each frame, as well as predicting, detecting and resolving interactions, such as collisions, that might occur between frames. The method distributes particular responsibilities amongst the functional units in a unique way. The method also supports the run time creation and modification of motions.

[0126] Another aspect of this invention is its motion control method. Quantities of motion are interrelated by derivatives or integrals, e.g., for a translation control, any one of position, velocity, or acceleration may be selected as the control parameter, the others being derived from this parameter. The control method dynamically fits a trajectory curve to any input control function, whether a discretely-sampled input, or a continuously-defined function. This model uniformly applies to other degrees of freedom, namely orientation, scaling, shearing, bending, stretching, and possibly others. This model allows us to detect interactions between motions that occur between rendering intervals, which would otherwise be missed.

[0127] Another aspect of this invention is its Boundary composition. Boundaries are seen as elemental units that can be combined to form complex structures. Algebraic operators are used to perform user-defined compositions and configurations. When a motion interacts with a Boundary, it is called a Constraint.

[0128] Still another aspect of this invention is its use of motion primitives and algebra. A specific set of motion primitives or elemental units can be defined. This set along with a motion algebra can then be used to form more complex motions.

[0129] Yet another aspect of this invention is its motion editor which provides visual feedback on the definition of complex and hierarchical motions that are supported by the API.

[0130] A more accurate summary of the invention is found in the claims that follow. It should be understood that the invention is meant to include apparatus, methods, computer programming recorded on computer readable media, and propagated signals capable of providing functionality of the type recited in each claim, even if there currently are not claim covering each of these different class of inventions below.

[0131] These and other aspects of the present invention will become more evident upon reading the following description of the preferred embodiment in conjunction with the accompanying drawings, in which:

[0132] FIG. 1 is a simplified schematic block diagram of one embodiment of the invention, the OpenMotion™ System;

[0133] FIG. 2 is simplified schematic block diagram of a possible relationship between the OpenMotion™ System, an application program, and a 3D rendering program;

[0134] FIG. 3 is another simplified schematic block diagram of the possible relationships between programs shown in FIG. 2, illustrating some of the representational information used by each program.

[0135] FIG. 4 is a simplified schematic block diagram of the interface between the OpenMotion™ System and motions stored in a file storage device;

[0136] FIGS. 5-8 illustrate pop-up menus for use with the OpenMotion System's Motion Editor/Viewer;

[0137] FIGS. 9-12 illustrate, respectively, the Basic, Advanced, Temporal, and Specialized tabs of the Shake Motion dialog box provided by the Motion Editor/Viewer;

[0138] FIGS. 13-15 illustrate, respectively, the Basic, Temporal, and Specialized tabs of the Shift Motion dialog box provided by the Motion Editor/Viewer;

[0139] FIGS. 16-18 illustrate, respectively, the Basic, Temporal, and Specialized tabs of the Spin Motion dialog box provided by the Motion Editor/Viewer;

[0140] FIGS. 19-22 illustrate, respectively, the Basic, Advanced, Temporal, and Specialized tabs of the Swing Motion dialog box provided by the Motion Editor/Viewer;

[0141] FIGS. 23-24 illustrate the Basic tab of the Composite Motion dialog box provided by the Motion Editor/Viewer, with the drop down list box of the edit box/list box used to list and select component motions closed in FIG. 23 and open in FIG. 24;

[0142] FIGS. 25-28 illustrate, respectively, the Initial Condition, Translation, Rotation, and Scaling tabs of the New Motion dialog box provided by the Motion Editor/Viewer;

[0143] FIG. 29 shows the Motion Editor/Viewer's dialog box for setting the parameters of the Sway Controller;

[0144] FIG. 30 is a diagram for illustrating several of the parameters of the Sway Controller;

[0145] FIG. 31 is a diagram for illustrating several of the parameters of the Wind Controller.

[0146] FIG. 32 shows the Motion Editor/Viewer's dialog box for setting the parameters of the Wind Controller.

[0147] FIG. 33 illustrates the visual user interface of the Mojo motion editor, displaying the editor's scene view window showing animated image of a 3D hierarchical model of a man;

[0148] FIG. 34 illustrates the same interface when a tree view window is displayed, showing the hierarchical structure of the human model shown in FIG. 33;

[0149] FIG. 35 and FIGS. 36A and 36B provide tree view graphs of the hierarchical model illustrated in FIGS. 33 and 34;

[0150] FIGS. 37 through 44 are similar to FIG. 34 except that each illustrates a different aspect of the user interface provided by the Mojo motion editor;

[0151] FIG. 45 is similar to FIG. 34 except that it illustrates the Mojo interface when a 3D hierarchical model of a bee is displayed, and the tree view graph illustrates a sound node and several graphic model nodes attached to the bee model;

[0152] FIG. 46 displays a tree view of a model, belonging to a class of simplified humanoid models;

[0153] FIG. 47 illustrates the appearance of the simplified humanoid model shown in FIG. 46;

[0154] FIGS. 48 and 49 are similar to FIGS. 46 and 47, respectively, except they are for a different model from the same class of simplified humanoid models;

[0155] FIGS. 50 and 51 illustrates two different motion package is, each of which is designed for use with models of the simplified humanoid class illustrated with regard to FIGS. 46 through 49;

[0156] FIGS. 52 and 53 are tree views of the model shown in FIG. 48 after the motion package is a FIGS. 50 and 51, respectively, have been attached to it;

[0157] FIG. 54 is similar to FIG. 45 in that it shows the same bee model, but in FIG. 54 the tree view graph is expanded to show all the model nodes within the hierarchical bee model;

[0158] FIG. 55 is similar to FIG. 54 except that it shows the tree view graph of the bee model after a complex motion package has been added to it;

[0159] FIGS. 56A, 56B, and 56C provide a highly simplified pseudo code description of some of the programming of the Mojo motion editor;

[0160] FIGS. 57 through 60 provide highly simplified pseudo code descriptions of subroutines called by the programming of FIGS. 56A through 56C;

[0161] FIGS. 61 through 63 illustrates how motions can be deleted from and/or dragged to different locations within Mojo's tree view window;

[0162] FIG. 64 is a simplified schematic diagram of a system for selling and buying 3D motions and models over a computer network, such as the World Wide Web;

[0163] FIGS. 65 and 66 illustrate steps performed by the e-commerce client and server computer, respectively, illustrated in FIG. 64;

[0164] FIG. 67 illustrates in networked computer system allowing multiple users at different computers to share interactive 3D models and motion, such as for use in a computer game;

[0165] FIGS. 68 and 69 illustrate steps taken by the client computers and server computer shown in FIG. 67, respectively;

[0166] FIG. 70 illustrate steps which can be taken by a computer application to automatically changed the motion package associated with a 3D model in response to program instructions; and

[0167] FIG. 71 illustrates additional user interface features that can be provided with a tree view graph according to certain aspects of the present invention.

[0168] 6.1.1 The OpenMotion System

[0169] The embodiment of the invention described in greatest detail in this application is the OpenMotion ™ System (or the “System”). FIG. 1 provides a simplified overview of the OpenMotion System 100. The System is a collection of software modules which is being produced by 3D OpenMotion LLC, the assignee of this provisional application. OpenMotion, ClipMotion, and eMotions are trademarks of 3D OpenMotion LLC. As of this writing many of the features described below are currently written and operating. Others of these features are expected to be operating soon.

[0170] The OpenMotion System provides a dedicated mechanism for the creation of basic and sophisticated interactive 3D motions independently of any specific object or geometry and independently of any specific rendering environment. These motions can be edited, extended, blended, stored, retrieved, imported and exported and can be independently attached to any number of different 3D objects using the System.

[0171] 6.1.2 The API and its Relation to Application, Modeling, and Rendering Programs

[0172] The System includes an Application Program Interface (“API”) 102 used for creating and invoking 3D motion and 3D interactive motion control capabilities and their constraints (spatial and temporal), independently of, but in association with, existing or new 3D modeling and/or 3D rendering development environments. Currently the System is written in C++, and an application program 104 which uses the System interfaces to it largely through the C++ API which is defined in greater detail below.

[0173] FIGS. 3 and 4 illustrate some of the ways in which the System can be used by an application program 104 in conjunction with a rendering program 106. The application program can be any program, whether written by an individual or a software company which models 3D spaces. As shown in FIG. 2, the application program interfaces with the System through the API, by defining motion related objects, variables, and constants through the API and by calling API functions. This includes using the API to define, combine, or manipulate primitive motions 108, shown in FIG. 2. Primitive motions are basic pre-defined motions which are part of the System.

[0174] In FIGS. 1-3, the application program 104 uses a separate rendering program 106 to perform the graphic rendering of the 3D world that it is modeling. Although the System can be used easily by an application that performs its own rendering, many of the applications which it is used will use have a separate rendering program perform the task of manufacturing screen images of the 3D worlds they are modeling. This is because drafting the programming for efficient 3D rendering is difficult, and excellent rendering programs are available from several third party vendors, such as OpenGL from Silicon Graphics and Direct3D from Microsoft. Application programs use such third party rendering programs through an API.

[0175] An application 104 which is using the OpenMotion System and a separate rendering program 106 can communicate to the OpenMotion system and to the Renderer, as is indicated by the solid arrows in FIGS. 2 and 3. Alternatively, the application could instruct the OpenMotion System to set variables directly in the renderer, as is indicated by the arrow 112 shown in broken lines in FIGS. 2 and 3.

[0176] As is shown in FIG. 3, the Application 104 maintains 3D object models 114. These models may have been purchased, authored by an individual user, coded as part of the application program 104, or created by a third party modeling program 116. Creating such models with third party modeling programs, such as SoftImage by AVID Technology Inc. or 3D Studio Max by Kinetix, a business unit of Autodesk Inc., often makes sense, because such programs often greatly increase the speed at which 3D models can be made. Normally, once these models have been created, they can be stored on a mass storage device, such as the hard disk 118 shown in FIG. 3, and then read back into the application program 104 as desired.

[0177] Commonly the rendering program 106, maintains a representation of these 3D objects including object matrices 120 which represent the position and orientation of each object model in 3D space. The OpenMotion System also represents the spatial attributes of each 3D object whose motion it is to model in a set of motion matrices 122, which represent the spatial attributes of each object, such as its 3D position, 3D orientation, and scale. Normally the application 104 tells the OpenMotion System the initial spatial attributes, the motions, the boundaries, and the rules for motion interaction for each of its object models. This is indicated by the arrow 121 in FIG. 3. Then it asks the OpenMotion System to give it the updated value of the spatial attributes of each such object at each screen redraw time. This is also indicated by the arrow 121. Then the application provides this information to the renderer program 106. This information flow is indicated by the arrow 123 in FIG. 3. The renderer then calculates a screen image of the objects, given their just-updated spatial attributes. The renderer then sends this updated screen image to the computer screen 124, where it can be seen by a user of the application.

[0178] Some applications and some renderers may allow the renderer program to directly obtain updated positional or other attributes for the modeled objects from the OpenMotion System. In such a case the information flow between the renderer and the OpenMotion System is indicated by the arrow 125 in FIG. 3, which is drawn with a broken line to indicate that it is not always used.

[0179] The application programs with which the OpenMotion System can be used can be of virtually any type. For example, it is possible that the program might even be used with application programs that do not even require 3D rendering. This might be the case where the application program is performing physical modelling not to produce a visual output, but rather to obtain simulated test results, and uses the OpenMotion system to perform the motion computation of the objects it is modeling.

[0180] 6.1.3 The Run Time Engine

[0181] The basic functional code of the System is called the Run-time Engine 130, shown in FIG. 1. The Run-time Engine (“RTE”) contains all of the functions, types, and classes which are accessible through the API 102 as well as some which are accessible only from within the code of the RTE itself .

[0182] The RTE contains pure motion and kinematic algorithms (referred to as the “The Math Engine” or TME) for both basic and complex 3D motion generation and control. These include algorithms required for: a) procedural or pre-defined motion generation and control such as in 3D animation creation; b) interactive motion and motion control such as in real-time manipulation and navigation; and c) behavioral or responsive motion and motion control such as in proximity, flocking, blending and time-sensitive (temporal) situations.

[0183] FIG. 1 shows some of the more important object oriented classes in the RTE, and how they are divided into object oriented groups, called packages.

[0184] The motion package 132, includes the motion class 134, sub-classes of the motion class called primitive motion classes 136, and the trajectory class 138. The motion class is the class which is used to defined the motion of an object. Each object for which the OpenMotion System is to compute a motion is given an instance derived directly from the motion class or from a subclass of this class. The motion class is accessible through the API and is explained below in greater detail. The primitive motion classes are sub-classes of the motion class which model particular types of relatively simple motions. These primitive motion classes come predefined as a standard part of the System. Many of them are explained in greater detail below. The trajectory class are used to define the path through space defined by a motion over a given portion of time.

[0185] The dispatch package 140 is a group of related classes which the RTE uses to update motions, schedule events, and represent time. These include the Clock class 142, the Dispatcher class 146, and the ScheduledItem class 148. The clock class is used to define a computational object which represents the passage of time. The dispatcher class determines the order in which Behavior, ScheduledItem, and Motion class instances are updated. The ScheduledItem class, defines items which require updating at a given time.

[0186] The Behavior package deals with state changes and various predicates. This includes the Behavior class 152, the Group class 154, the When class 156, and the Command class 160. The Behavior class 152 controls and changes the spatial attributes of a motion, including the first and second derivations of at least some such attributes. This class is accessible through the API and is defined in much greater detail below. The Group class enables behavior commands to be grouped together. The When class monitors an OpenMotion-defined predicate that controls state transitions. The Command class is an encapsulation of certain actions, such as the setting of a spatial controller.

[0187] The boundary package 162 deals with boundaries. The boundary class 166 defines surfaces and volumes with which motions can interact. It is accessible through the API and is discussed in more detail below. The pShape class 164, which stands for “primitive shape”, is a class used by boundary predicates, that is, predicates that test whether or not a given point, such as that associated with a motion, has a particular spatial relationship with a boundary. Such predicates are described in greater detail below in section 6.2.7.5.

[0188] 6.1.4 The Motion Translator, the OpenMotion Format, ClipMotions and the CMF

[0189] The System also contains the Motion Translator 170, shown in FIG. 1. This includes an interpreter for reading, and a translator for writing, statements in the OpenMotion Format, a language and file format described in more detail below. The OpenMotion Format enables one to make statements which are equivalent in their effect on the OpenMotion System to virtually any statements which can be made using the C++ OpenMotion API. The OpenMotion Format allows statements to be made which are interpreted, and, thus, do not require compilation. Its statements can also be stored on disk and then interpreted by the Translator in real time, as desired by an application program. The translator's interpreter parses the OpenMotion Format and then calls the corresponding functions of the API.

[0190] The ClipMotion Format (“CMF”) is a subset of the OpenMotion Format used for the efficient storage, retrieval, and classification of motions created using the OpenMotion System. Files written in CMF which record motions are called ClipMotions. The OpenMotion System will normally include a ClipMotion Library 172, which defines a relatively large number of motions. A user or an application developer can create more ClipMotions 174. When a ClipMotion file is read and translated by the Motion Translator, the Translator will interpret the file into a set of corresponding calls to the OpenMotion API.

[0191] FIG. 4 is a simplified block diagram showing how ClipMotions provide for persistance in the definition and specification of motions in the OpenMotion System, by allowing motions to be read from and written to a mass storage device 178. These saved Clipmotions 176 shown in FIG. 4 can included the ClipMotion Library 172 of FIG. 1, which comes as part of an OpenMotion System, as well as the new ClipMotions 174 of FIG. 1, which are programmed by a user, by an application programmer, or, in the future, by third-party ClipMotion vendors.

[0192] Some embodiments of the System will include a Motion Classifier 180. The Classifier 180 will sort motions created using the API or read from an external source into discrete classes. The classification used by such sorting can be used to label motions and ClipMotions, and to sort them when stored in a file system. For example, motions can be classified as to whether they are primitive or composite, as defined below, by what motion classes they are derived from, or by the class of motion-enabled model they are designed to provide motion to, such as four legged animals. These motions are accessible for assignment to any objects in different programming environments that contain the OpenMotion System. For example, all kinds of motions for specific types of objects such as animals, vegetation, terrain vehicles, flying vehicles, flags, balls, people, two legged animals, insects, fish, monsters, etc., can be generated. Similarly different classes of motions such as walking, jumping, flying, spinning, or combinations of these can be created.

[0193] The OpenMotion System also includes a class of motions that represent emotions, called ClipEmotions. These are a specific class of ClipMotions that, through motion, characterize emotional behavior like “doze-off”, “act frightened”, “act disappointed”.

[0194] 6.1.5 Motion-Enabled Objects

[0195] The invention includes the extension of the OpenMotion Format to provides for the storage and access of Object Models. A motion-enabled object is one that is modeled in such a way as to facilitate the addition of separately defined motion to it and its parts. For example, a motion-enabled object representing a four legged animal will contain sufficient information to instruct the System how to connect it to any ClipMotion designed to provide motion to such a model. The motion-enabled object is built in a hierarchical manner with a classic tree structure that contains additional information at each of the nodes. This additional information, or metadata, incorporated in the model or added to the model facilitates the attaching of motions in an intelligent manner. The metadata includes the object's physical structure, its hierarchical structure, its geometry, its pivot points, and various coordinate systems. For example in the case of the fish, each connected pair of parts is built as a ball and socket pair. It is intended that representations of all this information will be supported by future extensions to the OpenMotion Format.

[0196] We have identified a number of items that are part of the motion-enabling of models and these include the hierarchy structure (node structure and naming), pivot point specification, axes normalization (X, Y, or Z), and scaling information.

[0197] A motion-enabled model may be directly connected to a ClipMotion file. It is possible that the object with which a motion has been associated may change its form as an application progresses, or that a user or application might well try to apply a motion to an object having a very different structure than the structure of the class of objects for which it was originally designed. For example, a user might apply a ClipMotion file representing a fish motion to an airplane motion-enabled model. There will be rules which provide for mapping the object hierarchy of a ClipMotion to node hierarchy of a motion-enabled object, when those two hierarchies do not have the same structure.

[0198] We call a motion-enabled model and its associated motion Motion Art.

[0199] 6.1.6 Encryption

[0200] The design of the OpenMotion System provides the equivalent of a “watermark”, an encryption technique, in our ClipMotion, motion-enabled object, and Motion Art format. This will help in guaranteeing that models can be owned and that purchasers with the right key could have access. This will also support making models available only for certain application but not others. This should help provide incentive for the development of a rich and varied collection of ClipMotions and motion-enabled objects.

[0201] 6.1.7 Motion Editor/Viewer

[0202] The Motion editor/viewer 182, shown in FIG. 1, is a part of the OpenMotion System which allows programmers to interactively access, view and edit motions, whether these motions are created by programming in C++ or have been recorded in the ClipMotion Format. The motion editing dialog boxes described below are a subset of this interface

[0203] 6.1.8 Description of the Types of Motions Supported by the System

[0204] Tables 1, 2, and 3, below, respectively, show some of the Procedural Motions and Kinematics, some of the interactive motions, and some of the behavioral. motions which are included in the design of the OpenMotion System.

[0205] The OpenMotion API 102, shown in FIG. 1, is a C++ class library made available to programmers by the OpenMotion System for incorporating motion into real-time interactive 3D applications. The API provides a powerful and flexible set of functions for adding motions to applications.

[0206] 6.2.1 Scope

[0207] The OpenMotion API is designed to work with 3D applications, such as the application 104 shown in FIG. 1, without trying to become the center of an application's architecture. It does not provide graphic rendering. It is independent of and compatible with any graphic API or game engine the programmer may choose. It does not incorporate physical dynamics. It does provide general-purpose motion parameters that may be driven interactively or under control of the application. These parameters may be linked to variables representing anything ranging from game play to a physical simulation.

[0208] 6.2.2 Overview

[0209] The OpenMotion API comprises a set of C++ classes that the programmer uses to do the work of motion calculations, including synchronization, interaction, and update of multiple simultaneous motions. The classes implement new data types that are declared in the program code. Expressions embedded in the program using these types are, updated automatically during motion processing.

[0210] Although OpenMotion API has an object-oriented class architecture, it is neither intended nor necessary for the programmer to derive new classes from OpenMotion API base classes. Instead, OpenMotion API provides a complete set of flexible types that can be reconfigured to any situation. If necessary, the programmer may extend OpenMotion API by supplying callbacks that add application-specific functionality.

[0211] 6.2.2.1 Namespace

[0212] The classes and functions that comprise OpenMotion API are declared in a header file:

[0213] #include “om.h”

[0214] Naturally, class names used in the OpenMotion API are not completely unique. To avoid the chance that other APIs or the application might use the same names, we use the C++ namespace feature. All OpenMotion API classes and functions are encapsulated. in a namespace. In most cases, the application program simply declares use om; and the OpenMotion API can be accessed directly. In the event of a naming collision, the programmer qualifies OpenMotion API names with the om::scoping operator.

[0215] Within this document, the scoping operator om:: indicates a class or function that is visible at global scope. Member values and functions are indicated by the scoping operator for that class, e.g., Motion::

[0216] 6.2.2.2 Update Loop

[0217] At the top level, most applications, such as the application 104 shown in FIGS. 1-3, have some kind of update loop. In many circumstances, this loop is part of the application program. In other situations a graphic API or game engine supplies the update loop, and the application supplies a callback function. OpenMotion API is compatible with both cases.

[0218] 6.2.2.2.1 Loop in Application

[0219] OpenMotion API is integrated into an application which can be programmed to call its functions directly using one or two function calls:

[0220] while (!done) // application's update loop

[0221] {

[0222] // . . . application-specific code here . . .

[0223] om::update( ); // required: update all motions

[0224] // . . . graphic display-list updates here . . .

[0225] om::frame( ); // recommended: mark end of graphic-update The first call, om::update( ), is required. It performs all the actions and motion calculations for the next update. The second call, om::frame( ), is recommended. This call is normally used to provide OpenMotion with an indication of the actual time at which the objects for which it has calculated positions are shown to users on the screen 124 shown in FIG. 3. Thus, it allows OpenMotion API to compensate for latency in the update cycle due to both application and graphic processing time, by delaying the actual time for which it calculates the position of objects to reflect such delays. If om::frame( ) is not used, the motions will not be affected, but the display will lag behind the time for which OpenMotion calculates positions by the graphic and application update latency.

[0226] The above programming represents one way of achieving part of the information flow indicated by the arrow 121 shown in FIG. 3. After om::update( ) updates the frame, the application would then ask the API for the spatial attributes for each object or sub-object in the display list, as indicated by the comment after the call to om::update( ), and then call the renderer 106 over the communication path represented schematically by arrow 123 in FIG. 3 to render the display list.

[0227] 6.2.2.2.2 Loop in Graphic System

[0228] Many graphic programs, such as commercial, graphic system or game engine, interface with other programs though callbacks. This is commonly the case for commercial program which are sold in object code form, and, thus, which cannot be programmed to include calls to OpenMotion. With programs which use a callback architecture, OpenMotion API can be integrated like this:

[0229] This code represents another way of communicating between the programs shown in FIG. 3.

[0230] Simulation Clock

[0231] The OpenMotion API internally maintains a simulation clock. This clock can be configured by the programmer as a real-time clock, or as a fixed-increment frame clock. A unique instance of the class Dispatcher 140, shown in FIG. 1, maintains the Clock (see table 4 below), which is an instance of the clock class 142 also shown in that figure. The simulation time begins at zero when the system is initialized and increases monotonically.

[0232] The real-time mode is recommended for most interactive applications, and it is the default configuration. The fixed-increment mode is useful for debugging and certain animation applications.

[0233] The simulation clock can also run faster or slower than real-time, by configuring a speed factor.

[0234] 6.2.3 Principal Types

[0235] There are three principal families of classes in the API. Various specialized classes are derived from these families.

[0236] I. An instance of the motion class, or motion, contains state information for multiple spatial degrees of freedom, such as translation, rotation, and scaling which can be associated with an object. Each degree of freedom has a characteristic trajectory. Motions may be combined by expressions that specify hierarchical parent-child relationships, expressions that specify blending of simultaneous motions, and expressions that extract attribute information.

[0237] II. An instance of the behavior class, or a behavior, is used as a parameter of an instance of the motion class. A motion's behavior describes how one or more degrees of freedom changes over time. Behavior expressions can specify a state machine in which each state has a set of initializing actions, and a set of transitions.

[0238] The actions set up the controllers for the motion using that behavior. The state transitions are triggered by predicates, expressions, and callbacks.

[0239] III. An instance of a boundary class, or a boundary, is geometrical entity defining a surface or volume that can interact with one or more motions. A motion may have a boundary attribute, which defines a surface or volume that travels with the point defined by the motion's position attribute. A motion's boundary attribute. is commonly used to model the surface of the object being represented by a motion.

[0240] The programmer specifies interactions in terms of boundary predicates that are combined in Boolean. expressions to form triggers. The programmer may use boundaries to represent graphic objects, or the boundaries may represent virtual objects or regions of space.

[0241] These principal types, together with Vars discussed below, have built-in memory management, so that memory is automatically deallocated when the object is no longer used.

[0242] 6.2.4 Expressions

[0243] Expressions, referred to as “algebraic expressions” above, may be regarded as the “cables” that interconnect the elements of the motion model. The programmer uses expressions to link motions, behaviors, and boundaries to each other, and to transmit data to and from application-specific code. Expressions look just like regular arithmetic operations; they differ because OpenMotion API retains each expression, and defers the actual calculation until the results are needed. Then the expression is automatically evaluated, and the results are passed on to any other expression that might require the information using an “observed-observer” architecture, somewhat similar to that used to propagate related changes through a spreadsheet.

[0244] When an expression is used as input to one of the principal types (i.e., motion, behavior, or boundary) it is called a parameter. A parameter is a configurable value or expression that affects the character of an object.

[0245] A characteristic or output of a principal type is called an “attribute”. An attribute is a value that is derived from the current state of an object. Functions are provided to extract attributes from objects. In general the form object.attribute( ) returns a value, whereas the form attribute( object ) returns a Var expression. This is explained further in the next few sections.

[0246] 6.2.4.1 Values

[0247] The simplest kind of expression is a value. The OpenMotion API has its own arithmetic value types:

[0248] Boolean. True or False.

[0249] Scalar. Representation of a real number.

[0250] Vector. A 3D vector implemented in terms of Scalar.

[0251] Quaternion. An arithmetic entity uniquely suited to represent orientation.

[0252] Matrix. A 4×4 homogeneous transformation matrix.

[0253] Value types are generally available to the programmer for use as automatic variables anywhere in the application. A full complement of basic math functions (table 5) and C++ operators (table 6) are overloaded and behave as expected. Operations among these types provide immediate numerical results.

[0254] Value operations have been optimized with efficient C++ implementations. Although low-level optimization using assembly-language is possible, experience has shown that motion processing is insignificant compared to graphic rendering.

[0255] 6.2.4.2 Notation

[0256] In this document, value types are written using literal values and punctuation illustrated in table 7. In source code, values use explicit class constructors. Multiple constructors are offered for conversions between types, only some of which are shown in this table.

[0257] 6.2.4.3 Predefined Values

[0258] Each value type has predefined constant values that are available for general use. To access these constant values, they must be qualified using that type's scoping operator. For example, Scalar::Zero has the value 0. These constants are given in table 8.

[0259] Two directions may be configured by the programmer to reflect the orientation of the application's prevailing coordinate system. The default coordinate system is right-handed, with X axis to the right, Y axis forward, and Z axis up. The programmer may specify an alternate coordinate system by supplying direction vectors forward and up, and rotation convention of counter-clockwise (right-handed) or clockwise (left-handed). This configuration determines the values for the standard orientations, as well as the interpretation for Euler angles of roll, pitch, and yaw. The programmer optionally configures the coordinate system using

[0260] om::defineCoordinates(const Vector& forward,

[0261] const Vector& up,

[0262] const Boolean& isLeftHanded=False);

[0263] This setting remains in effect until reset by the user. The default values are: forward=Vector::Yaxis, up=Vector::Zaxis, isLeftHanded=False.

[0264] 6.2.4.4 Vars

[0265] Retained expressions are typed to correspond to their value type, suffixed with “Var”:

[0266] BooleanVar

[0267] ScalarVar

[0268] VectorVar

[0269] QuaternionVar

[0270] MatrixVar

[0271] When Var types are used in an expression, that expression is automatically retained until needed. Vars and values can be mixed indiscriminately; value types will be promoted to Var types. Vars are memory-managed objects, so that if all references to a Var have been deleted, the expression is de-allocated and its memory is returned to the heap.

[0272] Just as the C++ compiler combines a constant sub-expression into a single constant value, OpenMotion API will evaluate a value sub-expression once, convert it to a constant-valued Var, and use it repeatedly as needed.

[0273] 6.2.4.5 Callbacks

[0274] Most functionality can be specified directly using retained expressions. However, the program may embed application-specific algorithms and values in retained expressions by defining a callback function. The callback function must return an OpenMotion API value type.

[0275] Callbacks support a finite number of function signatures. The programmer must choose. to implement one of the supported signatures. Many useful signatures are available. User-defined data may be passed to function signatures supporting void*.

[0276] const Boolean fcn(void);

[0277] const Boolean fcn(const Boolean&);

[0278] const Boolean fcn(const Boolean&, const Boolean&);

[0279] const Boolean fcn(const Scalar&);

[0280] const Boolean fcn(const Scalar&, void*);

[0281] const Boolean fcn(const Scalar&, const Scalar&);

[0282] const Boolean fcn(const Vector&, const Vector&);

[0283] const Boolean fcn(const Quaternion&, const Quaternion&);

[0284] const Boolean fcn(const Matrix&, const Matrix&);

[0285] const Scalar fcn(void);

[0286] const Scalar fcn(const Scalar&);

[0287] const Scalar fcn(const Scalar&, void*);

[0288] const Scalar fcn(const Scalar&, const Scalar&);

[0289] const Scalar fcn(const Vector&);

[0290] const Scalar fcn(const Quaternion&);

[0291] const Vector fcn(void);

[0292] const Vector fcn(const Vector&);

[0293] const Vector fcn(const Vector&, const Vector&);

[0294] const Vector fcn(const Scalar&);

[0295] const Vector fcn(const Scalar&, void*);

[0296] const Vector fcn(const Scalar&, const Scalar&, const Scalar&);

[0297] const Vector fcn(const Vector&, const Scalar&);

[0298] const Vector fcn(const Quaternion&);

[0299] const Vector fcn(const Quaternion&, const Vector&);

[0300] const Vector fcn(const Matrix&);

[0301] const Vector fcn(const Matrix&, const Vector&);

[0302] const Quaternion fcn(void);

[0303] const Quaternion fcn(const Quaternion&);

[0304] const Quaternion fcn(const Quaternion&, const Quaternion&);

[0305] const Quaternion fcn(const Scalar&);

[0306] const Quaternion fcn(const Scalar&, void*);

[0307] const Quaternion fcn(const Scalar&, const Scalar&, const Scalar&);

[0308] const Quaternion fcn(const Vector&, const Scalar&);

[0309] const Quaternion fcn(const Matrix&);

[0310] const Matrix fcn(void);

[0311] const Matrix fcn(const Matrix&);

[0312] const Matrix fcn(const Matrix&, const Matrix&);

[0313] const Matrix fcn(const Scalar&);

[0314] const Matrix fcn(const Scalar&, void*);

[0315] const Matrix fcn(const Matrix&, const Scalar&);

[0316] const Matrix fcn(const Matrix&, const Vector&);

[0317] const Matrix fcn(const Matrix&, const Vector&, const Scalar&);

[0318] const Matrix fcn(const Vector&);

[0319] const Matrix fcn(const Quaternion&);

[0320] The programmer references a callback using the syntax,

[0321] call(fcn, <args> . . . )

[0322] where fcn is the programmer-defined function and <args> are the arguments. OpenMotion API automatically takes care of calling the function when the arguments have been updated and value is needed.

[0323] For example, the programmer defines a callback implementing an application-specific function of time that calculates an angle. This angle is input to a dynamic expression containing a sinusoid:

[0324] 6.2.4.6 Temporal Predicates

[0325] Predicates are functions that return a Boolean expression. Temporal predicates allow the programmer to sequence movements based on time.

[0326] 6.2.5 Motions

[0327] Motions are idealized points traveling through space. Each point has a number of degrees of freedom. Mathematically, these degrees of freedom (DOFs) naturally fall into groups. For example, translation is a 3D group that is described by vector algebra. The Rotation group behaves quite differently; it is described by quaternion algebra. Scaling is yet another 3D group; vector algebra does not handle scaling, but it is a subset of homogeneous (4×4) transformation matrices. In other embodiments, additional degrees of freedom will be considered (shearing, bending, tapering, to name a few).

[0328] One creates a Motion by declaring a variable using the Motion type.

[0329] Motion myMotion;

[0330] 6.2.5.1 Parameters

[0331] The parameters of Motion specify its initial conditions and behavior. The initial conditions specify values for the trajectory at simulation time zero. The behavior sets up which parameters are interactive and which are derived, and how a Motion responds to new conditions.

[0332] 6.2.5.1.1 Initial Conditions

[0333] Internally, each DOF group is assigned its own trajectory. All trajectories must “start” somewhere. By default, all motions begin at the origin, with zero velocity. The programmer may specify an initial condition for each motion to make it unique to its intended application. This is done using the syntax

[0334] myMotion.<initial condition>(<initial value>);

[0335] where <initial condition> specifies which parameter, and <initial value> specifies its value.

[0336] Methods that set a parameter return a reference to the modified object. This feature allows “chaining” of parameter configurations into one C++ statement. For example,

[0337] myMotion

[0338] .initialPosition(Zero)

[0339] .initialVelocity(Vector::Xaxis)

[0340] .initialSpin(Vector::Yaxis*Scalar::Pi2/10);

[0341] This statement causes myMotion to turn about its Y axis once every 10 seconds, while translating along the X axis. Of course, this specification is not very flexible because the initial conditions remain constant over time. However, many useful motions can be created using nothing but initial conditions.

[0342] If multiple initial conditions are set for a given action which are derivatives or integrals of each other, the last set of these initial conditions will dominate. Thus, if an initial value is set for acceleration and then for velocity of a given motion, the velocity value will dominate, meaning the velocity will stay constant, and acceleration will become zero until either a new initial condition affecting position is set for the motion or a behavior attached to the motion changes this condition. If, on the other hand, velocity is set first and then acceleration, the acceleration value would dominate and the motion would accelerate continuously until a new initial condition or a behavior changes this condition.

[0343] 6.2.5.1.2 Behavior

[0344] Motions have a parameter called Behavior, which is described in a subsequent Motion objects. For example, p1 myMotion.behavior(myBehav); // chainable

[0345] . . . or . . .

[0346] myMotion.behavior( )=myBehav; // as assignment

[0347] Each object gets its own copy, or instance, of the Behavior.

[0348] 6.2.5.2 Attributes

[0349] A trajectory, corresponding to an instance of the trajectory class 138 shown in FIG. 1, for each DOF group is always defined, and we can use the trajectory to get the current conditions, to predict short intervals into the future, or even to extrapolate into the past. Functions are provided to get these attributes from a Motion object.

[0350] 6.2.5.2.1 Current Attributes of Motion

[0351] The current attributes of a motion are given in table 12.

[0352] 6.2.5.2.2 Predict

[0353] A motion's trajectory is constantly adapted to inputs from control variables (more on this later). At any moment, the trajectory represents a curve in space that goes through the current position, and continues into the future (and past). Of course, some kind of interaction may occur in the near future that could drastically change the prediction. However, based on the current situation, barring drastic changes, the prediction is pretty accurate. All the predictors take one argument representing the simulation time.

[0354] 6.2.5.2.3 Boundary

[0355] Boundaries are discussed in detail in their own section, below. The boundary attribute of an individual motion affects the character of the motion indirectly. A motion's boundary participates in interactions between the motion. and other boundary objects. Multiple motion objects may have different instances of the same boundary class as their boundary attribute. A given motion's boundary is defined within the motion's individual frame of reference. A boundary is associated with a motion as shown in the example:

[0356] VectorVar origin;

[0357] ScalarVar radius,

[0358] Boundary myBounds=Sphere(origin, radius);

[0359] Motion myMotion;

[0360] myMotion.boundary( )=myBounds; // parameter assignment

[0361] . . . or . . .

[0362] myMotion.boundary(myBounds); // equivalent form

[0363] Note that origin and radius are expression types, which may be bound to dynamic expressions. For example, radius might be defined by a formula or user-input value.

[0364] 6.2.5.3 Complex Motions

[0365] Motions may be combined into complex forms by use of hierarchy and blending. C++ operators have been overloaded to make this notation easier.

[0366] 6.2.5.3.1 Hierarchy

[0367] Motions may be assembled into hierarchies. This could be used, for example, to dynamically assemble various graphical parts, all of which move independently. A Motion object may have any number of children, but only one parent. The parent establishes a frame of reference for its children. The global( ) attribute discussed above composes the local transformation with its parent's global transformation.

[0368] There are two ways to specify a hierarchy using member functions or the operator /. These forms are equivalent:

[0369] The notation for hierarchy uses operator / because it is similar to a file system hierarchy path name. A complex system of motion hierarchies can be set up simply by stating the various path specifications. When a hierarchical. motion is associated with a given object by a program or programmer using the API, expressions can be used to define the length between nodes of the hierarchy, so that, for example, as a parent node rotates its child nodes will be moved by the appropriate amount.

[0370] 6.2.5.3.2 Blending

[0371] Sometimes a complex motion is a blend of two motions acting independently. In contrast to hierarchy, blending is accomplished by weighted averaging of degree-of-freedom parameters, not by composing transformations. This is written using operator + and operator *. The multiplication operator is used to scale a motion by a constant or variable scalar value, and the addition operator combines the degrees-of-freedom of different motions.

[0372] This technique can be used, for example, to effect smooth transitions between a set of concurrent motions. As one may infer, a MotionVar is a variable that refers to a Motion value or a motion expression. A MotionVar has the same attribute interface as Motion value, except no input parameters.

[0373] 6.2.6 Behaviors

[0374] Although one can specify simple motions using nothing but initial conditions, they cannot change or respond to external stimuli. To specify this, OpenMotion API uses a class called Behavior. A Behavior is an action that changes a Motion's parameters. Behaviors are constructed by declaring Behavior variables and assigning descriptions to them.

[0375] BehaviorVar myBehav;

[0376] myBehav=<action>;

[0377] . . . or . . .

[0378] myBehav=(<action>, . . . );

[0379] A Behavior expression may be a single action, or a list of actions enclosed in parentheses and separated by commas. Behavior expressions are sequenced to produce complex responses. The ordering of actions is significant; the actions are executed in the specified order. Taken together, a collection of related Behaviors comprise a finite state machine.

[0380] 6.2.6.1 Interactive Controllers

[0381] One of the most useful actions is to effect an interactive controller. A controller links a parameter of motion to an expression. While the controller is in effect, the trajectory of the motion is continually adapted to this input parameter.

[0382] Only one controller for each DOF group can be active at one time. For example, a velocityControl means that the position is derived from the velocity by integrating over time. Actions occur in the order specified, so that the last action for each DOF group will be the one that remains in effect.

[0383] 6.2.6.2 Constant Controllers

[0384] Sometimes one needs to set a motion parameter to a fixed value. This is done by a sub-class of behaviors called a constant controller. These actions evaluate their expressions just once when they come into effect. They modify the trajectory to keep this value constant. For continuity, previous conditions remain in effect. For example, setting a constant velocity would set the acceleration to zero (since nonzero acceleration would change the velocity), but the calculated position would join the old and new trajectories at that point in time.

[0385] Note that constant attributes can still effect interesting motions. For example, projectile motion is simply constant acceleration, namely gravity.

[0386] 6.2.6.3 Reset Parameter

[0387] Sometimes a discontinuity is desired. For example, a ball bouncing off the floor has a sudden change in its velocity vector. In this case one must reset a motion parameter. These actions evaluate their expression arguments once, and reset. the corresponding. parameter. If a controller is in place, the reset action does not remove that controller. Derived parameters, if any, are not affected. For example, one may reset the velocity while maintaining continuity of position.

[0388] 6.2.6.4 States and Transitions

[0389] So far we have discussed simple actions. These actions change the trajectories of the motion that uses the behavior. We now consider actions that change the behavioral state of the motion.

[0390] A behavior may represent a complex finite state machine. Each state contains a list of actions to be performed when entering that state. Each state also has a list of state transitions. Each transition is guarded by a Boolean condition. The conditions are activated when the state is first entered. When the condition evaluates to true, the corresponding action is triggered. The action is either a callback, a state transition, or both.

[0391] The basic syntax for a state transition is:

[0392] when(<condition>).perform(<behavior>);

[0393] The when (<condition>) clause contains the trigger condition; the .perform(<behavior>) clause specifies the new state when and if that trigger fires. Such a when conditional is an instance of the when class 156, discussed above with regard to FIG. 1. The when clause may also have an optional callback attached. For example:

[0394] BehaviorVar state1;

[0395] BehaviorVar state2;

[0396] BooleanVar cond1(false);

[0397] VectorVar p;

[0398] VectorVar v;

[0399] state1=(positionControl(p), when(cond1).perform(state2));

[0400] state2=(velocityControl(v), when(!cond1).perform(state1));

[0401] myMotion.behavior( )=state1;

[0402] These statements set up two states controlled by a Boolean variable. Under state1, the motion exhibits a position control attached to expression p; under state2, the motion has a velocity control attached to expression v. A state transition occurs whenever cond1 changes value under control of expressions attached to cond1, or by direct control of the application.

[0403] 6.2.6.5 Assignments and Callbacks

[0404] There are several actions that allow the programmer to set application-specific variables within each behavioral state. They include assignment and callbacks. The action is performed once each time a state is entered.

[0405] For example:

[0406] BooleanVar cond1;

[0407] ScalarVar myVar;

[0408] BehaviorVar yourState;

[0409] . . .

[0410] BehaviorVar myState=(

[0411] assign(myVar, <expr>),

[0412] call(fcn),

[0413] when(cond1).perform(yourState));

[0414] Callbacks can also be attached to trigger conditions. In this case the callback occurs only when the trigger condition becomes true, as the Behavior is about to exit the state. The forms are:

[0415] when(<condition>).call(fcn, pdata ).perform(<behavior>)

[0416] when(<condition>).call(fcn).perform(<behavior>)

[0417] when(<condition>).call(fcn, pdata)

[0418] when(<condition>).call(fcn)

[0419] where pdata is an optional void* pointer to programmer-defined data, and fcn is a programmer-defined function having the signature void fcn (void* p=0). In other words, the callback is optional, and it may occur with or without a state transition.

[0420] 6.2.6.6 Attributes and Predicates

[0421] Behaviors have attributes that may participate in expressions and parameters. These attributes are useful for calculating movements, setting up triggers, and coordinating triggers between different states.

[0422] For example, a state that is active for 10 seconds, then transitions to another state:

[0423] BehaviorVar state0;

[0424] BehaviorVar state1;

[0425] state10=(velocityControl( . . . ), . . .

[0426] when(localTime(state10)>10).perform(state0));

[0427] 6.2.7 Boundaries

[0428] Although Motions are independent of graphics, a Motion object must know something about geometrical shapes in order to interact within a 3D scene that is represented graphically to the user. A Boundary is a geometrical shape which works within the dynamic OpenMotion API environment.

[0429] 6.2.7.1 Primitive Shapes

[0430] In the embodiment of the invention described here there are seven primitive Boundary shapes. In other embodiments there can be more or different primitive Boundary shapes. These shapes share a common Boundary interface, each adding its own unique properties

[0431] PointShape

[0432] Point(const Vector& origin);

[0433] Edge Shapes

[0434] Line(const Vector& origin, const Vector& direction);

[0435] LineSegment(const Vector& origin, const Vector& end);

[0436] Surface Shapes

[0437] Plane(const Vector& origin, const Vector& normal);

[0438] Rectangle(const Vector& origin, const Vector& h, const Vector& w),

[0439] Volume Shapes

[0440] Sphere(const Vector& origin, const Scalar& radius);

[0441] Box(const Vector& origin, const Vector& size);

[0442] These elementary shapes may be regarded as value types in a Boundary expression. To get different orientations, one may either associate the Boundary with a Motion by making it the boundary attribute for a motion, or one may construct Boundary expressions using regular transforms to modify the primitive shapes. These primitive shapes are available for general use within the application. They support functions to get attribute values, as well as spatial computations.

[0443] In other embodiments of the invention, the API will be modified to allow Boundaries to be defined by more complex shape modeling methods such as splines, NURBS, and parametric solid geometry; and Boundaries will be able to be derive their shape definition directly from standard data structures used to define shapes by major 3D modeling programs.

[0444] 6.2.7.2 Attributes

[0445] The attribute functions in table 19 apply to all boundaries regardless of shape.

[0446] 6.2.7.3 Composite Boundaries

[0447] Composite boundaries allow the programmer to construct complex boundaries by using Boolean logic to combine basic shapes. These composite boundaries permit logical groupings of motions, which will allow collision detection among groups of objects, without having to specify all combinations explicitly.

[0448] 6.2.7.4 Expressions

[0449] A BoundaryVar is a variable that may take on Boundary values and dynamic expressions. Boundary expressions support functions that can be used as parameters to Behaviors and other Boundaries. The functions in table 20 are dynamic analogues to the attributes and functions of the primitive shapes.

[0450] In the following example, the Boundary is controlled by a dynamic expression, which is also transformed by the Motion's frame of reference:

[0451] VectorVar origin;

[0452] Motion myMotion;

[0453] myMotion.boundary( )=point(origin);

[0454] 6.2.7.5 Predicates

[0455] Predicates are especially useful when constructing conditions that trigger state transitions. Boundaries support their own set of predicates.

[0456] First we consider proximity predicates. These predicates test the relationship between a Motion's boundary attribute and another boundary.

[0457] Spatial predicates test the relationship between a Motion and another boundary. For spatial predicates the Motion is an idealized point, defined by the Motion's position, regardless of its boundary attribute.

[0458] For edge or point boundaries, isInside and isBelow are always False; isOutside and isAbove are always True.

[0459] 6.2.7.6 Behaviors

[0460] Sometimes a Motion's behavior is constrained by interactions among shapes. There are special Behavior actions for these situations.

[0461] The parameters, gain and bias, affect the normal and tangential components of a boundary interaction. These parameters can be adjusted to simulate effects of gain or loss of momentum, for example due to elasticity and friction. To set these parameters, they are chained off the end of the boundary Behavior,

[0462] VectorVar myGain;

[0463] VectorVar myBias;

[0464] BoundaryVar myBounds;

[0465] BehaviorVar myBehav=reflect(myBounds).gain(myGain).bias(myBias);

[0466] 6.2.8 Examples

[0467] 6.2.8.1 Swing

[0468] Swing is extracted from our motion editor prototype. Swing has lots of parameters so that everything is dynamically adjustable through the user interface. It is a typical example of the use of the cyclic( ) temporal predicate. Also note the use of simTime( ), which makes signal a dynamic function of time. Swing is one of the most often-used motions, because it can be applied to any articulated joint to create animated models. Most of this code is declaration of parameters; the motion specification itself is three statements.

[0469] 6.2.8.2 Two Balls Bouncing in Box

[0470] This example uses boundaries. Two balls of equal radius are in a box. They bounce elastically off each other and the box, with gravity. The box can be dynamically moved and resized, as can the radius of both balls. (For brevity, we do not validate the input parameters.)

[0471] const Vector gravity(0, 0, −9.8);

[0472] VectorVar boxOrigin(0, 0, 0);

[0473] VectorVar boxSize(10, 10, 10);

[0474] ScalarVar ballSize(1);

[0475] BoundaryVar myBox=box(boxOrigin, boxSize);

[0476] BoundaryVar Ball=Sphere(Vector::Zero, ballSize);

[0477] Motion ball1;

[0478] Motion ball2;

[0479] ball1.initialPosition(2, 2, 2)

[0480] .initialVelocity(1, 1, 1)

[0481] .initialAccleration(gravity)

[0482] .boundary(aBall)

[0483] .behavior((reflect(myBox), reflect(ball2.boundary( ))))

[0484] ball2.initialPosition(8, 8, 8)

[0485] .initialVelocity(−1, 1, 1)

[0486] .initialAcceleration(gravity)

[0487] .boundary(aBall)

[0488] .behavior((reflect(myBox), reflect(ball1.boundary( ))));

[0489] 6.2.8.3 Motion Derivatives

[0490] Motion Derivitive shows how the 0-2^ndorder motion derivatives can be used as parameters to create complex, interrelated motions

[0491] 6.2.8.4 Motion Blending

[0492] Blend shows how motions can be combined

[0493] 6.2.8.5 Boundary Bounce

[0494] Boundary Bounce shows how boundaries can be created and combined with constraints and spatial predicates to create complex behaviors. For this example, we construct a “room” by combining simple boundary shapes and define a motion with behavior for bouncing off the “walls” or surfaces of the room. We further define the “balls” behavior by the “room”, we also define a “goal” volume within the room and stopping the ball's motion if it penetrates the “goal”.

[0495] The OpenMotion File Format (of which the ClipMotion Format is a subset) is a universal motion description language. Ideally, the OMF is completely equivalent and interchangeable with a C++ representation, or with a protected binary representation. The only differences are those imposed by the syntactical rules and conventions of the host language, such as C++ (or Java). The Translator 170, shown in FIG. 1, can read and write the OMF in ASCII or binary. Furthermore, the translator can export a set of OMF statements, such as a ClipMotion, into a C++ source program that may be immediately compiled and linked with a user's motion-enabled program.

[0496] 6.3.1 Notation

[0497] BNF (Backus-Naur Form) is a meta-language for specifying the syntax of computer languages. Here are the conventions we are using:

[0498] 6.3.2 Syntax Specification

[0499] The following is a current specification of the OMF. This specification will be expanded to include more functionality as the functionality of the API is extended.

[0500] 6.3.3 Examples

[0501] Here are a few examples (use case scenarios) of the ClipMotion format.

[0502] 6.3.3.1 The Simplest Motion

[0503] A constant linear velocity.

[0504] motion myMotion is {initial velocity is [1 0 0];};

[0505] 6.3.3.2 Velocity Controller Restricted to Plane p In the example below a six-degree-of-freedom input device is used as a velocity controller and the motion is constrained to the xz plane. In this example the 6 DOF input device is a Spaceball, a 6 DOF input device made by Labtec, a company located at 1499 SE Tech Center Place, Ste. 350, Vancouver, Wash. 98683-9575. The Spaceball is only one of multiple input devices which the OpenMotion API will support.

[0506] 6.3.3.3 Flag Waving (Hierarchical Motion)

[0507] Flag is modeled as three “hinged” hierarchical sections. Each section moves relative to the local frame of reference defined by the enclosing, or parent, motion. Nesting the sections effects articulated hierarchical motion.

[0508] 6.3.3.4 One Ball Bouncing in Box

[0509] The ball (radius 0.1) moves at constant linear velocity inside a box and bounces off the walls of the box.

[0510] 6.3.3.5 Two Balls Bouncing in Box

[0511] Two balls bounce in a box. When they collide, they exchange momentum in proportion to their mass.

[0512] parameter radius1 is 0.75;

[0513] parameter radius2 is 1.5;

[0514] boundary myBox is box [0 0 0] [10 10 10] [0 1 0] 0 rad;

[0515] 6.3.3.6 Harmonic (Spring) Motion

[0516] A particle oscillates on the x-axis between [−1 1].

[0517] 6.3.3.7 Circular Motion

[0518] One can specify circular motion using sine and cosine, however this version uses the relationship from physics, which states that the centripetal acceleration is equal to the square of the tangential velocity divided by the radius.

[0519] 6.3.3.8 Orbital Motion

[0520] Orbital motion traces an ellipse, however the velocity varies along this path. We can use Newton's law of gravitation to synthesize this path from initial conditions.

[0521] 6.4.1 Overview

[0522] This section identifies a set of primitive motion classes 136, shown in FIG. 1. These are motions that are available for use as a standard part of the OpenMotion Library. The primitive motions are a sub-set of the OpenMotion Library, a possibly larger set of motions which may include ClipMotions, which will commonly be sold as part of the OpenMotion System. The. primitive motions are constructed using the C++ API functions described above.

[0523] This section also describes part of the interfaces of the Motion Editor/Viewer 182, shown in FIG. 1, which can be used to produce a wide variety of complex actions.

[0524] Giving character to any 3D object is what makes 3D interesting. The object's character is determined by its actions and reactions and these are represented by motion. The motions can be created using some fundamental building blocks. Cartoon-like gestures and emotional expressions are complex but they can be broken down into relatively few elemental or atomic motions. These motions can be adjusted easily in real time using the Motion Editor/Viewer until they look just right.

[0525] The Motion Editor/Viewer interface designs described below are part of the current design for the OpenMotion System. In other embodiments of the invention other design tools could be used. When the OpenMotion Library is incorporated into different products, it might be desirable to provide it with different design tools. For example, motion design tools used with a child's product might be very different than that described above. The idea of what motion is and how it is used is the focus of this section.

[0526] 6.4.2 Foundation Motions for Action Definition

[0527] 6.4.2.1 Atomic Motions

[0528] In the OpenMotion Library there are four primary atomic motions: shake, shift, spin, and swing. In. other embodiments of the invention there can be a differing number and a differing selections of atomic motions and the motions which are derived from them.

[0529] Each of the Open Motion Library's primary atomic motion will be used as a template for all other elementary motions within each category. For example, with various settings of the parameters on spin we can derive new primitives, that is derived primitives like turn, gyrate, and revolve, which are simply variations of spin. All the derived atomic motions indented under spin in Table 24 below are available as part of the OpenMotion Library, but they are simply variations of spin. This makes it easier for the user to select and apply a more precisely defined motion, since such motions can be selected by means of pop-up menus, as is shown in FIGS. 5-8. This variety of motions with pre-defined settings provides a head start on what the various settings should be for a particular motion. Thus gyrate has a different look than spin as a result of the different initial values but each of the motions' parameters can be changed to any specification,

[0530] Each of the above four primary motions depends on the transformations translate and rotate. Some can have spatial constraints such as along a certain axis, vector, or path. Each can have some temporal constraint such as the number of times the motion is to be repeated or whether it is to run continuously.

[0531] The OpenMotion Library currently contains 4 primary and 25 secondary atomic motions (see Table 24), although this number may well be changed in the future.

[0532] 6.4.2.2 Composite Motions

[0533] Motions that comprise more than one primitive motion are called composite motions. An example of this would be spiral which is made up of one spin and two shifts (see Table 25, below, and FIG. 24). Altering any one of these three primitive motions' parameters can change that component of spiral.

[0534] Currently there are 14 primary and 7 secondary composite motions as shown in table 25, although this number may well be varied in the future.

[0535] 6.4.2.3 Reactive Motions

[0536] Reactive motions are atomic or composite motions triggered by a specific motion trigger. These motions generalize object control. Reactive motions are intended to be used in interaction and are defined to facilitate the development of interactive applications. Motion triggers are generalizations of triggers, which typically only have an on/off setting, and provide access to a parameter with varying magnitude and direction (a force). As is indicated by Table 26, the OpenMotion Library currently has 14 primary and 17 secondary reactive motions.

[0537] For example, referring to Table 26, if the reactive motion push/pull is applied to an object, the object will either swing or shift around relative to the mouse interaction. If the reactive motion wind is applied to say a field of daisies then these will sway in response to the direction and magnitude of the parameter.

[0538] 6.4.2.4 Deformations

[0539] Distortions of the geometry such as twist or stretch are known as deformations. As is indicated in Table 27, there are currently 10 primary and 10 secondary deformations in the OpenMotion Library.

[0540] 6.4.2.5 New Motions

[0541] The user can create new motion from motions defined in the OpenMotion Library in several different ways. One is by using the New Motion dialogue boxes shown in FIGS. 25-28. These enable a user to set the Initial Conditions of motion or input an expression for Transition, Rotation, and Scaling.

[0542] 6.4.2.6 Saved Motions

[0543] The OpenMotion Library supplies lists of standard motions the user can work with. A user may change the setting of such a standard motion and save it under a new name. To retrieve it, he or she either uses the pop-up menu to open a “Saved” motion or accessed it from the menu bar commands. Saved motions are any motions that are different than the standard motions originally provided by the library. Saved motions can be a composite of any number or combined motions, even previously saved ones.

[0544] 6.4.2.7 Serial Motions

[0545] Any motion that uses temporal. predicates to control a series of motions falls into this category with the exception of eMotions and a few of the Atomic motions such as “Turn”. An example of this would be the current “Flap-Stop” or “Orbit-Stop”. With the new design of motions, these would actually be created using the Atomic motion “Swing” and “Spin” respectively. This category is generally composed of motions that have been customized for a particular purpose, such as “Butterfly Wing”. Even though “Butterfly Wing” is a composite of three “Swing” motions, its components still only have two states, active and static.

[0546] 6.4.2.8 eMotions

[0547] Motions that tend to give the appearance of exhibiting emotional qualities are called eMotions. With the combination of primitive motions and deformations with state changes, these motions are able to make an object act as if it had believable human emotions. There are 25 primary and 88 secondary eMotions (see Table 28).

[0548] 6.4.2.9 Special or Behavioral Motions

[0549] Special motions are basically motions that have more than one behavioral state using more than just temporal predicates. These motions contain several states changes and triggers, unlike Serial Motions which cycle between active and static. An example would be “Bounce On” which uses proximity predicates and is linked directly with another unique motion node.

[0550] 6.4.2.10 Controlled Motions

[0551] Wind and Gravity are two global controllers that influence Controlled motions. The variable parameters on a controlled. motion are derived from the controllers. For example, Sway is applied to the three segments that make up the stem. Sway has the same Basic parameters as Swing except a dialog box is never seen. Instead we see the Wind dialog box and the changes we make to it affect Sway. This is discussed further in the Dialog Boxes section.

[0552] There are currently 4 Controlled motions: Sail, Sway, Waver, and Whirl, as is indicated in Table 29. They complement the four Atomic motions (Shift, Swing, Shake, and in) respectfully.

[0553] 6.4.3 Dialog Boxes

[0554] FIGS. 9-29 and 32 illustrate some of the Motion Editor/Viewer's dialog boxes. Additional dialog boxes will be added to the Editor/Viewer in the future. For example, additional controllers will probably be needed for Deformations and Special behavioral motions.

[0555] The top of each motion dialog box, of the types shown in FIGS. 9-28, contain the name of the motion and, although not shown in these figures, the name of object it is associated with, if any. The dialog boxes can be minimized for ease of viewing. At the bottom of each there are four buttons: Info, Code, Apply, Reset, and Save. The motion will automatically update as the sliders are moved. The Apply button is used when typing in a numerical value to indicate the current typed value is to be applied. The Code button causes an editable window to be popped up showing the C++ or OpenMotion Format code corresponding to the current motion. The Reset button returns the Motion to the values it had when read from the OpenMotion Library or when last saved. Pressing the Save button causes a dialog box to pop up which allows the motion to be saved and, if the user has changed any of its values, renamed.

[0556] Within each motion dialog box there are three or more tabbed parameter groups; Basic, Advanced, Temporal, Specialized. In the future, additional tabs can be added, such as for “Triggers”.

[0557] 6.4.3.1 Atomic Motions

[0558] Primitive motion dialog boxes currently have the three or four tabbed parameter groups just discussed: Basic, Advanced, Temporal, and Specialized.

[0559] 6.4.3.1.1 Basic Parameters

[0560] The Basic tab contains the common controls for a particular motion. FIGS. 9, 13, 16, 19, and 23 show examples of Basic setting tabs. They control the axis, speed, and displacement of the motion. In the axis selections X, Y, or Z may be chosen. “Other” indicates an axis vector other than the standard XYZ. Random chooses an axis at random but only when the object has reached the end of its active cycle or a maximum or minimum displacement.

[0561] As shown in FIG. 13, Shift has a unique way to determine its motion, either through the Maximum Distance and Speed sliders or Speed and Time. The Scaling feature appears in the Basic tab instead of the Advanced tab.

[0562] 6.4.3.1.2 Advanced Parameters

[0563] The Advanced tab controls what is called a motion's variance range and scale. FIGS. 10 and 20 provide examples of an Advanced tab. The variance range controls the variation of movement between the maximum and minimum displacement of a motion. For example, shake may have a max. distance of 10 units and a min. of 10. With the variance set at 0%, the object will move rhythmically from side to side 10 units left and right. Setting the variance to 50% will create a variance range of 5 units at either end of the motion. When the object shifts to the right, it may travel between 5 and 10 units. The distance it determined randomly by a noise function. The same thing happens when it shifts to the left. The numbers may look like this: 8 right, 6 left, 10 right, 7 left, 7 right, 9 left . . . .

[0564] This feature allows us to change “Shake” into “Shimmy”. If the displacement of Shimmy needed to be amplified, the max. and min. values could be changed or the Scale slider could be adjusted.

[0565] 6.4.3.1.3 Temporal Parameters

[0566] The Temporal tab overrides the continuous action of a motion if the Active button in checked. Examples of temporal tabs are shown in FIGS. 11, 14, 17 and 21. The Start Time and Stop Time are tied to the sim time. The Period is how long a particular motion cycle lasts, and the Active Cycle controls how long its active within the Period. For example, a Spin motion may have a Period of 10 sec. and an Active Cycle of 20%. The object will spin (become active) for 2 sec. and remain still for 8 seconds.

[0567] 6.4.3.1.4 Specialized Parameters

[0568] Specialized settings have Ramp and Ease controls. Examples of the Specialized tab are shown in FIGS. 12, 15, 18, and 22. The Ramp sliders control the acceleration and deceleration in and out of an entire motion. The Ease sliders control the acceleration and deceleration to and from a displacement sequence. For example, a “Swing” motion may have a lifetime of 10 minutes with a displacement of 90 degrees left and right at a speed of one swing per second, a 1-second swing to the left followed by a 1-second swing to the right. A choice is given in the Ramp setting of AM or FM. AM controls the amplitude of the motion. At 50% IN the swing displacement will start at 0 degrees and ramp up to 90 degrees after 5 minutes. FM controls the frequency. At 50% IN the swing action will slowly ramp up from less than 1-second for a displacement cycle to the full 1-second per displacement at 5 minutes.

[0569] Setting the Ease From Min. Angle slider to 50% will cause the swing to accelerate towards its Max. Angle. Checking the Speed-up button causes the swing to decelerate towards its Max. Angle (it will do the opposite of Ease).

[0570] 6.4.3.2 Composite Motions

[0571] Composite Motions are motions comprised by combining multiple other motion, by sequencing them and/or blending them. FIGS. 23 and 24 show some views of the dialog box used to help a user edit composite motions. If the composite motion was selected from a library or a previously saved motion, the name of the previously defined motion will appear at the top of the dialog box, along with the name of the object it is attached to, if any. Otherwise the name will read “New Composite Motion” if it is a new motion that is being developed. Adding a motion to either a saved or library picked motion will result in the name changing to “New”.

[0572] All the component motions of the composite motion can be accessed from a pull down list box at the top, shown expanded in FIG. 24. The component or sub-motions are ordered in this list by the chronological order of their start time. As shown in FIG. 23, the composite Motion Dialog Box has Basic, Temporal, and Specialized tabs, similar to those defined above, which provide information for a given composite motions sub-motion selected in the list box. If it is appropriate to the motion shown in the list box, the Advanced tab will appear. If the motions shown in the list box is, itself, a composite motion, a nested composite dialog box corresponding to the listed composite motion will appear.

[0573] The Composite Motion Dialog box also has an Add/Subtract Motion button, which if pressed will pop up a dialog box allowing for the addition or subtraction of component motions. A motion is added either by importing it from a library or file, or by making a new motion using the New Motion Dialog Box of FIGS. 25-28. A motion is deleted merely by selecting it and pressing a delete button.

[0574] In addition, the Composite Motion dialog box includes a Sequence/Blend button, which if pressed pops up a dialog box which displays a variable-scale time line of the motions. In this time line the start and stop times of a motion can be altered by dragging the start and end of the labeled temporal line representing each motion. A box highlights any two motions which overlap in time, and when that box is selected, a blend dialog box pops up allowing the user to specify the blend to be performed between any motions with overlap in that time frame.

[0575] 6.4.3.3 New Motions

[0576] The New Motion Dialog Box is shown in FIGS. 25-28. It contains four tabs, an Initial Conditions tab, a Translation tab, a Rotation Tab, and a Scaling tab.

[0577] The Initial Conditions tab is shown in FIG. 25. This is used to set initial conditions of a new motion in the same manner that the functions shown in Table 11 can be used to set such initial conditions.

[0578] The Transition tab is shown in FIG. 26. It has Four radio buttons in its upper left portion labeled Constant, Position, Velocity, and Acceleration. If Constant is selected the motion looks to the Initial Conditions for values. If not, the user can choose one of the three: Position, Velocity, or Acceleration. A Function List buttons gives the User access to a list of previously defined OpenMotion algebraic expressions for insertion into the Expressions edit box shown in the figure. Evaluate checks to see if the motion will run. Debug finds the scripting error if the motion fails to run. The user can edit an expression entered into the Expression edit box through use of the Function List, or enter his own expression from scratch. The Description edit box enables the user to provide a description to document the statement in the Expression edit box. The motion is not applied until the Apply button is clicked.

[0579] The Rotation and Scaling boxes are similar to Transition with the exception of choices with radio buttons.

[0580] Once a new motion has been created an saved, it can be edited with a dialog box similar to that shown in FIGS. 9-12, having Basic, Special, Temporal, Specialized, and other tabs, if appropriate to the particular motion.

[0581] 6.4.3.4 Wind Controller

[0582] For each atomic motion (Shake, Shift, Spin, and Swing) there is a complimentary motion that is affected by global controllers. FIG. 32 shows a dialog box for controlling the Wind controller. The Wind controller has four parameters: Strength, Direction, Focus, and Gust. Each parameter affects the Controlled Motions differently. FIG. 31 is a diagram illustrating the meaning of each of these four parameters. The dialog box provides a slider and edit box for setting each of these parameters.

[0583] 6.4.3.5 Sway Controller.

[0584] FIG. 29 shows a dialog box for the Sway Controller. The sway controller has four parameters: Minimum Angle, Maximum Angle, Speed, and Phase Angle. FIG. 30 illustrates the significance of some of these parameters.

[0585] 6.4.3.6 Other Controllers

[0586] Although not shown in the figures, it is planned that the Motion Editor/Viewer will have dialog boxes for controlling at least the following additional controllers: a Gravity Controller, a Viscosity Controller, and an Attraction Controller.

[0587] The Gravity Controller will have the following as parameters: Strength, Direction, Location, Shape, and Focus. The Viscosity Controller will have parameters: Strength, Location, Shape, and Focus. Finally, the Attraction ° Controller will have parameters: Strength, Direction, Location, Shape, and Focus.

[0588] FIGS. 33 through 63 represent one possible embodiment of a motion editor/viewer, such as the motion editor/viewer 182 shown in FIG. 1, which can be used with many aspects of the present invention.

[0589] FIG. 33 represents a screen of a graphical user interface of this motion editor, which is called Mojo. In the example of FIG. 33 the Mojo screen's scene view window 503 is shown displaying a scene that has in it only a 3D hierarchical graphic model 502 of a running man. Since Mojo is a real-time motion editor, the model of the running man is shown moving according to a set of motions that have been applied to the individual nodes of its hierarchical model.

[0590] The Mojo graphical user interface window 500 includes a toolbar having buttons 504 through 529.

[0591] The button 504 creates a new scene.

[0592] The button 505 opens a previously saved scene, model, motion, or package of motions.

[0593] The button 506 saves the current scene, or the currently selected model, motion, or motion package.

[0594] The button 507 cuts the current selection, which can be a model, such as the running man 502, a node of that model, a motion, a motion package, or other element attached to a model.

[0595] The button 508 copies the current selection to a clipboard, for temporary storage.

[0596] The button 509 pastes a selection that has been previously copied to the clipboard to a currently selected location in this. scene or in a model located in the scene.

[0597] The button 510 prints the view shown in the scene view window 503 or the contents of any window within it, such as the tree view window.

[0598] The button 511 provides help on how to use the Mojo editor.

[0599] The button 512 allows a user to select a sound file so that sounds can be attached to elements of the scene by operation of the button 524 described below.

[0600] The group of buttons 513 through 517 allows the user to select one of five different types of predefined motions that can be added to a node of a hierarchical 3D graphical model. These include the shift, shake, spin, and swing motions described above under heading 6.4.2.1, which are associated with buttons 513 through 516, respectively, as well as a turn motion associated with the button 517.

[0601] If the user selects the shift button 513, an instance of a predefined translation motion class can be created. This motion will define a translation in position a given distance in a given direction. Alternatively, the motion can be defined by specifying a given continuous speed of translation in a given direction.

[0602] If the user selects the shake button 514, an instance of an oscillating translation motion class can be created. This motion is a composite motion that repeatedly performs a succession of two motions: first a translation motion for a given distance in a first direction; and then a second translation motion for a given distance in a second direction; with a default being that the first and second motions are of equal length, but opposite direction.

[0603] If the user selects the spin button 515 an instance of a rotation motion class can be created, which defines a motion that rotates an orientation and direction about a given axis continuously at a given angular velocity. Alternatively, the motion can be defined to specify such rotation for a given angular distance at a given angular speed.

[0604] If the user selects the swing button 516, an instance of an oscillating rotation motion class can be created, which defines a composite motion that repeatedly performs a succession of two motions: a first rotation motion in a first direction about a first axis over a first angular distance, and then a second rotation motion in a second direction about a second axis over a second angular distance. The default values for such an oscillating rotation motion will cause the first and second rotation motions to be of equal angular distance about the same axis in opposite directions.

[0605] If the user selects the turn button 517 an instance of a turn motion class can be created, which is a composite motion consisting of repeatedly performing a succession of two motions: a first motion that is similar to the first motion of the oscillating rotation motion class, described in previous paragraph, and a second motion that merely resets the angular position associated with the motion to the position it had before the turn's first motion took place.

[0606] The toolbar shown in FIG. 33 includes four additional buttons 518 through 521 related to motions.

[0607] The add-motion button 518, if clicked, will display a dialog box allowing a user to select a named, previously saved motion or motion package, and add it, respectively, to a selected node of a hierarchical model or to select a hierarchical model or sub-portion thereof.

[0608] The button 519 is a delete-motion button that allows a user to delete a currently selected motion or motion package, respectively, from a node of a model, or from entire model or a sub-portion of such a model.

[0609] The button 520 is a parameters-window button that will display a window showing some of the editable parameters of a currently selected motion.

[0610] The button 521 hides the bounding box which indicates the currently selected portion, if any, of a graphic model in the scene view window 503.

[0611] The buttons 522 and 523 of the toolbar are related to model hierarchies.

[0612] The select-parent-node button 522 causes the parent node of the currently selected node in a hierarchical model to be selected.

[0613] The tree-view button 523 causes a tree view window 530 of the type shown in FIGS. 34 and 37 through 45 to be displayed.

[0614] The add-sound button 524 enables a user to place a selected sound file, such as a wav file 592 shown in FIG. 45, to be associated with a currently selected node of a model.

[0615] The buttons 525 and 526 are the eye-view-control and object-view-controlled buttons, respectively. They allow, respectively, a user to select between having the arrow keys of his or her computer keyboard either control the position of the viewpoint used to generate the view in the scene window 503, or to control the position of an object within that scene view relative to a fixed viewpoint.

[0616] The buttons 528 causes the change simulation clock speed control 532 shown in FIGS. 43 and 44 to be shown, which allows the user to stop or change the speed of the animation shown in the scene view window 503. The buttons 529 enables a user to turn on and off the animation shown in the scene view window 503.

[0617] FIG. 34 is similar to FIG. 33 except that in it the tree view window 530 is shown. This window is a scrollable window, as indicated by the scrollbar 534 shown in FIG. 34. As can be seen by observing the various portions of the tree view shown in FIGS. 34, 37, 38, and 39, the model 502 of the running man is comprised of a hierarchy of graphic model nodes 536 and 536A each of which is preceded by a graphic symbol 537 to make it readily apparent to users which nodes in a hierarchy of the tree view are model nodes and which are other types of nodes.

[0618] FIG. 35 illustrates the complete hierarchy of model nodes 536 and 536A associated with the man4 model 502 shown in FIG. 34. It also includes for each of the nodes 536 and 536A a more intuitive name 539 that identifies its corresponding body part in the model.

[0619] Certain of the model nodes 536 are parent nodes, indicated by the numeral 536A. In the particular form of model hierarchy used with Mojo any set of model components which do not move together rigidly as one unit, but which do all have their motion defined relative to the motion of a given parent node, should depend from a model node 536A designated as a parent node. Each such parent node has associated with it a parent location node 538 indicated by a box with a diagonal line through its in FIGS. 34, 367, 38 and 39, from which child nodes that are to have their position defined relative to such a parent node are depended. Any motion 540 place between a parent node and such a parent location node in the tree-view graph will cause all of the child nodes whose positions are defined relative to the parent location node to move as a function of such motions. As described below in more detail with regard to FIGS. 46 through 49, in a hierarchical, linked 3D model, a parent node 536A is often associated with either a point or shape (such as one representing a shoulder or knee joint) about which child nodes swing or rotate.

[0620] FIGS. 36A. and 36B illustrate the complete hierarchy of model nodes 536, including parent model nodes 536A and their associated a parent location nodes 538 in the hierarchical model of the running man 502 shown in FIG. 34 with each node being labeled with the more intuitive name 539 shown in FIG. 35.

[0621] In the scene-view window 503 a user can click on a portion of the model 502 to be selected (a double-click on a part selects the parent of that part). In FIG. 34 the cursor 542 has clicked on the node S_Leg_UR Parent 536AA shown in the tree view of FIG. 34, which corresponds to the running man's upper right leg parent node. As a result, the bounding box 544 is drawn around that node and, since it is a parent node, everything that depends from it in its model. Also the selected node 536AA is highlighted in the tree-view window 530 to indicate its selection.

[0622] FIG. 37 is similar to FIG. 34 except that it illustrates a view of the Mojo user interface after the user has clicked on the node 536BB of the running and model, which corresponds to his head. This causes the bounding box 544 to be displayed around the selected head node 536BB; if necessary, automatically scrolls the tree-view window so the head node will be shown in that window; and highlights the representation of that node in the tree-view window.

[0623] FIG. 38 illustrates the appearance of the Mojo interface 500 after a user has clicked the mouse cursor 542 on the left sleeve parent node 536AC causing it to be selected and causing a bounding box 544 to be displayed around it, and all model nodes which depends from it. In FIG. 38 the left sleeve parent node 536AC is not highlighted, because the user has right-clicked on the swing motion node 540AA shown in the tree view 530 of that figure, causing a motion-options menu 546 to be displayed. As can be seen, this option menu provides a user with a plurality of options 548 through 556.

[0624] The show-parameters option 548, if clicked, will display a parameters dialog box 558 like that shown in FIG. 42. This will enable a user to vary the parameters defining the selected motion and to interactively see their effects upon the selected motion, as such changes are made, in the scene window 503.

[0625] The delete-motion option 550 will remove the selected motion, such as the currently selected motion 540AA, from its associated node in its current model.

[0626] The copy-motion option 552 will copy the currently selected motion to a clipboard from which it can be pasted to another node in the same model or even another node in a different model.

[0627] The add-constraints option 554 allows a user defined or select and then add a given constraint or boundary, of the type described above in section 6.2.7, to the selected motion.

[0628] The add-behavior option 556 allows a user to define or select and then add a given behavior, of the type described above in section 6.2.6, to the currently selected motion.

[0629] FIG. 39 is similar to FIG. 38, except that it assumes that the user after having selected the left sleeve parent node 536AC has right-clicked on the mouse cursor 542, rather than making selections in the tree view 530. In this case a node-option menu 546A is displayed. This option menu is somewhat similar to the motion option menu 546 shown in FIG. 38, except that it is designed to provide options available for a model node rather than ones available for a motion.

[0630] The show-parameters option 548 of the menu 546A is identical to the show-parameters option described above with regard to FIG. 38.

[0631] The add-motion option 558 allows a user to add a motion to the currently selected node, which will motivate the entire part of the model shown in the bounding box 544.

[0632] The delete-last-motion option 550A is similar to the delete-motion option 550 described above with regard FIG. 38, except that at the time the node-option menu 546A is shown, there is no currently selected motion, and thus the deletion takes place on the last motion to have been added to the currently selected model node.

[0633] The select-parent-node option 560 has the same effect as pressing the select-parent-node button 552 described above with regard to FIG. 33; that is, it selects the parent node of the currently selected node. As can be seen from FIG. 35 if the currently selected. node is the left sleeve parent node 536AC, then the parent node of that node will be the torso parent node 536AD shown in FIG. 35. If the torso parent node were selected, the resulting bounding box would cover everything in the running man model 502 above his pelvis.

[0634] The remove-bounding-box option 562 would-remove the display of the bounding box 554 from the currently selected node.

[0635] The show-associations-tree-view option 564 shown in FIG. 39 would have the same effect as pressing the button 523 described above with regard FIG. 33, which is to cause the tree-view window 530 shown in FIG. 39 to be displayed, if it is not already shown.

[0636] FIG. 40 is similar to FIG. 39 in that it shows the scene view 503 after the user has used the cursor 542 to select a model node, in this case the node 536EE, which represents the running man's upper leg, or thigh, by right clicking on it with the cursor 542 and then selecting the add motion option 558 shown in FIG. 39. In this case an add motion dialog box 566 will be displayed which contains a scrollable list 568 of previously defined motions. The user can select one of these motions and then click the apply button 570 which will then add the selected motion to the currently selected node.

[0637] In the embodiment of the invention shown, desired results are normally obtained by attaching motions only to parent nodes 536A or to non-parent model nodes 536 which have no children nodes, which are intended to move with the them. That is because if a motion is applied to a first child node that has other sibling nodes depended from the same parent node, the motion will only move the first child node, and not the other sibling nodes or any node that depends from them. This is because in the particular tree graph used with Mojo, a given child node's position is defined relative to its parent location node 538, and, thus, motion of any node other than its parent location node will not affect its siblings. Thus, in the example of FIG. 40, if a swing motion was applied to the selected thigh model node 536EE, the thigh model would become disconnected from the lower leg parent node 536AF, the lower leg model 536FF, the foot parent node 536AG, and any nodes which depend from the foot parent node, because they would not be linked to its motion.

[0638] It should be appreciated that in other embodiments of the invention the add motion option could use other methods of selecting motions, including pop-up menus allowing a user to select a. motion from a previously defined hierarchy of motion classes, such as those shown in FIGS. 5 through 8.

[0639] FIG. 41 illustrates that the Mojo interface 500 includes a top-level menu 572 which provides additional user interface capability, including an edit menu 572A, which if clicked allows a user to cut, copy, or paste a selection, which can be one or more nodes, models, motions, or motion packages in the scene window 503 or the tree view window 530, as well as text in some of the text related windows of the Mojo interface, such as those which allow one to edit the code or comments associated with a motion. Although not shown, the File menu allows users to open and save scenes, motions, motion packages, and .wav files. Since Mojo is a motion editor it assumes that a user creates and edits models and their graphic nodes including image files and texture maps in a separate model editing and creating program.

[0640] FIG. 42 illustrates the motion parameter window 558 which will be displayed for a selected node if the user clicks on the show-parameters button 550 shown in FIGS. 33 and 42 or if the user selects the show-parameters option 548 shown in FIGS. 38 and 39.

[0641] In the example of FIG. 42, the user has selected as the currently selected node the left lower leg parent node 536AF and then has selected the show-parameters option for that node. When the show-parameters option is selected for a model node that has had a plurality of motions associated with it, the window will be shown for the, top-most motion node under that model node, which will be the most recently added motion. As can be seen from many of the figures illustrating Mojo's tree view, it is common to associate multiple motions with a model node 536 or 536A. When this is done, the respective positions, orientations, and scaling attribute values of all the models associated with a given node are summed to form, in effect, one composite motion which is applied to the given model node.

[0642] The show-parameters dialog box 558 shown in FIG. 42 includes maximum and minimum angle sliders 558A and 558B, respectively, which define how far the lower leg parent node can swing back at the knee and how far forward it can swing at the knee. In FIG. 42 the maximum angle slider 558B has been set to a much higher value than is appropriate for the motion of a human runner, as indicated by the fact that in that figure the runner's left leg is shown bent forward by more than 90 degrees from the thigh. As the user changes any of the controls shown in the show-parameters window 558, a corresponding interactive change is made to the animation of its associated motion in the scene-view window 503.

[0643] The show parameter window also includes a frequency slider 558C, which defines the speed at which the swing motion being edited will be performed, a phase angle slider 558D that shifts the phase of the swing motion. Each of the sliders 558A through 558D includes a corresponding edit box 558AA through 558DD, which enables a user to see a numerical representation of the current value entered by a slider, or which enables the user to enter an exact desired numerical value.

[0644] The show parameters window 558 also includes a set of radio buttons 558E that enable the user to select the axis of the selected swing motion defined relative to the orientation of the motion's associated model node. The show parameters window also includes an info button 558F that enables a user to enter, edit, or see comments about the purpose or function of the current motion. A code button 558G enables a user to see or edit the actual code that defines the motion. The apply button 558H causes the show-parameter window to be exited with any changes which have been made within it to the selected motion remaining in effect. The dismiss button 5581 allows the user to exit the show-parameters window with the selected motion being reset to the state it had before that window was last evoked.

[0645] It should be appreciated that in other embodiments of the invention other types of parameter windows could be associated with motions. For example parameter windows such as those described above with regard to FIGS. 9 through 24 could be used for this purpose.

[0646] FIGS. 43 and 44 illustrates the change-simulation-clock-speed window 532 that will be displayed if the user clicks the clock button 528. The window 532 includes a slider control 532A which enables a user to very the speed of the simulation clock for the entire animation shown in the scene view window 503. It also includes an numerical display box 532AA which shows the current speed of the clock in numerical form, and a close button 532B which will close the window 532 with the animation continuing to proceed at the clock speed which has been set in that window.

[0647] In FIG. 43 the simulation clock has been set to a speed of zero, which has the effect of stopping all motion in the scene view window 503. This can be very helpful when one is trying to select a particular node on a hierarchical model which might otherwise be moving so fast that would be difficult to accurately click on. This effect is similar to that which can be achieved by clicking on the run/stop toggle button 529.

[0648] FIG. 44 is similar to FIG. 43, except that in it the speed slider control 532A has been set to its maximum value, which in the current embodiment is a speed of three times normal speed. In other embodiments of the invention different ranges of simulated clock speed change could be allowed, including negative speeds, which would make motions run backwards.

[0649] FIG. 45 illustrates that the Mojo editor can be used to see and edit the motions of other hierarchical models besides the running man 502 described above. In this figure a model 580 of a bee is displayed with its hierarchical tree shown only partially expanded in the tree view 530. In this figure the bee model 580 is shown as a child node of a world node 582, which represents the parent node in the bee.x scene file. The path of other scene file is displayed at node 584 of the tree-view graph. Although not shown in any of these figures multiple models can be shown in the same scene, depending as sibling child node from the world node 582 or having motions defined relative to each other. For example, the running man model 502 and the bee model 580 could be placed in one scene, moving relative to each other and to their common scene by motions placed between their root nodes 502 and 580, respectively, and their associated top level parent.

[0650] In FIG. 45 the tree view 530 also shows a camera node 588. The camera node 588 in this figure is shown as fixed. In other embodiments of the invention motions could be applied to cameras, as well as model objects, so as to provide arbitrarily complex viewpoint motions in a 3D modeling application.

[0651] In FIG. 45 the thorax node 536HH has been expanded to show that it has associated with it a .wav file sub-node 592, which has been added by operation of the select-wave-file button 512 and the add-wave-file button 522. In the example shown in FIG. 45, the wave file 592 is one of the sound of a bee buzzing and the sound has been associated with the thorax of the bee which is located near its center.

[0652] The thorax node 536HH also has associated with it a 3D model node 594 that defines the three-dimensional shape associated with that node in the bee model. As can be imagined, most model nodes in a 3D hierarchical graphic model have 3D model nodes 594 associated with them. In some models, however, such as the one shown in FIGS. 48 and 49, all, or some, parent nodes 536A do not have 3D models associated with them. This is often the case when the graphic model for one of the child nodes of the parent node is defined to have a size that causes it to overlap, and, thus, fill the space that might otherwise have been occupied by a parent node.

[0653] FIGS. 46 through 55 are used to described how packages of parallel motions can be attached to 3D hierarchical models.

[0654] FIGS. 46 and 47 described a first model 600 that belongs to a given class 602 of 3D hierarchical models, and FIGS. 48 and 49 described a second model 600A which belongs to the same model class. In FIGS. 46 and 48 a tree graph view of the model nodes of the models 600 and 600A, respectively, are shown in a format corresponding to that in which the running man model 502 was shown in FIGS. 36A and 36B. As can be seen from FIGS. 46 and 48, the model nodes 536 and 536A of the models 600 and 600A are virtually identical to each other, both in the topology of the tree graph and in the naming that they use for corresponding model nodes. The only difference is that instead of the single head node 536II which occurs in the model 600, the model 600A includes a head parent node 536AK which has depending from it a parent location node 538 and two child nodes, i.e., a head node 536II and a hair node 536JJ.

[0655] The major difference between the model 600 of FIG. 46 and model 600A of FIG. 48 is the difference between their respective graphic model nodes 594. As can be seen in FIG. 46 the model 600 includes graphic model nodes 594A through 594M, whereas the model 600A includes graphic model nodes 594AA, 594CC, 594EE, 594GG, 594II, 594KK, 594MM, AND 594NN. As can be seen by comparing FIGS. 47 and 49, which display images of the model 600 and 600A, respectively, the graphic nodes of these two models are quite different in appearance.

[0656] Another distinction between the model 600 and model 600A is the fact that in the model 600 most of the parent model nodes 536A have corresponding graphic models, with the exception of the body parent node 536AI and the neck parent node 536AJ which are represented as dots in FIG. 47. Such parent model node graphic models are indicated by the numerals 594B, 594D, 594F, 594J, and 594L in FIGS. 46 and 47. In the model 600A, on the other hand, none of the parent nodes 536A have graphic models associated with them. In FIG. 49 all these parent nodes are represented by dots labeled 536A, or 536AI, or 536AK. In the model 600A graphic models are not use for parent model nodes because the child nodes of such parent nodes overlap with a sibling node of the parent node, so as to make the model appear fully connected. It should be understood, however, that models could be made in which the graphic models of some or all of its child nodes are neither connected to, nor overlapping with, their parent nodes.

[0657] As shown in the tree view of FIG. 48, the head node 536II includes not only a graphic model 594II, but also an image model 608, which represents a graphic image which is wrapped across the surface of the shape of the head node's graphic model 594II. In FIG. 48 the hair node 536II includes, in addition to its graphic model 594NN, a hair texture model 610 which defines a texture to be applied to the hair graphic model 594NN. Although not shown in the figures, Virturally any desired graphic model 594 can have image and texture models associated with them.

[0658] As can be seen in both FIGS. 46 and 48 each of the models 600 and 600A include a model class designation data object 606 which identifies the common model class 602 to which both such models belongs. If models share a common model class designation, that indicates they share a set of commonly named model nodes which enable them each to have the individual motions of a motion packages designed for use with that model class to be automatically associated with their individual model nodes.

[0659] FIGS. 50 and 51 illustrates the content of two possible motion packages 612 and 612A, respectively, which can have their individual motions automatically associated with models of the class 602 shown in FIGS. 46 and 48.

[0660] In the preferred embodiment, each of the motion packages 612 include a model class designation data object 606, similar to that contained in model 600 and 600A, which identifies the class 602 with which the motion package is designed for use. Such model class designations make it easy for a system using them to know whether or not motion packages and models are designed for use with each other.

[0661] As shown in FIGS. 50 and 51 each of the motion packages includes a list of motions 540C, in the case of FIG. 50, and 540D, in the case of FIG. 51, which are associated with metadata, in the form of model node tags, which identify the model node 536 or 536A with which each such motion is intended to be associated. As indicated under the body parent node tag 536AL all shown in FIG. 50, is possible to have multiple motions associated with a given model node tag in such a motion package. When this is the case, it indicates that the multiple motions associated with the model node tag are to be associated with the corresponding model node of any model to which the motion package is added.

[0662] The motion packages shown in FIGS. 50 and 51 each have a default model data object 614, which identifies a default model 616 which can be used in association with the motion package to illustrate its motion even if the user has not associated the motion package with another model. Normally the default model will be a very simple model, comprised largely of stick figure elements so that the model definition will take as few bytes as possible.

[0663] When a motion package is attached to its default model, it can be added to a scene and be viewed in the scene's tree view window 530 shown above with regard FIGS. 34 and 37 through 45. This will enable a user to compare its node structure with that of another model, which is particularly useful if the motion package does not belong to the same model class as the other model, and to copy motions from nodes of the motion package's default model to what the user believes should function as corresponding nodes in the other model.

[0664] In some embodiments of this aspect of the invention, a motion package or a 3D model could have a separate simulation clock associated with it, off of which all of the functions of time contained in the individual motions of the motion package or 3D model could be driven. This would enable all the motions of a given motion package or all of the motions associated with a given model to have their time sped up or slow down in unison, independently of other models in the same seen. A motion package having motions defined using the OpenMotion API defined above can not only all share a simulation common clock object, but can also share common events or variables, so as to communicate state between such motions, enabling complex parallel motion behaviors to be associated with such motion packages.

[0665] FIG. 52 illustrates the model 600A described above with regard FIG. 48, after it has had the motion package 612 of FIG. 50 associated with it, and FIG. 53 illustrates the same model after it has had the motion package 612A of FIG. 51 associated with it. FIG. 52 corresponds to the tree view graph shown in FIG. 48 with the motions 540C shown in FIG. 50 attached to the individual nodes of the model 600A corresponding to their respective model node tags 536 or 536A shown in FIG. 50. Similarly, FIG. 53 shows the tree view of FIG. 48 after the motions 540D of FIG. 51 have been added to its model nodes corresponding touch motion's respective model node tag fields shown in FIG. 51. As indicated in both FIGS. 52 and 53, it is preferred that a motion package node 618, or 618A, respectively, be added to the tree view graph of a model to which a motion package has been added to represent each of the one or more motion packages which have been added to that model. This enables a user to select such a motion package node in the tree view for such purposes as selecting, copying, or deleting it. In the preferred embodiment, when such a motion package node 618 in a tree view graph is selected, all of the motions belonging to that motion package will also be shown as currently selected, so as to better enable a user to see which motions are associated with which motion packages.

[0666] FIGS. 54 and 55 provide an illustration of a complex of motions that a motion package can impart to model in one step.

[0667] FIG. 54 corresponds to FIG. 45, except that it shows the tree graph of the bee model 580 with all of its model node 536 and 536A expanded and with its parent location nodes 538 also shown. It illustrates this tree view graph before any motions have been added to this model.

[0668] FIG. 55 is a view of the same tree view after the user has added a motion package, beeMotion1, indicated by the motion package node 618 shown as the fifth node from the top of that tree view. As a result of the one step of adding such a motion package, a complex set of individual motions 540E have been added to the bee model, with each such motion being associated with its corresponding model node 536 or 536A in the bee model. FIGS. 56A through 56C and FIGS. 57 through 60 provide a highly simplified a pseudo code representation of some of the program instructions which are executed by the Mojo motion editor. It should be understood that similar functionality could be used in other applications besides a motion editor, such as games, educational software, virtual reality programs, 3D browsing software, and other types of 3D applications.

[0669] FIGS. 56A through 56C represent just some of the instructions in an event loop 640 of the Mojo editor. The event loop defines the response which the program will make to each of a plurality of different events, including events corresponding to user input, events generated by the Mojo program itself, events generated by the OpenMotion API with which the Mojo motion editor is used, and events generated by the operating system of the computer on which the Mojo editor is running. The instructions contained in the portion of event loop shown in FIGS. 56A and 56B are performed in response to user selections made through the Mojo user interface, as indicated by line 642 of FIG. 56A.

[0670] If a user selects to open a model instance, which could, for example, be selected through a Edit>Add Geometry option in Mojo's main menu discussed above with regard to FIG. 41, step 644 cause steps 646 through step 650 to be performed. Step 646 displays an interface that lists available models and lets the user select open one of them. Then, if the user selects to open one of the listed models, step 648 causes steps 650 to open the selected model and to display it in the current scene shown in the Mojo's scene window 503, described above with regard FIGS. 33 and 34.

[0671] If the user selects to add a motion instance to a node, either by using an add-motion option 558 of the type shown in FIG. 39, a copy motion option 552 of the type shown in FIG. 38, or a paste option with regard to a selected motion using the edit menu 574 shown in FIG. 41, or by having dragged a motion to a selected node, then step 654 calls associateMotionWithNode function 778 shown in FIG. 58.

[0672] As shown in FIG. 58 the associatedMotionWithNode subroutine causes steps 780 through 784 to be performed.

[0673] In step 780, Mojo creates an association between the motion to be added to the node and the node in its own software, so it will know that the position, orientation, and scaling of that node is to be updated by corresponding values produced by the OpenMotion API for the given motion at each successive screen update time.

[0674] Then a step 782 tests to see if the motion has already been created in the API. If not step 784 calls the API's de-serialization interpreter with the serialized data that defines the motion. In Mojo the de-serialization interpreter corresponds to the translator 170 described above with regard FIG. 1. This translator has the capability to receive a string of characters containing the definition of a motion defined in the OpenMotion format described above with regard to section 6.3 of this application, and to translate that string of characters into a corresponding set of calls to the OpenMotion API. This process enables the OpenMotion system to dynamically create motions from a motion definition that has been recorded in OpenMotion format. Once the de-serialization interpreter has made calls to the API to create the motion, it makes a call in step 784 to schedule the motion. This informs the run-time engine 130 shown in FIG. 1 that the motion is active and should be updated at each om::update( ) call.

[0675] Returning to FIG. 56A, if a user selects to change a parameter of a motion instance, such as by use of a show-parameters window 558 of the type shown in FIG. 42, steps 656 and 658 call the API with the corresponding change to the motion instance. After such call is made with such a change to the parameter of a motion, the next time a call is made to om::update( ) the position, orientation, and scaling value of the motion will be updated, taking such a change into account. As a result, the system enables a user to see the effect of changes in the definition of the motion upon the operation of that motion as such changes are made.

[0676] If the user selects to change the simulation clocks speed, such as by use of the change simulation clocks speed dialog box 532 shown in FIGS. 43 and 44, step 660 and 662 call the API to make a corresponding change in the simulation clocks object.

[0677] If the user selects to save a motion instance, such as through use of the save button 506 shown in FIG. 33 when a given motion is selected, step 664 of FIG. 56A will cause steps 666 through 670 to be performed.

[0678] Step 666 displays an interface allowing the user to select a name and path under which the motions to be saved.

[0679] Step 668 then calls the API's serialization function for the motion. The API uses the translator 170 described above with regard FIG. 1 to perform such serialization. It has the capability to take an object which has been created by calls to the API 102 shown in FIG. 1 and to create a corresponding description of that object, as currently defined, written in the OpenMotion format described above with regard to section 6.3. This serialization function is provided by having a .print( ) method associated with each of the object classes used to define motions. This function returns a string identifying the current state of that object for which has been called. When this .print( ) function is called for a given motion, its in turn will cause the .print( ) function for each object which is involved in the current definition of the motion to be called in a recursive manner so as to produce a complete definition of the state of the object for which the .print( ) function was first called. In other embodiments of the invention other forms of serialization could be used, such as ones based on the .serialized( ) function which is supported by the Microsoft foundation class library.

[0680] After the call to the serialization function in step 668 returns with a string defining the current motion in OpenMotion format, step 670 saves that string on hard disk under the file name and path selected in step 666.

[0681] If the user selects to delete a motion instance from a node of a hierarchical model, step 672 causes steps 674 through 680 to be performed. A user can select to delete such a motion instance by use of the delete motion button 521 described above regard FIG. 33, by use of the delete motion option 550 discussed above regard FIG. 38, by use in the delete last motion. option 550A described above regard FIG. 39, or by use of the cut option shown in the edit menu 574 described above with regard FIG. 41.

[0682] If the user has made the option to delete motion, step 674 tests to see if the user has made the selection by use of a cut option. If so, it will cause the motion to be copied to a clipboard from which it can be subsequently pasted.

[0683] Next step 676 tests to see if the selected motion is part of a motion package. If so, step 678 deletes the current motion from the current instance of the motion package. Finally step 680 calls a function disassociateMotionFromNode 792 of FIG. 60.

[0684] As shown in FIG. 60, the disassociateMotionFromNode function includes steps 794 through 798.

[0685] Step 794 removes any association stored in the Mojo application between the given motion and the node from which. it is being disassociated.

[0686] Then step 796 tests to see if the motion is not associated with any other node, and if it is not, step 798 calls the OpenMotion API to un-schedule the motion and then to delete it.

[0687] If the user selects to add a motion package, such as a motion package is described above with regard to FIGS. 50 through 54, to a model, step 682 causes steps 684 through 700 to be performed.

[0688] Step 684 through 692 display a user interface, which as is indicated by line 686 enables the user to list available motion package is unless the user selects to open a selected one of them. As indicated by line 690 the interface also allows the user to open a Tree view of a motion package on its associated default model to 616 of the type shown in FIGS. 50 and 51. As indicated by line 692 the interface also lists all the models currently in the currently open seen and lets the user selects add a motion package to a selected one of them.

[0689] If the user selects to open a tree graph showing the selected motion package on its associated default model 616 of the type shown in FIGS. 50 and 51, steps 694 and 696 will display a tree view of the selected motion package's associated default model showing the motion package and its associated individual motions attached to the model and its individual model now is in the same manner as is illustrated in FIGS. 50 to through 54 described above.

[0690] If the user chooses to add a selected model package to a selected model, step 698 and 700 call the addMotionPackageToModel subroutine for the selected model and selected motion package.

[0691] As shown in FIG. 57 the addMotionPackageToModel subroutine 770 is comprised of steps 771 through. 776.

[0692] Step 771 causes a motion package node 618 similar to the nodes 618, 618A and 618B shown in FIGS. 52, 53, and 55, respectively, to be placed as a child node under the top node of the model to which the motion package is being attached.

[0693] Then step 772 performs a loop for each individual motion in the motion package. For each such motion, step 774 checks to see if there is a node in the model to which the motion package is being attached which has the same name as the model node tag associated with that motion. If such a node can be found in the model, then steps 776 calls the associatedMotionWithNode function, described above with regard to FIG. 58, for the current motion and found node.

[0694] Although not shown in FIG. 57, if there are one or more motions for which the step 774 cannot find a model node having a name corresponding to the motion's model node tag, the system checks if the model contains any model node having a name which is identical to that of the motion's model node tag except for the presence or absence of the word “parent” in the model node name. If such a model node is found, the motion will be associated with it. This, for example, will allow the head motion 1 of FIG. 50 or both the head motion 2A and head motion 2B shown in FIG. 51 to be associated with the head node 536II of the model 600 shown in FIG. 46, even though the model node tag associated with such motions has the name “head parent” rather than “head”.

[0695] Also not shown in FIG. 57, if at the end of the loop 772 there are any motions in the motion package that have not been assigned to model nodes in the model, the system can display a tree view of the motion package on its default model with an indication of all the motions that have been left unassociated. If desired, the user is then free to drag or cut and paste those remaining motions to desired nodes in the new model.

[0696] In other embodiments of this aspect of the invention, users will be allowed to select whether a new motion package is to immediately replace a motion package currently on the selected model, and whether there is to be a ramp-in or ramp-out of the new and old motion packages, respectively. In some embodiments, there is a browser that allows a user to see models and motion packages grouped by their associated model class 602, of the type shown in FIGS. 50 and 51. It is also preferred that such browsers enable a user to browse mapping transformations which can be used to automatically map a motion package into models of a model class other than one having model node names corresponding to the motion packages model node tags.

[0697] FIG. 56B shows that if the user selects to redefine the topology of a motion package instance, step 702 will cause steps 704 through 722 to be performed. By redefine the topology of a motion package means redefining the structure of what individual motions are associated with what model node tags in the package. The modification of a given motion definition by changing its parameters is described above with regard to steps 656 and 658 of FIG. 56A and the show-parameters window of FIG. 42.

[0698] If the user selects to redefine a motion package's topology, step 704 displays a user interface described by-lines 706 through 712.

[0699] As is indicated in line 706, this user interface prompts the user to select the motions desired for inclusion: in the motion package and place motions under the model nodes with which it is desired that they be associated, if they are not already located under such nodes. When this interface is displayed the selected motion package node 618, of the type shown in FIGS. 52, 53, and 55 will be highlighted, as will all the other individual motions associated with currently selected motion package. The user can then click on any unselected individual motion to add it to the currently selected motion package, or click on any currently selected individual motion to deselect it and subtract it from that package. If the user wishes to change the selected/deselected status of more than one individual motion he or she can used a shift-click to do so.

[0700] As indicated by line 708, the interface displayed by step 704 provides features for allowing a user to select to rename the model node tag associated with the one or more motions under a given model node in the current tree graph view. This will not change the name of the model node to which the motions are attached, unless the model on which the motion package is being displayed is the selected motion package's default model 616, of the type described above with regard FIGS. 50 and 51.

[0701] As indicated by line 710, the user interface displayed by step 704 includes controls that allow a user to rename the selected motion package's model class, if the user desires to do so. If there currently is no selected motion package, these controls allow the user to create a name for a new motion package which is to have associated with it any motions that are selected at the time the user selects to redefine the motion package, as described in the next paragraph. This, for example, would allow a user to define a new motion package having all of the motions shown associated with the running man mode 502 described above with regard to FIGS. 33 through 44.

[0702] As indicated by line 712, the user interface lets a user select to re-define the motion package in its current state after having made any changes in response to the aspect of the user interface described above with regard line 706 through 710. If the user so selects to redefine the motion package, step 714 causes steps 716 to redefine the current motion package as indicated incline 718 through 722.

[0703] As indicated by line 718, this redefinition process includes causing the redefined motion package to have a model node tag for each model node under which there is an individual motion currently displayed in the current tree view graph, with any renaming of model node tags which has been performed in response to the user interface described above with regard to line 708.

[0704] As indicated line by 717, the redefinition process includes placing under each such model node tag each individual motion associated with that node tag's corresponding model node in the currently displayed tree view graph.

[0705] As indicated by lines 718 and 720, if the name of the model class associated with the motion package has been changed in response to the user interface described above with regard to line 710 then the redefined motion package will have a corresponding change in the name of its associated model class.

[0706] If there have been any changes in the structure of the model node from which motions depend, lines 721 and 722 will give the user the option to have the system automatically create a new default stick figure model for the motion package having the same hierarchical structure as that of all the nodes directly moved by the motion package's motions in the current model.

[0707] If a user selects to save a motion package, then step 724 of FIG. 56B causes steps 726 through 742 to be performed. Such a selection can be made by selecting a motion package node 618, of the type shown in FIGS. 52, 53, and 55, and then choosing to save that motion package by pressing the button 506 shown in FIG. 33 or by selecting a save option under the file menu shows in the top-level menu 572 described above briefly with regard FIG. 41.

[0708] Once a user has selected to save a motion package, step 726 displays an interface that allows the user to select the name and path under which the motion package is to be saved.

[0709] Then step 728 performs a loop comprised of steps 730 through 736 for each node tag in the motion package being saved.

[0710] Step 730 adds the current node tag to a temporary motion package data string being created in association With the save.

[0711] Then step 732 performs a loop comprised of steps 734 and 736 for each individual motion associated with the name tag in the motion package being saved. Step 734 calls the API serialization function, described above, for the motion to create a description of that motion in the OpenMotion format described above with regard to section 6.3. After this call returns, step 736 adds the OpenMotion format string returned by it to the temporary motion package data string.

[0712] Once the loop 728 has been performed for each node tag in the current motion package, step 737 adds the model class and default model definitions 606 and 614 shown in FIGS. 50 and 51 to the motion package data string being created.

[0713] Once all these steps have been performed, step 738 saves the motion package data string to hard disk under the selected file name and path.

[0714] If the user selects to delete motion package instance, model instance, step 740 causes steps 742 to call the removeMotionPackageFromModel function. A user can make such selection by selecting the motion package node 618 shown in FIGS. 50 to 53 and 55 in a tree view of the model to which the motion package has been attached, and then select the motion delete button 519 shown in FIG. 33 or the cut option shown on the edit menu 574 in FIG. 41.

[0715] FIG. 59 shows the removeMotionPackageFromModel function 786 that is called if the user selects to delete a motion package instance. This function includes a step 787 which removes the motion package node 618, shown in FIGS. 52, 53, and 55, from under the top model node of the model from which the motion package is being removed. Then step 788 performs a loop comprised of calling the disassociateMotionFromNode function described above with regard FIG. 60, for each individual motion in the motion package.

[0716] FIG. 56C illustrates that if it is time for a screen update in this scene view window 503 of the Mojo interface (an event that occurs many times each second), step 746 through 768 will be performed.

[0717] Step 746 calls the om::update( ) function described above in section 6.2.2.2.1.

[0718] As indicated in step 748, this causes the opened motion API to update the position, orientation, and scaling value of each currently scheduled motion, including updating any temporal or spatial predicates of the type described above.

[0719] Once the of om::update( ) call has returned, step 750 performs a loop for each model in the current scene. This loop performs an inner loop 754 for each model node in current model of loop 750 having an associated motion. This inner loop 754 is comprised of steps 756 through 764.

[0720] Step 756 of this inner loop performs an innermost loop for each motion associated with the current node of the loop 754 comprised of steps 758 through 760. Step 758 calls the position( ), orientation( ), and scale( ) functions for the current motion. Then step 760 adds the value returned in each of the calls a step 758 to a sum which is calculated, respectively, for the position, orientation, and scale of the current node over all of the iterations for the current node in loop 754.

[0721] Once the current iteration of loop 754 has added the position, orientation, and scale for each motion associated with the current node, step 762 updates the position, orientation, and scale of the current node with the resulting sum.

[0722] Next, step 764 calls the renderer program 106, shown in FIG. 1, to have the renderer make a corresponding change to the position, orientation, and scale of the node in the renderer's object matrix 120, shown in FIG. 3, which corresponds to the current node.

[0723] Once the loop 750 has caused each node of each model to be updated, step 766 calls the renderer 106 to have it render an image of the scene with a current state of all of its nodes.

[0724] Then step 768 calls the om::frame( )function described above with regard to section 6.2.2.2.1 to help the OpenMotion API take into account the time delay between each om::update( ) call and the time at which the motion values calculated by that call are actually rendered.

[0725] FIGS. 61 through 63 provide a graphic illustration of how the Mojo interface allows motions to be dragged from a location under an old model node in a tree view graph to a location under a new model node in that graph, so as to disassociate the motion from the old model node, as described above with regard to steps 672 and 680 of FIG. 56A, and to associated the motion with the new model node, as described above with regard to steps 652 and 654 of that same figure.

[0726] FIG. 61 illustrates a part of a tree view 530 of the model 600A after it has had motion package 618 added to it, as shown in FIG. 52. In the view of FIG. 61, the user has right-click on a selected motion 540C with the cursor 542, causing it to be highlighted, as is indicated by the box 793 in FIG. 61. This result in the display of a motion-options menu 546 of the type described above regard FIG. 38. Then it is assumed that the user clicks on the delete-motion option 550 which causes the display of the selected motion 540C to be removed from the tree view as indicated in FIG. 62.

[0727] As shown in FIG. 62, the user has taken the cursor 542 and used it to drag the head motion 1540C shown up to the position 799 shown in dotted lines in that figure. If the user then releases the drag, the head motion 1540C will be repositioned at the location at which the drag was released causing the tree view to have appearance shown in FIG. 63.

[0728] The ability to reposition motions in a tree view graph can be very useful. This is particularly true if a user desires to copy a given motion to multiple nodes of one or more models, or if a user desires to drag motions from a motion package displayed in a tree view of its associated default model to desired nodes in a new model.

[0729] FIGS. 64 through 66 illustrate one of many possible e-commerce system which can use aspects of the present invention to enable users to purchase and download, as well as to upload and sell, motions and models of the general type described above.

[0730] FIG. 64 illustrates a network computer system 800 including a server computer 802 and a client computer 804 which are connected over a computer network, which can be virtually any computer network, but would preferably be the Internet.

[0731] The client and server computers 804 and 802, respectively, can be virtually any type of computer capable of functioning as clients or servers on a computer network. In many embodiments such computers will include random access memory 806 for storing computer instructions endowed in a form which can be rapidly accessed by the computer CPU 812, a video interface 808 which enables the computer to generate video displays to be seen on a screen or monitor 809, and I/O interface 810 which can receive user input, such as from a keyboard 811 or a pointing device 813, a CPU 812 which can execute computer instructions stored in the random access memory 806 and which can read and write data to that memory, and a network interface 814 which can communicate data and instructions between one such computer and another over a computer network, such as the Internet 805 shown in FIG. 64. Each such computer also includes a hard disk 118, which includes an operating system 818. It should be understood that the components of the client and server computer could vary both among various clients, among various servers, and between such clients and servers.

[0732] The client computer 804 includes a web browser 820. It also includes an OpenMotion directory 822 in which users can store OpenMotion 3D models such as the models 600 and 600A described above, preferably in a model directory 824. The OpenMotion directory also preferably includes one or more motion packages such as the motion packages 612 and 612A described above, and one or more motions 540 of the type described above. Preferably such motions are stored in a motion directory 826. Commonly a client computer 804 of the type which will be using the e-commerce site describe with regard FIGS. 64 through 66 will include one or more 3D motion application 828 which can use 3D motions and models of the type sold by such in the e-commerce site. It is also preferred that such clients include the OpenMotion library 100, including the classes which define the OpenMotion system shown in FIG. 1, including the Mojo motion view/editor 182 shown in FIG. 1 and a renderer program 106 of the type shown in FIG. 1.

[0733] Preferably the server computer 802 in which the e-commerce web site 801 is located includes a server program 830 which can respond to HTTP requests. Preferably the e-commerce web site 801 also includes a plurality of web pages 832 that provides an appropriate user interface for the site. Preferably the site also includes a models directory 824A that includes a plurality of models that can be demonstrated and/or purchased through the web site. Similarly, it is preferred that the web site also include a motion directory 826A, which includes motion packages 612 and individual motions 540 which can be demonstrated and/or purchased through the web site. In addition, it is preferred that the web site include an e-commerce back end 834 to perform the standard functions associated with an e-commerce site.

[0734] FIG. 65 provides a high-level pseudo code description of the steps 900 performed by the e-commerce client computer 804 shown in FIG. 64. These includes a step 902 in which the client uses the browser 820 shown in FIG. 64 to display and interact with web pages of the e-commerce server 802 shown in FIG. 64.

[0735] If a model or motion (including a motion package) is downloaded to the client computer by the e-commerce server as a result of interactions performed in step 902, step 904 cause steps 906 to store the model or motion in the disk drive 816, preferably in the appropriate models or motion directory 824 or 826 shown in FIG. 64. Although it is not necessary, it will normally be the case that a step 910 will be performed in which the user subsequently uses a motion or graphic model downloaded and stored in steps 904 and 906 with an OpenMotion application, such as the application 828 shown in FIG. 64. For example, the user can use the Mojo motion viewer/editor 182 as such and application.

[0736] FIG. 66 describe some of the steps 912 which can be performed by the e-commerce server 802, shown in FIG. 64, including steps 913 through 962.

[0737] If a user on a client connected to the commerce server indicates that he or she wishes to purchase motions or models from the commerce web site a step 913 will cause steps 914 through 936 to be performed.

[0738] Step 914 includes the display of one or more web pages designed for use by potential. purchasers of motions and models. As shown by line 916, these pages explain to a potential purchaser how to use the system. As shown by line 918, these pages display models, preferably grouped by model class, with their associated price. As indicated by line 920, these pages display motions, including motion packages an/or individual motions, organized by model class with indications of their associated price. As indicated by line 922, these pages allow a user to select to downloaded a given motion or model for trial use. And as indicated by step 924, these pages also allow a user to select to purchase a given motion or model.

[0739] If a user selects to download a selected motion are model for trial step 926 causes step 928 to download a time-limited copy of the selected motion are model.

[0740] If the user selects to purchase a selected motion or model step 930 causes steps 932 through 936 to be performed. Step 932 records the transaction in the account of the user, charging the cost of the purchased item to the user's account. Then step 934 records the transaction in account associated with the vendor of the purchased model, crediting the cost of the purchased item minus any commission charged by the e-commerce site, itself, to the account of the vendor. Then step 936 downloads a copy of the purchased motion or model to the client.

[0741] If the user of a client connected to the e-commerce server indicates he or she wants to sell models or motion, step 938 causes steps 940 through 962 to be performed.

[0742] Step 940 displays one or more web pages designed for use by such potential model or motion vendors. As shown by line 942, these web pages include explanations of how the system can be used by a potential vendor of motions are models. As indicated by line enable a user to, select to upload a given motion or model which is to be sold by the vendor on the web site.

[0743] If a user selects to upload a motion or model, step 946 causes step 948 to display one or more web pages which, as indicated by lines 950 and 952, allow the user to specify a name, description, and price for the uploaded motion or model, and which allow the user to actually give a command to upload the given item.

[0744] If a user selects to upload a given motion are model, step 954 causes steps 956 through 962 to be performed. Step 956 actually uploads the given motion or model. If the uploaded item is a model or motion package, step 958 checks if the uploaded item's model classification 606, shown in FIGS. 50 and 51, is accurate. Although not shown, if the classification label associated with the uploaded model or motion packages an accurate the server will notify the potential vendor and instruct him or her to either fix the item so that its current classification is correct or given a new classification which is correct. Then step 960 places the uploaded motion or model in a list of motions or models with the associated uploaded price in association with its corresponding model class. Then step 962 records an association between the uploaded model or motion and the vendor who uploaded it.

[0745] FIGS. 67 through 69 illustrate a type of network interaction that is made possible by aspects of the present invention. In the particular embodiment shown in these figures, this network interaction is in the form of a network 3D game.

[0746] FIG. 67 displays a network system that makes such a network game possible. It includes a plurality of client computers 804A, which are virtually identical to the client computers 804 described above regard FIG. 64, except that the motion application associated with each such client is a 3D game application 828A and except that its model involved in such a system has a game server program 838 associated with it.

[0747] FIG. 68 illustrates some of the steps performed by the game application 828A shown in FIG. 67. These include steps 1002 through 1044.

[0748] If the client application adds a model to, alters a model in, or deletes a model from, a shared seen in the 3D network game, then step 1002 causes step 1004 to upload a change description to the server computer identifying the model which has been changed and the change which has been made to it.

[0749] If the client application alters a motion, including an individual motion or a motion package, associated with a given model instance in a shared scene of the game, such as by changing the motion's parameters or behaviors, or by adding or deleting a motion or motion package to or from a model, then step 1006 causes step 1008 to upload a change description to the server identifying the motion change and any model affected thereby. In the preferred embodiment of this aspect of the invention, this is done by uploading an OpenMotion format description of the motion as changed, or, if separately serializable, of the individual change which is been made to the motion, in the OpenMotion format described above with regard to section 6.3.

[0750] If the client application receives a downloaded change description from the server, step 1010 causes steps 1012 through 1044 to be performed.

[0751] If the downloaded change description relates to a motion which has already been created on the client computer receiving the download, step 1012 causes step 1014 the call the de-serialization .print( ) function for OpenMotion format string downloaded as part of the change description. This will generate calls to the client computer's API causing the motion identified in the downloaded change description to. have a change corresponding to

[0752] If the downloaded change description is for a motion package to be added to a model, step 1016 causes step 1018 to call the addMotionPackagetoModel subroutine described above with regard FIG. 57. This will cause the individual motions associated with the downloaded motion package to be associated with the appropriate notes of the model referred to in the downloaded change description.

[0753] If the downloaded change description indicates that an individual motion is to be added to an individual node of a model, step 1020 causes step 1020 to the call the associatedMotionWithNode function described above with regard to FIG. 58 for the motion and node.

[0754] If the downloaded change description indicates that a motion package is to be deleted from a given model, step 1024 causes step 1026 to call the removeMotionPackageFromModel subroutine described above with regard to FIG. 59 for the motion package and the model. This will cause the individual motions associated with motion package to be removed from the model.

[0755] If the downloaded change description indicates that an individual motion is to be deleted from a node of a model, step 1028 will cause step 1030 to call the disassociateMotionFromNode function described above with regard FIG. 60 for the motion and node.

[0756] If the downloaded change description indicates that a new model is to be added to the scene, step 1032 will cause step 1034 to add the model to the scene.

[0757] If the downloaded change description indicates that a change is to be made to one or more nodes of a model already in the current scene, step 1036 will cause step 1040 to make a corresponding change to that model.

[0758] If the downloaded change description indicates that a model is to be removed from the scene, step 1042 will cause step 1044 to remove the model from the scene.

[0759] FIG. 69 illustrates that the server. program 838 shown in FIG. 67 includes steps 1048 and 1050 that respond to the uploading of any change description from one client by downloading that same change description to all other clients having the same model in a common shared scene.

[0760] The network game system shown in FIGS. 67 through 69 allows a plurality of client computers to share an interactive 3D scene in which any change made to the motions and models on one client computer will be replicated on other client computers having the same motions and models in their own versions of the same. In other embodiments of the invention, a significant number of motion packages and motions could be stored on each client computer, and interaction between various network systems could involve commands to merely link motion package already stored on a given client computer to a given model also stored on that client. This would allow complex changes in behavior to be communicated between separate computers with relatively little communication. In such systems, only if there is a command or action related to a model or motion not currently on a client would there be a need to access such a motion or model over the network.

[0761] The capability which the present invention provides to enable complex motions and complex changes in motions to be communicated between computers can be used for many applications other than network games, including, to name just a few: shared virtual reality applications, 3D network browsing applications, virtual travel, and 3D educational applications. It should also be understood that this aspect of the invention can be used other than in applications in which changes in motion are merely copied from one client through a server to another client. For example, the network aspect of the invention can be used in peer-to-peer computer networks as well as in client/server computer networks. Similarly, in other aspects of the invention motion or model changes may be originated in the server computer.

[0762] FIG. 70 provides a highly simplified example of how in application program, such as, for example, the game application 828A shown in FIG. 67 can use programmed instructions to automatically cause a succession of different motion packages to be associated with a given model. It illustrates a small portion of the event loop 1068 associated with such a 3D motion application comprised of steps 1062 through 1076.

[0763] In this highly simplified example, it is assumed that if a 3D model of a lion, called lionModel, comes within 20 feet of a model of a man, such as the man model 502 shown in FIG. 33, called manModel in FIG. 70, a spatial predicate of the type described above regard to section 6.2.7.5 will generate an event. If this event is generated, a step 1062 shown in FIG. 70 will cause steps 1064 through 1068 to be performed. Step 1064 tests to see whether the manModel currently has other than a run motion package associated with it. If so, it causes steps 1066 and 1068 to be performed. Step 1066 calls the removeMotionPackageFromModel function, described above with regard FIG. 59, to remove any motion package currently associated with the manModel, and then step 1068 calls the addMotionPackageToModel function, described above regard FIG. 57, to add the run motion package to the manModel with an initial direction causing the running motion to be away from the lionModel. This will cause the manModel to start running away from the lionModel.

[0764] If the application program in which the loop of FIG. 70 occurs were the game application 828A shown in FIG. 67, change descriptions corresponding to the changes made in steps 1066 and 1068 would be uploaded to the server 862A shown in FIG. 67 according to steps 1066 and 1068 shown in FIG. 68, so that the resulting changes in motions would be modeled on each client computer sharing the scene including both the manModel and lionModel.

[0765] In the example of FIG. 70 it is also assumed that a spatial predicate has been defined which will generate an event if the lionModel is more than 300 feet from the This latter step will test to see if the manModel has other than the walk motion package associated with it. If so, steps 1074 and 1076 will be performed. Step 1074 will call the removeMotionPackageFromModel function to remove any previous motion package associated with the manModel from that model, and step 1076 will call the addMotionPackageToModel function so as to add the walk motion package to the manModel.

[0766] As the simple example of FIG. 70 illustrates, the present invention makes it possible to remove and add complex sets of motions from and to hierarchical 3D models with relatively simple programming steps. In other, more complex, uses of this aspect of the invention, the motion packages removed from or added to hierarchical models by a given sequence of programmed instructions, and/or the hierarchical models themselves identified in such instructions, could be represented by variables so that such instructions could add or remove different motion packages from different models at different times as a function of programmed context. For example, the instructions that would carry out steps 1016 and 1018, as well as steps 1024 and 1026, of FIG. 68 would preferably be written using such variable motion package and model names.

[0767] FIG. 71 displays additional features which can be represented in the tree view window 530, described above with regard to FIGS. 33 through 45, in some embodiments of the invention.

[0768] In this figure the node labeled “Attachment Point” will normally be a model node 536 or a model parent node 536A and the motion node 540 will be a motion node, all of the type described above.

[0769] In this enhanced version of the tree view window, the motion node 540 can have depended from it the following types of nodes: an actor definition node 1080, an actor node 1088.

[0770] An actor definition node 1080 would specify the actor database fields and classifications that apply to the motion under which the node 1080 has been placed. An “actor” is a term applied to a motion-enabled object, that is a model object comprised of one or more nodes to which one or more motions have been applied. Normally in such an actor, the motions attached to its nodes exhibit autonomous or interactive behaviors.

[0771] The actors attributes node 1082 defines data values or expressions for the actor database fields specified under the actors definition node.

[0772] The excitables node 1084 specifies variables and expressions that drive the behavioral model. This includes transient and time based variables which reflect the current state of reaction to dynamic interactions with the environment, with other actors, or with the user.

[0773] The pivot point node 1086 represents the offset, with respect to the model node to which this nodes parent motion is attached, of the origin for the position and orientation calculation for that motion. For example if the motion is a swing or spin, it will define the pivot point for such swinging or spinning.

[0774] A behavior node 1088 specifies the finite-state machine, of the type described above regard to section 6.2.6.4, that controls motion behavior. Such a behavior node is an expandable node which can have stored underneath it one or more user-named state nodes, each representing a state transition of the type described above in section 6.2.6.4.

[0775] Each such state in node 1090 can have associated with it one or more transition nodes 1092 and a single controller node 1098. Each such transition represents a When class of the type described above in section 6.2.6.4, and can have as child nodes a and behavior associated with an instance of the When class.

[0776] The controller node 1098 located as a child under a state node 1090 has as children three degree-of-freedom nodes 1100, 1102, or 1104 which represent, respectively, the mathematical function of time which controls position, orientation, or scaling for the current motion in the state represented by the controller's parent state node 1090. Each degree-of-freedom node indications which the 0^ththrough 2^ndderivative of its corresponding degree of freedom it drives.

[0777] The Geometry node 594 is identical to the graphic nodes 594 shown above in FIGS. 38, 39, 45, 46 through 49, 52, and 53.

[0778] The texture node 610 is the same type of node as the hair texture model 2610 shown in FIG. 52. It is used to define a texture which can be applied to the surface of the 3-D shape defined by a geometry node.

[0779] The sound node 952 is the same type of node as the node 592 described above regard FIG. 45.

[0780] In the embodiment of the invention providing the tree view shown in FIG. 71, a user can click on, or otherwise select, any one of these nodes to see more about the information they represent. In cases where the information can be edited from within the application providing such a display, controls will be provided to enable a user to interactively edit such information.

[0781] It should be understood that the foregoing description and drawings are given merely to explain and illustrate the invention and that the invention is not limited thereto. Those skilled in the art who have the disclosure before them will be able to make modifications and variations therein without departing from the scope of the invention.

[0782] For example, it should be understood that the behavior described in the claims below, like virtually all computer behaviors, can be performed by many different programming and data structures, using substantially different organization and sequencing. This is because programming is an extremely flexible art in which a given idea of any complexity, once understood by those skilled in the art, can be manifested in a virtually unlimited number of ways. Thus, the claims are not meant to be limited to the exact steps and sequence of steps described in the figures.

[0783] This particularly true since the pseudo-code described in the text above has been highly simplified to enable it to more efficiently communicate that which one skilled in the art needs to know to implement the invention without burdening him or her with unnecessary details. In the interest of such simplification the structure of the pseudo-code describe above often differs significantly from the structure of the actual code that a skilled programmer would want to use when implementing the invention.

[0784] Furthermore, many of the functions which are shown being performed in software in the specification could be performed in hardware in other embodiments.

[0785] Furthermore, it should be understood that the invention of the present application is not limited to use with any one type of operating system or computer hardware. For example, although the OpenMotion System described above operates with the Microsoft Windows operating system, it is independent of any operating system and, thus, can be used with other operating systems such as variants of Unix or Microsoft Windows NT Similarly, although the OpenMotion System described above is written in C++, it should be understood that in other embodiments of the invention all or a part of it can be written in other languages, although for certain aspects of the invention object oriented procedural languages are preferred.

[0786] One should understand that the interface of the invention is not limited to the above. Similarly, in other embodiments the API could include different functions and differently defined functions and still accomplish many aspects of the current invention.

[0787] The aspects of the invention described above with regard to FIGS. 64 through 69 use the World Wide Web. It should be understood that the aspects of the invention shown in these figures are applicable to other types of network servers. It should also be understood that these aspects of the invention can be used not only on the Internet, but also with other forms of computer networks, including local area networks and wide area networks, as well as host-terminal network systems. In particular, the invention is not limited to use with HTML based systems, but could operate with other types of network protocols, or even proprietary protocols or future network protocols. Of course many aspects of the invention can be performed on a non-networked computer.