Control object and user interface for controlling networked devices

[0002] The present invention relates generally to facility management systems such as home automation systems, and, more particularly, to an object oriented design software control object within such systems that provides for a logical representation and implementation of a network of physical devices.

[0003] In the past, environments having networked equipment were typically limited to commercial and industrial sites. Such equipment was typically complex and highly sophisticated computer controlled machines that utilized networks to coordinate manufacturing activities. However, unlike the commercial and industrial setting, the typical consumer setting included devices and appliances that were fairly unsophisticated and that were operated by a user on a one-to-one basis. To achieve the consumer device's specific function, the user had to physically interact with the device by, for example, pressing buttons on a coffee maker to preset the machine to turn ON at a specific time.

[0004] Early advances in consumer devices and appliances were limited to remote control devices that, at first, were wired to the device, and later used infrared (IR) communications to transmit commands. However, the remote control devices still forced the user to interact with their respective devices on a one-to-one basis. In addition, most hand-held remote control devices were proprietary, which required separate remote controllers for each consumer product. Further advancements included universal remote controllers that were capable of being programmed to control many consumer devices. However, universal remote control devices were limited in that they were typically not able to control devices other than entertainment equipment.

[0005] Lately, consumer appliances have become more sophisticated because of the decreased size and costs of computer-related components such as microprocessors. These new consumer devices include the so-called “smart” devices that, like the sophisticated commercial and industrial machinery, are capable of executing predefined tasks and capable of communicating over a network infrastructures. An example of a system is smart devices is CEBus®, which may utilize household powerlines to transport messages between compliant CEBus® household devices. CEBus® is implemented per EIA IS-60 for home automation. The CEBus® system utilizes controller and receptacle units to communicate and control CEBus® compliant devices. The receptacle units are connected to standard household receptacles and the command and control information is communicated over the powerlines. However, the CEBus® system is a hardware solution, which requires that the compliant devices be designed to be compatible with the standard. As a further limitation, the CEBus® controller is a special-purpose device. While this is an advancement over prior solutions, this hardware solution limits the upgradablity of the network and the devices, and fails to provide a solution for non-compliant smart devices.

[0006] A further example of a system that is intended to network devices is Jini™, which is based on Java™. Jini™ uses Java Remote Method Invocation protocols to move code around a network of devices. Network services run on top of the Jini™ software architecture such that devices and applications may register with the network via a lookup service. However, the Jini™ system is limited in that each device must register with the lookup service for other devices to discover them. Thus, each device is not aware of other devices on the network without first pointing to the lookup service. Further, there is no general purpose user interface that enables a user to interact and control all of the networked devices from a remote location.

[0007] In view of the above, there is a need for a system whereby once isolated and non-remotely controllable and configurable devices are able to share information with and about each other, and are able to provide and accept common command, control, and communication though a network. There is also a need for a system that provides the networked devices with the ability to operate autonomously in the case of a network failure, or for localized control and operation. There is also a need for a system that provides users with a method to control devices with a general purpose device (e.g., personal computer), and that provides a method to communicate over open network infrastructures. The present invention provides a solution to address the limitations and shortcomings of the prior art.

[0008] The present invention includes a set of software components and an application programming interface for the logical networked representation and implementation of a system of smart devices. The software components may be generally expressed as a control object component and a user interface component. The control object (CO) is a software object that may carry out communication, and accept and issue control messages. The control object allows a user to remotely control or query a physical device for which the specific control object has responsibility and also maintains logical and physical connections and control of the device.

[0009] The second component is the user interface, or consumer presentation display (CPD), which supplies a coherent and transparent interface across all network media and topologies. The user interface is capable of communicating with any control object that it identifies as being controllable in a given system. This communication may be across a logical network or may be local to a particular device. The user interface takes on the “personality” of the physical device for which the attendant control object carries a logical and physical mapping and control responsibility.

[0010] In accordance with an aspect of the invention, the control object may be adapted to bind to any designed physical transport mechanism for communication with other control objects and is adapted to be embedded in a container application and executed within a wrapper executable. In addition, the control object is preferably polymorphic, and thus is able to take on the logical attributes and command and control capabilities of any of the devices.

[0011] The control object may be a component object model object (COM) that exists within the operating system. This preferably provides wrappers for exposed clients and enables the use of method calls and exposed properties by the operating system or other application software programs. The control object may include extensions such that the control object may logically and physically pass a control signal to the devices and send a message to one of any electronic controllers or interfaces provided in the devices. In accordance with a feature of the invention, the first registered control object in the system may be designated a manager object, which performs list management to maintain and administer the list of control objects by periodically broadcasting the list to all registered control objects on the system. The list contains the logical attributes, an identifier and an address of all registered control objects on the system. The Identifiers are preferably globally unique random numbers generated at runtime, so as to identify each registered object “on-the-fly.” This globally unique identifier may be also used with directed object-to-object communication, where a specific object must communicate with another specific object. Preferably, the random generator is a linear congruential generator; i.e., f(z)=16807 z mod (2**31−1).

[0012] In accordance with another feature of the present invention, the user interface is adapted to control the devices either locally or across the network. The user interface may also be adapted to control all of the devices that are functioning on the network via the control object. In addition, the user interface preferably supplies a coherent and transparent interface across all network media and topologies. Further, if the devices include a display by which the device may be controlled, in accordance with an aspect of the invention, the display may be adapted to control other devices via the network and the control objects.

[0013] In accordance with an aspect of the present invention, a method of registering the control objects is provided, which includes broadcasting a control message from an enabled control object to inform other registered control objects that the enabled control object exists; determining if the enabled control object is a first registered control object; and appointing the first registered object as a manager object to administer the list.

[0014] The manager object may send the list to any new control object that registers with the system; periodically broadcast the list to all registered control objects; and perform a synchronization to force all other control objects to synchronize to the list. An election process is held if the manager object ceases to function. The election process may comprise promoting a second registered control object to the manager object; promoting all other control objects up one position on the list; and distributing the list to all registered control objects on the system. If any other control object ceases to function, the list is preferably distributed to all registered control objects if any control object other than the manager object ceases to function.

[0015] Other features of the invention are described below.

[0016] The foregoing summary, as well as the following detailed description of the preferred embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the invention, there is shown in the drawings an embodiment that is presently preferred, in which like references numerals represent similar parts throughout the several views of the drawings, it being understood, however, that the invention is not limited to the specific methods and instrumentalities disclosed. In the drawings:

[0017] FIG. 1 is a block diagram representing a computer system in which aspects of the present invention may be incorporated;

[0018] FIG. 2 is an exemplary environment in which aspects of the present invention may be incorporated;

[0019] FIG. 3 is a block diagram illustrating the internal architecture of a control object of the present invention;

[0020] FIG. 4 is a block diagram illustrating the interaction of the control object with external software components, an operating system and physical transport media;

[0021] FIG. 5 is block diagram illustrating the relationship of list and object management, application programming interfaces, transport dynamically linked libraries, physical media routines, and the physical media;

[0022] FIG. 6 is a flowchart illustrating the process by which the control object registers with the system and how a manager object is appointed;

[0023] FIG. 7 illustrates an election process whereby a new manager object is appointed when an existing manager is removed from the system;

[0024] FIG. 8 illustrates an election process whereby a new manager object is appointed when a periodic message is not received from the manager object; and

[0025] FIGS. 9A, 9B, 10A, 10B, 11 and 12 are exemplary Booch diagrams that express the static structure of the object classes, their internal structure, and the relationships in which they participate in a preferred embodiment of the system described above with regard to FIGS. 2-7.

[0026] The present invention provides for a software system that may be implemented on any network infrastructure such that networked devices may be remotely controlled by, e.g., a general purpose computer, and a system whereby the networked devices may share information with and about each other. The present invention provides and accepts common command, control and communication through the network while retaining the ability of each device to operate autonomously.

[0027] Computer Environment

[0028] FIG. 1 and the following discussion are intended to provide a brief general description of a suitable computing environment in which the invention may be implemented. Although not required, the invention will be described in the general context of computer-executable instructions, such as program modules, being executed by a computer, such as a client workstation or a server. A more detailed description of the invention will be presented with reference to the home area network of FIG. 2 and the description of the control object in FIGS. 3-12.

[0029] Generally, program modules include routines, programs, objects, components, data structures and the like that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the invention may be practiced with other computer system configurations, including hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers and the like. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0030] As shown in FIG. 1, an exemplary system for implementing the invention includes a general purpose computing device in the form of a conventional personal computer 20 or the like, including a processing unit 21, a system memory 22, and a system bus 23 that couples various system components including the system memory to the processing unit 21. The system bus 23 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The system memory includes read-only memory (ROM) 24 and random access memory (RAM) 25. A basic input/output system 26 (BIOS), containing the basic routines that help to transfer information between elements within the personal computer 20, such as during start-up, is stored in ROM 24. The personal computer 20 may further include a hard disk drive 27 for reading from and writing to a hard disk, not shown, a magnetic disk drive 28 for reading from or writing to a removable magnetic disk 29, and an optical disk drive 30 for reading from or writing to a removable optical disk 31 such as a CD-ROM or other optical media. The hard disk drive 27, magnetic disk drive 28, and optical disk drive 30 are connected to the system bus 23 by a hard disk drive interface 32, a magnetic disk drive interface 33, and an optical drive interface 34, respectively. The drives and their associated computer-readable media provide non-volatile storage of computer readable instructions, data structures, program modules and other data for the personal computer 20. Although the exemplary environment described herein employs a hard disk, a removable magnetic disk 29 and a removable optical disk 31, it should be appreciated by those skilled in the art that other types of computer readable media which can store data that is accessible by a computer, such as magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, random access memories (RAMs), read-only memories (ROMs) and the like may also be used in the exemplary operating environment.

[0031] A number of program modules may be stored on the hard disk, magnetic disk 29, optical disk 31, ROM 24 or RAM 25, including an operating system 35, one or more application programs 36, other program modules 37 and program data 38. A user may enter commands and information into the personal computer 20 through input devices such as a keyboard 40 and pointing device 42. Other input devices (not shown) may include a microphone, joystick, game pad, satellite disk, scanner or the like. These and other input devices are often connected to the processing unit 21 through a serial port interface 46 that is coupled to the system bus, but may be connected by other interfaces, such as a parallel port, game port or universal serial bus (USB). A monitor 47 or other type of display device is also connected to the system bus 23 via an interface, such as a video adapter 48. In addition to the monitor 47, personal computers typically include other peripheral output devices (not shown), such as speakers and printers.

[0032] The personal computer 20 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 49. The remote computer 49 may be another personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the personal computer 20, although only a memory storage device 50 has been illustrated in FIG. 1. The logical connections depicted in FIG. 1 include a local area network (LAN) 51 and a wide area network (WAN) 52. Such networking environments are commonplace in offices, enterprise-wide computer networks, Intranets and the Internet.

[0033] When used in a LAN networking environment, the personal computer 20 is connected to the local network 51 through a network interface or adapter 53. When used in a WAN networking environment, the personal computer 20 typically includes a modem 54 or other means for establishing communications over the wide area network 52, such as the Internet. The modem 54, which may be internal or external, is connected to the system bus 23 via the serial port interface 46. In a networked environment, program modules depicted relative to the personal computer 20, or portions thereof, may be stored in the remote memory storage device. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

[0034] Home Area Network Environment Employing a Control Object and User Interface

[0035] As noted, the computer 20 described above can be deployed as part of a computer network within which the system of the present invention may be implemented. In general, the above description applies to both server computers and client computers deployed in a network environment. As will become evident, the above description may also apply, in part, to appliances and devices attached to the network that have the ability to accept and execute software instructions. FIG. 2 illustrates one such exemplary network environment in which the present invention may be employed.

[0036] The present invention may be implemented in, for example, a household, a small office, or other similar environment. As illustrated, several devices are attached to the network 10 that may be controlled by users within the home or office. Such devices include, for example, a game device 112, a video device (VCR) 114, a telephone 116, a smart AC outlet 118, an answering machine 120, and a television 122, which may be connected to the network 10 as client devices. Other consumer devices, small office machines, and security devices may be connected to the network 10.

[0037] Also connected to the network are general purpose computing devices that may control the above-identified devices 112-122. Such devices include a notebook (laptop) computer 49, a hand-held computing device 63, and a desktop PC/server 20. The notebook computer 49 preferably operates utilizing Windows® 95/98 or Windows NT® 4.0 Workstation (or higher) operating systems. The hand-help computing device 63 preferably operates running Windows® CE 2.0 (or higher) operating system. The PC/Server 20 preferably runs the Windows NT® 4.0 Workstation or Server (or higher) operating system.

[0038] As illustrated, the devices 20, 49, 63 and 112-122 are connected to a network medium 136. The physical transport of the medium 136 is preferably Ethernet, since this topology is in ubiquitous use. However, the present invention is not limited to any particular physical network technology and may include fiber optic, X.10, and serial. This system 10 preferably utilizes TCP and Internet protocol (IP) as a common network layer, although it is not limited to the TCP/IP transport protocol, and the devices 20, 49, 63 and 112-122 are preferably identified by IP addresses.

[0039] In accordance with the present invention, each of the above consumer devices 112-122 is represented by a control object CO1-CO6. As will be described in greater detail below, the control objects CO1-CO6 are software objects designed to logically take on the physical attributes of devices 112-122 attached to the network 10.

[0040] Although not shown in FIG. 2, the devices 112-122 preferably include a microprocessor/microcontroller and a read-only memory containing an embedded operating system (e.g., Windows® CE 2.0 (or higher)). The embedded operating system and/or an application running on the operating system serves to control the functions of the devices 112-122. The methods provided by the control objects CO (discussed below) may be executed by the operating system or the software application. It is preferable that the operating system of the devices include application programming interfaces (APIs) to enable each device's attendant control object CO to control the device based on the information contained within the control object CO. Such a structure would allows device manufacturers to use the control object CO, application software, and/or the device operating system to implement functions that would be appropriate for the manufacturer's specific device or service (e.g., providing play and record functions within the VCR 114).

[0041] The present invention also provides for a user interface to allow users to interact and control the devices 112-122. The general purpose computing devices 20, 49 and 63 may act as the user interface or “consumer presentation displays” (CPD) 130, 132 and 134 to provide a coherent and transparent interface to users across all network media 136 and topologies. The user interface will be described in greater detail below.

[0042] Control Object Architecture and User Interface

[0043] Control Object

[0044] The control object and its operation within the network 10 will now be described in greater detail with reference to FIGS. 3-11. The control object CO logically represents the attributes of its associated physical device in order to provide for local and/or remote control of the device. The control object CO additionally maintains a list of other control objects CO registered on a given system such that any one control object may logically control any physical device connected to the system network. An exemplary implementation of the control object CO will now be described.

[0045] The control object CO supports a layered software model that allows multiple transport layers to bind to the core object at runtime. The control object CO is preferably polymorphic, meaning that it can take on the logical attributes and command and control capabilities of any particular real-world device. The control object CO also is adapted to bind to any designed physical transport mechanism for communication with other control objects or control programs. The control object is further preferably capable of being embedded in a container application and/or executed within a wrapper executable. When the control objects CO1-CO6 are active, each carries out communication, accepts and issues control messages, contains data specific to the devices on the network, and allows a user to remotely control or query the physical device for which the specific control object CO has responsibility for maintaining logical connections and control.

[0046] FIGS. 3 and 4 illustrate the inner architecture of the control object CO and the connectivity of the control object to a user interface 156, an operating system kernel 154, physical transport dynamically linked libraries (DLLS) 152, and the physical medium 36. As a summary, the control object CO is preferably embodied as a Component Object Model (COM) object that exists within the operating system (e.g., Windows® 95/98, Windows NT®, Windows® CE) that calls transport DLLs 152 through well known medium neutral application programming interfaces (APIs) 170 to communicate via the network medium 36. The data and events to be communicated by the control object CO, are embodied as callbacks and function (method) returns. Typically, the control object CO will load, activate, deactivate, and pass data to and from the transport.

[0047] Turning to the exemplary embodiment, the control object CO includes a core 150 that has a COM interface to access the operating system kernel and COM services 154. The operating system kernel schedules activities (threads) for the computer processor to perform, and it handles interrupts and exceptions. If the computer has multiple processors, the kernel synchronizes activity among the processors to optimize performance. COM provides a binary standard for component interoperability by allowing any programming language that can call functions via pointers (e.g., C, C++, Small Talk®, Ada, and Basic) to interoperate with other components written to the same binary standard. The control object CO preferably utilizes COM services 154 and the associated interfaces, because the interfaces have predefined behaviors and responsibilities that allow applications to interact with each other and the operating system, are programming language-independent, and are provided on multiple platforms (e.g., Microsoft® Windows®, Microsoft® Windows NT®, Apple® Macintosh®, UNIX®).

[0048] APIs 170 are called to translate to and from the transport DLLs 152 that provide for physical transport of information over the network medium 36. The APIs are a set of routines that request and carry out lower-level services performed by the operating system. In the case of Microsoft® Windows® operating systems, the APIs may also manage the application's windows, icons, menus, and dialog boxes.

[0049] The transport DLLs 152 are reusable functions that can be called from other executable code. Under the Win32® model used in Microsoft® Windows® 95/98 and Windows NT®, the operating system maps the DLLs into the address space of a process when the process is starting up or while it is running. The process then executes the functions of the DLL.

[0050] The control object core 150 also communicates to the user interface 156, which operates as the aforementioned consumer presentation display (CPD). The user interface 156 communicates with registered control objects COs for each of the physical devices 112-122 connected to the media 36. A more detailed description of the CPD is provided below.

[0051] The control object CO provides wrappers (container) 163 for exposed clients and enables use of method calls and exposed properties 166 that may be used by e.g., an operating system or other application software program. The wrapper 163 is provided to envelope (i.e., wrap) a call to a function or program inside another function or program. Methods and exposed properties 166 are logical operations provided by an object. In object-oriented programming, an object invokes a method by sending a message that contains the receiving object and the name of the specific method to invoke. Objects use messages as the mechanism through which they interact. Preferably, the methods are data neutral, in that they all take arguments of type VARIANT. Such typing allows the methods to be interoperable with Visual Basic, Delphi, and other ActiveX objects, as well as with C/C++, and applications such as Microsoft Internet Explorer (via HTML based scripting). An exemplary list of methods and exposed properties 166 available to the client container 164 are outlined in Table 1A below.

[0052] In general, when a COM object notifies its clients that something has happened, the COM object sends out a message. The message is called an event and the process of sending the message is referred to as “event firing.” In accordance with the present invention, Table 1B outlines an exemplary list of methods (i.e., logical operations) that cause event firings on behalf of the client container 164.

[0053] The interaction between the control object CO and the transport DLLs 152 is illustrated in greater detail with reference to FIG. 4 and Tables 2A and 2B below. As shown in FIG. 4, Input and Output APIs 170aand 170bcall transport DLLs that provide for core input and output management and the transport routines 152aand 152b. The APIs 170 and transport DLLs 152 are preferably provided by the operating system for the purpose of transporting data. The transport routines 152aand 152b, together with physical media routines 152c, provide access to the physical media 36 by providing the necessary transport protocol of the media (e.g., IP, serial, etc.). As noted above, the data and events that need to be communicated by the control object CO are embodied as callbacks and function (method) returns and the control object CO will typically need to load, activate, deactivate, and pass data to and from the transport.

[0054] In addition to the control object's inter-communication capabilities provided by the APIs 170 and transport DLLs 152, the control objects CO in a system are capable of upper and lower edge control functions. Upper edge control functions include interaction with the user interface 156, and lower edge control functions include interaction with the network media 36 and the devices 112-122. The control functions are related to the control object's logical representation of its associated physical device. In addition, by using extensions, a particular control object CO may logically and physically pass a control signal to a physical device directly, or send a message to the physical device's electronic controller or interface.

[0055] Referring again to FIG. 3, the control object CO also performs list management 162. List management is a feature of the invention whereby each control object CO maintains a list of its own state and the states of all other registered control objects existing in a live state on the network 10. An exemplary implementation of list management 162 is outline with respect to FIGS. 6 and 7.

[0056] Referring to FIG. 6, when a physical device 112-122 is first turned ON and functioning, its associated control object CO1-CO6 is enabled (S.100). The control object broadcasts a control message to inform other registered control objects that it exists (S.102). This will allow all other control objects within a given system to remain (or become) aware of each other's existence and world-state. If the control object CO is the first control object (S.104) to register with the system 10 (i.e., no other control objects are listed), the control object CO becomes a “manager object” (S.106). In addition to the control object's primary control duties with regard to the physical device it represents, the first control object CO (i.e., the manager object) maintains and administers a list of all other control objects. The list preferably contains an identifier (D) and an IP address for each registered control object on the networked system 10. This feature makes the list of control objects self-administering. The identifiers are preferably globally unique random numbers generated at runtime, so as to identify each registered object “on-the-fly.” This globally unique identifier may be also used with directed object-to-object communication, where a specific object must communicate with another specific object. Preferably, the random generator is a linear congruential generator; i.e., f(z)=16807 z mod (2**31−1).

[0057] The manager object responsibilities, however, are deliberately limited to allow each control object CO as much autonomy as it requires or is capable of having in view of the device it is controlling. The manager object sends a copy of the world-state list to any new control object CO that registers with the system 10. Also, the manager object periodically broadcasts the list to registered control objects COs and performs a synchronization (i.e., takes a census) to force all other control objects to synchronize to the updated list (S.108).

[0058] The manager object position is an elected position and all control objects COs have the ability to become the manager object should the manager object be disconnected or otherwise unable to perform its duties (S.110). The election of a manager object preferably is performed as illustrated in FIG. 7 (S.112).

[0059] Referring now to FIG. 7, when the manager object (e.g., CO1) drops out of the system 10, the second registered control object (CO2) becomes the manager object. All other control objects move up one position on the list (i.e., CO3 becomes CO2; CO4 becomes CO3; and CO5 becomes CO4, etc.) and a new list is created. The newly created list is then replicated to all control objects on the system 10. However, if, for example control object CO2 drops out of the system and control object CO1 remains, no election is held, but a new list is replicated and passed to all registered control objects. As further indicated in FIG. 8, when a “heartbeat” (i.e., a periodic message from the manager object indicating that the manager object is alive) is not provided from the manager object within the specified arbitrary time limit, the first device to timeout waiting for the manager heartbeat message will automatically elect itself manager and immediately send an updated list to all other objects. In the example of FIG. 8, if the timeout period expires for Object 2 prior to a heartbeat being received from the manager object. Accordingly, Object 2 elects itself as manager object and informs the other objects by sending a new list to all of the registered objects. This can mean, in cases where the manager is not actually inoperative, but rather just very busy, that manager object-related duties can pass to another registered object by way of default, rather than manager object failure.

[0060] Referring back to FIG. 6, after a new manager object is elected (S.112) and appointed as a manager object (S.114), the new manager object performs the duties discussed above with regard to S.108. It is noted that there are preferably cross checks to ensure that if an manager object is rudely pulled out of service without a normal shut down message (e.g., its respective device ceases functioning without notice at S.110), that the system 10 will perform the recovery election of FIG. 7. This will ensure the immediate determination and election of a manager object and that newly promoted manager object will take a new census of the remaining objects such that each control object will have an up-to-date list of other control objects.

[0061] Referring back to S.104, when the control object CO registering with the system 10 is not the first object, it receives a list of registered objects from the manager object and will respond to queries from the manager object or other objects (S.116) while in an active state. Each control object CO preferably provides information only if queried by another object. All subordinate objects receive a “heartbeat” message from the manager object. If the heartbeat is not sent to the subordinate object in a pre-determined amount of time, the subordinate objects will automatically hold an election and one of these subordinates will become the manager (in the case where the manager object ceases to exist in a non-failsafe way).

[0062] It is preferable that all control objects CO are peers in the system 10, and any physically dependent component that is less than a peer has a controller, which assumes the peer role. Further, when a physical device 112-122 is to be turned OFF and to cease functioning, it is preferable that the associated control object CO1-CO6 broadcast a message to inform other objects that it no longer exists and should be removed from the world-state list.

[0063] Each control object preferably will be able to query any other CO to retrieve the other CO's state, dependencies, and possible actions. If no response is received, the CO will be considered to have gone to sleep, which will cause the manager CO to rebuild and present a new list to the remaining COs.

[0064] User Interface

[0065] The user interface 156 (consumer presentation display 130, 132 and 134) enables a user to control all devices 112-122 attached to the network via registered and controllable control objects. The consumer presentation display 130, 132 and 134 (CPD) supplies a coherent and transparent interface across all network media and topologies. This communication and control may be across a logical network or may be local to a particular physical device. The CPD takes on the personality of a physical device, for which the attendant control object CO carries out logical and physical mapping and control responsibility. In the case of, e.g., the VCR 114, the CPD 130 contacts the VCR control object CO5, queries the control object CO5 for capabilities and state information, and then presents a user with an appropriate display. The appropriate display in this example may include tape operation buttons and programming menus, which would be displayed by the CPD 130 on the laptop 49. That CPD would be capable of showing status for, and issuing commands to, the VCR 114 (through the registered control object CO5).

[0066] The CPDs 130, 132 and 134 are primarily expressed in this invention as a component of the upper-edge control mechanism. It is preferable that the operating system running on the laptop 49, pen computer (hand-held computer) 63, or desktop PC 20 contain a set of APIs to retrieve information from the control object CO and to generate the display of the CPD 130, 132 and 134. In addition, the CPD preferably uses HTML formatting techniques and may be programmed in Visual BASIC, C/C++, or other programming language. Information from a particular control object CO may be retrieved by calling the appropriate API, and based on the information retrieved, the CPD formats an appropriate screen for the user to control the device.

[0067] Although the general purpose computing devices are shown as CPDs 130, 132, 134, it is noted that any device having display capabilities may act as a CPD. In particular, because the CPD retrieves information from the control objects CO1-CO6, and the control objects CO1-CO6 all perform list management 162 and are aware of their world-state and the world-state of all other objects, a display provided with, e.g., the game device 112 may be used to operate the video device 114. This feature is provided by the APIs included in the game device operating system or the software application controlling the game device 112, which enables the game device's attendant control object CO6 to control the video device 114 based on the world-state list information contained in the control object CO6. The control object CO6 would communicate command and control information to the control object CO5 to accomplish physical control of the video device 114. Further, the display on the game device 112 would be formatted to the personality of the VCR control display (i.e., display tape operation controls, etc.) in accordance with the world-state information.

[0068] FIGS. 9A, 9B, 10A, 10B, 11 and 12 are exemplary Booch (class) diagrams that express the static structure of the object classes, their internal structure, and the relationships in which they participate in a preferred embodiment of the system described above with regard to FIGS. 2-7 in order to further provide one of skill in the art with a description of the present invention. It is noted that references in FIGS. 9A, 9B, 10A, 10B, 11 and 12 to a “home object” are references to the “control object” above.

[0069] It is noted that the foregoing examples have been provided merely for the purpose of explanation and are in no way to be construed as limiting of the present invention. While the invention has been described with reference to preferred embodiments, it is understood that the words which have been used herein are words of description and illustration, rather than words of limitations. Further, although the invention has been described herein with reference to particular means, materials and embodiments, the invention is not intended to be limited to =the particulars disclosed herein; rather, the invention extends to all functionally equivalent structures, methods and uses, such as are within the scope of the appended claims. Those skilled in the art, having the benefit of the teachings of this specification, may effect numerous modifications thereto and changes may be made without departing from the scope and spirit of the invention in its aspects.

[0070] For example, while the devices shown to be clients and controllable by the CO have been illustrated as a game device, a video device, a telephone, a smart AC outlet, an answering machine, and a television, other devices may be controlled include, but not limited to personal computers, security systems, dishwashers, etc. In addition, other operating systems than the Microsoft® Windows® family may be provided with the appropriate APIs to implement the features of the invention.