Enhanced UPnP AV media renderer

[0001] This application is a continuation of and claims priority from U.S. patent application Ser. No. 11/397,841, filed on Apr. 4, 2006. U.S. patent application Ser. No. 11/397,841 is incorporated herein by reference.

[0002] The present invention relates generally to Universal Plug and Play (UPnP) technology. More particularly, the present invention relates to the use of overlay user interfaces (UIs) in a UPnP audio/video (AV) renderer.

[0003] This section is intended to provide a background or context to the invention that is recited in the claims. The description herein may include concepts that could be pursued, but are not necessarily ones that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, what is described in this section is not prior art to the description and claims in this application and is not admitted to be prior art by inclusion in this section.

[0004] UPnP technology defines an architecture for pervasive peer-to-peer network connectivity of intelligent appliances, wireless devices, and personal computers of all form factors. UPnP is designed to bring easy-to-use, flexible, standards-based connectivity to ad-hoc or unmanaged networks whether in the home, in a small business, public spaces, or attached to the Internet. UPnP technology provides a distributed, open networking architecture that leverages TCP/IP and Web technologies to enable seamless proximity networking, in addition to control and data transfer among networked devices.

[0005] The UPnP Device Architecture (UDA) is designed to support zero-configuration, “invisible” networking and automatic discovery for a breadth of device categories from a wide range of vendors. In other words, the UDA enables a device to dynamically join a network, obtain an IP address, convey its capabilities, and learn about the presence and capabilities of other devices.

[0006] In a UPnP network, such as a network established in a person's home, a wide variety of different devices can freely interact with each other. For example, a user may be able to play music or video stored on a computer or mobile device on a stereo or television system. A user could use his or her mobile telephone to program different devices, and a plethora of other functions could also be implemented among multiple devices.

[0007] In the UPnP environment, at least one device serves as a control point, through which various actions can be implemented. For example and in the context of a mobile telephone, a user could have images or video stored on his or her phone displayed on a television, could have audio played on a stereo, or could perform a myriad of other actions involving other rendering devices. Although the use of such mobile devices in a UPnP network has many advantages, the use of a mobile telephone or similar mobile device in the digital home, for example when acting as a UPnP/Digital Living Network Alliance (DLNA) control point, poses some problems. For example, if a user is watching a movie or television program, it can be difficult or even impossible to hear his or her mobile telephone when it is ringing. Likewise, if a user is watching a program, he or she is not likely to notice an indication of an incoming email or short message service (SMS) message.

[0008] Another problematic situation can arise in connection with “home automation” systems, such as alarm system, a heating system, or a cooling system. Some such systems, when UPnP-enabled, can transmit messages to other UPnP devices when certain actions occur, such as a heating unit not functioning properly or an alarm fault being detected. However, if such messages are programmed to be transmitted to a mobile telephone, and if the user is watching a television program when a message is transmitted, he or she may not become aware of the message until a much later time.

[0009] It would therefore be desirable to develop a system that would help to address the above-identified shortcomings.

[0010] The present invention comprises a system and method for providing various overlay features for use in a UPnP AV media renderer. According to the present invention, indications of messages and/or other information received by a UPnP mobile telephone or other UPnP control point can be directly transmitted to a renderer and exhibited in different manners. For example, information can be overlayed directly over the content being exhibited, it can be shown in a “picture in picture” format, or it can be exhibited in a “news ticker” fashion. The present invention also permits the UPnP control point to transmit an event subscription to the renderer for subsequent use. Additionally, and in various embodiments of the present invention, the renderer permits the user to accept or reject the receipt of various information at the particular time.

[0011] The present invention provides a number of advantages and benefits to a user that have not previously been available in the UPnP environment. The present invention improves the usability of a mobile device both as a UPnP control point and as part of the home network. The present invention also improves both the usability and interoperability of other UPnP devices within the UPnP network. The present invention can be implemented in a wide variety of UPnP devices and is particularly beneficial for mobile devices such as mobile telephones.

[0012] These and other advantages and features of the invention, together with the organization and manner of operation thereof, will become apparent from the following detailed description when taken in conjunction with the accompanying drawings, wherein like elements have like numerals throughout the several drawings described below.

[0018] The present invention comprises a system for providing various overlay features for use in a UPnP AV media renderer. According to the present invention, indications of messages and/or other information received by a UPnP mobile telephone or other UPnP control point can be directly transmitted to a renderer and exhibited in different manners. The present invention also permits the UPnP control point to transmit an event subscription to the renderer for subsequent use. Additionally, and in various embodiments of the present invention, the renderer permits the user to accept or reject the receipt of various information at the particular time.

[0019] FIGS. 1 and 2 show one representative UPnP control point device, in the form of a UPnP mobile telephone 12, within which the present invention may be implemented. It should be understood, however, that the present invention is not intended to be limited to one particular type of UPnP mobile telephone 12 or UPnP control point device. The features and circuitry depicted in FIGS. 1 and 2 can be incorporated into a wide variety of other UPnP devices. The UPnP mobile telephone 12 of FIGS. 1 and 2 includes a housing 30, a display 32 in the form of a liquid crystal display, a keypad 34, a microphone 36, an ear-piece 38, a battery 40, an infrared port 42, an antenna 44, a smart card 46 in the form of a UICC according to one embodiment of the invention, a card reader 48, radio interface circuitry 52, codec circuitry 54, a controller 56 and a memory 58. Individual circuits and elements are all of a type well known in the art, for example in the Nokia range of mobile telephones. The features and circuitry depicted in FIGS. 1 and 2 can be incorporated into a wide variety of other UPnP devices.

[0020] FIG. 3 shows a representative UPnP environment 100 within which the present invention may be implemented. The UPnP environment 100 may comprise, for example, a person's living room, kitchen, home office, or other space. The UPnP environment can also be larger in size, including most or all of a user's home, for example. In addition to the UPnP mobile telephone 12, the UPnP environment 100 may include UPnP devices such as a land-line telephone 110, a desktop computer 120, a laptop computer 130, a stereo system 140, a television 150, a speaker system 160, a video cassette recorder (VCR) 170, a digital versatile disc (DVD) player 180, and a personal video recorder (PVR) 190. Other types of UPnP devices that can exist within the UPnP environment 100 include, for example, personal digital assistants (PDAs), a combination PDA and mobile telephone, and integrated messaging devices. A server 200 may be used, for example, to store items such as movies, music, television shows, games, and text-based files for use by UPnP devices within the UPnP environment 100. The UPnP environment 100 can also include several automated home systems, including an alarm system 185 and a heating system 195, for example.

[0021] The following are a number of different potential use cases for the present invention. It should be noted, however, that the following use cases are only examples and are not intended to comprise an exhaustive list of such scenarios. In one use case, a user receives a telephone call. This call can comprise virtually any format, including cellular, VoIP, etc. This call is received by the user's mobile telephone 12 while the user is watching television or listening to music on UPnP AV-enabled equipment. Alternatively, the user could receive a message in virtually any format, including SMS, multimedia message service (MMS), instant message (IM), e-mail, etc. while watching or listening to video or audio content. In these situations, the mobile telephone 12, acting as a UPnP control point, transmits a message such as “An incoming call from (phone number) or “You have a new message” to a currently-active media renderer or renderers for exhibition to the user. The message can be exhibited in a variety of forms. For example, the message can overlay the video content being shown, it can appear in its own separate picture (creating a picture-in-picture format on the screen), or it could appear as scrolling text along the bottom of the screen, in a “news ticker” form. The particular format used can be based upon user preferences, the capabilities of the renderer, or some other criteria.

[0022] In another use case, a user can request an event notification from his or her PVR 190 when the hard disk is becoming full. When the control point receives the event notification “PVR, disk full,” it redirects the message to the currently-active media renderer or renderers such that a message is shown. As above, the message can be shown in overlay, PIP, news ticker, or some other format.

[0023] In yet another use case, a person's home includes UPnP devices other than standard AV devices. For example, a person's home can have UPnP-enabled home automation systems, security devices, etc. When one such device sends an event notification to a control point, such as a message that the heating system 195 is malfunctioning, the control point can redirect the message to the currently active media renderer or renderers. The control point can also store the messages in the phone for further reference.

[0024] For the implementation of the present invention, various embodiments involve the addition of a new “overlay message action” which should be recognizable for the UPnP AV media renderer at issue. This can be implemented through changes and/or additions to device and service description XML files that are used by the media renderer. Using this action, and having the message text, format, and other related information included as arguments, the control point can request that the renderer display the message as an overlay message on its screen. In this arrangement, the control point must have the appropriate application logic to send the action requests. In other words, the control point must be capable of transmitting the appropriate messages to the media renderer. In one embodiment of the invention, the addition to the UPnP AV media renderer specification is, by default, backwards compatible with earlier versions, as a control point receives all of the required information about the media renderer's capabilities in the device & service descriptions and will not attempt to send such actions to a renderer that does not support them.

[0025] The implementation of the control point depends upon the use cases that are desired to be accomplished. If the use cases are limited to mobile phone use cases (incoming calls, messages, etc), then the implementation of that particular embodiment of the invention requires integration of the mobile phone's functionality (or the functionality of any portable UPnP control point with connectivity that can be used for making/receiving calls (e.g. VoIP) or receiving messages) and the UPnP AV control point running in the same device. An example implementation in a mobile phone 12 involves the use of the phone application program interface (API) to discover an incoming call or message, and then the use of the UPnP AV action to send the proper message to be displayed as an overlay message on the renderer's screen.

[0026] In one embodiment of the present invention, a “feedback” mechanism is included with the renderer device, allowing the user to control or manipulate the messages that are displayed. The feedback mechanism can involve, for example, a user interface and/or remote control mechanism. As in the case of incoming messages, a user may want to receive an indication of a new message to appear on his television screen but, for privacy reasons, he or she might not want the message itself to be read by others in the room at that time. Such a situation is represented in FIG. 4. In this figure, the control point (the mobile telephone 12) first transmits an indication of a new message with a question to the renderer 205. This message can be transmitted by wireless LAN (WLAN), UPnP, etc. In this instance, the message states “A new message has arrived. Do you want to read it now? Y/N.” In this instance, the indication is provided as an overlay message 210 on the renderer 205. However, the indication could also be provided as part of a news ticker 215. In addition, the control point also transmits an event subscription for the reply to the yes/no question. The renderer 205 permits the user to select “Y” or “N” and press “OK” triggering the event to be sent back to the control point.

[0027] If the user selects “N,” then the message that has been received is stored for later use. If the user selects “Y,” then the message is exhibited. This exhibition can take a variety of forms. For example, the message could be displayed on the mobile telephone 12, or it could be transmitted to the renderer 205 for exhibition. The manner of exhibition can be determined based upon predefined and/or adjustable user settings. In addition, it is also possible that a “Y” response results in other actions to occur at the renderer 205. For example, it is possible for a “Y” response to cause a video program to pause, for the sound to be lowered or muted, or for other actions to occur. These actions can occur based upon predefined and/or adjustable user settings.

[0028] In related embodiments of the invention, the user can use the feedback channel to acknowledge the event that led to a message being displayed. For example, in a situation where a warning is displayed that the heating system 195 appears to be malfunctioning, the user can use his or her remote control to “cancel” or otherwise acknowledge the message. Alternatively, devices other than the remote control can be used to acknowledge the message. Depending upon the respective capabilities, virtually any device within the UPnP network 100 can be used for this purpose. Additionally, it is possible that the entering of an acknowledgment can result in actions being implemented in other devices within the network, in a manner similar to that discussed above.

[0029] In another embodiment of the invention, several UPnP device categories (e.g. home automation/security and AV) can be involved. In such a case, the UPnP control point needs to be capable of supporting each of the UPnP device control protocols (DCPs) in question. Therefore, the UPnP control point should be capable of implementing several UPnP DCPs in various embodiments of the present invention.

[0030] FIG. 5 is a flow chart showing the implementation of various features of the present invention. At 500 in FIG. 5, the UPnP control point device receives an indication of a particular action. As discussed above, this action may comprise, for example, a voice call, a text, multimedia or electronic message, a notice from the heating system 195 of a malfunction, etc. At 510 and in response to the receipt of this indication, the UPnP control point device accesses settings information to determine whether the indication should be transmitted to a renderer 205, to which renderer 205 or renderers 205 the indication should be transmitted, and how the renderer 205 should exhibit the indication. The various settings can be input and/or adjusted at 515. At 520, a message is transmitted to the desired renderers 205 regarding the indication and how it should be exhibited. Based on information collected by the UPnP control point device, the UPnP control point device knows which rendering devices are located within the UPnP environment 100, which such devices are activated, etc. Therefore, the message that is transmitted can be specifically targeted to one or more rendering devices, although it is also possible to use multicast or broadcast messages if so desired. At 530, the message is received by the appropriate renderer(s) 205, and the indication is displayed or otherwise exhibited at 540 in accordance with the settings. In the event that a feedback mechanism is provided, and an inquiry is presented with the indication, then the user responds to the inquiry at 550. The user's answer is transmitted by the renderer 205 at 560 and is received at the control point at 570. Depending upon the user's provided answer, appropriate action is taken at 580. For example, if the user responds to “Do you want to accept the call” (when the mobile telephone 12 receives an incoming call) with “yes,” then the renderer 205 would perform an action such as muting the sound, pausing a recorded program, etc., while the user answers the call on his mobile telephone 12.

[0031] Communication devices of the present invention may communicate using various transmission technologies including, but not limited to, Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Transmission Control Protocol/Internet Protocol (TCP/IP), Short Messaging Service (SMS), Multimedia Messaging Service (MMS), e-mail, Instant Messaging Service (IMS), Bluetooth, IEEE 802.11, WLAN/VoIP, etc. A communication device may communicate using various media including, but not limited to, radio, infrared, laser, cable connection, and the like.

[0032] The present invention is described in the general context of method steps, which may be implemented in one embodiment by a program product including computer-executable instructions, such as program code, executed by computers in networked environments. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps.

[0033] Software and web implementations of the present invention could be accomplished with standard programming techniques with rule based logic and other logic to accomplish the various database searching steps, correlation steps, comparison steps and decision steps. It should also be noted that the words “component” and “module,” as used herein and in the claims, is intended to encompass implementations using one or more lines of software code, and/or hardware implementations, and/or equipment for receiving manual inputs.

[0034] The foregoing description of embodiments of the present invention have been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the present invention to the precise form disclosed, and modifications and variations are possible in light of the above teachings or may be acquired from practice of the present invention. The embodiments were chosen and described in order to explain the principles of the present invention and its practical application to enable one skilled in the art to utilize the present invention in various embodiments and with various modifications as are suited to the particular use contemplated.