US 7,003,731 B1Grant

User control and activation of watermark enabled objects

Issue Date:2006-02-21

•10 Claims

•3 Drawing Sheets

Abstract

Methods and systems for associating watermark enabled objects with machine behaviors. Machine behaviors refer to actions by devices or systems in response to a triggering event. Examples of these behaviors include fetching a web page, opening an email client to send an email to a specific person, initiating a phone or video conference call, etc. A registration system enables users to associate machine behaviors with objects through a watermark embedded in the objects. The decoding of the watermark initiates a process for triggering the machine behaviors associated with the watermarked object. While the disclosure describes an implementation for stickers as a class of watermark enabled objects, the system and related methods apply to other forms of watermark enabled objects, including both electronic and physical objects.

Metadata

Assignee

Digimare Corporation

Inventors

Geoffrey B. Rhoads
Tony F. Rodriguez
Tyler J. McKinley
Marc D. Miller
Kirstin Hierholzer

Application Information

Application Number:US 09/690,773

Filing Date:2000-10-17

Priority Date:1995-07-27

Art Unit:2173

Classifications

IPC:

G09G5/00

Patent Drawings (3 sheets)

Description

Related Application Data

[0001] This patent application is a continuation in part of U.S. patent application Ser. No. 09/633,587, filed Aug. 7, 2000, which is a continuation in part of U.S. patent application Ser. No. 09/343,104, filed Jun. 29, 1999. U.S. patent application Ser. No. 09/343,104 claims priority from U.S. Provisional Application No. 60/134,782, filed May 19, 1999. This patent application is also a continuation in part of U.S. patent application Ser. No. 09/571,422, filed May 15, 2000, which claims priority to U.S. Provisional Application No. 60/134,782, filed May 19, 1999. These patent applications are hereby incorporated by reference. This patent application is also a continuation in part of U.S. patent application Ser. No. 09/560,976, filed Apr. 28, 2000 (Now U.S. Pat. No. 6,553,129), which is a continuation of U.S. application Ser. No. 08/746,613, filed Nov. 12, 1996 (now U.S. Pat. No. 6,122,403), which is a continuation in part of U.S. application Ser. No. 08/649,419, filed May 16, 1996 (now U.S. Pat. No. 5,862,260), PCT Application No. PCT/US96/06618, filed May 7, 1996 (Published as WO96/36163), and U.S. application Ser. No. 08/508,083, filed Jul. 27, 1995 (now U.S. Pat. No. 5,841,978). This patent application is also a continuation in part of U.S. patent application Ser. No. 09/314,648, filed May 19, 1999 (Now U.S. Pat. No. 6,681,028).

[0002] The subject matter of the present application is related to that disclosed in U.S. Pat. No. 5,862,260, and in application Ser. No. 09/503,881, filed Feb. 14, 2000 (now U.S. Pat. No. 6,614,914), which are hereby incorporated by reference.

Technical Field

[0003] The invention relates to multimedia signal processing, and in particular relates to hiding auxiliary information in media signals like audio, video and image signals.

Background and Summary

[0004] Digital watermarking is a process for modifying physical or electronic media to embed a machine-readable code into the media. The media may be modified such that the embedded code is imperceptible or nearly imperceptible to the user, yet may be detected through an automated detection process. Most commonly, digital watermarking is applied to media signals such as images, audio signals, and video signals. However, it may also be applied to other types of media objects, including documents (e.g., through line, word or character shifting), software, multi-dimensional graphics models, and surface textures of objects.

[0005] Digital watermarking systems typically have two primary components: an encoder that embeds the watermark in a host media signal, and a decoder that detects and reads the embedded watermark from a signal suspected of containing a watermark (a suspect signal). The encoder embeds a watermark by altering the host media signal. The reading component analyzes a suspect signal to detect whether a watermark is present. In applications where the watermark encodes information, the reader extracts this information from the detected watermark.

[0006] Several particular watermarking techniques have been developed. The reader is presumed to be familiar with the literature in this field. Particular techniques for embedding and detecting imperceptible watermarks in media signals are detailed in the assignee's co-pending application Ser. No. 09/503,881 and U.S. Pat. No. 5,862,260, which are hereby incorporated by reference.

[0007] The invention provides methods and systems for associating watermark enabled objects with machine behaviors. In this context, machine behaviors refer to actions by devices or systems in response to a triggering event. Examples of these behaviors include fetching a web page, opening an email client to send an email to a specific person, initiating a phone or video conference call, etc. In the detailed description to follow, a watermark system enables users to associate machine behaviors with objects through a watermark embedded in the objects. The decoding of the watermark initiates a process for triggering the machine behaviors associated with the watermarked object. While the detailed description focuses on an implementation for stickers as a class of watermark enabled objects, the system and related methods apply to other forms of watermark enabled objects.

[0008] Further features will become apparent with reference to the following detailed description and accompanying drawings.

Brief Description of the Drawings

[0009] FIG. 1 is a system diagram depicting a system for associating watermark enabled objects with machine behaviors, and for triggering those behaviors in response to decoding watermarks.

[0010] FIG. 2 is a diagram depicting a variant of the system shown in FIG. 1 showing system components for enabling users to perform watermark identifier registration and embedding.

Detailed Description

[0011] The following description details a system and related methods for associating watermark enabled objects with machine behaviors. To illustrate the system, the description focuses on an example of watermark enabled stickers. As noted, the system applies more broadly to watermarking both physical and electronic objects. In particular, aspects of the system may be used for watermarking media signals like images, video and audio, as well as applying watermarks to physical objects. Watermarks may be applied to physical objects by placing a watermarked image on a physical object, by modulating the surface topology of the object, etc. See U.S. Pat. No. 5,862,260, for more information about watermark embedding of and decoding from physical and electronic objects.

[0012] Stickers in all their varieties have found an enduring place in our society. From the workplace (Post-It® brand message notes), to kids in a classroom, stickers have an inherent value associated with them, whether it be functional (seals, labels, etc.) or just to identify yourself with a particular affinity group (bumper stickers on cars). By placing a watermark on stickers they can be used in novel ways. By encoding a set of stickers with a watermark during production, specific machine behaviors can be assigned to them. These behaviors can be associated or even possibly changed by anyone from the manufacturer through the distributor, all the way to the end-user. In addition, the users can create their own watermark enabled stickers by creating an image, embedding a watermark in it, and associating the watermark with one or more machine behaviors.

[0013] These behaviors may include, but are not limited to the following:

- [0014] Taking the user to a web-site linked to the watermark via a network address of the web-site or index to the network address.
- [0015] Opening an email client to email to a specific person (e.g., a person whose email address is stored in the machine behavior description associated with the watermark).
- [0016] Launching the user into an Interframe Relay Chat (IRC) session that other people with the same sticker can participate in.
- [0017] Authenticating the user as part of a process of accessing a network resource, such as account information or access to a computer network.
- [0018] Authentication the user in an electronic commerce transaction performed on a computer network.
- [0019] Sending an electronic card.
- [0020] Placing a phone or video-conference call.
- [0021] As props in a computer game. For example, the prop is a multi-sided, or multi-faceted object, where each side or facet has a watermarked image conveying a different message used to control the game. The computer game includes a watermark decoder for extracting the messages from image frames captured of the prop. The message may directly carry the message or act as an index to a more detailed game instruction in a database, such as an instruction that changes over time based on changes to the corresponding database entry by the user or game manufacturer.
- [0022] As a visual aide for disabled users.
- [0023] Anywhere where machine vision is not feasible.

[0024] In each of the above applications, the watermark carries information that links the watermarked object (e.g., sticker) with a machine behavior. To trigger this behavior, a watermark decoder application captures an image or images of the watermarked sticker, extracts the watermark, and uses information embedded in the watermark to determine the associated machine behavior. The watermark decoder then takes action to initiate the machine behavior associated with the watermark.

[0025] For some applications, it is useful to enable the user to control the behavior associated with a watermarked object. This type of management may be handled by creating accounts for users and providing access to the accounts via some authentication method (email, passwords, etc.). For a number of reasons, these access methods can be problematic (losing passwords, asking kids for their email addresses, etc.). As an alternative, watermarks may be used to manage the process of associating behaviors with a watermarked object.

[0026] For example, in the scenario where a user wants to assign behaviors to a set of watermarked stickers they have received, they can hold up the first sticker (or its packaging), and be taken to a registration application to activate the stickers in the pack.

[0027] FIG. 1 is a system diagram depicting a system for associating watermark enabled objects with machine behaviors, and for triggering those behaviors in response to decoding watermarks. The system depicted in FIG. 1 is implemented on a computer network, namely, the Internet. The user accesses the system via a computer 100 connected to the Internet. The computer broadly encompasses a variety of devices, such as Personal Computers, set-top boxes, personal digital assistants, Internet appliances, set-top boxes, telephones (including wireless devices), audio and video players, and imaging devices (CCD or CMOS cameras, camcorders, printers, fax machines, copiers, etc.). The computer is connected to an image capture device 102, such as a PC camera or scanner, and includes watermark decoder software for decoding watermarks from images captured from the image capture device.

[0028] The system architecture shown in FIG. 1 includes a system (106) for managing the process of assigning behaviors to watermarked objects as well as a database management system (108) for initiating behaviors in response to decoding watermarks from the objects. These two systems may be integrated or implemented separately. In the application depicted here, the registration system and database management system are accessible via a network interface using standard network technology, including HTML, XML, and TCP/IP. A watermark embedding system has previously embedded watermarks carrying watermark identifiers into stickers. The stickers (or packages of them) also include a registration identifier used to activate the behaviors associated with them.

[0029] The registration system maintains a registration database including records of the registration identifiers and corresponding watermark identifiers. The registration identifiers are serialized numbers corresponding to the watermarked stickers or packages of them. The watermark identifiers are a form of object identifiers that are encoded into the watermarks on the corresponding stickers. The registration system maintains a registration database 110 of data records indicating the watermark identifiers associated with each registration identifier. When a user selects a behavior to be associated with a watermarked object via the registration system, the registration system sends an update 112 to a behavior database 114 specifying the behavior to be associated with a corresponding watermark identifier. In response, the database management system 108 updates its database to include a record that indicates the behavior associated with a particular watermark identifier.

[0030] The database management system 108 is also responsible for supporting machine behavior associated with a watermarked sticker in response to detection of the watermark on the sticker. It has a network interface for communicating with other computers over the Internet. In particular, it receives requests in the form of an XML packet from a watermark decoding computer, extracts a watermark identifier from the packet and looks up the associated behavior or behaviors in the behavior database. It then initiates the associated behavior. The details of how this behavior is carried out depend on the application and type of behavior.

[0031] In a typical registration process, the user accesses the registration system via a registration web site, which presents an HTML interface to the users' computers. The user may fetch the HTML pages of this interface using an Internet browser or application program, like the watermark decoder application executing on the computer 100. This interface enables the user to enter a registration identifier to start a process of associating behaviors with watermark identifiers embedded in watermarked stickers. In response to a registration identifier, the registration system returns a page that enables the user to specify the behavior. In the case where the behavior is linking a watermarked sticker to a web site, the user specifies the network address of the web site, such as a URL or IP address. In the case where the behavior is linking a watermarked sticker to an email message, the user specifies the email address of the email recipient.

[0032] As noted above, there are many other types of watermark enabled behaviors. They can be classified as providing information to the watermark decoding computer, launching some software program or machine action, or a combination of both. Table 1 below gives some examples of behaviors, and the related information and actions.

[0033]

TABLE 1

Behavior	Information Returned to	Associated Machine or
Type	Decoding Computer	Software Actions

linking to	URL, web page	launching browser on client
web site		to fetch/render web page at
		URL
opening an	email address of target	launching email client and
email client	recipient	populating address field
		with target recipient
starting a	address of chat session	launching chat application
chat session		(watermarks on the stickers
		can be designed such that
		only those holding the
		stickers can gain access to
		the chat session, each by
		showing the sticker to his or
		her watermark decoder
		enabled camera)
accessing	address of account	launching of browser
account in-	information	application to access
formation or		account information
other		through a web interface;
network		supplying user
resources		authentication information
		from watermarked object
		and/or from user (user
		password, user name, log
		on, etc.)
sending an	card template	launching client application
electronic		to enable the user to design
card		the card and add personal
		message, launching email
		application to send
		electronic card (or link to
		electronic card)
placing a	phone number or IP address	launching application to
phone or	of destination	initiate phone call over the
video con-		internet or telephone
ference call		network
props in an	identifier of prop, and	game application receives
interactive	possibly other context	prop and context
computer	information, such as game	information and responds
game	player holding the prop,	accordingly
	etc.
visual aid for	returns information in the	browser, or other media
disabled	form of graphics, audio, or	player applications render
users	video (may provide address	the information (such as the
	of audio or video content at	streaming media) on the
	an audio or video server on	decoding computer
	the Internet)
machine	machine instruction	machine or software
control		executes instruction

[0034] For a given application, the registration system provides information to the user to enable the user to select the behavior and provide pertinent information, such as URL, IP address, phone number, email address, content file (e.g., audio, image or video file), etc. The registration system formulates a description of the behavior, associates it with the watermark identifier specified by the user, and creates an update 112 to the behavior database.

[0035] The user then uses the stickers or shares them with friends. To trigger the behavior of a sticker, a user captures an image of the sticker with an image capture device 102 using a watermark decoder application 104 executing on the computer 100. The watermark decoder extracts the watermark identifier from a watermark embedded in the image on the sticker. It then sends the watermark identifier to the database management system 108 via the Internet, which in turn, looks up the associated behavior. The database management system then triggers the associated behavior by sending information, or instructions back to the decoding computer. The decoding computer renders the information, and launches a software or other machine action associated with the instructions returned from the database. The database need not be implemented in a remote computer. For example, the database may be implemented in the watermark decoding computer or device.

[0036] As an enhancement to the registration process, objects may carry watermarks that automatically link the user to the registration web site. For example, one side of the sticker 116 or its packaging 118 may contain a watermark with the network address or an index to a network address of the registration web site. The user shows this part of the sticker or packaging to the image capture device. The watermark decoder extracts the watermark and looks up the network address in the behavior database, and launches a browser to fetch the registration web site. The watermark may also carry the registration identifier. In this case, the registration web site can tailor the web page returned to the user to be specific to the watermarked object. If the user or someone else previously associated a behavior with the sticker, the registration web site returns the current status associated with the registration identifier and the behaviors associated with the watermarked objects linked to that registration identifier. To get detailed information about particular watermarked objects during the registration process, the user can show the watermarked object to a camera, and use a watermark decoder to extract the watermark identifier and supply it to the registration system. In response, the registration system takes the watermark identifier, queries the behavior database via the database management system, and returns a description of the associated behaviors. This approach provides a simple and automated process of activating watermark enabled objects.

[0037] For more information about an object identifier registration system and system for linking objects with machine behaviors, see U.S. patent application Ser. No. 09/571,422, which is hereby incorporated by reference.

[0038] In some applications, the user may wish to create his or her own watermarked objects. FIG. 2 illustrates a system that enables users to assign watermark identifiers to corresponding behaviors and objects and update the behavior database. In this particular system, the user's computer includes a watermark embedder application (120). However, the embedder application may be implemented on a separate computer, such as a server on the Internet accessible via a client application on the user's computer 100. In the former case, the user embeds the watermark into the desired image content on his computer. In the latter case, the client supplies the image content to the server, which performs watermark embedding and returns watermarked images to the client. In both cases, the watermarked objects are created by printing the watermarked images on objects.

[0039] The process begins when an embedder 120 creates a registration request. In the system shown in FIG. 2, the embedder 120 is a software application running on the computer 100. The embedder formulates the request in a request file. The system provides a template for the request file. The request file specifies the number of watermark identifiers requested and the names of the media files to be embedded. The file may also specify the behaviors to be associated with each watermark identifier. Alternatively, the user can specify the behaviors to be associated with the watermark identifier at a later time using the methods described in this document. In the case where embedding is performed on a server as opposed to the user's computer, the request file may also include the media file (e.g., an image file) carrying the content to be embedded with the watermark identifier.

[0040] Next, the embedder connects, via a network connection, to the registration system 106. In particular, it connects to a registration web site via an Internet connection. This web site requests the embedder's username and password to authenticate it.

[0041] The user enters his username and password via a user interface displayed on the PC 100 and submits them to the web site for authentication.

[0042] Upon authentication, the registration website 106 returns an HTML page presenting the embedder with a user interface screen that allows the user to locate the embedder's registration request file for uploading to the web site. The user then enters a command to instruct the embedder to upload the selected request file.

[0043] The embedder provides the information required to locate the file on the embedder's computer and submits it for upload.

[0044] The registration request file is uploaded into a registration loader program 122.

[0045] The registration loader 122 performs a quick scan of the uploaded registration request file and reports back to the embedder any errors in format that it detects. If there are errors, the file is not processed.

[0046] If the registration request file is properly formatted, the embedder receives a confirmation from the registration website 106 that the request file has been successfully uploaded and will be submitted for processing by the registration loader 122.

[0047] The embedder may now either submit a new registration request file or logoff of the registration web site 106.

[0048] The registration loader 122 uses the information contained in the embedder's uploaded registration request file to automatically allocate (register) watermark identifiers in a registration database 110. The identifiers are in the form of serial numbers. Once this process is completed, the registration loader 122 initiates a request to a registration extractor 124 for these new registration entries.

[0049] Upon receipt of a request, the registration extractor 124 accesses the registration database 110 and creates embedder control files for each of these new registered watermark identifiers (e.g., serial numbers).

[0050] Upon completion of this process, the registration extractor 124 process sends the embedder control file(s) back to the embedder via Internet e-mail. In the event that the embedder is server based, the extractor sends the control file(s) (or a pointer to them) to the embedder server 126, which may be integrated with the registration system or implemented at a different Internet site. The extractor 124 also sends an update 128 to the behavior database 114 to create database records associating each of the watermark identifier with a behavior.

[0051] Once the embedder 120 has received the embedder control file(s), it uses these file(s), along with the media file(s) (in this case, image files) and a set of embedding instructions to the embedder 120 to instruct the embedder to automatically embed the list of watermark serial numbers included in the embedder control file(s) into the listed media files, producing a set of watermark-embedded media files. In the case where the embedder is server based, the client executing on the PC 100 uploads the media files to be embedded to the embedder server, either directly or as part of the registration process (e.g., as part of the request file). The embedder server then returns the watermarked files to the computer 100 via e-mail or other network file transfer protocol.

[0052] For detailed disclosure describing how to embed watermarks in media signals, including images, audio, and video, see U.S. Pat. No. 5,862,260, and co-pending application Ser. No. 09/503,881, filed Feb. 14, 2000, incorporated above.

[0053] The embedder may create watermarked objects by printing watermarked images on objects, such as stickers, documents, etc. The embedder sends the watermarked image to a printer 128, which in turn, prints the image on an object.

[0054] The above system provides a mechanism for linking objects to machine behaviors. As noted previously, this mechanism applies to both physical objects, like stickers and packaging, and electronic objects, like image, audio and video signals. It also applies to other forms of machine readable signal carriers that can be applied to such objects, including bar codes, magnetic stripes, Radio Frequency tags, integrated circuit chips, organic transistors, etc. These machine readable carriers can be used in the same way that watermarks are used in the example of watermarked enabled stickers above.

[0055] While these technologies provide a mechanism for linking objects to machine behaviors, there is a need for a tool that explicitly facilitates the creative coordination between the object and the behavior linked to it. The linking process results in a potentially complex database structure which not only embodies the fundamental object to behavior link, but also might include a hierarchy of delivered responses as a function of user implicit or explicit requests. Or, said in a quite different way, the creative process of linking an object to complex data-driven responses is itself a creative endeavor all to itself, involving thinking through the various different reactions that users will want and expect when using an object as a portal. The artist who is tasked with creating the choreography between an object and a simple or complex machine behavior will need explicit assistance from well designed tools, resulting in a database record which memorializes that creativity as an active element within a connectivity system described in this document and U.S. patent application Ser. No. 09/571,422. The immediate creative output is a database structure. The long term creative output is the active use of that structure as a stimulus-response hub.

[0056] Whether a link design tool be web-server based, or whether it be a stand-alone application similar in kind to an Adobe Photoshop or a Quark Express, it is possible to offer visual metaphors to a creative designer which literally presents that designer with an image of the to-be-linked object along with explicit visual links to one or more data responses.

[0057] One embodiment of this tool for linking printed objects to web pages is a local computer application which presents an image of a printed object on the left side of an application's window pane and the image of a web page on the right side of the application pane. The images of the printed objects may be stored locally or fetched from a remote device (e.g., a content database) and rendered to the left side of the screen. Similarly, the web pages may be stored locally or downloaded from web sites on the Internet or some other network. The user interface of the application displays a control such as a button, labeled, “Connect”, “Link” or some other active word representing the process of associating an object with a corresponding machine behavior. The user, having browsed through a series of objects to be linked, and browsed through a series of potential web site destinations, finding the best “matched-pair”, pushes the button and off this relational link goes into a queue waiting to “go live”, or, in other words, a temporary record is stored for a candidate link to be sent to the behavior database of the linking system described previously. A user can perform multiple links per session, queueing them up as they go, reviewing the queue at some point in time, then directing the links to become active at the behavioral database, as described previously and in the referenced documents.

[0058] An extension begins by generalizing the single printed item to be an icon or visual analogy to a related set of printed material. Graphical user interface methods can be employed to move, manipulate, view and otherwise process this icon in a fashion familiar to creative professionals. Likewise, surrounding this generalized icon representing the object(s) to be printed can be a whole series of icons representing a variety of potential data-delivered responses that are possible links. Existing web pages, placeholders for web pages to be designed, streaming media icons, Java application icons, “links to links” icons wherein a given response may explicitly point to a menu of actions presented to the end user. (end user=consumer doing the linking). This list of possible responses is incomplete but nevertheless representative of graphically displaying the possible relationships between printed material and data responses.

[0059] As in the baseline case, various relationships can be created between objects and responses, ultimately stored into a queue. The actual functionality and quality assurance of the links could be tested in the process. Once the creative artist is satisfied with their link or set of links, the queue can be sent to go live at the behavior database and further double checks on quality performed.

Concluding Remarks

[0060] Having described and illustrated the principles of the technology with reference to specific implementations, it will be recognized that the technology can be implemented in many other, different, forms. To provide a comprehensive disclosure without unduly lengthening the specification, applicants incorporate by reference the patents and patent applications referenced above.

[0061] While the invention is illustrated with reference to watermarked stickers, aspects of the invention apply to other object types including media signals like audio and video. There are number of different watermark embedding and decoding methods that may be used. The watermark embedding process may modulate features of a signal in the time, frequency, spatial or some other transform domain of the signal to be watermarked.

[0062] In addition to an object identifier, the watermark may be used to convey other information, such as an index to related metadata, rendering control instructions, etc. For example, the watermark can carry a network address or index to a network address to link the watermarked signal to a network resource such as a related web site.

[0063] Other machine readable codes may be embedded in an object and used to link the object to a machine behavior. Some examples include bar codes, magnetic stripes, RF tags, etc. The devices and methods used to extract an identifier from the machine readable code differ, yet the process for registering identifiers and associating behavior with objects may be similar.

[0064] The methods, processes, and systems described above may be implemented in hardware, software or a combination of hardware and software. For example, the auxiliary data encoding processes may be implemented in a programmable computer or a special purpose digital circuit. Similarly, auxiliary data decoding may be implemented in software, firmware, hardware, or combinations of software, firmware and hardware. The methods and processes described above may be implemented in programs executed from a system's memory (a computer readable medium, such as an electronic, optical or magnetic storage device).

[0065] The particular combinations of elements and features in the above-detailed embodiments are exemplary only; the interchanging and substitution of these teachings with other teachings in this and the incorporated-by-reference patents/applications are also contemplated.

Claims

We claim:

1. A system for enabling a user to associate a machine behavior with an object comprising:

a registration system enabling a user to specify a machine behavior to be associated with an object via an object identifier;

2. The system of claim 1 wherein the objects are watermarked stickers.

3. A system for enabling a user to associate a machine behavior with an object comprising:

a registration system enabling a user to specify a machine behavior to be associated with an object via an object identifier;

4. A system for enabling a user to associate a machine behavior with a physical object comprising:

a registration system enabling a user to specify a machine behavior to be associated with a physical object via an object identifier encoded in a machine readable signal carrier on the physical object;

a database management system in communication with the registration system for receiving and maintaining records associating object identifiers with machine behaviors, wherein the database management system is operable to receive object identifiers and in response, to initiate the machine behavior or behaviors associated with the object identifiers, including a user interface that enables a user to view an image of an object along with an image of a behavior to be associated with the object, the user interface including a control that enables the user to associate the object with the behavior, and the user interface being operable to communicate a database structure to the database management system specifying an object identifier for the object and a description of the behavior; wherein the object identifier is embedded into the object using a watermark.

5. A system for enabling a user to associate a machine behavior with an object comprising:

a registration system enabling a user to specify a machine behavior to be associated with an object via an object identifier;

a user interface that enables a user to view an image of an object along with an image of a behavior to be associated with the object, the user interface including a control that enables the user to associate the object with the behavior, and the user interface being operable to communicate a database structure to the database management system specifying an object identifier for the object and a description of the behavior; wherein the behavior is fetching a web page.

6. The system of claim 5 wherein the database management is responsive to a watermark identifier extracted from a watermarked object by a watermark decoding computer to initiate a process of directing a web page to the watermark decoding computer.

7. A method for enabling a user to associate a machine behavior with an object comprising:

presenting a user interface enabling a user to specify a physical object and a machine behavior to be associated with the object;

receiving a specification of a physical object and a behavior to the associated with that object; and

updating a database with an entry specifying an object identifier of the physical object and the behavior, wherein the database is operable to initiate the behavior in response to receiving an object identifier decoded from machine readable code on the physical object, wherein the specification of the object includes a registration identifier provided on the object or packaging of the object; and wherein the registration identifier is embedded in a watermark on the object.

8. The method of claim 7 wherein the user interface is launched in response to decoding the watermark on the object.

9. A method for enabling a user to associate a machine behavior with an object comprising:

presenting a user interface enabling a user to specify an object and a machine behavior to be associated with the object;

receiving a specification of an object and a behavior to be associated with that object; and

updating a database with an entry specifying an object identifier of the object and the behavior, wherein the database is operable to initiate the behavior in response to receiving an object identifier decoded from machine readable code on the object; the specification of the object includes a registration identifier provided on the object or packaging of the object, the registration identifier is embedded in a watermark on the object, the user interface is launched in response to decoding the watermark on the object; and the user interface is a web based interface, and the user establishes a connection with the web based interface automatically in response to decoding the watermark on the object.

10. A computer readable medium on which stores instructions for executing the method of claim 7.

Patent Citations (127)

Patent	Date	Inventor	Cited By
US4659919(A)	1987-04-01	Price	Applicant
US4807031(A)	1989-02-01	Broughton et al.	Applicant
US4846693(A)	1989-07-01	Baer	Applicant
US4953080(A)	1990-08-01	Dysart et al.	Applicant
US5013047(A)	1991-05-01	Schwab	Applicant
US5026058(A)	1991-06-01	Bromley	Applicant
US5314336(A)	1994-05-01	Diamond et al.	Applicant
US5331547(A)	1994-07-01	Laszlo	Applicant
US5385371(A)	1995-01-01	Izawa	Applicant
US5411259(A)	1995-05-01	Pearson et al.	Applicant
US5450490(A)	1995-09-01	Jensen et al.	Applicant
US5463209(A)	1995-10-01	Figh et al.	Applicant
US5495581(A)	1996-02-01	Tsai	Applicant
US5517336(A)	1996-05-01	Molee	Applicant
US5533124(A)	1996-07-01	Smith et al.	Applicant
US5575659(A)	1996-11-01	King et al.	Applicant
US5606609(A)	1997-02-01	Houser et al.	Examiner
US5612943(A)	1997-03-01	Moses et al.	Applicant
US5617119(A)	1997-04-01	Briggs et al.	Applicant
US5621864(A)	1997-04-01	Benade et al.	Examiner
US5638513(A)	1997-06-01	Ananda	Examiner
US5640193(A)	1997-06-01	Wellner	Applicant
US5640647(A)	1997-06-01	Hube	Applicant
US5644711(A)	1997-07-01	Murphy	Applicant
US5673316(A)	1997-09-01	Auerbach et al.	Applicant
US5680619(A)	1997-10-01	Gudmundson et al.	Examiner
US5689561(A)	1997-11-01	Pace	Applicant
US5689626(A)	1997-11-01	Conley	Examiner
US5742845(A)	1998-04-01	Wagner	Applicant
US5743801(A)	1998-04-01	Welander	Applicant
US5748731(A)	1998-05-01	Shepherd	Applicant
US5754981(A)	1998-05-01	Veeneman et al.	Applicant
US5765152(A)	1998-06-01	Erickson	Examiner
US5765176(A)	1998-06-01	Bloomberg	Applicant
US5768426(A)	1998-06-01	Rhoads	Examiner
US5778102(A)	1998-07-01	Sandford et al.	Examiner
US5795213(A)	1998-08-01	Goodwin	Applicant
US5804803(A)	1998-09-01	Cragun et al.	Applicant
US5810666(A)	1998-09-01	Mero et al.	Applicant
US5825892(A)	1998-10-01	Braudaway et al.	Applicant
US5841978(A)	1998-11-01	Rhoads	Applicant
US5848413(A)	1998-12-01	Wolff	Applicant
US5853327(A)	1998-12-01	Gilboa	Applicant
US5855483(A)	1999-01-01	Collins	Applicant
US5892900(A)	1999-04-01	Ginter et al.	Applicant
US5899700(A)	1999-05-01	Williams et al.	Applicant
US5900608(A)	1999-05-01	Iida	Applicant
US5902353(A)	1999-05-01	Reber et al.	Applicant
US5903729(A)	1999-05-01	Reber et al.	Applicant
US5905248(A)	1999-05-01	Russell et al.	Applicant
US5905800(A)	1999-05-01	Moskowitz et al.	Applicant
US5906369(A)	1999-05-01	Brennan	Applicant
US5915027(A)	1999-06-01	Cox et al.	Applicant
US5915115(A)	1999-06-01	Talati	Examiner
US5932863(A)	1999-08-01	Rathus et al.	Applicant
US5933798(A)	1999-08-01	Linnartz	Applicant
US5938726(A)	1999-08-01	Reber et al.	Applicant
US5940595(A)	1999-08-01	Reber et al.	Applicant
US5954332(A)	1999-09-01	Mero et al.	Applicant
US5954515(A)	1999-09-01	Iggulden	Applicant
US5956877(A)	1999-09-01	Raasch et al.	Applicant
US5959281(A)	1999-09-01	Domiteaux	Applicant
US5974548(A)	1999-10-01	Adams	Applicant
US5978773(A)	1999-11-01	Hudetz et al.	Applicant
US5982736(A)	1999-11-01	Pierson	Applicant
US5995105(A)	1999-11-01	Reber et al.	Applicant
US6052486(A)	2000-04-01	Knowlton et al.	Applicant
US6081827(A)	2000-06-01	Reber et al.	Applicant
US6082774(A)	2000-07-01	Schlauch	Applicant
US6084528(A)	2000-07-01	Beach et al.	Applicant
US6122403(A)	2000-09-01	Rhoads	Applicant
US6138151(A)	2000-10-01	Reber et al.	Applicant
US6160540(A)	2000-12-01	Fishkin et al.	Applicant
US6164534(A)	2000-12-01	Rathus et al.	Applicant
US6190174(B1)	2001-02-01	Lam	Applicant
US6199048(B1)	2001-03-01	Hudetz et al.	Applicant
US6200216(B1)	2001-03-01	Peppel	Applicant
US6256398(B1)	2001-07-01	Chang	Applicant
US6309690(B1)	2001-10-01	Brogger et al.	Applicant
US6311214(B1)	2001-10-01	Rhoads	Applicant
US6314457(B1)	2001-11-01	Schena et al.	Applicant
US6324573(B1)	2001-11-01	Rhoads	Applicant
US6332031(B1)	2001-12-01	Rhoads et al.	Applicant
US6345256(B1)	2002-02-01	Milsted et al.	Examiner
US6434561(B1)	2002-08-01	Durst, Jr. et al.	Applicant
US6448979(B1)	2002-09-01	Schena et al.	Applicant
US6449377(B1)	2002-09-01	Rhoads	Applicant
US6456725(B1)	2002-09-01	Cox et al.	Applicant
US6470096(B1)	2002-10-01	Davies et al.	Applicant
US6505160(B1)	2003-01-01	Levy et al.	Applicant
US6510234(B1)	2003-01-01	Cox et al.	Applicant
US6526449(B1)	2003-02-01	Philyaw et al.	Applicant
US6535614(B1)	2003-03-01	Kimura et al.	Examiner
US6542933(B1)	2003-04-01	Durst, Jr. et al.	Applicant
US6553129(B1)	2003-04-01	Rhoads	Examiner
US6591252(B1)	2003-07-01	Young	Applicant
US6614914(B1)	2003-09-01	Rhoads	Applicant
US6681028(B1)	2004-01-01	Rodriguez et al.	Applicant
US6735324(B1)	2004-05-01	McKinley et al.	Applicant
US2001/0001854(A1)	2001-05-01	Schena et al.	Applicant
US2002/0006212(A1)	2002-01-01	Rhoads et al.	Applicant
US2002/0012446(A1)	2002-01-01	Tanaka	Applicant
US2002/0023148(A1)	2002-02-01	Ritz et al.	Applicant
US2002/0049967(A1)	2002-04-01	Haseltine et al.	Applicant
US2002/0059162(A1)	2002-05-01	Shinoda et al.	Applicant
US2002/0062382(A1)	2002-05-01	Rhoads et al.	Applicant
US2002/0073214(A1)	2002-06-01	Iinuma	Applicant
US2002/0075298(A1)	2002-06-01	Schena et al.	Applicant
US2002/0080271(A1)	2002-06-01	Eveleens et al.	Applicant
US2002/0080396(A1)	2002-06-01	Silverbrook et al.	Applicant
US2002/0085759(A1)	2002-07-01	Davies et al.	Applicant
US2002/0102966(A1)	2002-08-01	Lev et al.	Applicant
US2002/0153661(A1)	2002-10-01	Brooks et al.	Applicant
US2002/0155893(A1)	2002-10-01	Swanberg et al.	Applicant
EP493091	1992-07-01		Applicant
EP975147	2000-01-01		Applicant
EP1001605	2000-05-01		Applicant
JP7115474	1995-05-01		Applicant
WO97/43736	1997-11-01		Applicant
WO9803923	1998-01-01		Applicant
WO/07356	2000-02-01		Applicant
WO/26749	2000-05-01		Applicant
WO115021	2001-03-01		Applicant
WO1/61987	2001-08-01		Applicant
WO173586	2001-10-01		Applicant
WO1/84438	2001-11-01		Applicant
WO2/03328	2002-01-01		Applicant

Non-Patent Literature (32)

Jian Zhao, “A WWW Service to Embed and Prove Digital Copyright Watermarks,” May 1996.Examiner
Jean-Henry Morin and Dimitri Konstantas, “Hypernews: A Media Application for the Commercialization of an Electronic Newspaper,” ACM, 1998.Examiner
James Kempf, “Cross-Address Space Dynamic Linking,” Sep. 1992.Examiner
U.S. Appl. No. 09/630,243, filed Jul. 31, 2000, McKinley et al.Applicant
U.S. Appl. No. 09/571,422, filed May, 15, 2000, Rhoads et al.Applicant
U.S. Appl. No. 09/709,255, filed Nov. 8, 2000, Hannigan et al.Applicant
U.S. Appl. No. 60/141,468, filed Jun. 29, 1999, Livermore.Applicant
U.S. Appl. No. 60/288,272, filed May 2, 2001, Sharma et al.Applicant
U.S. Appl. No. 60/297,229, filed Jun 7, 2001, Hannigan et al.Applicant
U.S. Appl. No. 60/355,856, filed Feb. 10, 2002, Hannigan et al.Applicant
“1999 NFL Rookies Featured in Upper Deck's PowerDeck Lineup,” Upper Deck press release, Aug. 17, 1999, 2 pages.Applicant
Acken, “How Watermarking Adds Value to Digital Content,” Communications of the ACM, vol. 41, No. 7, Jul. 1998, pp. 75-77.Applicant
Alattar, “‘Smart Images’ Using Digitmarc's Watermarking Technology,” Proc. SPIE vol. 3971: Security and Watermarking of Multimedia Contents II, Jan. 24-26, 2000, pp. 264-273.Applicant
“Digimarc Turns Online Images into Powerful Marketing Agents,” Digimarc press release, Jul. 20, 1998, 3 pages.Applicant
McDonald, “Making a MARC: Digimarc's MediaBridge fastens print advertising to the Internet,” Publishing & Production Executive, vol. 14, No. 6, Jun. 2000, pp. 40, 42 and 44.Applicant
“Newly Released Magic: The Gathering Interactive Encyclopedia CD-ROM Makes Every Magic Card Ever Printed Available for Electronic Play,” Wizards of the Coast press release, Nov. 10, 1999, 2 pages.Applicant
“Overlay Plug-In User's Guide,” Plug-in- Systems, 1996, 12 pages. the ACM, vol. 41, No. 7, Jul. 1998, pp. 67-72.Applicant
Yeung et al., “Digital Watermarks: Shedding Light on the Invisible,” IEEE Micro, vol. 18, No. 6, Nov./Dec. 1998, pp. 32-41.Applicant
Zhao et al., “In Business Today and Tomorrow,” Communications of the ACM, vol. 41, No. 7, Jul. 1998, pp. 67-72.Applicant
Arai et al., Retrieving Electronic Documents with Real-World Objects on InteractiveDESK, Nov. 14, 1995, UIST '95, pp. 37-38.Applicant
Arai et al., “InteractiveDESK: A Computer-Augmented Desk Which Responds to Operations on Real Objects,” CHI '95 Mosaic of Creativity, May 7-11, 1995, pp. 141-142.Applicant
Berners-Lee et al., “World-Wide Web: The Information Universe,” Electronic Networking, vol. 2, No. 1, Spring 1992, pp. 52-58.Applicant
Berners-Lee et al., “The World-Wide Web,” Computer Networks and ISDN Systems 25 (1992) pp. 454-459.Applicant
Buyens, “Building Net Sites with Windows NT,” Excerpt from Chapter 16, Advanced Web Services, 1996, 7 pages.Applicant
Davison, “Clickable Images in HTML,” Dr. Dobbs Journal, Sep. 1995, 9 pages.Applicant
Lungstrand et al., “WebStickers: Using Physical Tokens to Access, Manage and Share Bookmarks to the Web,” DARE 2000, Apr. 12, 2000, 9 pages.Applicant
Nickerson, “WorldWideWeb: Hypertext from CERN,” Computers in Libraties, Dec. 1992, vol. 12, No. 11, pp. 75-78.Applicant
NCSA Imagemap Tutorial, http://hoohoo.ncsa.uiuc.edu/docs/tutorials/imagemapping.html, Nov. 5, 1995, 5 pages.Applicant
Rekimoto et al., “CyberCode: Designing Augmented Reality Environments with Visual Tags,” DARE 2000, Apr. 12, 2000, 10 pages.Applicant
Rekimoto et al., “Augment-able Reality: Situation Communication Through Physical and Digital Spaces,” Proc. of 2d Int. Symp. on Wearable Computers, Oct., 1998.Applicant
Want et al., “Bridging Physical and Virtual Worlds with Electronic Tags,” CHI 99, May 15, 1999, pp. 370-377.Applicant
Webcom HTML Guide, http://www.webcom.com/html/tutor/imap—concepts.shtml, 1994, 2 pages.Applicant

Abstract

Description

Related Application Data

Technical Field

[0003] The invention relates to multimedia signal processing, and in particular relates to hiding auxiliary information in media signals like audio, video and image signals.

Background and Summary

[0008] Further features will become apparent with reference to the following detailed description and accompanying drawings.

Brief Description of the Drawings

[0009] FIG. 1 is a system diagram depicting a system for associating watermark enabled objects with machine behaviors, and for triggering those behaviors in response to decoding watermarks.

[0010] FIG. 2 is a diagram depicting a variant of the system shown in FIG. 1 showing system components for enabling users to perform watermark identifier registration and embedding.

Detailed Description

[0013] These behaviors may include, but are not limited to the following:

- [0014] Taking the user to a web-site linked to the watermark via a network address of the web-site or index to the network address.
- [0015] Opening an email client to email to a specific person (e.g., a person whose email address is stored in the machine behavior description associated with the watermark).
- [0016] Launching the user into an Interframe Relay Chat (IRC) session that other people with the same sticker can participate in.
- [0017] Authenticating the user as part of a process of accessing a network resource, such as account information or access to a computer network.
- [0018] Authentication the user in an electronic commerce transaction performed on a computer network.
- [0019] Sending an electronic card.
- [0020] Placing a phone or video-conference call.
- [0021] As props in a computer game. For example, the prop is a multi-sided, or multi-faceted object, where each side or facet has a watermarked image conveying a different message used to control the game. The computer game includes a watermark decoder for extracting the messages from image frames captured of the prop. The message may directly carry the message or act as an index to a more detailed game instruction in a database, such as an instruction that changes over time based on changes to the corresponding database entry by the user or game manufacturer.
- [0022] As a visual aide for disabled users.
- [0023] Anywhere where machine vision is not feasible.

[0033]

TABLE 1

Behavior	Information Returned to	Associated Machine or
Type	Decoding Computer	Software Actions

linking to	URL, web page	launching browser on client
web site		to fetch/render web page at
		URL
opening an	email address of target	launching email client and
email client	recipient	populating address field
		with target recipient
starting a	address of chat session	launching chat application
chat session		(watermarks on the stickers
		can be designed such that
		only those holding the
		stickers can gain access to
		the chat session, each by
		showing the sticker to his or
		her watermark decoder
		enabled camera)
accessing	address of account	launching of browser
account in-	information	application to access
formation or		account information
other		through a web interface;
network		supplying user
resources		authentication information
		from watermarked object
		and/or from user (user
		password, user name, log
		on, etc.)
sending an	card template	launching client application
electronic		to enable the user to design
card		the card and add personal
		message, launching email
		application to send
		electronic card (or link to
		electronic card)
placing a	phone number or IP address	launching application to
phone or	of destination	initiate phone call over the
video con-		internet or telephone
ference call		network
props in an	identifier of prop, and	game application receives
interactive	possibly other context	prop and context
computer	information, such as game	information and responds
game	player holding the prop,	accordingly
	etc.
visual aid for	returns information in the	browser, or other media
disabled	form of graphics, audio, or	player applications render
users	video (may provide address	the information (such as the
	of audio or video content at	streaming media) on the
	an audio or video server on	decoding computer
	the Internet)
machine	machine instruction	machine or software
control		executes instruction