Dynamic annotations for electronics documents

[0001] This invention relates to applying annotations to electronic documents. Electronic documents are collections or aggregations of electronic data, such as files or collections of files, that typically can be displayed with an appearance that resembles that of a paper document. Early electronic document applications—that is, applications in which electronic documents can be created, manipulated and viewed, in any combination—such as word processors, publishing applications, and the like, allowed a user to do little more than add and change text on an electronic document page with simple formatting selections. As electronic document applications have evolved, the number of options to alter the appearance of electronic documents has grown. Some applications allow users to add annotations, such as stamps, to an electronic document. One such electronic document application, Adobe® Acrobat®, available from Adobe Systems Incorporated of San Jose, Calif., allows a user to apply one or more of a number of stamps. The stamp can display a word, such as “Confidential”, “Draft”, “Approved” or any of a number of other available selections.

[0002] Adding a stamp to an electronic document can be as simple as selecting a stamping tool with a pointing device, moving a cursor controlled by the pointing device to the area of the document where the stamp is to be applied, and activating the pointing device. The selected stamp is applied by the application. The stamp can be altered by modifying one or more stamp properties—for example, selecting a new appearance for the stamp, dragging the stamp to another section of the document or deleting the stamp.

[0003] The invention provides computer-implemented techniques for applying a dynamic annotation to an electronic document. According to the techniques described herein, an instance of an annotation can be applied to the electronic document based on existing conditions at the time the instance is applied. When two of the same annotation are applied, the appearance of each annotations may be the same or different, depending on the state of the conditions upon which the annotations are based when the annotations are applied.

[0004] In general, in one aspect, the invention features methods and apparatus, including computer program products, implementing techniques for annotating an electronic document in an electronic document application. A request to add an annotation to an electronic document application is received. An annotation template for the requested annotation is retrieved. The annotation template is external to the electronic document application and the electronic document and includes fixed content, associated instructions defining variable content, and layout information for the fixed content and variable content. The associated instructions are executed to define the variable content for an instance of the requested annotation. The instance of the annotation is applied to the electronic document, such that the annotation instance has an appearance defined by the variable content, the fixed content and the layout information.

[0005] Particular implementations can include one or more of the following features. A condition based on a state of a system, the electronic document application or the electronic document can be determined and the variable content can be based on the determined condition. The instructions to define variable content can generate graphics or obtain graphics, text, an image, audio, video or a hyperlink. The appearance of vector art can be defined. The instructions can define a layout for the variable and fixed content. Content information can be received from a remote or local system or memory. The annotation template can include an electronic form and the electronic form can have form fields with associated scripts. The annotation content can be flattened to create a static instance of the annotation and the instance can be applied to the electronic document to create a fixed representation of the instance of the annotation. When the instance of the annotation is applied, the instance can be associated with a specific location in the electronic document.

[0006] The invention can be implemented to provide one or more of the following advantages. An annotation template can be created that includes fixed and variable content, and layout information that defines the spatial relationship between content. The fixed content includes content that will be incorporated in each instance of an annotation. The variable content is content that is dynamically generated at the time the annotation is applied, and can include, for example, dynamically generated text or graphics. The layout of the annotation content can be dynamically determined. The annotation content can be automatically retrieved from a source outside the electronic document or document application. For example, the logic that performs the steps to create the annotation can include instructions that cause the system to find and integrate data from a server into the annotation. Annotation templates can be added, edited and deleted by an author without altering the application, making the annotations fully customizable. The instructions to create the annotation are not a part of the electronic document application's logic that adds the annotation to the electronic document. The annotation template is also not a part of the document to which the annotation is being applied. The annotation templates can be customized for the end user, making each annotation template useful in a particular instance. The appearance of the annotation can depend on factors or values related to the document to which it is applied or the appearance can simultaneously or separately depend on factors independent of the electronic document. Because the annotation is instantiated just before placement, different instances of an annotation of the same type can have different appearances depending on conditions when the annotation is applied.

[0007] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.

[0013] Like reference symbols in the various drawings indicate like elements.

[0014] The invention provides techniques for defining annotations in the context of electronic documents. An annotation is an object that can be added to an electronic document to comment on, explain, expand upon, or otherwise mark the content of the electronic document, often in a way that mimics annotations in conventional paper documents. Thus, for example, an annotation can embody a reader's commentary on a particular passage of an electronic book, resembling notes penciled in the margin of or a “sticky note” attached to a page of a conventional paper book. Similarly, an annotation can provide a stamp that, when applied at a particular location in an electronic document, identifies content at that location as “Draft” or “Confidential”, just as might be done with a conventional rubber stamp in a paper document. Like conventional annotations, electronic annotations can include text and graphics; unlike-their conventional counterparts, however, electronic annotations can include any other form of electronic content, including, for example, audio and video information such as sounds and movies.

[0015] The annotations described herein have an appearance that is determined by defining some of the annotation content at the time the annotation is applied. To create this content, the annotation includes a series of instructions, for example, a script that can be executed to create an annotation instance based on data available when the annotation is selected to be applied.

[0016] FIG. 1 illustrates a method 100 for adding an annotation to an electronic document that can be implemented in an electronic document application according to one aspect of the invention. The method begins with a request to apply an annotation to an electronic document (step 10). A user can request the annotation by selecting a particular annotation type—for example, by selecting a tab, button, option from a pull down menu or other user interface that identifies annotation types that are available for the particular document or application. Alternatively, the request can be generated automatically in response to an event, such as the opening of a document. The request can specify a particular annotation type or a generic annotation of unspecified type. If the request is generic, the system can prompt the user to identify the type of annotation to be applied to the document—for example, by presenting a pull down menu, a list with check boxes or radio buttons, or some other interactive user interface through which the user can select a desired annotation type.

[0017] Each annotation type can have an associated annotation template that is used to define the appearance of instances of the corresponding type, as will be described in more detail below. An annotation template is an object, such as a file, that includes the instructions and optionally other information, including fixed content, such as vector graphics, and layout information, that will be used to define the appearance of an instance of the annotation. The annotation templates are external to the application and the document—i.e., the templates are not coded directly in the application and do not reside in the document to which the annotations are being applied. The templates can therefore be stored and manipulated separate from the application and document, as will be discussed in more detail below. The application can determine what annotation types are available by querying annotation template storage and presenting the available types for selection by the user. For example, the application can query a particular directory or a database in which annotation templates are stored.

[0018] In response to the request, the application retrieves fixed content, layout information and annotation instructions that are associated with the annotation type (step 120). The instructions define some of the content of the annotation, as described in more detail below. Additionally, the instructions can define in whole or in part how the content is to be arranged or implemented. For example, the instructions can determine some or all of the layout and the appearance of the content (e.g., instead of or in addition to any layout or appearance information defined as a fixed part of the annotation template). The application can retrieve the instructions in the form of a script, such as a script in the JavaScript language or another machine readable code. The instructions for each annotation type can be stored (and retrieved) in a unique annotation template.

[0019] In one implementation, the annotation template can be an electronic form. Forms typically include fields that can have associated instructions that specify actions to be performed in the form, such as calculating a value or values to be entered in a form field, performing an action (e.g., retrieving data from a specified location) in response to user input (e.g., selection of a button by a user), or the like. Forms can also be used to transmit data and receive a response. Thus, annotation templates can be implemented as files in a format that supports scriptable electronic forms, such as the Portable Document Format (“PDF”), introduced by Adobe Systems Incorporated of San Jose, Calif. Annotation templates can also include static content that will determine a portion of the appearance (e.g., a background) of annotations instantiated from the type.

[0020] The application interprets and/or executes the instructions to create content for a specific instance of the annotation (step 130). Executing the instructions can cause the application to define the variable content, generally by performing any combination of creating, retrieving, arranging, or transforming of data or elements from a source where some or all of the elements are stored, such as the system's memory, external memory, a server or other remote location. Because the annotation is instantiated with content and in some cases a layout, as described below, that is defined at the time the annotation is applied, each instance of a particular annotation type can be created with a different appearance, determined by a condition or conditions at the time it is created, as further described below.

[0021] The application creates an instance of the annotation according to the fixed layout information included in the annotation template and, if available, variable layout information defined by the instructions (step 140). The layout information defines the relationships between the different pieces of fixed, variable or fixed and variable content. The fixed content and the variable content are arranged according to the layout information. In one implementation, where the annotation is implemented as an electronic form, creating an annotation instance can include defining an instance of the form having defined values for one or more form fields provided in a template form.

[0022] The annotation instance is applied to the document (step 150). To apply the annotation instance, the defined instance is flattened, and the flattened instance is applied to the document. Flattening can include, for example, converting the now-defined annotation content, as well as any fixed content, into a fully static form, such as a bitmap. Generally, the flattened annotation cannot be transformed back into variable content. In some implementations, the user can be provided the option of changing the annotation before the annotation is flattened. Once the annotation is flattened, the annotation cannot be updated when an element on which the annotation was based changes, unlike form fields which may change when a change occurs in the document. Application of the annotation instance can include associating the annotation instance with the electronic document and storing the annotation instance in association with (e.g., within) the document. For example, if the document is a PDF document, the annotation instance can be added as a layer to the PDF document.

[0023] Thereafter, when the document is rendered, a representation of the annotation instance is rendered at the location at which the annotation was applied. The application can place the annotation instance in a variety of ways—for example, in response to user input specifying a location for the annotation, based on information included in the original request to apply the annotation, according to a value or rule (e.g., defined by the instructions), or in a predetermined location. For example, if the annotation is placed based on the request, a reference point (e.g., the annotation's center) can be located at a location on the document page where the user activates the pointing device. Alternatively, the application can invoke a rule that defines a location for the annotation—for example, a rule defining an offset distance from the location at which the pointing device was activated. In still another alternative, the instructions themselves can define a location for the annotation based on external conditions, such as the number of times an annotation of the particular annotation type has been previously applied to the electronic document. In some implementations, once the annotation is applied the user can modify the annotation—for example, by moving the annotation to another location or deleting the annotation. In particular implementations, the ability to modify annotations can be restricted according to rights or permissions, such as permissions provided with the application or with the particular document.

[0024] In some implementations, a request need not be received from a user, but can be generated automatically, such as in response to an event. Consider, for example, a document that is to be reviewed by more than one user before it is completed. The document can be created by a first user who includes, as a part of the document, a request to apply an annotation each time the document is opened. Generally the request specifies a specific annotation template—for example, a template for an annotation that includes an identification of the user opening the document and the date. Thus, each time that a user opens the document, the annotation template is retrieved and the instructions are executed to define variable content including the date and user identification. An annotation instance is created, as described above, and applied to the document. Each of these annotations is displayed in the document—for example, on the first page—such that each subsequent user who views the document can readily identify all other users who have previously viewed the document, and when the document was viewed.

[0025] The instructions for creating the annotation content can cause the application to retrieve data from a variety of locations, such as the system's memory, external memory, a server or other remote location over a network, the document that the annotation is to be applied to, or the user can be prompted to enter information. The data can include any information available to the application, such as the identity of the application user, the date, the document type, the document name, the number or contents of the page on which the annotation is going to be applied, or a serial number or other information retrieved from a local or remote location. The application can use the retrieved data to dynamically define annotation content based on one or more conditions or rules. For example, the application can define the variable content based on a condition related to the state of the document, such as how many times the document has been opened, how many annotations have been applied or how many other users have annotated the document. Alternatively, the condition can be unrelated to the document, or external to the document, such as the date, the time, or a number generated by the application or another application. Some of the content can be defined according to the condition(s) or retrieved information, while some portion of the content can be predefined (i.e., fixed).

[0026] The fixed or variable annotation content can include any item that can be presented in an electronic document, such as graphics, text, images, audio, video, hyperlinks, or a combination of two or more of these. Audio and video content can either be in an open (activated) or closed state when the document is opened. If the content is not activated automatically upon opening the document, a user can activate the audio or video by selecting the annotation. Hyperlinks generally must be activated by the user to jump to a linked object. The graphics can be in any suitable form for the application, such as bitmaps, vector art, or raster art. If the appearance of no fixed content is desired, the fixed content can include a transparent frame.

[0027] As discussed above, the template can include layout information that defines the spatial relationships between the content. Because this layout information is included in the annotation template, each time an annotation is rendered from a particular annotation template, the layout information causes the application to configure this content in the same way each time. The application template can additionally include instructions defining the annotation layout based on the conditions, such that how the content will be presented is defined and can change from one instance of applying the annotation to another. As an example, an annotation template can include instructions specifying a variable layout for an annotation including a text object and a graphic object, where the relationship of the two objects changes with each application of the annotation moving the text in a clockwise direction in relationship to the graphic each time the annotation is applied. In this example, when the annotation is first applied in a document, the text is positioned in one orientation relative to the graphic—such as above the graphic. When a second annotation is applied in the document using the same annotation template, the text is positioned in a different orientation relative to the graphic—e.g., to the right of the graphic. A global variable can be used to track the number of times the annotation is applied. Other aspects of the appearance of the content, such as the color, size, shape and effects applied to the content, can also be determined by the conditions.

[0028] In some implementations, the application can be configured to enable authors to define new dynamic annotation types. The author defines a new annotation type by defining an annotation template. In one implementation, where the annotation template is an electronic form, the author creates the annotation template using conventional form creation techniques, defining one or more form fields with associated instructions, such as scripts, that will be used to define content for the associated fields when an annotation of the type is to be applied. Because the annotation instructions are external to the application, they can be modified by simply modifying the annotation template or file using conventional applications, such as word processors or Adobe® Acrobat® software, without requiring modifications to the application code.

[0029] One specific example of applying an annotation to an electronic document will now be described, with reference to FIGS. 2-5. In the following example, an annotation 200 including a serial number 205, a user identifier 210 and a date 215, can be applied to an electronic document, as shown in FIG. 2. The serial number 205 is a unique number that is received from a centralized database, such that multiple users can access the database and each user can apply a unique number. Because the information compiled to create the annotation is retrieved just before application, the annotation will include the most current information available, which is the next number in the series.

[0030] As shown in FIG. 3, the user begins by opening the electronic document 300 in an electronic document application. To select an annotation to be applied in the document, the user selects a desired annotation tool—for example, by selecting a corresponding menu item from a pull down menu that identifies all annotation types that are supported by the application and/or the document 300. In this example, the user selects the annotation type “serial number”. When the user has selected the desired annotation type, the application changes the appearance of the cursor to indicate that an annotation tool has been selected. The user then specifies where the annotation will be placed by navigating the cursor 305 over a desired paragraph and activating the pointer tool.

[0031] The application receives the selection and retrieves an annotation template 400 associated with the “serial number” annotation, as shown in FIG. 4. The template 400 contains the fixed content, a set of instructions that can be executed to define the variable content, and layout information. The application executes a first instruction 405 to retrieve a serial number from a database. A second instruction 410 is executed, which determines a user identifier. In this example, each user identifier includes a job title and number and is a part of the user's login information. The third instruction 415 causes the current time at which the annotation is instantiated to be retrieved.

[0032] In addition to the instructions that determine the variable content, the template also includes fixed content 420. In this case, the fixed content 420 defines a rectangular yellow frame. Referring back to FIG. 2, the layout information 425 specifies that the serial number 205 should be placed at the upper center, the user identifier 210 in the middle and the date 215 on the bottom center of the text field. The variable content is then centered on the fixed content 220. In this example, the instructions do not specify additional layout information.

[0033] Once all of the instructions that determine the appearance of the annotation have been performed, the annotation content is flattened to define a static appearance of the annotation. The application performs a script that designates the placement of the annotation, which in this example applies the annotation an inch to the right of the paragraph in which the cursor was located when the annotation was selected. The flattened annotation is associated with this location so that each time the document is rendered the annotation appears at the same location. The document, including a layer corresponding to the serial number annotation, can be saved for future access.

[0034] As shown in FIG. 5, when the user or another viewer later opens the annotated electronic document 500, the annotation is displayed at the location at which it was previously applied.

[0035] The invention can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The invention can be implemented as a computer program product, i.e., a computer program tangibly embodied in an information carrier, e.g., in a machine-readable storage device or in a propagated signal, for execution by, or to control the operation of, data processing apparatus, e.g., a programmable processor, a computer, or multiple computers. A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

[0036] Method steps of the invention can be performed by one or more programmable processors executing a computer program to perform functions of the invention by operating on input data and generating output. Method steps can also be performed by, and apparatus of the invention can be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in special purpose logic circuitry.

[0037] A number of embodiments of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. For example, other commands can be executed by the annotation type's script, including setting button form fields to an arbitrary appearance, changing text, changing form fields, changing the appearance of vector art-based form fields and moving form fields. The shape of the annotation can be non-rectangular. The annotation can be scalable. The annotation can be chosen by selecting a button that causes the actions to be performed. The actions can call the commands that create an annotation. Accordingly, other implementations are within the scope of the following claims.