Method of processing an E-mail message that includes a representation of an envelope

[0002] This application is a Division of Ser. No. 08/845,722 Apr. 25, 1997 U.S. Pat. No. 6,014,688.

[0003] The present invention relates to an E-mail software program enhancement, and, more particularly, an e-mail program capable of transmitting, opening and presenting a container having digital content (e.g. Visual images, audio) using embedded executable software.

[0004] Conventional E-mail systems transmit textual messages from a sender to a receiver. It is also customary to send attachments with messages, the attachments containing files that the sender desires forwarding to the receiver.

[0005] Despite the vast usage of E-mail and E-mail systems, certain drawbacks still exist. Among these drawbacks include lack of privacy and verification of message receipt. While encryption systems exist that can provide for privacy and verification of message receipt, such encryption systems become tedious to properly use, and are, therefore, avoided in many circumstances due to the fact that they are difficult to use. Since typical E-mails are not secure in a heterogeneous environment and there is no verification of message receipt, E-mail messages are typically used only for low priority communications.

[0006] Furthermore, messages that are sent via one E-mail platform cannot be reliably transmitted to another E-mail platform and still ensure that different types of content appear as desired. For this and other reasons, E-mail messages typically lack interesting different types of content, such as visual images, audio, multimedia, and other functional attributes, such as, for instance, links to a sender's home page on the world wide web. Accordingly, since only text based content can be easily transmitted and received in a reliable manner, most E-mail transmissions tend to use textual content that looks similar, with attachments of files that are similarly uninteresting.

[0007] As a result, a more robust enhancement to conventional E-mail platforms is needed in order to reliably overcome the above mentioned drawbacks.

[0008] It is an object of the present invention to provide an E-mail message enhancement apparatus that allows for privacy and verification of message receipt.

[0009] It is another object of the present invention to provide an E-mail message enhancement apparatus that can be reliably transmitted from one E-mail platform to another E-mail platform.

[0010] It is a further object of the present invention to provide an E-mail message enhancement apparatus that allows for the transmission, reception and layout of interesting different types of content.

[0011] The above objects of the present invention, among others, are provided by the present invention through the E-mail message enhancement apparatus which is preferably implemented through a sequence of program instructions and achieved as attachments to e-mail message. On the sender side, creation software implements a data format that allows for differing types of digital content. Furthermore, in addition to such content in the E-mail message, there is included attachments, header information, including the E-mail address of the sender, and recipient executable embedded software that ensures the recipient will be able to view contents and perform the functional attributes that make up the E-mail message.

[0012] In a preferred embodiment of the invention, the executable software that is transmitted as part of the E-mail message is written using Java programming language and contains a self-opening message feature, such that the container automatically opens and is initially viewed with all of the desired content appearing. Thereafter, depending upon the content originally included by the sender, and the action taken on the part of the recipient, different portions of the content and functional attributes are highlighted. Such content and functional attributes will be view able, regardless of the E-mail platform of the recipient, due to the inclusion of the executable software in the E-mail message itself that is transmitted.

[0013] Through the use of a data structure that allows for a plurality of component objects to be included in a single E-mail message, as well as a plurality of sub-component objects to be associated with each component object, an E-mail message having robust content and functional attributes can be achieved.

[0014] In a specific embodiment of the invention, the E-mail message that is transmitted has a plurality of component objects, with one of the component objects representing the front face of an envelope, and different other component objects including, but not limited to, recipient address, sender address, stamp, digital time stamp, and an envelope open attribute so that the E-mail message appears on the recipient's computer screen like an envelope. When the E-mail message is opened using the envelope open attribute, or subsequently closed, verification of message receipt is automatically forwarded back to the original sender. In a specific form of this embodiment, both sides of the envelope can be represented, with another component object representing the back side of the envelope and the envelope open attribute that allows for attachments or messages within the envelope to be viewed appearing on the back side of the envelope.

[0015] In another specific embodiment of the present invention, the E-mail message that is transmitted has a plurality of component objects, with one of the component objects representing the front face of a postcard, and different other component objects represented on the back face of the postcard being the recipient address, sender address, stamp, senders letter and digital time stamp.

[0016] Still furthermore, the size and shape of component objects can be varied as desired by the sender, overlap with each other and vary in number within a single E-mail message. Accordingly, the appearance of the E-mail message can be varied as desired by the sender.

[0017] The structural and functional aspects of the invention will become understood by those skilled in the art by reference to the following detailed description when taken in conjunction with the appended drawings, in which:

[0018] FIG. 1 is a diagram illustrating an implementation of the invention in a computer network;

[0019] FIG. 2 illustrates an E-mail message format for an enhanced E-mail message according to the invention;

[0020] FIG. 3 illustrates the format of an envelope data structure according to the invention;

[0021] FIG. 4 illustrates the contents of an envelope header in an envelope data file according to the invention;

[0022] FIG. 5 illustrates envelope data;

[0023] FIG. 6 is a diagram illustrating a component objects file according to the invention;

[0024] FIG. 7 illustrates the contents of a component object header data structure according to the invention;

[0025] FIG. 8 illustrates the contents of a component object data structure according to the invention;

[0026] FIG. 9 illustrates the contents of a component object property data structure according to the invention;

[0027] FIGS. 10A and 10B illustrate the process of creating and sending an enhanced E-mail message according to the invention;

[0028] FIG. 11 is a diagram illustrating the process of receiving an enhanced E-mail message according to the invention;

[0029] FIG. 12 illustrates an enhanced E-mail message initial container display according to the invention;

[0030] FIG. 13 is a diagram illustrating-the process of handling a user action according to the invention;

[0031] FIG. 14 is a diagram illustrating the process of conditional display of contents if conditions set by the sender according to the invention;

[0032] FIG. 15 is a diagram illustrating the process of sending return receipt E-mail messages according to the invention;

[0033] FIGS. 16A, 16B1-16B2 and 16C1-16C4 illustrates example data structures for an envelope container having a company logo as an object component according to the invention; and

[0034] FIGS. 17A-17D illustrate views and actions that result from the data structures such as set forth in FIGS. 16A, 16B1-16B2 and 16C1-16C4.

[0035] FIG. 1 illustrates a computer network 10 which operates upon the E-mail enhancement program according to the present invention. A sender computer 20 communicates with a recipient computer 30 through a network 12. Of course, additional computers, represented as computers 32(1), 32(2). . . 32(n) can also be connected to the network 12 and be used to send and receive E-mail messages.

[0036] Sender computer 20 contains, as is known, a computing portion 22 that contains, as is known, I/O 22A, memory 22B and some type of processor 22C, which can be in the form of, for instance, a microprocessor. Attached to the computing portion 22 is a keyboard 24, a mouse 26, and a display 28. Similarly, recipient computer 30 contains, as is known, a computing portion 32 that contains, as is known, I/O 32A, memory 32B and some type of processor 32C, which can be in the form of, for instance, a microprocessor. Attached to the computing portion 32 is a keyboard 34, a mouse 36, and a display 38.

[0037] Creation executable software according to the present invention contains sequences of program instructions that allow for the creation of an enhanced E-mail message according to the present invention, as well as the transmission of the created message through the network to the recipient. Recipient executable software according to the present invention contains a different sequence of program instructions that allow for the receipt, and thus visual, audible and functional attributes to be obtained by the recipient.

[0038] There currently exist two different embodiments of the software according to the present invention. In a first, version of the software, the creation and recipient executable software is loaded into the memory 22B of the sender computer 20. When this embodiment of the software is used, the E-mail message structure contains, as described hereinafter, a portion that attaches the recipient executable software to the E-mail message transmitted from the sender to the recipient. So long as the recipient computer 30 can operate as a Java virtual machine, the recipient computer 30 will receive the E-mail message containing the recipient executable software, preferably written in Java, and use that recipient executable software to display the enhanced E-mail message on the display 38 of the recipient computer 30.

[0039] In a second, native version of the software, the creation executable software is loaded into the memory 22B of the sender computer 20, and the recipient executable software is loaded into the memory 32B of the recipient computer 32. In this embodiment, the transmitted E-mail message will not contain the recipient executable software, since the recipient executable software will already reside on the recipient computer 30.

[0040] It should of course be understood that the present invention is not to be interpreted in a manner that limits the present invention to a specific sequence of program instructions. The descriptions given herein, as well as a review of the source code attached hereto, provide the teachings necessary to implement the present invention in different programming languages and on different operating system platforms.

[0041] FIGS. 2-9 will now be described, which figures illustrate the data format which the Java executable software version implements. Those specific fields which are believed to be self explanatory will not be described in detail. Before describing this data format in further detail, however, reference is made to FIG. 12, which illustrates an enhanced E-mail initial container display 100 made up of component objects 102, 104, 106 and 108. It should be noted that the number of component objects in any initial display can be varied, as can the size and shape of each of the component objects. Also, it should be noted that the component object is also referred to in certain of the drawings and potentially the specification text as a “Vixel”, which term is a trademark of PostX Corporation. The initial display 100 provides a display that is also referred to in certain of the drawings and potentially the specification text as “Active View”, which term is a trademark of PostX Corporation.

[0042] FIG. 2 illustrates the E-mail message format for the enhanced E-mail message according to the present invention. This data format 40 includes a native stub code 40A that provides for a self-opening envelope at the present time, despite the fact that Java classes are not presently defined so that self-opening can occur without the use of such a native stub. Thereafter, data 40B indicates the number of recipient executable files, which files then follow as files 1, 2. . . n, which are labeled as 40C-1, 40C-2. . . 40C-n, the number of which will be determined by the different number and type of images, sounds, and functional attributes that the sender inserts into the enhanced E-mail message that is sent. Container/envelope data file 40D follows thereafter, which file is further described with reference to FIG. 3.

[0043] As illustrated in FIG. 3, the envelope data file 40D contains an envelope header 42, the contents of which are provided in detail in FIG. 4. Of note with respect to the contents of the FIG. 4 header is the return E-mail address of the sender, as well as the recipient name that is used in the return receipt E-mail function described further hereinafter, as well as an indication of whether the E-mail message is encrypted as described further hereinafter. It should be noted that the header is not encrypted.

[0044] As also illustrated in FIG. 3, the envelope data file 40D contains envelope data 44, which includes the list of attachments sent by the sender, and recipients and sender's letter. Senders Attachments 50-1, 50-2. . . 50-n represent attachments that are included within the enhanced E-mail message, as will be described in more detail hereinafter, as will be the contents of the component objects file 54, also referred to as the Active View file, the contents of which are set forth in FIG. 6.

[0045] As shown in FIG. 6, component objects file 54 includes a component object header data structure 56, followed by component object or Vixel data structure 58-1, 58-2. . . 58-n, and component object property or Vixen property data structures 60-1, 60-2, 60-2-n . . . 60-n. The contents of the component object header data structure 56 are set forth in FIG. 7, the contents of the component object property data structures 58 are set forth in FIG. 8, and the contents of sub-component object property data structures 60 are set forth in FIG. 9.

[0046] Of note with respect to the contents of the header data structure illustrated in FIG. 7 is that the “platinum envelope” entry for the file information is used as a description information, no conditional processing is done with this field. The identifiers for “Container Type” correspond to different representations of default containers.

[0047] A default “envelope” container contains component objects representing the front face of an envelope, and different other component objects being recipient address, sender address, stamp, digital time stamp, the back side of the envelope and the envelope open attribute that allows for attachments or messages within the envelope to be viewed appearing on the back side of the envelope so that the E-mail message appears on the recipient's computer screen like an envelope.

[0048] A default “postcard” container contains component objects representing the front face of a postcard, and different other component objects represented on the back face of the postcard being the recipient address, sender address, stamp, and digital time stamp.

[0049] A default “newsletter” container contains a component object representing each page of a newsletter, as well as various component objects on each page.

[0050] A default “CD” container contains a component object for each different audio track for a component object that has a functional attribute of playing music.

[0051] A default “Rolled up Newspaper” container contains a component object representing a newspaper, a component object representing the index to the newspaper, and a component object representing each section of the newspaper, as well as component objects for each article if desired.

[0052] A default “stack of card” container contain a component object representing each card.

[0053] A default “coupon” contains a component object for each coupon that could appear in, for example, a newspaper.

[0054] The Front Color indicates the background color that will be used for the container 100, whereas the Envelope Size is used by the default envelope container to represent the width and height dimensions.

[0055] The Data Location refers to whether data for the component object requires looking to a data field, fetching data using a URN, or an absolute offset. The data field may be a digital data field that contains digital data, such as digital images from, for instance, a GIFT, a video sequence, an audio sequence, or other representations of images.

[0056] For each component object, there exists the data structure illustrated in FIG. 8, which data structure contains a Component ID that allows that component object to be uniquely identified, as well as Component Attributes that identify whether the component object appears as an image on the front and back of a container, or is invisible. If it appears on the front or back of a container, the size of the component object, its location, and the location of data are also specified. If it is an invisible attribute, the size and location data may not be necessary. Also a hidden component object can, for instance, trigger a return receipt without the knowledge of the recipient, such that a mouse over would cause a return receipt if the hidden component object were configured to respond to a mouse over by providing a return receipt when a mouse-over occurs.

[0057] Potentially associated with each component object identified by a data field of FIG. 8 are a plurality of sub-component objects. Each sub-component object associated with a component object contains data fields provided in FIG. 9.

[0058] Each sub-component object is initiated by a User Action, which User Action is specified by, but not limited to the following, the position and state of the mouse pointer, which state can be that it is not over the component object (no action), appears over the component object (mouse over), there is a single mouse click when it is over the component object, or there is a mouse double click when it is over the component object. Each User Action can cause a Sub-Component-Type to be displayed/initiated. A plurality of sub-components can be associated with a single component object, and be initiated by the same or different User Action.

[0059] Each sub-component object also has an ID, as well as an ID type that has associated with it a primary or a secondary sub-component object. The secondary sub-component object will only be initiated if the primary sub-component object is unavailable, as will be described in more detail hereinafter.

[0060] Having now described the preferred data format that is used to transmit the enhanced E-mail message according to the present invention, FIGS. 10 and 11 will be referred to in describing the creation and sending, and the receipt, respectively, of an enhanced E-mail message, so that a container such as the container 100 illustrated in FIG. 12 is displayed on the display 38 of a recipient computer 30 after having been sent from a sender computer 20.

[0061] The creation and sending of an enhanced E-mail message according to the present invention, as well as initial receipt of an enhanced E-mail message prior to a user event, will now be described with respect to FIGS. 10 and 11.

[0062] FIGS. 10A-10B illustrate the process of creating and transmitting an enhanced e-mail message with a container attachment that is self opening and has the other characteristics described herein. In step 62, header information is first designated to define the container. Thereafter, component definitions are created by the sender in a step 64. There can be any number of component definitions desired, with it being understood that the more definitions that exist, the larger the size of the attachment that will ultimately be transmitted. Thereafter, sub-component definitions can be created for each component object, as described previously, is step 66. In step 68, the component and sub-component files are combined into an Active view file, which thereafter is combined with header information and attachments to create a container file in step 70. The container file is encrypted in step 72. Once being encrypted, in the preferred embodiment, the Java class files, collectively termed the recipient executable software, and the native stub are attached to the encrypted container file in step 78. Mime encoding follows in step 80, which mime encoded file is then attached to an e-mail message in step 80 and transmitted.

[0063] Upon receipt at the recipient computer in step 84 with reference to FIG. 11, double clicking the received message in step 86 if on a java virtual machine will cause self opening to occurs in step 88, or, alternatively, opening the container attachment. Thereafter, the files are extracted from the attachment in step 90 and invocation of the application (the java virtual machine and the java class files) and extraction of the senders container data files in step 94 will cause the display of the container in step 96. Thereafter, in step 98, the computer 30 awaits the recipients action, as will be further described hereinafter.

[0064] As a result, of the display on the display 38 of the recipient computer 30, a container, such as the envelope container 100 illustrated in FIG. 12 is obtained. At this time, the recipient executable software then proceeds to a state in which it waits for the user to take an action, as will now be described with respect to FIG. 13.

[0065] In step 110, a first event takes place. Step 112 follows and a determination is made in step 112 as to whether the event that takes place is an event that is directed to one or more object components. If it is not such an event, then step 114 follows and the recipient executable software does nothing, and waits for a next event in step 120, Types of nothing events would be, for instance, if the mouse cursor 122 illustrated in FIG. 12. Moves to an area that is not over any object component.

[0066] If, however, step 112 determines that the event is directed to a component object/Vixel, then step 116 follows and it is determined which component objects have events, and then in step 118 the action associated with the event intended for each component object takes place.

[0067] For instance, if component object 102 illustrated in FIG. 12 contains an User Action of FIG. 9 in the form of a mouse over, then the action of placing the mouse cursor 122 over the component object 102 will cause in step 112 a determination is made that the action involves a component object, and in step 116 a determination that the action pertains to component object 102. Thereafter, step 116 to results and the actions associated with that event will be executed. A specific example of such events will be provided hereinafter.

[0068] FIG. 14 illustrates one functional attribute of the present invention, which functional attribute can be set forth at the initiation of receiving an enhanced E-mail message according to the present invention, or after the initiation of an action stemming from a component object. An event 124 initiates the conditional display of contents functional attribute. If this conditional display of contents functional attribute occurs at the initiation of the enhanced E-mail message, step 126 follows and the enhanced E-mail message is opened as an attachment, as has been previously described. Step 128 follows, in which terms and conditions which must be agreed upon are first made known to the recipient. Thereafter, step 130 follows and the recipients response is reported back to the sender, in a manner using some form of return receipt as will be described hereinafter. If the conditions responded to in a manner deemed appropriate by the sender, then step 132 follows, and the appropriate contents are then made accessible to the recipient. Step 134, which follows thereafter, waits for another event to occur, as has been previously described with reference to FIG. 13.

[0069] FIG. 15 illustrates another functional attribute of the present invention, referred to as a return receipt. If a return receipt is requested by the sender, step 136 is initiated at an appropriate time. This could be when the attachment that forms the enhanced e-mail according to the present invention is initially transmitted to the recipient, when the recipient closes the enhanced e-mail that has been received, or it can be triggered by an event that relates to a sub- component of a component object according to the present invention. If the sender did not request a return receipt when initially sending the message, then step 140 follows, no action is taken, and then step 142 waits for another event to occur, as has been previously described with reference to FIG. 13. If, however, sender requested a return receipt, step 144 follows and a determination is made whether a specific format of response was requested. If not, step 146 follows and a default response, such as “[email protected] has opened the envelope at 10:p.m. Pacific daylight time” is transmitted, this response being delivered using the sender address information, and other time information on the system. Step 148 follows and the computer 30 waits for another event to be initiated by the recipient, as has been previously described with reference to FIG. 13.

[0070] If a specific format of response was requested in step 144, then step 150 follows and determination is made whether the sender's request requested a specific input from the recipient type of response. If it did not, then the senders configured response is sent in step 152 to addresses specified by the sender, which may be the senders own e-mail address, the address of another, or the address of many others. Thereafter a step 154, like step 148 previously described, follows.

[0071] If a response from recipient was requested, then the requested response is made known in step 156 and collected and compiled in step 158. Thereafter, the recipient's response is automatically transmitted in step 160 and another wait step 162 follows.

[0072] The return receipt responses transmitted back to the sender in steps 146, 152, and 160 are each automatically sent using the APE provided by respective E-mail client's. For STP, routines to send mail can be written using public domain information.

[0073] FIGS. 16A, 16B1-16B2 and 16C1-16C4. Illustrate example data structures for an envelope container having a company logo as an object component according to the invention. FIGS. 17A-17D illustrate views and actions that result from the data structures such as set forth in FIGS. 16A, 16B1-16B2 and 161-16C4. So that the capabilities of the present invention can be appreciated, it should be noted that FIG. 17A is represented by the data structure of FIGS. 16a, 16C1 and 16C2; FIG. 17B is represented by the data structure in FIG. 16A; FIG. 17C is represented by the data structure in FIG. 16C3; and FIG. 17D is represented by the data structure in FIG. 16C4.

[0074] Of particular note, a mouse over 180 in FIG. 17A results in a bubble 180-A, which corresponds to data associated with the bubble as defined by mouse over action if FIG. 16C1. That has a text sub-component. Similarly, FIG. 16C2. Data defines that the mouse over 182 will cause a process launch attribute that will result in audio music being played by a program containing program instructions to launch the audio content, which programs can also have been transmitted from the sender with the originally sent enhanced e-mail message.

[0075] A digital time stamp 184 is also included and indicates either sender transmission time or, in the alternative, a certified time stamp from a dedicated time stamp server. Object components 188-204 are also illustrated on both sides of the envelope, and represent the ability of the present invention to depict, as desired by the user, many different forms, in many different manners, as well as attributes that then lead the recipient to other areas. For instance, FIG. 16C3., when a mouse click over the stamp object component 186 occurs, if the user is on line at the time, will result in the user automatically being connected to the home page 206 illustrated in FIG. 17A. If, however, that mouse click over the stamp object component 186 occurs, but the user if off line, then the FIG. 16C3. Primary sub-component is not used, but the alternate sub-component that is identified by FIG. 15C4. Is launched, which in this case results in a textual message being displayed.

[0076] The above example is intended to show how robust the enhanced e-mail message enhancement apparatus according to the present invention actually is. By being able to be sent as an attachment through E-mail, the appearance of the E-mail message can be enhanced at the option of the sender, and the recipient will be ensured of being able to receive it in the intended form, since the executable recipient program instructions are preferably transmitted as part of the attachment.

[0077] Furthermore, even though the executable recipient program instructions can ensure receipt of the message in its intended form, the architecture of the present invention also provides that if a more robust program or hardware element exists on the recipient computer 30, which can even further enhance the content that is transmitted by the sender, then that more robust program or hardware element exists on the recipient computer 30 can be used instead of the executable recipient program instructions that were transmitted by the sender.

[0078] While the invention has been particularly shown and described with reference to the preferred embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.