US 6,816,835 B2Grant

Electronic mail system and device

Issue Date:2004-11-09

•7 Claims

•10 Drawing Sheets

Abstract

Insertion of an attachment file identifying character code in a text of an electronic mail enables an electronic mail device to automatically reproduce (display) data from the attachment file during vocal reproduction of the text. When a text of a received electronic mail was selected and a vocal reproduction start key of the device was pressed, the device stores vocally readable text data to be sent to a voice synthesis control portion and attached reproducible music data in a buffer portion of a random access memory. The device then transmits a voice synthesis and reproduction command to a voice synthesis control portion to vocally reproduce the text. When the device detected an attachment file identifying character code in the text being displayed and vocally reproduced, it converts the text sentence into voice synthesis/reproduction command data and sends the command data to the control portion to vocally reproduce the text. On completion of reproduction of the textual portion of the electronic mail, the device automatically reproduces music data of the attached music file and image data of the attached image file referring to the corresponding files by detected reference codes.

Metadata

Assignee

Sharp Kabushiki Kaisha

Inventor

Kiyoko Hayashi

Application Information

Application Number:US 09/800,480

Filing Date:2001-03-08

Priority Date:2000-06-15

Art Unit:7

Classifications

IPC:

G10L 1300

Field of Search:

704208704275704251704258704278704770709206709260707101

Patent Drawings (10 sheets)

Description

Background of the Invention

[0002] The present invention relates to an electronic mail system and, more specifically, to an electronic mail device having vocally reproducing function, which is capable of reading aloud text of a received electronic mail and automatically reproducing audio data or displaying visual data in files attached to the electronic mail.

[0003] In Japanese Patent Publication No.10-198615, the present applicant has disclosed a technique of reading aloud electronic mail text, which enables electronic mail devices to display an image and characters of a text on the same screen by inserting an icon of image information and an icon of a program for controlling the image display points (beginning and ending timings) in character strings of the electronic mail text. Unfortunately, this technique has to insert control information in every mail text, which information is, however, unnecessary for usual electronic mails and devices unable to use these control codes. In addition, the insertion of unnecessary codes makes the electronic mail text be unpleasant to see and difficult to read.

Summary of the Invention

[0004] Accordingly, a primary object of the present invention is to provide an electronic mail system that has insertion of only a character code referring to an attached music or image data file in electric mail text and enables people to enjoy vocally audio-visually reproducible electronic mail with easier operation and pleasant display, which is capable of vocally reproducing electronic mail text, detecting in the electronic mail text a character code referring to an attached thereto music or image data file and automatically reproducing (or display) the detected file content.

[0005] To realize the above object, the present invention, instead of an icon representing an image display program used in the prior art, inserts in a usual electronic mail text only a character code (character string) informing of a timing to refer and reproduce an attached file of music data or image data.

[0006] When the attached file reference character code is recognized in the process of reading aloud (by synthesized voice) the electronic mail text, the attached file data is automatically reproduced (or displayed). This can eliminate the use of unnecessary character codes, thereby making the text easier to read and reducing the quantity of data to be processed by the system.

[0007] In vocal reproduction of the electronic mail text being displayed on a display screen, the electronic mail device can reproduce music data and/or image data at timing designated by an electronic mail sender, enabling an electronic mail receiver to subsequently reproduce and display text, music or image data by a single operation. Namely, there is no need for the receiver to open each of attached files to be reproduced.

[0008] Since such music data and image data can be easily attached to an electronic mail in a well-known manner, such data may be easily transmitted and received and electronic mails may also be easily prepared.

[0009] Another object of the present invention is to provide an electronic mail system that enables electronic mail devices having function of vocally reproducing an electronic mail text to exchange electronic mails with an attachment file identifying character code (character string) inserted in a text of each electronic mail so that the devices may reproduce or display the data from the attached file designated by the character code in a given timing while vocally reproducing the electronic mail text.

[0010] A further object of the present invention is to provide an electronic mail device comprising a memory for storing an electronic mail data and attached file data, an input portion such as a keyboard for entering characters, a liquid crystal display portion for displaying characters (text) and images, a voice synthesizer control portion and a speaker and having a vocally reproducing function, which is capable of reproducing the text of a received electronic mail with an associated music data file in such a way that in process of reading aloud a text being displayed on a mail image screen it recognizes a reference character code (character string identifying the music file) inserted in the text and at this point reproduces the music data from the file identified by the character code.

[0011] Another object of the present invention is to provide an electronic mail device comprising a memory for storing an electronic mail data and attached file data, an input portion such as a keyboard for entering characters, a liquid crystal display portion for displaying characters and images, a voice synthesizer control portion and a speaker and having a vocally reproducing function, which is capable of reproducing a text of a received electronic mail with an associated image data file in such a way that, in process of reading aloud a text being displayed on a mail image screen, it recognizes a reference character code (character string for the image file) inserted in the text and at this point displays the image data from the file identified by the code.

[0012] A further object of the present invention is to provide an electronic mail device that, in the process of vocally reproducing an electronic mail content, can refer to an attached music file and reproduce music data therefrom and, after completing or stopping the reproduction of the file, can automatically return to reading the next phrase of the mail text and repeat the above cycle operation.

[0013] A further object of the present invention is to provide an electronic mail device that, in the process of vocally reproducing an electronic mail content, can refer to an attached image file and displays image data from there and, after completing or stopping the reproduction of the file, can automatically return to reading the next phrase of the mail text and repeat the above cycle operation.

Brief Description of the Drawings

[0014] FIG. 1 is a plan view of an electronic device in its open state, which relates to an embodiment of the present invention.

[0015] FIG. 2 is a block diagram of a hardware construction of an electronic device according to an embodiment of the present invention.

[0016] FIG. 3 is a flowchart depicting a procedure for preparing a text of an electronic mail by an electronic device according to an embodiment of the present invention.

[0017] FIG. 4 illustrates an exemplary layout of a text on a screen image for preparing an electronic mail text.

[0018] FIG. 5 illustrates an exemplary layout image of a menu (a list of selectable items).

[0019] FIG. 6 illustrates an image of an electronic mail text with an insertion of reference code (character string) of attached file as a text character string in a text image of FIG. 4.

[0020] FIG. 7 is a flowchart depicting a procedure for reproducing a content of a received electronic mail prepared and transmitted by an electronic mail device according to the present invention.

[0021] FIG. 8 is a flowchart depicting another procedure for reproducing a content of a received electronic mail prepared and transmitted by an electronic mail device according to the present invention.

[0022] FIG. 9 illustrates an exemplary data structure of music data registered in a music data portion 93 of a random access memory (RAM) 9.

[0023] FIG. 10 illustrates an exemplary data structure of image data registered in an image data portion 94 of the random access memory (RAM) 9.

[0024] FIG. 11 illustrates an exemplary data structure of an electronic mail text and a reference ID table for an attachment file.

[0025] FIG. 12 illustrates a practical example of an electronic mail text, an attachment file and a procedure for reproducing voice message of an electronic mail prepared by an electronic mail device according to the present invention.

Preferred Embodiment of the Invention

[0026] A preferred embodiment of the present invention will be described bellow with reference to accompanying drawings.

[0027] FIG. 1 is a plan view of an electronic device in its open state according to the present invention.

[0028] Device body 1 has a liquid crystal display portion 2 for displaying messages and images and an input portion 6 comprising a group of various operation keys 6.

[0029] The group of keys 6 includes a power supply key 61 for switching on and off a power source, a mail application key 62, a menu key 63, a voice reproduction start/stop key 64, an OK key 65, a cancel key 66, character input keys (a keyboard) 67 and so on.

[0030] The electronic mail application key 62 is used for driving an electronic mail application and the menu key 63 is used for displaying/switching off the menu (a list of available functions) on each image screen.

[0031] The OK key 65 and the cancel key 66 are used for executing (or recognizing) and canceling, respectively, during the data input operation or alert information on the display.

[0032] Characters can be entered into the device through input keys 67 such as a keyboard or a touch panel. The electronic device incorporates a CPU and memories (ROM and RAM) in its body.

[0033] FIG. 2 is a block diagram of a hardware structure of an electronic mail device according to the present invention. The components identical to those shown in FIG. 1 are respectively given the same numerals.

[0034] In the shown device, the CPU 4 controls the input/output activities of the device and performs all the arithmetic and logical operations. It includes a voice synthesizer control portion 41, a data control portion 42 and an arithmetic and logic portion 43.

[0035] The voice synthesis control portion 41 controls the synthesis and reproduction of voice signals. The synthesized/reproduced voice is output through the speaker 12 when the vocal reproduction start/stop key 64 was pressed.

[0036] The display portion 2 is, for example, a dot-matrix type liquid crystal display unit (hereinafter referred to as LCD), which displays on its screen a result of arithmetic and logic operation and a message etc. when display data is written in a display buffer 31 within the display control portion 3 (hereinafter referred to as LCD driver) based on a control signal from the CPU 4.

[0037] Data input by the input portion 6 is transferred via an input/output port 5 to the CPU 4 that in turn processes the input data and recognizes which function key was pushed.

[0038] The read-only memory ROM 8 has a program storage portion 82 for storing programs to realize various functions of the device body 1 and mail display facility according to the present invention and CG data storage portion 81 for storing a character font usable for displaying characters.

[0039] The random access memory RAM 9 consists of a buffer portion 91, a mail data portion 92 for storing mail data to be registered in the device body 1, a music data portion 93 for storing a music data file attached to the electronic mail, a image data portion 94 for storing an image data file attached to the electronic mail and a reference ID table 95 for attachment files. The memory stores a variety of data in the corresponding portions.

[0040] The data control portion 42 in the CPU 4 controls retrieval of music and image data, related to mail data in the electronic mail data portion 92, respectively, from the music data portion 93 and image data portion 94, referring to the reference ID table 95 for the attachment files in the RAM 9.

[0041] The RAM 9 is backed up by a battery 11 not to lose data contained therein in the event of power switched OFF.

[0042] A battery 11 is a power source of the device.

[0043] A communication unit 7 is used for transmitting and receiving data to and from other parties via a telephone network and/or by radio communication.

[0044] FIG. 3 is a flowchart depicting a procedure of processing for preparation of an electronic mail text by an electronic mail device according to the present invention.

[0045] Further, FIG. 3 illustrates the procedure of preparing an electronic mail text by selecting a file of music data or image data to be automatically reproduced (displayed) while vocally reproducing by voice the electronic mail text and inserting an attachment file identifying character code (character string) of the selected file in the electronic mail text.

[0046] The electronic mail application key 62 (see FIG. 1) of the input portion 6 is first pressed to activate the electronic mail application (Step S301) and presents an image for preparation of the electronic mail text (Step S302).

[0047] FIG. 4 shows an exemplary layout of an image for preparation of the electronic mail text.

[0048] In the state waiting for input data, the device examines whether a character was input (Step S303) or not. If so, the device displays the input character on an image of a text input editor screen 21 (see FIG. 4) (Step S308).

[0049] When a menu key 63 for displaying the menu was pressed rather than any character key in the waiting state of the device (Step S304), the menu (a list of available options) is displayed on the screen (Step S305).

[0050] FIG. 5 shows an exemplary layout of an image of a menu presenting a list of options selectable by using a suitable key of the input portion 6.

[0051] The user selects (sets) any one of music (or image) files from the menu for selection of attached files of music data (or image data) to be automatically reproduced (displayed) (Step S306), inserts a character code (a character string representing the selected attached file) in a mail text (Step S307), which text is presented on an image of the editor 21 as shown in FIG. 4 (Step S308).

[0052] FIG. 6 illustrates an image of the electronic mail with insertion of attachment file identifying character codes (character strings) selected by the user in a text image of FIG. 4.

[0053] In FIG. 6, each one of character codes (character strings) is indicated in square brackets.

[0054] Attachment file identifying character codes (character strings) are not limited to the shown examples and may also be any other character codes, pictorial symbols, icons and so on.

[0055] FIG. 7 is a flowchart depicting a procedure of vocally reproducing a received electronic mail by an electronic mail device according to the present invention.

[0056] Referring to FIG. 7, the procedure of automatically reproducing an electronic mail text with insertion of attachment file identifying character codes (character strings), prepared by the procedure of FIG. 3, is described below.

[0057] The electronic mail text to be reproduced by synthesized voice is first selected (Step S701) and the voice reproduction start key 64 (FIG. 1) is pressed (Step S702). Text data to be vocally reproduced by the voice synthesis control unit 41 (FIG. 2) or associated music data is stored in the buffer portion 91 of the RAM 9 (FIG. 2).

[0058] A voice synthesis/reproduction command data stored in the buffer portion 91 is transferred to the voice synthesizer control unit 41 to reproduce a speech of the text or music data through a speaker 12.

[0059] When an attachment file identifying character code (character string) inserted in the electronic mail text was detected in Step S703, the text sentence before the detected character code is converted to voice synthesis data and reproduction command data that are then transferred to the voice synthesis control unit 41 (Step S704) and reproduced by voice (Step S705).

[0060] On completion of voice reproduction of the text portion, the attachment music (or image) file designated by the detected attachment file identifying character code (character string) is reproduced (Step S706) and music data (or image data) is reproduced (displayed) (Step S705).

[0061] Again, upon completion of voice reproduction or image displaying for the attachment file, the next reference character codes (character strings) are detected to vocally reproduce the text characters or reproduce (display) for the attachment file.

[0062] FIG. 8 is a flowchart depicting another procedure of vocally reproducing a received electronic mail by an electronic mail device according to the present invention.

[0063] To vocally reproduce text data of an electronic mail according to the procedure of FIG. 8, the beginning of the electronic mail text is first designated by the user with a pointer (Step S801). The voice reproduction processing is started with the designated beginning of the text (Step S802) and the beginning part is examined whether or not it is a character code (character string) identifying an attachment file (Step S803).

[0064] If an attachment file identifying character code was detected, the attachment file is retrieved (Step S808) and music data (or image data) from the file is reproduced (or displayed) (Step S809). When the reproduction of the music data (or image data) was finished or stopped by interruption with the interrupt key (Step S810), the following text portion of the electronic mail is vocally reproduced (Step S804). If the following part is a character code (character string) identifying another attachment file, the user is requested to point the code position (Step S807).

[0065] If the character string was an ending code (not attachment file identifying code), the reproduction of the electronic mail text is finished (Step S806). If not, the device is put in the state waiting for another process (Step S811).

[0066] Thus, the process automatically returns to the vocal reproduction of a next phrase of the electronic mail text after reproducing music data or displaying image data and restarts reading aloud the text data until the next attached music or image data file is detected.

[0067] The structure of data to be used by the above procedure will be described below.

[0068] The electronic mail text prepared or received is stored in the electronic mail data portion 92 of the RAM 9 (FIG. 9). Music data and image data to be attached to the electronic mail text are separately (from the text data) stored together with respective ID numbers in the music data portion 93 and image data portion 94 of the RAM 9.

[0069] FIG. 9 illustrates an exemplary structure of music data registered in the music data portion 93 of the RAM 9, while FIG. 10 illustrates an exemplary structure of image data registered in the image data portion 94 of the RAM 9.

[0070] When an attached melody (or image) reference code was detected in a received mail text being read aloud, it must be decided which music data in the music data portion 93 of the RAM 9 or image data in the image data portion 94 of the RAM 9 is requested by that code. For this purpose, an ID table is prepared for respective one of mail data and stored/registered in an ID reference table 95 (FIG. 2) in the RAM 9.

[0071] The ID reference table 95 contains ID numbers of attached files in the order of precedence of their character codes (character strings) in the electronic mail text.

[0072] FIG. 11 illustrates an exemplary data structure of mail text data and an exemplary data structure of a reference ID table for attachment file.

[0073] When, the device, in the process of reading aloud a received electronic mail, detected an attachment file reference character code (character string) in the text as shown in FIG. 11 for example in Step S706 (FIG. 7) and Step S808 (FIG. 8), it can get a corresponding ID number from the ID table 95 for data files attached to the electronic mail, retrieves data registered in the music data portion 93 or the image data portion 94 of the RAM 9 and reproduce (display) the data from the identified attachment file.

[0074] The above ways enable the device to read aloud the electronic mail text, refer to the attached music file or image file and reproduce the music or image file.

[0075] FIG. 12 illustrates practical exemplified text and attached files of an electronic mail prepared by an electronic mail device and a process of reproducing by voice the electronic mail text with the associated attached files according to the present invention.

[0076] In the case of vocally reproducing the electronic mail text with attached files, the electronic mail text (first sentence) is read aloud before [melody 1] being an attachment file reference character code (character string) (Step S121). As soon as the reading aloud of the first sentence was completed or interrupted, the attached music file (melody 1) corresponding to the character code [melody 1] is retrieved with reference to an attachment file ID table (FIG. 11) and the content of the file is reproduced (Step S122).

[0077] When the music reproduction was finished or interrupted, the process returns to reading aloud the electronic mail text and a next sentence before [image 2] being a next attachment file reference character code (character string) is vocally reproduced (Step S123). When the vocal reproduction of the next sentence was finished or interrupted, an attachment file ID table (FIG. 11) is examined and an attached image file (image 2) corresponding to the character code [image 2] is accessed and reproduced (Step S124).

[0078] When the image reproduction was finished or interrupted, the process returns to vocal reproduction of the electronic mail text and a remaining part of the text before [melody 3] being a next attachment file reference character code (character string) is read aloud (Step S125). When the vocal reproduction of the next sentence was finished or interrupted, the attachment file reference ID table (FIG. 11) is examined and an attached music file (melody 3) corresponding to the character code [melody 3] is retrieved and reproduced (Step S126).

[0079] Thus, the vocal reproduction of the electronic mail text and accessing to and reproducing of the attached files can be performed.

[0080] As apparent from the foregoing, an electronic mail system according to the present invention enables a mail sender to prepare an electronic mail and attached data files, which text is not only displayed but also vocally reproducible and which files can be automatically reproduced at intended timing by inserting respective character codes (character strings) identifying respective files in designated positions in a mail text. Thus prepared electronic mail can express sender's massage with enhanced impression, pleasure and variety of music and images assembled in addition to vocal reproduction of the electronic mail text, which can not be achieved by the electronic mail text only.

[0081] Furthermore, the system according to the present invention requires insertion of only attachment file identifying character codes (character strings) in the form of usual textual components without using any unnecessary and strange internal code in the mail text, thereby giving no affection to usual electronic mail content.

[0082] An electronic mail system according to the present invention enables a mail receiver (or a mail sender) to view and hear with reality an electronic mail text displayed and vocally reproduced and associated music and/or image data which is reproduced at specified timing by detecting each attachment file identifying character code (character string) inserted in the electronic mail text and retrieving the designated file.

[0083] The receiver may feel more real the sender's message with enhanced impression, pleasure and variety that may be produced by automatically reproducing music and image data at timing designated by the sender, which effect cannot be achieved by the electronic mail text only.

[0084] The music and image data to be inserted in the electronic mail may be treated with as a usual attachment file that can be open and reproduced separately at any time at the receiver terminal.

[0085] An electronic mail system according to the present invention can automatically detect an attachment file identifying character code arranged in a received mail text being displayed and reproduced by voice and can reproduce designated music and image data from each detected attached file at timing specified by the sender and then repeat the above sequence after returning to vocal reproduction of the electronic mail text. This enables the receiver (or sender) to feel more real the sender's message with enhanced impression, pleasure or variety at timing specified by a sender. The above effects cannot be achieved by the electronic mail text only.

Claims

What is claimed is:

1. An electronic mail device for an electronic mail text with an attachment file attached thereto, comprising:

a memory for storing the electronic mail text and attachment file;

a display portion for displaying characters and images;

a speaker or said display portion for reproducing the attachment file; and

a character, a pictorial symbol or an icon for representing the attachment file, the character, the pictorial symbol or the icon being inserted in the electronic mail text to designate the timing when the attachment file should be reproduced, the inserted character, pictorial symbol or icon not representing a control program;

wherein when the electronic mail device recognizes the character, the pictorial symbol or the icon inserted in the electronic mail text is detected, the electronic mail device discriminates the kind of the attachment file in the memory and reproduces the attachment file according to the result of the discrimination.

2. The electronic mail device as defined in claim 1, wherein the character, the pictorial symbol or the icon is inserted in the electronic mail text during a preparation of the electronic mail text whereby a user selects the attachment file from a menu of available options.

3. The electronic mail device as defined in claim 2, wherein the character, the pictorial symbol or the icon is indicated in brackets.

4. An electronic mail device having function of vocal reproduction for an electronic mail text with a music data file attached thereto, comprising:

a memory for storing electronic mail text data and attachment file data;

an input portion;

a liquid crystal display portion for displaying characters and images;

a voice synthesis control portion for controlling synthesis of voice; and

a speaker;

wherein if the mail device recognizes in vocal reproduction of the electronic mail text that a character, a pictorial symbol or an icon designating the music data file is inserted in the electronic mail text, the electronic mail device reproduces the music data in the music data file in the memory, wherein the inserted character, pictorial symbol or icon does not represent a control program.

5. An electronic mail device as defined in claim 4, wherein if the reproduction of the attached music data file is finished or interrupted, the electronic mail device returns to vocal reproduction of the electronic mail text and repeats vocal reproduction of the text data and reproduction of music data of a next attachment file.

6. An electronic mail device having function of vocal reproduction for an electronic mail text with an image data file attached thereto, comprising: