Storage medium playback system and method

[0002] 1. Field of the Invention

[0003] The present invention relates to a storage medium having a plurality of storage areas distinguished by stored data such as a format, etc., and to a technique for playing back data stored in the storage medium.

[0004] 2. Description of the Related Art

[0005] In recent years, remarkable progress in information compression technology and in semiconductor technology has been made. For example, the international standards such as MPEG 1 and 2 have been made, so that standards for storage media for storing information are frequently revised. Recently, the standardization of DVDs (Digital Video Discs) has been completed. For CDs (Compact Discs), multimedia-conscious standards are frequently proposed. A CD which is conscious of multimedia is called being an enhanced CD.

[0006] The enhanced CD includes two storage areas distinguished by data format. One area is for storing the same CD-DA (Digital Audio) data as that of a conventional music CD, while the other is an area for storing CD-ROM data. Image data (including data encoded with the MPEG encoding method) such as a promotional video, record jacket, etc., can be stored as the CD-ROM data. Therefore, the enhanced CD attracts public attention as a tool for providing entertainment over and above the conventional music CD.

[0007] As is known, enhanced CDs are classified into three major types depending on their format. The first of these types is called mixed mode, the second is called multi-session, and the third is called CD-ROM ready. The multi-session type can store CD-DA data in tracks 1 through 99 in a similar manner as in a music CD, unlike the other types. It allows a conventional music CD player to play back music. This is the reason why the multi-session type is of the greatest interest to the general public. The multi-session type is normally called a CD-EXTRA.

[0008] In the meantime, the number of personal computers (hereinafter abbreviated to PCs) has been remarkably increasing in recent years, and PCs have been equipped with a CD-ROM device as standard. The PCs equipped with the CD-ROM device can only use a CD-EXTRA (enhanced CD) by adding the MPEG decoding capability. The number of PCs equipped with the MPEG decoding capability as standard has also been growing in recent years. Therefore, the number of CD-EXTRAs (enhanced CDs) sold for PCs is expected to significantly increase.

[0009] Because PCs are equipped with a CD-ROM device as standard, a device for accessing a DVD is expected to be included in PCs, by adding a capability for accessing a DVD in addition to a capability for playing back a CD-ROM (including an enhanced CD), etc.

[0010] The enhanced CD such as the above described CD-EXTRA, etc. allows high quality sound and a variety of videos to be enjoyed in a similar manner as with a music CD. Conventionally, sound and video are played back by basically reproducing data sequentially read from a CD. In other words, data of various representation media such as video (still pictures, moving pictures, photos, etc.), music, etc. can be recorded, but those data are not fully utilized. The same thing is said about storage medium other than the enhanced CD.

[0011] The object of the present invention is to allow data stored in a storage medium to be fully utilized, and to be reproduced in various forms.

[0012] A storage medium according to the present invention includes first and second storage areas distinguished by stored data, and stores reproduction management information for synchronizing and reproducing data respectively stored in the first and second storage areas, in one of the first and second storage areas, and a program storing a capability for reproducing data stored in one of the first and second storage areas in synchronization with the data stored in the other of the first and second storage areas, in another storage medium, in one of the first and second storage areas.

[0013] It is desirable that the above described program should include the capability for storing the data stored, from the storage medium including the first and second storage areas distinguished by stored data, in another storage medium. However, since this capability may sometimes be implemented by another program (such as an operating system, etc.), it is not essential.

[0014] A storage medium playback system according to the present invention comprises a first reading unit for reading data stored in one of the first and second storage areas from a storage medium including the first and second storage areas distinguished by stored data, and storing the data in another storage medium; a first reproducing unit for reproducing data which is stored in the other of the first and second storage areas read by the first reading unit; a second reading unit for reading data stored in another storage medium; a second reproducing unit for reproducing the data read by the second reading unit; and a controlling unit for synchronizing the reproduction of the data by the first reproducing unit with the reproduction of the data by the second reproducing unit, based on reproduction management information, which is prepared in advance, for synchronizing and reproducing the data respectively stored in the first and second storage areas.

[0015] The data respectively stored in the first and second storage areas can simultaneously be reproduced in parallel by storing the data stored in one of the first and second storage areas in another storage medium. The controlling unit synchronizes and reproduces those data based on the reproduction management information. With these operations, the data respectively stored in the first and second storage areas can arbitrarily be combined, and the proceeding status of the reproduction of combined data can be managed.

[0016] To synchronize and reproduce a plurality of data such as CD-DA data, there arises the demand for obtaining a reproduction point in a data stream (including a relationship of relative reproduction points in combined data in this case), depending on the data. The above described reproduction management information is intended to obtain the reproduction point. With the reproduction management information, the reproduction point in a data stream can be obtained. Therefore, reproduction can be started from an arbitrary point in the data, or data to be reproduced can be switched depending on a reproduction point. In other words, the number of options for data reproduction is increased, and data can be reproduced in a variety of forms.

[0017] The above described feature can also be applied to the storage medium playback method, according to the present invention.

[0018] FIG. 1 is a block diagram showing the system configuration of a first embodiment;

[0019] FIG. 2 is a block diagram showing the system configuration of a second embodiment;

[0020] FIG. 3 is a schematic diagram showing the concept of the fundamental operations of data reproduction according to the second embodiment;

[0021] FIG. 4 is a schematic diagram explaining the structure of data included in a CD-EXTRA;

[0022] FIG. 5 is a schematic diagram explaining the structure of data loaded into a PC;

[0023] FIGS. 6A and 6B exemplify the contents stored in a first session in the CD-EXTRA;

[0024] FIG. 7 exemplifies extracted data corresponding to one phrase;

[0025] FIG. 8 is a schematic diagram showing the structure of a display screen according to the second embodiment;

[0026] FIGS. 9A and 9B are schematic diagrams explaining a method for specifying playback contents of a performance, and a display state of words;

[0027] FIG. 10 exemplifies a displayed word area;

[0028] FIG. 11 is a schematic diagram explaining a flow of playing back the CD-EXTRA according to a user operation;

[0029] FIG. 12 is a flowchart showing a CD auto-play process performed by an operating system;

[0030] FIG. 13 is a flowchart showing a reproduction point change process;

[0031] FIG. 14 shows another playback example of a CD-EXTRA (No.1);

[0032] FIG. 15 shows a further playback example of the CD-EXTRA (No.2);

[0033] FIG. 16 is a flowchart showing a playback process of associated data;

[0034] FIG. 17 shows a further playback example of the CD-EXTRA (No.3);

[0035] FIG. 18 is a schematic diagram explaining a modification of the method for playing back a CD-EXTRA; and

[0036] FIG. 19 is a simplified block diagram showing the system configuration of a third embodiment.

[0037] Provided below are the explanations about the details of the embodiments according to the present invention, by referring to the drawings.

[0038] <First Embodiment>

[0039] FIG. 1 is a block diagram showing the configuration of a storage medium playback system (hereinafter referred to as a sound playback/display system) according to the first embodiment of the present invention. The explanation about the system configuration of the first embodiment is provided below by referring to FIG. 1.

[0040] A storage medium 1 includes first and second storage areas 1aand 1brespectively, which are distinguished by stored data. The distinction by stored data is made by the difference between data reproduction forms such as the difference between sound and video, the difference between representation media such as text, sound, moving pictures, still pictures, etc., or the difference between data formats. As the storage medium 1, an enhanced CD (such as a CD-EXTRA), DVD, optical disc, magneto-optical disc, etc. are available.

[0041] A first reading unit 2 reads data stored in the second storage area 1bof the storage medium 1, and stores the read data in a storage medium 3. The storage medium 3 may store all of data stored in the second storage area 1b, or data extracted from among the data stored in the second storage area 1b. Assuming that the system shown in FIG. 1 is configured by using a PC, a main storage device or an auxiliary storage device (hard disk, etc.) correspond to the storage medium 3.

[0042] The first reading unit 2 further reads data stored in the first storage area 1aof the storage medium 1, and outputs the data to a first reproducing unit 4. The first reproducing unit 4 reproduces the data received from the first reading unit 2.

[0043] A second reading unit 5 reads data stored in the storage medium 3 by the first reading unit 2, and outputs the read data to a second reproducing unit 6. The second reproducing unit 6 reproduces the data received from the second reading unit 5.

[0044] The data respectively stored in the first and second storage areas 1aand 1bcan be reproduced simultaneously (in parallel), by storing the data included in the second storage area 1bof the storage medium 1 in the storage medium 3. In other words, a combination of data of various representation media stored in the storage medium 1, and a reproduction timing of those data, can arbitrarily be selected and controlled. In this way, a wider variety of user entertainment options and use forms are realized. As a result, the entertainment value of the storage medium 1 can be increased.

[0045] A controlling unit 7 synchronizes the reproduction of data that the first reading unit 2 reads from the first storage area 1aof the storage medium 1, using the first reproducing unit 4, with the reproduction of data that the second reading unit 5 reads from the storage medium 3 (data stored in the second storage area 1bof the storage medium 1), using the second reproducing unit 6.

[0046] If there is a correspondence between the data to be reproduced by the first and second reproducing unit 4 and 6, for example, between music and a video corresponding to that music, the controlling unit 7 synchronizes and reproduces those data based on the prepared reproduction management information. The reproduction management information is information for defining a correspondence between associated data for each reproduction unit of data (such as one phrase of music, one frame of a still picture, etc.). With such reproduction management information, specification of an arbitrary reproduction point (based on a reproduction start time, reproduction end time, etc.) such as an intermediate point of a musical piece, can be made, and the reproduction of the different data at the specified reproduction point can be synchronized.

[0047] According to this embodiment, the controlling unit 7 obtains the reproduction management information from the storage medium 3 by preparing the reproduction management information stored in the second storage area 1bof the storage medium 1, and storing it in the storage medium 3 together with the data stored in the second storage area 1b. This process eliminates the need for preparing a further storage medium to store the reproduction management information. Additionally, preparation of only the storage medium 1 enables data stored in different storage areas to be synchronized. This leads to advantages in cost, operability, etc.

[0048] PCs currently on the market are equipped with all of the components corresponding to the above described devices and storage medium 3 in terms of hardware, or can easily be equipped with all of them, even if they are currently equipped with none. Since most PCs are now being shipped as multimedia PCs, they are equipped with all of the capabilities required for configuring a sound playback/display system which implements the present invention.

[0049] Accordingly, a program including the capabilities for implementing the above described operations of each of the devices may be stored in the storage medium 1, and a user may purchase the storage medium 1. If such a program is stored in the storage medium 1, a user having a personal computer can configure the sound playback/display system only by purchasing that storage medium 1. Such a user can easily configure the system, at less cost to the user. As a result, the user can get benefits in cost, operability, etc.

[0050] <Second Embodiment>

[0051] FIG. 2 is a block diagram showing the configuration of a sound playback/display system according to the second embodiment.

[0052] In FIG. 2, 101 is a PC (main body) equipped with multimedia capabilities. The sound playback/display system is configured by using the PC 101. As shown in FIG. 2, the PC 101 comprises a CRT 102 as a display device, CD-ROM device 104, input device (such as a keyboard, pointing device, etc.) 105, auxiliary storage device 106, and a sound output device (such as a speaker) 107. An enhanced CD and a music CD in addition to a CD-ROM are available for the CD-ROM device 104. A storage medium according to the second embodiment is a CD-EXTRA 103 which is one type of enhanced CD, and one type that the present invention is applied to.

[0053] FIG. 3 is a schematic diagram showing the concept of the fundamental operations for reproducing data according to the second embodiment. Prior to the detailed explanation about FIG. 2, the explanation about the fundamental operations for reproducing the contents and data stored in the CD-EXTRA 103 according to the second embodiment, is provided by referring to FIGS. 3 through 7.

[0054] The program area of the CD-EXTRA (storage medium) 103 is partitioned into two areas, that is, a first session 103a, and a second session 103b, according to the format of stored data. CD-DA tracks are arranged in the first session 103a, while data tracks are arranged in the second session 103b. Additionally, the innermost track (read-in area) of the CD-EXTRA 103 stores TOC (Table of Contents) information indicating the start address etc. of each of the tracks, which is not shown in the figure, in a similar manner as in a music CD.

[0055] FIG. 4 shows the data structure of the CD-EXTRA 103.

[0056] As shown in FIG. 4, the first session 103astores data (music data) of CD-DA tracks, while the second session 103bstores data in the same format as that of a CD-ROM. The second session 103bcan also store MPEG data (audio/video data). These sessions are delimited by read-in and read-out markings. These markings enable a CD player for normal music CDs to play back the data stored in the first session 103a. Hereinafter, the data stored in the first and second sessions 103aand 103bare respectively referred to as CD-DA data and CD-ROM data.

[0057] The data stored in the CD-EXTRA 103 are managed as files classified in a hierarchical structure, similar to a CD-ROM. An OS (operating system) 111 shown in FIG. 2 installed in the PC 101 is assumed to comprise a capability (auto-play capability) for determining whether or not a file having a predetermined name (such as an autorun.inf file) is included when a CD is loaded into the CD-ROM device 104, and performing a process according to the described contents if the file is included.

[0058] Accordingly, the file having the predetermined name, which describes the contents to be executed by the OS 111 is stored in the second session 103bof the CD-EXTRA 103. The second session 103bstores also viewer software 110 which is the application software for reproducing (displaying) data stored in the second session 103bin synchronization with the reproduction of the CD-DA data stored in the first session 103a.

[0059] The file having the predetermined name includes the description for loading (copying) the viewer software 110, specified data in the second session 103b, for example, the information about a disc (album), text data such as the title or/and words of the musical piece, and picture data (MPEG encoded data (including video or/and audio data)), into the main storage device 201 (part of the above described data is also loaded into the auxiliary storage device 106 depending on the capacity or use state of the main storage device 201) in the PC 101, so that the viewer software 110 is invoked. Accordingly, the contents of the description are implemented by the OS 111. Note that the data loaded into the PC 101 is hereinafter referred to as load data 202, in order to make a distinction from other data.

[0060] By loading the data stored in the second session 103binto the PC 101, the data stored in the different areas, such as the first and second sessions 103aand 103b, can be simultaneously processed, (reproduced), and arbitrarily combined, and the timing for reproducing those data can arbitrarily be controlled. Furthermore, since the viewer software 110 is stored in the CD-EXTRA 103, a user having the required items of hardware can implement the sound playback/display system according to the second embodiment, only by purchasing the CD-EXTRA 103. This reduces the cost to the user.

[0061] According to the second embodiment, the OS 111 copies the load data 202. However, the viewer software 110 may include the capability for determining whether or not the load data 202 is copied to the main storage device 201, and for copying it to the main storage device 201 if it has not yet been loaded, for an OS which does not include the above described auto-play capability. In this way, the load data 202 is copied to the main storage device 201 regardless of the existence/non-existence of the auto-play capability. That is, it improves the operability.

[0062] After the viewer software 110 is loaded into the main storage device 201 according to the description of the above described file, the OS 111 instructs the CPU to invoke the viewer software 110 as shown in FIG. 3. Once the viewer software 110 is invoked, moving pictures, still pictures, and sound depending on a user instruction are reproduced according to a predetermined reproduction procedure. The TOC information stored in a read-in area in the CD-EXTRA 103 is stored in the main storage device 201, and used to identify or specify a reproduction point of the CD-EXTRA 103.

[0063] When a user instructs the reproduction of the data (CD-DA data) stored in the first session 103a, the viewer software 110 starts reproducing the CD-DA data that the user desires by determining the reproduction start point in the first session 103aaccording to the contents of the instruction, and instructing the CD-ROM device 104 of the reproduction start point.

[0064] In the meantime, the viewer software 110 retrieves associated information (data) to be reproduced together with the CD-DA data, from the load data 202, and displays the retrieved information on a screen.

[0065] FIG. 5 is a schematic diagram showing the structure of data loaded from the second session 103bin the CD-EXTRA 103 to the PC 101.

[0066] As shown in this figure, time information (reproduction management information), words, pictures, additional information, and other data are loaded into the PC 101 as the load data 202. The time information among these information is intended to synchronize CD-DA data stored in the first session 103awith the corresponding (associated) data, and reproduce them simultaneously.

[0067] As is known, the time unit “MSF” (M: minute, S:second (00-59), F:frame (00-74)) is used td indicate the position of the CD-DA data stored in a track of a musical CD. Since a normal music CD player can play back the CD-DA data in the CD-EXTRA 103, that time unit is employed for the first session 103ain which the CD-DA data is stored. Accordingly, this time unit is employed for representing the time information. Other units, such as a number of sectors, etc., may also be employed as the time unit.

[0068] In the meantime, the reproduction unit for music is employed because “MSF” is employed for representing the time information. Specifically, a phrase is employed as the reproduction unit. The reproduction point of a phrase is managed using reproduction start and end times of each phrase, as shown in FIG. 5. In this way, the reproduction point of a musical piece can be specified in phrases.

[0069] Data to be reproduced in synchronization with the CD-DA data, such as words, pictures, etc., are divided into phrases, and their correspondences are defined. The definition of a correspondence is made by putting the time information into array variables, and putting the data such as words, pictures, etc., or corresponding storage information (file name, etc.) about the data into array variables. With this process, the reproduction of the data to be reproduced in synchronization with the CD-DA data, can be managed in phrases.

[0070] The correspondence between a musical piece and data to be reproduced in synchronization with the musical piece can be identified, for example, by preparing a file defining the correspondence between them.

[0071] FIG. 6 (FIGS. 6A and 6B) exemplify the contents stored in the first session 103ain the CD-EXTRA 103, and FIG. 7 exemplifies extracted data corresponding to one phrase. Provided below is the specific explanation about the correspondence between data, which is defined as described above, by referring to these figures.

[0072] In FIGS. 6A and 6B, numerals (1) through (4) indicate track numbers, and tracks storing music data for a karaoke version are shown shaded. That is, FIG. 6A shows the first session 103ain which each of the tracks numbered (1) and (2) stores each of original versions of musical pieces, and the track numbered (3) stores the karaoke version of the musical piece stored in the track numbered (1). FIG. 6B shows the first session 103ain which each of the tracks numbered (1) and (2) stores each of original versions of musical pieces, and each of the tracks numbered (3) and (4) stores each of the karaoke versions of the musical pieces stored in the tracks numbered (1) and (2).

[0073] Here, karaoke is a performance which is a musical accompaniment of a popular song. Additionally, karaoke means singing a song to the tune of the music accompaniment.

[0074] The reason why the karaoke versions are stored in addition to the original versions of musical pieces as shown in FIGS. 6A and 6B is that karaoke has become a popular form of entertainment. FIG. 7 specifically exhibits part of the data (text data such as words, translated words, and pronunciation) extracted from the load data 202 with a user's specification of a certain phrase in a musical piece stored in the tracks numbered (1) and (3), on the condition that the contents shown in FIG. 6A or 6B are stored in the first session 103a.

[0075] In FIG. 7, numerals (1) through (4) indicate time information (refer to FIG. 5). As shown in this figure, when different performance types of an identical musical piece are stored in a plurality of tracks, the time information in each track is linked. This is because data synchronized with the reproduction of CD-DA data is made common to the original and karaoke versions, and a user can select to reproduce either of the original and karaoke versions.

[0076] The four numerical values of the time information (1) through (4), which are represented and delimited by a comma “,”, and colons “:”, respectively indicate a track number, minutes (M), seconds (S), and a frame number (F) from the left side of the drawing. Accordingly, the time for performing the phrases shown in FIG. 7 is 5(=25−20) seconds 26(=70−44) frames.

[0077] As described above, the correspondence between the data stored in the first and second sessions 103aand 103bis defined in the reproduction units (phrases according to this embodiment). Such a definition allows those data to be synchronized in reproduction units and reproduced, and a reproduction point to be specified in the reproduction units.

[0078] Once the CD-ROM device 104 starts reading and reproducing the CD-DA data, the viewer software 110 sequentially obtains the reproduction information (information indicating a current reproduction point) from the data received from a controller 109 in the CD-ROM device 104, monitor its progress, identifies data to be displayed next by comparing the reproduction information with the time information, obtains the identified data from the load data 202, and displays it, as shown in FIG. 3. With this process, the CD-DA data and its associated data in the load data 202 are synchronized and reproduced.

[0079] Referring to FIG. 2.

[0080] If a CD-EXTRA 103 is loaded into the CD-ROM device 104, for example, the controller 109 detects it. The controller 109 then makes a read head 108 perform seek operations in order to read TOC information stored in a read-in area in the CD-EXTRA 103, and makes the read head 108 perform seek operations outside the read-in area in order to identify the type of the CD (whether or not it is an enhanced CD) loaded into the CD-ROM device 104 based on the markings (refer to FIG. 4). The controller 109 notifies the OS 111 via driver software 112 that a CD is loaded into the CD-ROM device 104, and at the same time, it passes the TOC information and the information indicating the type of the CD, to the OS 111.

[0081] The data stored in the CD is modulated, for example, with the modulation method, called “EFM” (Eight to Fourteen Modulation). After the controller 109 demodulates the data read by the read head 108 from the CD, it passes the demodulated data to the OS 111 via the driver software 112.

[0082] When receiving the above described information via the driver software 112, the OS 111 reads the data stored in the CD loaded into the CD-ROM device 104 based on the TOC information, examines the contents stored in the CD, and identifies the type of the CD. If the CD is not determined to be a music CD, the OS 111 determines whether or not a file having a predetermined name is included in the files stored in the CD. If the file having the predetermined name is stored, the OS 111 performs a process according to the contents described in that file. With this process, the viewer software 110 and load data 202 specified in the second session 103bare loaded into the main storage device 201, and the viewer software 110 is invoked as described above.

[0083] If there is not enough space for storing the data in the main storage device 201, the OS 111 stores the excess data in the auxiliary storage device 106 via the driver software 114.

[0084] After the viewer software 110 is invoked, it stores the TOC information in the read-in area of the CD-EXTRA 103 in the main storage device 201. Then, it reproduces the data according to a predetermined reproduction procedure. For example, the viewer software 110 displays an explanation about the contents (title, and information about an artist (photo, name, etc.), information about a CD maker, etc.) stored in the CD-EXTRA 103 on the CRT 102 as still pictures or moving pictures, immediately after it is invoked, and then displays pictures on a display screen of CRT 102 having a predetermined structure. After that, the viewer software 110 identifies the data to be reproduced according to a user instruction, and displays the data to be displayed on a screen having a predetermined structure.

[0085] FIG. 8 shows a structure of the display screen.

[0086] As shown in FIG. 8, a display screen 800 is composed of three areas. That is, a command area 801 positioned at the top of the display screen 800, a selection list area 802 positioned on the left of the display screen 800, and an information area 803 occupying the remaining area. Each of the areas 801 through 803 displays the following.

[0087] The command area 801 displays the title of the CD-EXTRA 103, the title of a musical piece or a video being reproduced, and each of different types of buttons (not shown in this figure) for specifying the reproduction of the CD-EXTRA 103 by a user. The buttons relating to the reproduction operations such as a playback, stop, fast forward, rewind, repeat, etc., and the buttons for setting various modes are displayed. Also an EXIT button for instructing the termination of the viewer software 110 is displayed.

[0088] The selection list area 802 is a selection list box for displaying the list of musical pieces, videos, etc. stored in the CD-EXTRA 103. A user moves a mouse cursor, for example, to the title of a displayed musical piece, and clicks the mouse, so that he or she can specify that musical piece.

[0089] The information area 803 is an area for displaying general-purpose information including text information such as words and translated words of the musical piece whose reproduction is currently specified, and reproduced pictures (still pictures or moving pictures). In the information area 803, for example, the words are displayed in the form shown in FIG. 9. “M” shown in FIG. 9 is a mouse cursor.

[0090] The user moves the mouse cursor “M”, and clicks using the input device 105 (mouse, etc.). The contents of the user instruction operation performed with the input device 105 is transmitted to the OS 111 via the driver software 115. The OS 111 analyzes the contents of the instruction operation, performs a process according to a user instruction (move operation of the mouse cursor “M”, etc.), and passes the result of the analysis to the viewer software 110 depending on need. With this process, the viewer software 110 reproduces the data that the user desires by changing the mode setting, switching the data to be reproduced, etc.

[0091] The input device 105 that the user operates the most often is a pointing device. Since a mouse is the most frequently used pointing device, the input device 105 is hereinafter referred to as the mouse.

[0092] If the user clicks the title of a video displayed in the selection list area 802, and instructs the playback of the video (MPEG audio/video data), the viewer software 110 reproduces it according to the following procedure.

[0093] The viewer software 110 identifies the name of the file storing a corresponding video according to the title that the user clicked. After identifying the file name, the viewer software 110 references the file management information stored in a predetermined area in the second session 103b, and identifies its reproduction start point (normally the beginning of the file). Then, the viewer software 110 passes the identified reproduction start point to the CD-ROM device 104 as a control command via the OS 111 and driver software 112. The controller 109 which receives the control command makes the read head 108 perform seek operations, and starts reading data.

[0094] After the controller 109 demodulates the data read by the read head 108, the demodulated data is transmitted to the viewer software 110 via the driver software 112 and OS 111. The viewer software 110 transmits the data transmitted from the controller 109 to the decoder 116 via the OS 111 and driver software 113. The data transmitted to the decoder 116 at this time is an MPEG bit stream in which audio and video data encoded with the MPEG 1 encoding method are interleaved.

[0095] The decoder 116 is an MPEG board equipped with, for example, a D/A conversion capability (DA (Digital to Analog) converter) for an output of an analog audio signal, and an overlay capability for synthesizing a graphic screen from the main body of the PC 101 with decoded pictures. The decoder 116 separates the MPEG bit stream into audio and video streams, and decodes the respective streams. Additionally, the decoder 116 performs the D/A conversion of the decoded data (CD-DA data), and outputs analog audio signals.

[0096] The decoded audio data is output to a sound output device 107 via the DA converter. The sound output device 107 is, for example, a speaker or headphones, and sound is output according to an audio signal input from the decoder 116. The decoded picture data is combined with the graphic screen data generated by the viewer software 110 and OS 111. The combined data is then displayed on the CRT 102.

[0097] As is generally known, there is a difference in a time required for the decoding to the reproduction process between MPEG audio data and MPEG video data. The decoder 116 eliminates this difference using a buffer, not shown in this figure, in order to synchronize the sound and pictures.

[0098] If a user clicks the title of a musical piece displayed in the selection list area 802, and instructs the playback of the musical piece, the viewer software 110 reproduces it according to the following procedure.

[0099] First of all, the viewer software 110 obtains the number of the track (CD-DA track) storing the musical piece having the clicked title, and the data to be reproduced in synchronization with that musical piece, from the load data 202 based on the clicked title. The data to be obtained is identified, for example, by referencing the file defining the data synchronized with each musical piece. To facilitate understanding, the following explanation is provided based on the assumption that the data synchronized with the playback of the musical piece is its words.

[0100] After obtaining the word data from the load data 202, the viewer software 110 divides the word data into, for example, phrases, and displays the divided phrases in the information area 803, as shown in FIG. 9. Additionally, the viewer software 110 references the TOC information, transmits the start point of the track identified according to the title of the musical piece, to the controller 109 in the CD-ROM device 104, and makes the controller 109 start reading the data stored in that track. As a result, the CD-DA data is transmitted from the controller 109 to the viewer software 110 via the driver software 112 and OS 111.

[0101] The viewer software 110 transmits the CD-DA data to the decoder 116 via the OS 111 and driver software 113 to play back the musical piece. Furthermore, the viewer software 110 monitors the proceeding status of the playback by obtaining the current reproduction point, for example, from the data of a Q channel in a subcode included in the CD-DA data, and notifies the user of the current reproduction point by highlighting the phrase currently being reproduced in synchronization with the progress of the musical piece.

[0102] As shown in FIG. 5, the time information is data including the reproduction point in phrases. Therefore, after the current reproduction point is obtained from the CD-DA data, the associated data is displayed on the CRT 102 in synchronization with the reproduction of the CD-DA data (music) in phrases.

[0103] According to the second embodiment, the user can specify the reproduction point in phrases, since the reproduction point is defined as the time information in phrases. If the original and karaoke versions of an identical musical piece are stored in the first session 103aas shown in FIG. 6, also the specification of which of the versions to play back can be made. Provided below is the explanation about how to make this specification, by referring to FIGS. 9A and 9B.

[0104] In FIGS. 9A and 9B, 901 is a word area in which words of one phrase are displayed; (1) through (6) indicate the playback order of the phrases displayed in the word areas 901, for example, in number from the beginning of the musical piece. The word area 901 currently being played back is shaded. An eighth note “K” displayed at the beginning of each of the word areas 901 is a karaoke button for specifying the performance of the karaoke version.

[0105] FIG. 10 exemplifies a display of the word area 901. This example displays words, and corresponding pronunciations and translated words as shown in FIG. 10. The detailed explanation is omitted here, but characters to be displayed in the word area 901 can be selected from the words, pronunciations, and translated words.

[0106] If a user specifies a musical piece, that is, if the user clicks one of the titles of musical pieces displayed in the selection list area 802, the words of that musical piece are displayed in the information area 803 as shown in FIG. 9A, and the playback is started from the word area 901 (the first phrase of the musical piece) numbered (1). The musical piece to be played back at that time is the original version.

[0107] The reproduction point is specified by clicking the word area 901 in which the phrases desired to be played back exist, that is, by operating the mouse 105 to move the mouse cursor “M” to that word area 901, and pressing the left button of the mouse 105. The content of the playback is specified by whether or not the mouse cursor “M” is positioned on the Karaoke button “K” at the time of the above described clicking operation. If the original and karaoke versions of an identical musical piece are stored in the first session 103a, the time information are linked between them, as shown in FIG. 7. The viewer software 110 identifies the phrase and the track of the original or karaoke version that the user specifies, according to the position at which the mouse cursor “M” is displayed at the time of the clicking operation, and instructs the controller 109 in the CD-ROM device 104 of the position at which the data read is to be started.

[0108] As described above, the reproduction point in phrases, and the contents of the reproduction, are changed according to a user instruction. In the example shown in FIG. 9B, the reproduction point of a phrase is switched from the word area 901 numbered (1) to the word area 901 numbered (5), in which the mouse cursor “M” is displayed, and the playback of the phrase is started in the original version. Once the playback of the phrase is started, the playback of the data is continued according to a predetermined playback procedure, for as long as the user does not give another instruction. According to the second embodiment, the phrases succeeding that phrase are sequentially played back.

[0109] As described above, the reproduction point in a musical piece is specified by the words of the phrases displayed in the word area 901. Therefore, the user can easily specify the reproduction point that he or she desires. As a result, the user load for the change operation of the reproduction point is alleviated.

[0110] FIG. 11 is a flowchart showing the process for reproducing the data of the CD-EXTRA 103 according to a user operation. This figure, which is intended to exhibit the reproduction of data stored in the first session 103a, shows the extracted flow of data (limited to a karaoke version) switched according to a user operation and reproduced from when the user loads the CD-EXTRA 103 into the CD-ROM device 104 till when the viewer software 110 invoked by the loading operation is terminated. The operations of the above described viewer software 110 are explained by referring to FIG. 11.

[0111] When the user loads the CD-EXTRA 103 into the CD-ROM device 104 (step S1), this is notified from the controller 109 in the CD-ROM device 104 to the OS 111 via the driver software 112.

[0112] The OS 111 receives the notification, identifies the type of the CD loaded into the CD-ROM device 104, and checks to see if a file having a predetermined name is stored. If the user loads the CD-EXTRA 103 into the CD-ROM device 104, the CD-EXTRA 103 includes the file having the predetermined name. Accordingly, the OS 111 starts the process according to the description of that file (step S2). The load data 202 specified in the viewer software 110 and second session 103bis copied from the CD-EXTRA 103 to the main storage device 201 (step S3). The viewer software 110 is then invoked (step S4).

[0113] The viewer software 110 asks the user whether or not the succeeding operations are permitted immediately after the invocation, and starts reproduction if the user gives permission. If the user does not give permission, the flow shown in FIG. 11 is stopped at step S4.

[0114] If the user gives permission, the viewer software 110 starts reproducing the load data 202 copied to the main storage device 201, and the data stored in the first session 103ain the CD-EXTRA 103. For example, after the viewer software 110 displays the information about the CD-EXTRA 103 and the information about the contents stored in the CD-EXTRA 103, and reproduces a short introduction video, it displays the display screen 800 shown in FIG. 8 on the CRT, 102, and the words of the first musical piece stored in the first session 103a, in the information area 803 (step S5). Then, the viewer software 110 starts the reproduction of the CD-DA data of that musical piece (step S6), and enters the state in which a user operation is accepted (step S7).

[0115] Steps S8 through S29 succeeding step S7 show the operations to be performed according to the contents of the reproduction of data that the user desires, and the switching of the reproduction of the data made by the operations.

[0116] The user clicks the title of his or her desired musical piece among the titles of the musical pieces displayed in the selection list area 802, when he or she desires to change a musical piece to be played back (steps S7→S8→S9). After the title of the desired musical piece is clicked, the viewer software 110 obtains the number of the track storing the CD-DA data of the musical piece, and the data to be displayed in synchronization with the reproduction of the musical piece from the load data 202 in the main storage device 201, starts playing that musical piece, and displays the obtained data from the load data 202 on the CRT 102 (steps S10 and S11). Then, the viewer software 110 waits for a user operation while reproducing the musical piece in synchronization with the reproduction of its associated data (step S11 to step S7).

[0117] The musical piece is played back by transmitting the CD-DA data received from the controller 109 via the OS 111 and the driver software 112, to the decoder 116 via OS 111 and driver software 113, after the reproduction start point of the CD-EXTRA 103 is transmitted to the controller 109 of the CD-ROM device 104. The playback of the data associated with the musical piece is synchronized with the playback of the musical piece, for example, by obtaining the current reproduction point according to data of the Q channel in a subcode included in the above described CD-DA data, and comparing the reproduction point with the time information.

[0118] If the user desires to change a reproduction point in a musical piece, that is, to fast-forward or rewind the musical piece currently being played back, he or she clicks the word area 901 displaying his or her desired phrase in the information area 803 (steps S7→S12→S13). When the word area 901 is clicked, the viewer software 110 obtains the reproduction start point of the phrase displayed in the word area 901, and its associated data to be reproduced, from the load data 202 in the main storage device 201, starts reproducing that phrase, and displays the data obtained from the load data 202 on the CRT 102 (steps S14 and S15). For example, the viewer software 110 displays highlighted characters indicating the reproduction of the phrase, or highlights such as a background (including a change of the background color) in the word area 901. Then, it enters the state for waiting for a user operation while synchronizing the reproduction of the musical piece with the reproduction of its associated data (step S15→S7).

[0119] If the user desires to play back the original version of a musical piece (including the switching from the karaoke version to the original version), he or she clicks the word area 901 displaying the desired phrase, other than the area of the karaoke button “K” (steps S7→S16→S17). When the word area 901 is thus clicked, the viewer software 110 obtains the reproduction start point of the phrase displayed in the word area 901 by referencing the time information, and its associated data to be reproduced for that phrase from the load data 202 in the main storage device 201, starts playing back the phrase, and displays the data obtained from the load data 202 on the CRT 102 (steps S18 and S19). Then, the viewer software 110 enters the state for waiting for a user operation while synchronizing the reproduction of the musical piece with the reproduction of its associated data (step S19→S7).

[0120] When the user desires to switch from the original version of a musical piece to its karaoke version, he or she clicks the karaoke button “K” in the word area 901 displaying the desired phrase (steps S7→S29→S21). When the karaoke button “K” in the word area 901 is clicked, the viewer software 110 obtains the reproduction start point of the phrase of the karaoke version displayed in the word area 901 from the time information (refer to FIG. 7), and its associated data to be reproduced for that phrase from the load data 202 in the main storage device 201, starts playing back the karaoke version of that phrase, and displays the data obtained from the load data 202 on the CRT 102 (steps S22 and S23). The viewer software 110 then waits for a user operation while performing the playback of the musical piece in synchronization with the playback of its associated data (step S23→S7).

[0121] When the user desires to repeatedly play back a specified musical piece, he or she clicks a repeat button displayed in the command area 801 (steps S7→S24→S25). When the repeat button is clicked, the viewer software 110 restarts playing back the musical piece from its beginning upon termination of the playback of the musical piece, and displays the associated data on the CRT 102 in synchronization with the playback (Steps S26 and S27). The viewer software 110 then waits for a user operation while performing the playback of the musical piece in synchronization with the playback of the associated data (step S27→S7).

[0122] Clicking the repeat button again cancels the setting of the repeated playback after it has been clicked to set the repeated playback. However, since the cancellation of that setting does not directly affect the playback operations of data, the process relating to the cancellation of the setting is omitted in FIG. 11.

[0123] When the user desires to terminate the viewer software 110, he or she clicks an EXIT button displayed in the command area 801 (steps S7→S28→S29). When the EXIT button is clicked, the OS 111 cancels the allocation of the CPU to the viewer software 110, and terminates the viewer software 110.

[0124] Then, the OS 111 erases the viewer software 110 loaded into the main storage device 201, and the load data 202 stored in the main storage device 201, and any excess data stored in the auxiliary storage device 106. This is because the size of the viewer software 110 is relatively small (it does not require a lot of time to read from the CD-EXTRA 103, and the capacity required for its storage is small), and the CD-EXTRA 103 must be loaded into the CD-ROM device 104 in order to synchronize the playback of the load data 202 with the playback of the audio data (CD-DA data) which is read from the CD-EXTRA 103. This process prevents the use efficiency of the auxiliary storage device 106 from deteriorating. However, since the capacity of an auxiliary storage device 106 has become larger recently, the viewer software 110 and the load data 202 read from the CD-EXTRA 103 may remain stored in the auxiliary storage device 106, depending on need.

[0125] FIG. 12 shows the details of step S2 in FIG. 11. As described above, this is the process performed by the OS 111 when a user loads a CD into the CD-ROM device 104.

[0126] When the user loads a CD into the CD-ROM device 104, the controller 109 in the CD-ROM device 104 detects it, and identifies the type of the loaded CD. The controller 109 notifies the OS 111 of, for example, the loading of the CD into the CD-ROM device 104, TOC information, and the type of the CD (whether or not the CD is an enhanced CD). FIG. 12 shows the flow of the process when'the notification is received. The detailed explanation about this process is provided below by referring to FIG. 12.

[0127] Whether or not the CD is an enhanced CD is determined as follows: the controller 109 makes the read head 108 perform seek operations over the entire CD in order to check to see if there are markings indicating a separation between areas in the CD.

[0128] In step S31, it is determined whether or not the CD loaded into the CD-ROM device 104 is an enhanced CD, that is, whether or not the CD includes the second session 103b, based on the information transmitted from the controller 109.

[0129] If “YES”, the process goes to step S32 to examine the type of data stored in the second session 103b. As is known, the formats of data stored in partitioned areas are different, depending on a data type of the enhanced CD. Therefore, the type of the enhanced CD can be identified by examining its contents. Whether the data is either the CD-ROM data or the CD-DA data can be determined based on the data of the Q channel in a subcode, or the contents of a header in a sector.

[0130] In step S33, it is determined whether or not the CD stores a file having a predetermined name. Assume that the predetermined name is, for example, “Autorun.inf”. If a file having that name is stored, the process goes to the execution of the process according to the contents described in that file (S33:YES). That is, the process transfers from step S2 to step S3, shown in FIG. 11. If “NO”, the process relating to the CD is terminated (step S33:NO).

[0131] If it is determined that the CD does not include the second session 103bin step S31, that is, “NO”, the process goes to step S34 where the data stored in the track numbered 1 is read, and its format is examined. This data read is implemented as follows: the OS 111 transmits the data at the beginning of the track obtained according to the TOC information, to the controller 109 in the CD-ROM device 104 as a control command.

[0132] In step S35, it is determined whether or not the data transmitted from the controller 109 is the CD-DA data. If CD-DA data is stored in the track, the result of the determination is “YES”, and control is transferred to the process for starting the playback (performance) as a music CD. If “NO” (for example, if the CD is a CD-ROM), the process goes to step S32 to determine whether or not the file having a predetermined name is stored. The succeeding operations are performed according to its result.

[0133] FIG. 13 is a flowchart showing the process for changing a reproduction point, which is performed by the viewer software 110 when a user clicks the word area 901 including the karaoke button “K”. In the flowchart shown in FIG. 11, the operations from step S14 to step S15, from step S18 to step S19, and from step S22 to step S23, are implemented by performing the process for changing a reproduction point.

[0134] Once the viewer software 110 starts playing back a musical piece, it is in a state of waiting for an event that a user selects on the display screen 800 of the CRT 102 while performing the necessary process. When the user selects on the display screen 800 in this state, the viewer software 110 obtains the information about the position of the clicking operation of the mouse 105 (display position information of the mouse cursor “M”) via the driver software 115 and the OS 111 (step S41). After the viewer software 110 obtains the display position information, it determines whether or not the mouse cursor “M” is displayed in the word area 901 at the time of the clicking operation according to the display position information of the mouse cursor “M” (step S42).

[0135] If it is determined that the display position of the mouse cursor “M” at the time of the clicking operation is outside the word area 901 (step S42:NO), the process goes back to step S41 where the viewer software 110 waits for a new clicking operation. If it is determined that the display position is inside the word area 901 (step S42:YES), it is then determined whether or not the display position is inside the word area 901 currently being played back (step S43). If the display position is determined to be inside the word area 901 currently being played back (step S43:YES), the process goes back to step S41.

[0136] If it is determined that the display position of the mouse cursor “M” at the time of the clicking operation is outside the word area 901 currently being played back (step S43:NO), the word area 901 corresponding to the phrases currently being played back is normally displayed (step S44). Then, the word area 901 that the user specified with the clicking operation is highlighted (step S45).

[0137] With this process, the display screen 800 (information area 803) of the CRT 102 varies from FIG. 9A to FIG. 9B. That is, the word area 901 numbered (5), which is clicked by the user, is highlighted, and at the same time, the word area 901 numbered (1) is changed to a normal display.

[0138] When the word area 901 specified by the user is highlighted, it is then determined whether the mode which is currently set, that is, the specified playback, is either the karaoke version or the original version (step S46). Then, the data required for playing back the musical piece is obtained from the load data 202 according to the determined mode (step S47). The reproduction start time of the phrases in the specified word area 901 is obtained at this time.

[0139] Then, the viewer software 110 instructs the controller 109 of the data read position via the OS 111 and the driver software 112, based on the obtained data, and displays the data obtained in step S47 on the display screen 800 of the CRT 102. Then, the viewer software 110 enters a state of waiting for a next user selection (step S48).

[0140] As described above, the viewer software 110 synchronizes the reproduction of the CD-EXTRA 103 as a music CD, that is, the reproduction of the CD-DA data stored in the first session 103awith the reproduction of the data stored in the second session 103b. This operation is common to the case in which a musical piece is played back from its beginning.

[0141] According to the second embodiment, words are displayed in phrases, and a reproduction point is changed by specifying a phrase. However, the reproduction point may be changed, for example, by specifying a reproduction elapsed time from the beginning of a musical piece. Additionally, the reproduction point may be changed according to a specified number among numbers, each of which is assigned to each of changeable reproduction points. Furthermore, an arbitrary reproduction point may be specified not only in one musical piece, but also in a plurality of musical pieces.

[0142] Still further, a user may specify the contents of the karaoke version or the original version to be reproduced in each word area (phrase) 901, for example, when a plurality of word areas 901 are displayed on the CRT 102 as shown in FIG. 9

[0143] In both of the above described cases, associated data stored in different areas in the CD-EXTRA 103 are synchronized and reproduced, so that the stored data can effectively be used. As a result, a variety of data reproduction forms can be provided to users, and at the same time, the CD-EXTRA 103 can be provided to the users as a more sophisticated and entertaining product.

[0144] Provided next is the explanation about another example of the reproduction of the CD-EXTRA 103, and the operations performed by the viewer software 110 for implementing the reproduction.

[0145] The fundamental operation of the previous example is to perform the reproduction of text data such as words, etc., in synchronization with the reproduction of a musical piece (CD-DA data). In the meantime, this example is intended to reproduce still pictures in addition to text data. The explanation about this example is provided below by referring to FIGS. 14 through 17.

[0146] FIGS. 14 and 15 show another example of the reproduction of the CD-EXTRA. These figures exemplify the case in which pictures and words corresponding to a musical piece in the load data 202 are displayed in synchronization with the reproduction of the CD-DA data stored in the first session 103a.

[0147] In the example shown in FIG. 14, the musical piece is played back by reproducing the CD-DA data in the first session 103a, and the still pictures and words associated with this musical piece are obtained from the load data 202, and displayed in synchronization with the playback of the musical piece. The still pictures are intended to match the story of the musical piece. As a result, a user can enjoy karaoke while experiencing the atmosphere of the musical piece from the still pictures.

[0148] The example shown in FIG. 15 is similar to that shown in FIG. 14 in which the vocal reproduction is synchronized with the reproduction of still pictures and words, but FIG. 15 exemplifies the case in which the language of the words to be displayed is different from that of the words to be uttered. Specifically, this is the case in which the language of the words to be uttered is English, while the language of the words to be displayed is Japanese.

[0149] Such reproduction allows a user to learn audiovisually, which leads to attaining a great effect from the learning. When the scenes (still pictures) representing and accompanying the contents (story) of the reproduced sound are sequentially displayed as shown in FIG. 15, the visual information is added to the correspondence between English and Japanese. Therefore, English can be learned more easily, and a great learning efficiency of the foreign language can be obtained. Accordingly, the CD-EXTRA 103 can be provided as a teaching material for performing an effective learning of a foreign language.

[0150] Note that words to be displayed may be pronunciations or words in another language, instead of the translated words of the words to be uttered. Further, a user may select the type of the words to be displayed.

[0151] Provided next is the explanation about the operations performed by the viewer software 110 which synchronizes the above described vocal reproduction (data in the first session 103a) with the reproduction of associated data in the load data 202, by referring to FIG. 16. FIG. 16 is a flowchart showing the process for reproducing associated data. To avoid confusion, the explanation provided below is based on the assumption that the CD-DA data in the first session 103ais data of a musical piece.

[0152] With the process for reproducing associated data, the number of a musical piece currently being played back (its playback start has been instructed) is obtained (step S51). Next, the text data (words, translated words, and pronunciations) and picture (still picture) data which are associated with the obtained number of the musical piece are obtained from the load data 202 in the main storage device 201 (step S52).

[0153] When the associated data are obtained, they are displayed on one screen in order to match the portion of the musical piece currently being played back (step S53). The operation performed in step S53 switches between a still picture and a display of words. Then, the current reproduction point of the CD-EXTRA 103, for example, the time elapsed from the beginning of the playback of the musical piece having the obtained number (reproduction elapsed time), is obtained as reproduction information (step S54). The reproduction elapsed time is a reproduction point in a musical piece currently being played back, which is transmitted from the controller 109 in the CD-ROM device 104 to the viewer software 110, that is, a performance point. The performance point is data indicating what minute, what second, and which frame is being played back. When such reproduction information (reproduction point) is obtained, the change of a display position such as one caused by highlighting words corresponding to the reproduction information (step S55), is made.

[0154] After words are highlighted in real time as a musical piece is being played back as described above, it is then determined whether or not the playback of the musical piece is terminated (step S56). This determination is made, for example, by making a comparison between the performance time of the musical piece obtained from the TOC information and the above described reproduction elapsed time obtained in step S54.

[0155] If it is determined that the performance of the musical piece is terminated in step S56 (step S56: YES), the process is terminated. If it is determined that the performance of the musical piece is not terminated (step S56: NO), the process goes to step S57.

[0156] According to the second embodiment, the entire picture (still picture and words) displayed on the display screen 800 (information area 803) is changed (redisplayed) each time one phrase is reproduced. Therefore, whether or not a picture must be redisplayed is determined depending on whether or not the reproduction of one phrase is terminated in step S57. This determination is made, for example, by comparing the reproduction end time of the phrase in the time information and the reproduction elapsed time indicating the reproduction point, which is obtained in step S54.

[0157] If it is determined that the performance of one phrase is not terminated in step S57 (step S57: NO), the process goes back to step S54. This is because the redisplay of a picture is regarded as being unnecessary. If it is determined that the performance of one phrase is terminated (step S57:YES), the process goes back to step S53. That is, the redisplay of a picture is regarded as being necessary. With the operation performed in step S53, another still picture and new words are displayed in the information area 803.

[0158] As described above, the viewer software 110 synchronizes the reproduction of the data stored in the first session 103ain the CD-EXTRA 103 with the reproduction of data in the load data 202, that is, the data of different representation media (still pictures and words), which are stored in the second session 103b. With this process, the reproduction of the data shown in FIGS. 14 and 15 can be realized.

[0159] The above described example is one implemented by synchronizing the reproduction of the CD-DA data stored in the first session 103awith the reproduction of display data such as still pictures and words. However, data for outputting sound (such as MPEG data, MIDI data, etc.) may be stored in the second session 103bin the CD-EXTRA 103. FIG. 17 shows the example in which the reproduction of the CD-DA data is synchronized with the reproduction of the data for outputting sound in the load data 202.

[0160] As shown in FIG. 17, the phrase to be played back next to the phrase currently being played back is output as vocal sound by reproducing the data in the load data 202, that is, the data stored in the second session 103b. This aims at, for example, helping people who are visually handicapped learn a phrase to be sung next in advance, and sing the song. With this process, the CD-EXTRA 103 can be provided to more people as a more useful form. The phrase to be sung next may be output to different channels such as earphones intended for phrases, a speaker intended for sound output by reproducing the CD-DA data, etc., in order to prevent the phrase from becoming inaudible due to the sound output of the reproduction of the CD-DA data.

[0161] As described above, the correspondence between the CD-DA data and the associated data is defined in phrases. Therefore, the reproduction shown in FIG. 17 can be implemented by reproducing the associated data in the load data 202, and advancing the correspondence by one phrase. In other words, this reproduction can fundamentally be implemented by performing the process exactly the same as that of the flowchart shown in FIG. 16. Accordingly, the explanation about the process performed by the viewer software 110 in order to implement the example shown in FIG. 17, is omitted here.

[0162] Although the words of the phrases are not displayed in the example shown in FIG. 17, words as well as sound may be displayed. Additionally, such reproduction may be performed not only for music, but also for learning a foreign language.

[0163] Conventionally, when pictures and sound are desired to be reproduced simultaneously, those data are compressed with the MPEG encoding method, etc., and stored. However, the reproduction by making various combinations of the load data 202 and the data stored in the first session 103acan be performed by loading the data stored in the second session 103bin the CD-EXTRA 103 into the PC 101 as described above. As a result, even if picture and sound data are stored in different storage areas, the pictures and sound can simultaneously be reproduced. This means that the restriction depending on the contents to be reproduced of data stored in the storage medium, can be reduced. Therefore, data stored in a storage medium can be selected depending on need. According to the second embodiment, sound of high quality can constantly be obtained by reproducing the CD-DA data (uncompressed data) stored in the first session 103ain synchronization with the display of pictures.

[0164] In this embodiment, a user arbitrarily selects the data to be reproduced in synchronization with the reproduction of the CD-DA data, among the data associated with the CD-DA data. However, a mode for mainly displaying artist information and musical piece information (artist/musical information mode), a mode for mainly displaying words, translated words, and pronunciation (word display mode), and a mode for mainly displaying discography (discography mode), may be prepared, and at the same time the basic structure of data to be displayed in the information area 803 may be predetermined for each mode, FIG. 18 shows examples of the display screen 800 in each mode. A display screen 800ais in the artist/musical piece information mode; a display screen 800bis in the word display mode; and a display screen 800cis in the discography mode.

[0165] <Third Embodiment>

[0166] According to the above described second embodiment, the data stored in the second session 103bis reproduced in synchronization with the CD-DA data stored in the first session 103a. In the meantime, according to the third embodiment, a user can add data to be synchronized with the reproduction of the CD-DA data to the data stored in the second session 103b.

[0167] FIG. 19 is a simplified block diagram showing the configuration of a sound playback/display system according to the third embodiment. In this figure, the same portions or the portions almost the same as those in the second embodiment, are denoted by the same reference numerals. The CRT 102 is omitted in FIG. 19. Provided below is the explanation about the third embodiment, by referring to FIG. 19.

[0168] In the third embodiment, an A/D converter 1902ain a converter 1902 converts a user's voice input from a microphone 1901 sung to the tune of the music into digital audio data; and the viewer software 110 receives the digital audio data via a corresponding driver software among a group of device driver softwares 1903, and the OS 111, and can store the data in the auxiliary storage device 106 via the OS 111, and the corresponding driver software for the auxiliary storage device 106 among the group of device driver softwares 1903.

[0169] The viewer software 110 stores the audio data in the auxiliary storage device 106 in phrases in synchronization with a reproduction point in the CD-EXTRA 103. With this process, the input user's voice can be played back in synchronization with the CD-DA data, and the load data 202 stored in the second session 103b.

[0170] To play back the user's audio data stored in the auxiliary storage device 106, the viewer software 110 outputs the data to a D/A converter 1902bin the converter 1902 via the OS 111 and the corresponding driver software among the group of device driver softwares 1903, after it receives the user's audio data via the driver software for the auxiliary storage device 106 and the OS 111. The audio data input to the D/A converter 1902bis converted into analog audio signals, output to a mixer 1905, and output from a sound output device 107 as vocal sound.

[0171] An audio signal generated based on MIDI data that a MIDI reproducing device 1904 receives from the viewer software 110 via the OS 111 and the corresponding driver software among the group of device driver softwares 1903, and an audio signal generated by D/A converting the CD-DA data read by the CD-ROM device 104, are input to the mixer 1905 in addition to the audio signal input from the D/A converter 1902b. The mixer 1905 synthesizes these signals according to predetermined contents, and outputs the synthesized signals to the sound output device 107.

[0172] As described above, when a user can prepare data to be reproduced in synchronization with data in the CD-EXTRA 103, a wider variety of entertainment using the CD-EXTRA 103 can be realized, in comparison with those realized according to the second embodiment. Furthermore, the user can display his or her originality, thereby obtaining deeper satisfaction.

[0173] Data such as photos, text (including the latest information about an artist), MIDI data, etc. may be added in addition to the above described audio data, as the data that a user can prepare. The prepared data may be erased upon termination of the viewer software 110. Otherwise, it may be stored in the auxiliary storage device 106.

[0174] As is evident from the explanations about the above described first through third embodiments, data stored in a storage medium is stored in another storage medium, and other data stored in a storage medium is reproduced in synchronization with data stored in another storage medium, according to the present invention. Accordingly, the use forms of the storage medium can be diversified, and a wider variety of users' entertainments can be realized. As a result, the storage medium can be provided to a user as a more satisfying and entertaining tool.