Reproducing apparatus with sub-picture processing function

[0001] This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2000-283382, filed Sep. 19, 2000, the entire contents of which are incorporated herein by reference.

[0002] This invention relates to a reproducing apparatus with a sub-picture processing function useful in applying to, for example, a DVD (digital versatile disk) video reproducing apparatus.

[0003] In techniques related to DVD video systems, video data and management information necessary to reproduce the video data are recorded on an optical disk. A DVD reproducing apparatus reads management data and controls the reproducing position of video data or sets the signal processing mode of a decoder or the like according to the attributes of the video data.

[0004] Video data includes main picture data obtained by compressing picture signals by the MPEG (Moving Pictures Experts Group) scheme, audio data obtained by PCM (pulse code modulation) processing audio signals or processing audio signals by the AC-3 system, and further sub-picture data obtained by run-length compressing signals of subtitles or the like. The main picture data, audio data, and sub-picture data are packed and recorded in the form of video packs, audio packs, and sub-picture packs. These packs are put together to form a video object unit (VOBU) to make it convenient to specify a playback range or a playback sequence. Moreover, a management unit called a video object (VOB), a collection of a plurality of VOBUS, is set to make it convenient to select or specify a playback chapter.

[0005] Management information includes VOB information to specify the VOB and program chain (PGC) information to determine the order in which a plurality of VOBUs included in the VOB are reproduced. The program chain (PGC) information has cell (Cell) information that specifies one or more VOBUs. Actually, the PGC specifies the order of the Cell. Then, the Cell information has entry information about the VOBUs.

[0006] Furthermore, the management information includes information about conditions necessary to decode the sub-picture data. The condition information is actually included in a sub-picture pack. The sub-picture signal has to be displayed in synchronization with the main-picture signal. For this reason, it includes display timing information (or information about the display position and display area on the screen) with respect to the main picture data, display period information, contrast information, and display color information.

[0007] As described above, in the DVD system, the display position and display color of the sub-picture signal with respect to the main picture signal have been determined. Thus, the magnitude, color, display position, and the like of the sub-picture signal (subtitles) are realized in the state set by the DVD video producer.

[0008] Depending on the users, however, it is desirable that the sub-picture signal (subtitles) should be smaller or larger. Moreover, they may make various requests, such as watermarked display or the change of the display position.

[0009] It is, accordingly, an object of the present invention is to provide a reproducing apparatus with a sub-picture processing function which enables the user to change the display state of the sub-picture signal on the reproducing apparatus side and obtain the display state of the sub-picture signal according to the user's taste.

[0010] The foregoing object is accomplished by providing a reproducing apparatus with a sub-picture processing function, comprising: a main picture decoder for decoding compressed main picture data and outputting a main picture signal; a sub-picture decoder for decoding sub-picture data; sub-picture processing means for receiving the sub-picture signal decoded at the sub-picture decoder and changing the layout position and/or size of the sub-picture signal; and combining means for combining the processed sub-picture signal outputted from the sub-picture processing means with the main picture signal.

[0011] In the reproducing apparatus of the present invention, the sub-picture processing means includes display area position judging means for judging the position of the display area of the inputted sub-picture signal with respect to a main picture frame, and position changing means for the position of the display area with respect to the main picture frame.

[0012] Furthermore, in the reproducing apparatus, the sub-picture processing means includes means for judging the size of the display of the inputted sub-picture signal with respect to a main picture frame, and means for changing the size of the display on the basis of the result of judgment made by the size judging means.

[0013] In addition, in the reproducing apparatus, the sub-picture processing means has the function of changing the display color of the inputted sub-picture signal.

[0014] Moreover, in the reproducing apparatus, the sub-picture processing section has display color changing means for changing the display color according to the display position of the inputted sub-picture signal.

[0015] Additional objects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out hereinafter.

[0033] Hereinafter, referring to the accompanying drawings, embodiments of the present invention will be explained.

[0034] FIG. 1 shows a DVD video reproducing apparatus according to an embodiment of the present invention.

[0035] The control of the reproducing apparatus is supervised by a main microprocessor unit (main MPU) 12. In this case, the main MPU 12 not only receives the request of the user operating a remote controller 14 via a remote control signal receiving section 13 and controls the reproducing apparatus but also displays the signal of the result of the control on a display section 15.

[0036] A DVD video (or an optical disk) 11 is controlled by a disk driver 16 driven under the control of the main MPU 12. The disk driver 16 has the function of rotating the optical disk 11 at a specific rotational speed by means of a disk motor (not shown) and its rotary servo circuit and the function of reading the data from the optical disk 11 by means of an optical head (not shown) and its servo circuit.

[0037] At the time of playback start, the main MPU 12 resets an STC (system time counter) 17, a timer for measuring a reference time in carrying out a reproduce operation.

[0038] The operation of reproducing the data from the optical disk 11 will be explained.

[0039] When receiving the playback instruction from the remote controller 14, the main MPU 12 reads the data in the management area on the optical disk 11 via the disk driver 16 and a data processing section 25 and determines a read address for reproducing the data.

[0040] Thereafter, the main MPU 12 outputs the determined address and read instruction to the disk driver 16. Then, the disk driver 16 reads the sector data from the optical disk 11 on the basis of the inputted read instruction and outputs the data to the data processing section 25.

[0041] In this case, the data processing section 25 subjects the inputted sector data to a demodulation process (16-8 conversion) and error correction process and converts the resulting signal into pack data. The pack data is inputted to a main picture, sub-picture, and audio separating section 30. The separating section 30 separates the inputted pack data into main picture packet data, sub-picture packet data, and audio packet data.

[0042] The separated main picture packet data, sub-picture packet data, and audio packet data are supplied to a main picture decoder 31, a sub-picture decoder 32, and an audio decoder 33 respectively. In this case, a navigation pack is stored in the built-in memory (not shown) of the separating section 30 for subsequent processing by the main MPU 12 and is ready to be accessed by the main MPU 12.

[0043] When the individual packet data separated at the separating section 30 are transferred to the decoders 31, 32, and 33 respectively, a PTS (presentation time stamp) is loaded into the STC 17. The loading process is realized when, for example, the main MPU 12 loads the PTS in the navigation pack into the STC 17 or when the main picture decoder 31 loads the PTS of the picture data automatically into the STC 17.

[0044] The navigation pack is placed at the head of a video object unit (VOBU) and is referred to when special playback (such as high-speed normal or reverse playback, or angle change). The navigation pack is a kind of management information including address information about the VOBUs in front of and behind the VOBU currently being reproduced and interleave unit information.

[0045] Each of the decoders 31, 32, 33 compares the value of the PTS in the packet data it is processing with the measured value of the STC 117. That is, each of the decoders 31, 32, 33 reproduces the packet data in synchronization with the value of the PTS.

[0046] The main picture data and sub-picture data reproduced at the main picture decoder 31 and sub-picture decoder 32 are supplied to a combining section 34, which combines the main picture data and sub-picture data. Thereafter, the combined picture data is converted by a D/A (digital-to-analog) conversion section 35 into analog data. The audio data reproduced at the audio decoder 33 is converted by a D/A conversion section 36 into analog data. Then, the output of each of the D/A conversion sections 35, 36 is supplied to the outside world via an A/V output section.

[0047] The output of the D/A conversion section 35 is supplied to a monitor 40 and the output of the D/A conversion section 36 is supplied to speakers (SPs) 41, 42.

[0048] The apparatus of the present invention is characterized by comprising a sub-picture processing section 50 which enables the user to process the sub-picture signal outputted from the sub-picture decoder 32.

[0049] Hereinafter, the signal processing function including the sub-picture processing section 50 and its operation will be explained.

[0050] Explanation will be given about the operation and function in a case where the contents of movies and karaoke have been recorded on a DVD video, a case where their subtitles are to be enlarged, or a case where the subtitles are to be moved.

[0051] FIG. 2 shows the control keys of the remote controller 14 necessary for the above operations, the order in which the user operates the keys, and the way the subtitles on the monitor change.

[0052] When the user carries out operations related to enlargement, reduction, or movement and the enlarged, reduced, or moved subtitles are displayed, the flow of the entire system is as follows.

[0053] The user operates the remote controller 14 to effect enlargement, reduction, or movement of the subtitles. The remote control signal reception section 13 recognizes the contents of the user's operation transmitted from the remote controller 14. The remote control signal reception section 13 transfers the remote control key data to the main MPU 12. Recognizing the received remote control key data and controls the main decoder 26 to effect the enlargement, reduction, or movement of the subtitles requested by the user. On the basis of the main MPU control data, the main decoder 26 controls the sub-picture processing section 50, which effects the enlargement, reduction, or movement of the subtitles.

[0054] Next, an example of the output state on the monitor corresponding to operation by the user will be described by reference to FIG. 2. The order in which the user operates the remote controller 14 to effect the enlargement or movement of the subtitles will be explained.

[0055] Choice 1: The user presses a process enter button 141 on the remote controller 14 to tell the reproducing apparatus that it is going to enter the subtitle enlarging mode (screen 61). At this time, to inform the user that the subtitle enlarging mode is on, the reproducing apparatus displays a character string, such as “ENLARGING MODE,” or the corresponding symbol on the monitor (screen 62).

[0056] Choice 2: The user presses an enlarging button 142, thereby causing enlarged subtitles to appear (screen 63).

[0057] Choice 3: The user presses a mode switching button 143 to move the subtitles. At this time, to inform the user that the subtitle moving mode is on, the reproducing apparatus displays a character string, such as “MOVING MODE,” or the corresponding symbol on the monitor (screen 64).

[0058] Choice 4: The user presses a moving button 144, thereby causing the subtitles to move (screen 65).

[0059] Choice 5: The user presses a process output button 145, thereby causing the apparatus to exit from the enlarging mode, the subtitle processing mode, or the moving mode (screen 66).

[0060] FIG. 2 is a mode transition diagram to help explain the relationship between the controlled states of the main decoder 26.

[0061] The main MPU 12 has to judges whether the present state is in the enlarging/reducing mode, the moving mode, or the normal mode in which subtitles will not be processed, and give a suitable instruction to the main decoder 26.

[0062] As shown in FIG. 3, the modes include the normal mode, enlarging/reducing mode, and moving mode and are decided as follows. Only in the enlarging/reducing mode and the moving mode, subtitles are processed. In the normal mode, they are not be processed. The default mode is the normal mode.

[0063] (a) When the contents of the user's request (the transmission from the remote controller 14) is the start of subtitle processing, the mode changes to either the enlarging/reducing mode or moving mode.

[0064] (b) When the contents of the user's request is the end of subtitle processing, the mode changes to the normal mode, even if the present mode is in either the enlarging/reducing mode or moving mode.

[0065] (c) When the contents of the user's request is the change of the subtitle processing mode, the present mode will change to the moving mode if it is in the enlarging/reducing mode, whereas the present mode will change to the enlarging/reducing mode if it is in the moving mode.

[0066] FIG. 4 is a flowchart for controlling the main decoder 26 of the main MPU to realize the transition of the above states.

[0067] The present state is checked and it is judged whether the present state is in the normal mode, enlarging/reducing mode, or moving mode, or another mode (step S71).

[0068] If the contents of the transmission from the remote controller 14 are the start of subtitle processing, the main decoder 26 (or sub-picture processing section 50) is set in the enlarging/reducing mode (steps S72, S73). If it has been judged at step S71 that the enlarging/reducing mode is on, the contents of the transmission from the next remote controller 14 are checked (step S74). Here, if the contents indicate the mode change, the main decoder 26 (or sub-picture processing section 50) is set in the moving mode (step S75). If the contents indicate the enlarging mode, the main decoder 26 (or sub-picture processing section 50) is set in the enlarging mode (step S76). If the contents indicate the reducing mode, the main decoder 26 (or sub-picture processing section 50) is set in the reducing mode (step S77).

[0069] If at step S71, it has been judged that the moving mode is on, the contents of the transmission from the next remote controller 14 are checked (step S78). Here, if the contents indicate the mode change, the main decoder 26 (or sub-picture processing section 50) is set in the enlarging/reducing mode (step S79). If the contents indicate the moving mode, the main decoder 26 (or sub-picture processing section 50) is set in the moving mode (step S80).

[0070] FIG. 5 shows the timing with which the main MPU 12 exchanges information with the main decoder 26. At time t1, the main decoder 26 transfers subtitle area data to the main MPU 12. Then, at time t2, the main MPU 12 transfers movement (or enlargement) control data to the main decoder 26. The range from time t3 to time t4 is the subtitle display period of the moved (or enlarged) subtitles. If necessary, the main decoder 26 transfers the subtitle area data to the main MPU 12 at time t5. Then, at time t6, the main MPU 12 transfers the movement (or enlargement) control data to the main decoder 26. Then, the range from time t7 to time t8 is the subtitle display period of the moved (or enlarged) subtitles.

[0071] FIG. 6 shows various cases in the direction of the enlargement of the subtitle area.

[0072] As shown at left in FIG. 6A, it is assumed that the subtitle area has more length than width and that when it is located in the lower part of the screen, an enlarging instruction is given from the remote controller 14. In this case, an enlargement is made only in the vertical direction and the subtitle area is enlarged upward by an increase in the size in the vertical direction as shown at right in FIG. 6A.

[0073] As shown at left in FIG. 6B, it is assumed that the subtitle area has more length than width and that when it is located in the upper part of the screen, an enlarging instruction is given from the remote controller 14. In this case, an enlargement is made only in the vertical direction and the subtitle area is enlarged downward by an increase in the size in the vertical direction as shown at right in FIG. 6B.

[0074] As shown at left in FIG. 6C, it is assumed that the subtitle area has more width than length and that when it is located on the right side of the screen, an enlarging instruction is given from the remote controller 14. In this case, an enlargement is made only in the horizontal direction and the subtitle area is enlarged leftward by an increase in the size in the horizontal direction as shown at right in FIG. 6C.

[0075] As shown at left in FIG. 6D, it is assumed that the subtitle area has more width than length and that when it is located on the left side of the screen, an enlarging instruction is given from the remote controller 14. In this case, an enlargement is made only in the horizontal direction and the subtitle area is enlarged rightward by an increase in the size in the horizontal direction as shown at right in FIG. 6D.

[0076] FIG. 7 shows the principle on which the main MPU 12 analyzes the subtitle area data.

[0077] The sub-picture processing section 50 of the main decoder 26 includes a subtitle area judging section.

[0078] The main MPU 12 receives the subtitle area data from the subtitle area judging section as explained in FIG. 5. Then, the main MPU 12 judges whether the subtitle area has more length than width or more width than length. Thereafter, it judges in which part of the screen the subtitle area is located, the upper, lower, right, or left.

[0079] As shown in FIG. 7, the subtitle area data represents the coordinate information of x1, x2, y1, y2 with respect to the origin (0, 0) (corresponding to the upper left corner of the effective screen of the monitor). Let the horizontal size of the subtitle area be h, its vertical size be v, the horizontal size of the overall display area (or monitor screen size) be hh, and its vertical size be vv.

[0080] (A) In the setting, the algorithm for judging whether the subtitle area has more length than width or more width than length is as follows: (1) h is compared with v; (2) if h is larger than v, it is judged that the area has more length than width; and (3) if v is larger than h, it is judged that the area has more width than length. In this way, a judgment whether the subtitle area has more length than width or more width than length is made.

[0081] (B) Next, the algorithm for judging in which part of the monitor screen the subtitle area is located, the upper, lower, right, or left part, is as follows: (1) when the subtitle area has more length than width, it is judged whether it is located in the upper part or lower part, and when the substitute area has more width than length, it is judged whether it is located in the right part ore left part; (2) when the subtitle area has more length than with, if the vertical coordinate (y1) of the left side of the subtitle area is larger than half (vv/2) the vertical screen size, it is judged that it is in the lower part, otherwise, it is judged that it is in the upper part; and (3) when the subtitle area has more width than length, if the horizontal coordinate (x1) of the left side of the subtitle area is larger than half (hh/2) the horizontal screen size, it is judged that it is in the right part, otherwise, it is judged that it is in the left part.

[0082] (C) Next, a judgment whether the subtitles can be enlarged is made as follows. The main MPU 12 judges only whether the subtitles can be enlarged, because it has to enlarge the subtitles only when they can be enlarged. Here, let e (e>1) be a magnifying power.

[0083] When the subtitles have more length than width, if the vertical size of the enlarged subtitle area is smaller than the vertical screen size, it is judged that enlargement is possible, and if the former is larger than the latter, it is judged that enlargement is impossible. It can be verified by making a judgment whether the expression v×e<vv holds. When the subtitles have more width than length, if the horizontal size of the enlarged subtitle area is smaller than the horizontal screen size, it is judged that enlargement is possible, and if the former is larger than the latter, it is judged that enlargement is impossible. It can be verified by making a judgment whether the expression h×e<hh holds.

[0084] FIG. 8 shows the flow of the operation when the main MPU 12 controls the main decoder 26.

[0085] When control of the main decoder 26 is started, the magnifying power e of the subtitle area is set (step B1). Then, the subtitle area data is received from the main decoder 26 (step B2). Next, an analysis of the subtitle area data is made (step B3). The analysis has been explained earlier. Then, it is judged whether the subtitles can be enlarged (step B4).

[0086] If the subtitles can be enlarged, the movements dh, dv of the subtitles are calculated (step B5). After the calculation, the calculated values dh, dv are transmitted to the main decoder 26 (step B6).

[0087] The movements of the-subtitles are calculated at step B5 as follows.

[0088] When the subtitle area has more length than width and is in the lower part, then dv=v−v×e (step C1).

[0089] When the subtitle area has more length than width and is in the upper part, then dv=v×e−v (step C2).

[0090] When the subtitle area has more width than length and is in the right part, then dh=h−h×e (step C3).

[0091] When the subtitle area has more width than length and is in the left part, then dh=h×e−h (step C4).

[0092] Next, an example of creating subtitle area data (x1, x2, y1, y2) at the main decoder 26 will be explained.

[0093] The subtitle data is composed of a 2-bit bit map. When the value of 2 bits is 00b, it indicates background; when the value is 01b, it indicates pattern; and when the value is 10b or 11b, it indicates emphasis. Then, x1, x2, y1, and y2 are calculated according to the following procedure and set as the subtitle area data:

[0094] (1) Let the first position of 01b, 10b, 11b to appear in the horizontal direction be x1.

[0095] (2) Let the last position of 01b, 10b, 11b to appear in the horizontal direction be x2.

[0096] (3) Let the first position of 01b, 10b, 11b to appear in the vertical direction be y1.

[0097] (4) Let the last position of 01b, 10b, 11b to appear in the vertical direction be y2.

[0098] Next, the operation of enlarging the subtitle data at the main decoder 26 will be explained.

[0099] FIGS. 9A to 9C show the subtitle area and the way the data changes when the subtitle area is enlarged. Specifically, FIG. 9A shows the whole of the bit map data including the subtitle area. FIG. 9B shows part of the subtitle area. FIG. 9C shows an enlarged example of the data in FIG. 9B. The lines marked with * are added lines. In this example, the subtitle area is enlarged twice in the horizontal direction. One line is added to each line by copying the data of each line. The adding process is carried out using the buffer memory and memory control section of the sub-picture processing section 50.

[0100] The present invention is not limited to the above embodiment.

[0101] As shown in FIG. 10, the main MPU 12 may set the video processing/subtitle processing synchronizing mode in the main decoder 12.

[0102] Specifically, the output of the main picture decoder 31 is supplied to the video processing section 51, which supplies the output to the combining section 34. The main MPU 12 sets the main decoder 26 in the video processing/subtitle processing synchronizing mode.

[0103] FIG. 11 shows an example of the transition of the video processing/subtitle processing synchronizing mode.

[0104] Specifically, the video processing/subtitle processing synchronizing mode is turned on or off as follows:

[0105] (1) When the menu display comes on, the video processing/subtitle processing synchronizing mode is automatically turned off (because the subtitles work as highlights at this time).

[0106] (2) When the menu display for the present volume comes on, the video processing/subtitle processing synchronizing mode is automatically turned off (because the subtitles work as highlights at this time).

[0107] (3) When the user turns on the video processing/subtitle processing synchronizing mode, processing is forced to be allowed.

[0108] FIG. 12 shows an example of a monitor screen before enlargement and an example of a monitor screen after enlargement, when the video processing/subtitle processing synchronizing mode is on. When the video processing/subtitle processing synchronizing mode is on, the magnifying power of video is the same as that of the subtitles.

[0109] The present invention is not limited to the above embodiment. While in the embodiment, the video processing/subtitle processing synchronizing mode can be turned on and off, the smoothing mode may be set in the main decoder 26 via the remote controller 14 and main MPU 12.

[0110] At this time, the execution of the smoothing mode prevents the edges of the subtitles from becoming serrate when the subtitles are enlarged.

[0111] FIGS. 13A and 13B shows part of the subtitles vertically enlarged twice the size of those before enlargement on the monitor. Each small square block in the figure represents a pixel. FIG. 13A shows a case where the smoothing mode has not been set. FIG. 13B shows a case where the smoothing mode has been set. The shaded parts in FIG. 13B represent the pixels interpolated for smoothing.

[0112] There are various methods of determining the interpolating positions of pixels. For example, there are case patterns as shown in FIGS. 14A, 14B, 14C, and 14D. In this example, an interpolating position is determined using nine pixels at the edge of a subtitle pattern as a block. In each case, the shaded parts indicate the subtitle data, the dotted-line part indicates subtitles or background data, and the pixel at the position where the word “INTERPOLATION” is written indicates the interpolated data part.

[0113] The present invention is not limited to the above embodiment.

[0114] When the subtitles are enlarged, one line of subtitles occasionally fails to fit in the monitor screen as it is. To overcome this drawback, the present system is provided with a line feed function.

[0115] FIG. 15 shows an example of enlarging the subtitles in the subtitle area with more length than width. When the subtitles fail to fit in the screen as a result of enlargement, the value x2 smaller than and closest to the horizontal size of the screen is calculated. Then, the screen size is subtracted from the horizontal data of the subtitles at the positions greater than x2, whereas y is added to the vertical data corresponding to x2 to shift the data one line. This makes it possible to fit all the subtitles corresponding to the screen into the screen.

[0116] FIG. 16 shows an example of simply moving the subtitles with more length than width in the upper part of the screen downward.

[0117] Furthermore, the color of the subtitles can be changed. This is done by operating a color change key to change the color data included in the subtitle data.

[0118] While in the above embodiment, the explanation has been given on the assumption that the sub-pictures and main pictures reproduced from a DVD video can be processed, the present invention is not limited to this. For instance, the invention may be applied to various sub-picture processes of sub-pictures and caption data sent with broadcasting signals. In addition, while in the above embodiments, the smoothing function, video processing function, and video processing/sub-picture processing synchronizing function have been explained in the separate embodiments, they may be combined together in a single embodiment.

[0119] Still furthermore, while in the above embodiments, the DVD video reproducing apparatus has been explained, the reproducing apparatus may be provided with means for recording data on a recordable disk. Moreover, while explanation has been given, provided that the recording medium is a DVD video, an optical disk, the recording medium may be a hard disk acting as a magnetic recording/reproducing medium.

[0120] As has been described above, the present invention enables the user to process and set the display state of sub-picture signals arbitrarily on the reproducing apparatus side and obtain the display state of the sub-picture signals according to the user's taste.

[0121] Additional advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.