US 5,111,505 AGrant
System and Method for Reducing Distortion in Voice Synthesis Through Improved Interpolation
Issue Date:1992-05-05
•11 Claims
•7 Drawing Sheets
Abstract
A voice synthesizing device which compiles wave segments, such as pitch wave segments, in order to synthesize speech. Speech is synthesized by connecting wave segments to form a contiguous waveform. Each wave segment is assigned one or more connection types which describe the connection to be made between points on that wave segment and points on adjacent wave segments. A wave segment connector uses information on the connection types of adjacent wave segments to connect the end point and lead point of the adjacent wave segments using a normal sampling period or a normal sampling period compressed or expanded by 1/2 of the sampling period. The period used depends on the connection type stored in the connection type memory.
Metadata
Assignee
- Sharp Kabushiki Kaisha
Inventors
- Atsunori Kitoh
- Yoshiji Fujimoto
Application Information
Application Number:US 5988268
Filing Date:1990-10-16
Priority Date:1988-07-21
Art Unit:231
Classifications
IPC:
G10L 502
Field of Search:
38136436-40;51-53513.5
Patent Drawings (7 sheets)
Description
Brief Description of the Drawings
FIG. 1 is a block diagram of a preferred embodiment of a voice synthesizing device according to the present invention;
FIG. 2 is a diagram showing the format of storage of pitch wave segment data in a read-only memory (ROM);
FIG. 3 is a flow chart showing the sequence of operation for the voice synthesizing operation;
FIG. 4a, FIG. 4b, FIG. 4c and FIG. 4d are descriptive drawings of the wave types;
FIG. 5a, 5b, and FIG. 5c are explanatory diagrams showing the wave types and their connection methods;
FIG. 6a, FIG. 6b, FIG. 6c and FIG. 6d are explanatory diagrams showing wave types according to an alternative embodiment of the present invention; and
FIG. 7a and FIG. 7b are explanatory diagrams showing the wave types and their connection methods according to the alternative embodiment of the present invention.
Detailed Description of the Invention
A first preferred embodiment of the present invention will now be described with reference to FIG. 1 which shows a block diagram of a voice synthesizing device according to the present invention.
Reference number 1 is a control ROM (read-only memory) which stores a control program used by CPU (central processing unit) 5 for voice synthesis; reference numeral 2 is a RAM (random access memory) used as a work memory during voice synthesis; reference numeral 3 is a data ROM used to store voice coding data; reference numeral 4 is an I/O interface through which input/output signals pass at the start of voice synthesis and other processes; reference numeral 6 is a D/A converter used for digital-to-analog conversion of voice wave data synthesized under the control of CPU 5; and reference numeral 7 is an amplifier which amplifies an input analog voice wave and outputs it to a loudspeaker 8.
The control ROM 1, RAM 2, data ROM 3, I/O interface 4, CPU 5, and D/A convertor 6, all used in the voice synthesizing device of the above construction, can be integrated together on a single chip. It is also possible to employ an external data ROM 9 for storing voice coding data for systems expansion.
When a start signal necessary to initiate the voice synthesis is input to a voice synthesizing device of the above construction from an external source through I/O interface 4, CPU 5 begins the voice synthesizing operation based on the control program stored in the control ROM 1. Thus, a voice synthesis wave data is generated by CPU 5 based on the voice coding data stored in the data ROM 3. The generated voice synthesis wave data is converted to an analog signal by D/A convertor 6, then amplified by amplifier 7 and is finally outputted as a synthesized voice from the loudspeaker 8.
As described below, the voice synthesizing device according to the present invention generates a synthesized voice free of distortion in the pitch wave rise by connecting wave segments such as pitch wave segments or quasi-voice wave segments to generate the synthesized voice.
According to a first method as shown in FIG. 5(a), when the time axis zero cross point of the interpolated waveform for the end sampled value of the preceding pitch wave segment and the time axis zero cross point of the interpolated waveform for the top sampled value of the following pitch wave segment are both within the range P2 when the waves are connected due to the connection of similar waves of type (1) or of dissimilar waves of waves of type (1) and type (3) as shown in FIG. 4(a) and FIG. 4(c), and when the time axis zero cross point of the interpolated waveform for the end sampled value of the preceding pitch wave segment and the time axis zero cross point of the interpolated waveform for the top sampled value of the following pitch wave segment are both within the range P1 when the waves are connected due to the connection of similar waves of wave type (2) or dissimilar waves of wave type (2) and type (4), the end sampled value and top sampled value of the pitch wave segments are output at the conventional sampling point and the pitch wave segments are connected. Then, the interpolated values between the end sampled value and the top sampled value (indicated by a solid triangle) are computed at a point equal to 1/2 sampling interval Ts and are outputted so that the two pitch wave segments can be connected smoothly. Hereinafter the connection of such pitch wave segments as just described shall be referred to as connection type 0a.
As shown in FIG. 5(b), when the time axis zero cross point of the interpolated waveform for the end sampled value of the preceding pitch wave segment is within the range P1 and the time axis zero cross point of the interpolated waveform for the top sampled value of the following pitch wave segment is within the range P2 when the waves are connected due to the connection of dissimilar waves of type (2) and type (1) or waves of type (2) and type (3), the wave segments are not connected at the conventional sampling point; the conventional sampling interval between the end and top sampled values is compressed by one-half and is then outputted to connect the pitch wave segments. Hereinafter the connection of such pitch wave segments as just described will be referred to as connection type 1a.
As shown in FIG. 5(c), when the time axis zero cross point of the interpolated waveform for the end sampled value of the preceding pitch wave segment is within the range P2 and the time axis zero cross point of the interpolated waveform for the top sampled value of the following pitch wave segment is within the range P1 when the waves are connected due to the connection of dissimilar waves of type (1) and type (2) or of waves of type (1) and type (4), the wave segments are not connected at the conventional sampling point; the conventional sampling interval between the end and top sampled values is expanded by one-half and is then outputted to connect the pitch wave segments. The period between the end and top sampled values of the pitch wave segments is interpolated as follows.
Specifically, assuming the end sampled value of the preceding pitch wave segment is .vertline.x1 .vertline. and the top sampled value of the following pitch wave segment is .vertline.x2.vertline., if .vertline.x1.vertline.>.vertline.x2.vertline., the interpolated value x1/2 is computed following the end sampled value .vertline.x1.vertline. (specifically, the higher peak value), and is then outputted at intervals of Ts/2. Next, the period between this interpolated value x1/2 and the top sampled value .vertline.x2.vertline. (specifically, the lower peak value) is interpolated and is then outputted. Hereinafter the connection of such pitch wave segments as just described shall be referred to as connection type 2-(a). Furthermore, if .vertline.x1.vertline.<x2, the interpolated value x2/2 of the prior top sampled value .vertline.x2.vertline. is computed and is then outputted at intervals of Ts/2. Next, the period between this interpolated value x2/2 and the top sampled value .vertline.x1.vertline. (specifically, the lower peak value) is interpolated and is then outputted. Hereinafter the connection of such pitch wave segments as just described shall be referred to as connection type 2-(b).
According to a second method, sampling is performed at a cycle twice (twice the frequency) that defined by the Nyquist theorem. Whether at an even-numbered sampling point or an odd-numbered sampling point, the sampling data used for voice synthesis is resampled at the standard Nyquist theorem cycle from the sampling point which is nearest the pitch segment rise. This wave is illustrated in FIG. 6(a)-FIG. 6(d). Here, the even-numbered sampling points are the sampling points (those shown by a solid line in FIG. 6(a)-FIG. 6(d)) occurring in the Nyquist theorem cycle, and the odd-numbered sampling points (those shown by a dotted line in FIG. 6a-FIG. 6(d)) are the sampling points occurring between even-numbered sampling points. In this case, sampling data obtained at the sampling points indicated by a double circle are the sampled values (which are hereinafter referred to as object samples) which will be the object of voice synthesis. These segments may be either wave type (1) or type (2).
As shown in FIG. 7(a), when the time axis zero cross point of the interpolated waveform for the end sampled value which will be the object of voice synthesis for the preceding pitch wave segment (hereinafter referred to as the end object sample) and the time axis zero cross point of the interpolated waveform for the leading object sample of the following pitch wave segment are both within the range P2 due to the connection of similar waves of type (5) or dissimilar waves of type (5) and type (6), the end object peak which is the object of voice synthesis and the leading object sample are outputted at the sampling point which will be the object of voice synthesis to connect the pitch wave segments. Then, at the half point of the object sampling period, the end sampled value q of the preceding pitch wave segment is outputted as the interpolated value so that the two pitch wave segments can be connected smoothly. Hereinafter, connection of such pitch wave segments will be referred to as connection type 0b.
As shown in FIG. 7(b), when the time axis zero cross point of the interpolated waveform for the end object sample of the preceding pitch wave segment is within the range P1 and the time axis zero cross point of the interpolated waveform for the leading object sample of the following pitch wave segment is within the range P2 due to the connection of similar waves of type (6) or dissimilar waves of type (6) and type (5), the pitch wave segments are not connected at the sampling point which is the object of voice synthesis; the period between the end object sample and the leading object sample of the pitch wave segments is compressed by one-half and is then outputted to connect the pitch wave segments. Hereinafter, connection of such pitch wave segments will be referred to as connection type 1b.
FIG. 2 shows one example of the data format when, for example, the pitch wave segments are analyzed and the resulting pitch wave segment data is stored in ROM 3 (see FIG. 1). The illustrated data format is comprised of encoding data of multiple pitch wave segments, each of said encoding data for each pitch wave segment including interpolation data and voice data. The interpolation data consists of end segment data 11 identifying whether the pitch wave segment is the last pitch wave segment or not, encoding method data 12 identifying the method used to encode the sampled data of the pitch wave segment, repeat number data 13 telling how many times the pitch wave segment was repeated, connection type data 14, as shown in FIG. 5 and FIG. 7, for use when the same pitch wave segment is repeated, and connection type data 15 (hereinafter referred to as a next pitch wave segment connection type) for when the given pitch wave segment is connected to the next adjacent pitch wave segment. The voice data includes a sample number data 16 specifying the number of encoded datum included in the pitch wave segment, and a series of multiple encoded data 17 to 19 for each sampling point used in voice synthesis. This encoded data is stored as a bit string according to the encoding method (e.g., pulse code modulation (PCM) or adaptive differential pulse code modulation (ADPCM)) stored in the encoding method data 12 for the interpolation data.
Referring now to the flow chart of FIG. 3, the voice synthesizing operation whereby pitch wave segments which are wave segments are connected and a voice is synthesized by the methods 1 and 2 described above will be described in detail below.
At step S1, 1 byte of interpolation data is read from the pitch wave segment data stored in the data ROM 3 according to the format shown in FIG. 2, and the byte is divided into the end segment data 11, the encoding method data 12, the repeat number data 13, the connection type data 14, and the next pitch wave segment connection type 15. Based on the obtained information, the end segment data flag, encoding method flag, repeat counter, repeat connection type, and next pitch wave segment connection type are each set in RAM 2. RAM 2 has an area for storing the repeat connection type for wave segment connection and a pitch wave segment connection type for wave segment connection, and the repeat connection type of the preceding pitch wave segment data and the next pitch wave segment connection type are both set therein.
At step S2, sample number data 16 specifying the encoded datum number of one pitch wave segment is read from the data ROM 3, and this number is set in RAM 2 as the sample number count.
At step S3, the first coded datum is read from data ROM 3.
At step S4, the first coded datum is decoded according to the encoding method set in the encoding method flag of RAM 2, and the top sampled value of the pitch wave segment is computed. The interpolated value of the period between this top sampled value and the following sampled value (based on the second encoded datum) is then computed. Next, the interpolation processing required for connection with the preceding pitch wave segment is then executed according to the next pitch wave segment connection type of the preceding pitch wave segment data set in the repeat connection type for pitch wave segments in RAM 2. Furthermore, the timing of the output of the computed the top sampled value to the D/A convertor 6 (if connection type 0a and 0b, the normal timing is outputted; if connection type 1a and 1b, the timing of a sampling cycle advanced by one-half is outputted; if connection type 2a and 2b, the timing of a sampling cycle delayed by one-half is output) is computed.
At step S5, the top sampled value computed at step S4 and the output timing of the preceding and following interpolated values computed in step S4 are outputted to D/A convertor 6.
In other words, it is interpolated according to the four connection types shown in FIG. 5 whether the period between the end sampled value of the preceding pitch wave segment and the top sampled value of the current pitch wave segment is expanded or compressed by one-half sampling cycle, and then D/A converted.
At step S6, the next encoded data (second encoded datum) is read from data ROM 3.
At step S7, the next encoded datum is decoded according to the encoding method, and the next sampled value is computed. Then, the interpolated value of the period to the next sampled value is computed. The computed sampled value and the interpolated value are outputted to D/A convertor 6 at the normal timing (specifically, the normal sampling point).
At step S8, the sample counter is decremented by 1, and it is determined based on this value whether the processing of the encoded data of the current pitch wave segment has been completed or not. If the result is that all processing has been completed, the flow advances to step S9; if not, the flow returns to step S6; and in both cases processing of the next encoded data is executed.
At step S9, the repeat connection type of the preceding pitch wave segment data set at the repeat connection type for pitch wave segments in RAM 2 is reset to the repeat connection type of the current pitch wave segment data set in the repeat connection type in RAM 2.
At step S10, the repeat counter in RAM 2 is decremented by 1, and it is determined based on this value whether all repetitions of the current pitch wave segment are completed or not. If the result is completion, the flow advances to step S11; if not, the flow returns to step S3, the first encoded data of the current pitch wave segment is again inputted, and repeat processing is executed.
At step S11, the next pitch wave segment connection type of the preceding pitch wave segment data set in the next pitch wave segment connection type for pitch wave segments in RAM 2 is reset to the next pitch wave segment connection type of the current pitch wave segment data set in the next pitch wave segment connection type of RAM 2.
At step S12, the end segment data flag in RAM 2 is referenced to determine whether the current pitch wave segment is the end segment. If the result is "yes", the voice synthesis operation is completed; if "no", the flow returns to step S1, the next pitch wave segment data is read, and processing of the next pitch wave segment data begins.
Thus, wave segment connection types are categorized by the combination of the connections of the pitch wave segments of differing wave types. Based on the connection type, the period between the end sampling point and the leading sampling point of connected pitch wave segments may be compressed or expanded by one-half of the normal sampling period, or the normal sampling period may be used to connect the wave segments. Therefore, pitch wave segments can be connected smoothly by a simple operation without producing any phase shift in the connection of the pitch wave segments. In other words, in a voice synthesizing device according to the present invention, distortion does not occur in the rise of the pitch wave segment and sound quality deterioration is not produced.
In the foregoing preferred embodiment as described above, a pitch wave segment is used as the wave segment, but the present invention shall not be so limited, and a voice wave segment conforming to a pitch wave segment may also be used.
As will be known from the foregoing description of the present invention, no phase shifts occur in the connection of wave segments in the synthesized voice generated by the voice synthesizing device according to the present invention This advantage results due to the voice synthesizing device being provided with the wave segment connector which stores a connection type which expresses the type of connection between the wave segments in the voice in a connection type memory. Further, when said wave segments are connected to synthesize a voice, the end and leading sampling points of said wave segments are connected by a normal sampling period or by a sampling period compressed or expanded by one-half period depending upon the connection type stored in the connection type memory.
As a result, the period between pitch wave segments can be interpolated and the segments smoothly connected by a simple operation. Therefore, according to the present invention, voice synthesis free of distortion in the rise of connected wave segments and with no deterioration of sound quality can be achieved.
Although the present invention has been fully described in connection with the preferred embodiments thereof with reference to the accompanying drawings, it is to be noted that various changes and modifications are apparent to those skilled in the art. Such changes and modifications are to be construed as included within the scope of the present invention defined by the appended claims, unless they depart therefrom.
Claims
What is claimed is:
1. A device used with a voice synthesizing device which connects wave segments such as pitch wave segments in speech input to the device, comprising: a connection type memory for storing a plurality of wave segment connection types; means for assigning a connection type to a connection between a preceding wave segment and a following wave segment; and a wave segment connector which, when said wave segments are connected, connects an end sampling point of the preceding wave segment and a lead sampling point of the following wave segment utilizing a preferred sampling period between the end sampling point of the preceding wave segment and the lead sampling point of the following wave segment with an interval determined by the connection type assigned to the connection between the preceding wave segment and the following segment.
2. A device according to claim 1 wherein said preferred sampling period is selected from the group consisting of a predetermined sampling time period, three-halves a predetermined sampling time period, and one half of a predetermined sampling time period.
3. A device used with a voice synthesizing device for connecting wave segments, comprising: a) a connection type memory for storing a plurality of preferred connection types for wave segments, said connection types each representing a connection of an interpolated waveform for an end sampled value of a preceding wave segment of a particular type with an interpolated waveform for a lead sampled value of a following wave segment of a particular type, each of said preferred connection types determining a preferred sampling period for use during connection of said wave segments; b) means for assigning a connection type to a connection between a preceding wave segment and a following wave segment by interpolating a time axis zero cross point for said interpolated waveform for said end sampled value of said preceding wave segment and a time axis zero cross point for said interpolated waveform for said lead sampled value of said following wave segment and c) a wave segment connector providing connection of said preceding and following wave segments using one of said preferred sampling periods as determined by the connection type assigned to the connection between said preceding and following wave segments.
4. A device according to claim 3 wherein said preferred sampling period has one of the following three values: a predetermined sampling time period, three-halves a predetermined sampling time period, and one half of a predetermined sampling time period.
5. A device according to claim 3 wherein said plurality of preferred connection types comprises: a) a first connection type in which both the time axis zero cross point of said interpolated waveform for said lead sampled value of said following wave segment and the time axis zero cross point of said interpolated wave segment for said end sampled value of said preceding wave segment are located within a second half of a predetermined sampling time period; b) a second connection type in which both the time axis zero cross point of said interpolated waveform for said lead sampled value of said following wave segment and the time axis zero cross point of said interpolated wave segment for said end sampled value of said preceding wave segment are located within a first half of a predetermined sampling time period; c) a third connection type in which the time axis zero cross point of said interpolated waveform for said lead sampled value of said following wave segment is located within a second half of a predetermined sampling period and the time axis zero cross point of said interpolated waveform segment for said end sampled value of said preceding wave segment is located within a first half of a predetermined sampling time period; and d) a fourth connection type in which the time axis zero cross point of said interpolated waveform for said lead sampled value of said following wave segment is located within a first half of a predetermined sampling time period and the time axis zero cross point of said interpolated wave segment for said end sampled value of said preceding wave segment is located within a second half of a predetermined sampling time period.
6. A device for connecting wave segments according to claim 3 wherein said wave segments comprise pitch wave segments.
7. A device for connecting wave segments according to claim 3 wherein said wave segments comprise voice wave segments.
8. A device for connecting wave segments according to claim 7 wherein said voice wave segments comprise quasi-voice wave segments.
9. An improved voice synthesizing device of the type in which a read only memory device stores a control program for use by a central processing unit for voice synthesis, a random access memory device is used as a work memory during voice synthesis, a data read only memory device is used to store voice coding data, an input/output interface is provided through which input/output signals pass at the start of voice synthesis and using other processes, a digital to analog convertor is used for conversion of voice wave data synthesized under the control of the central processing unit, and in which an amplifier amplifies an input analog voice wave and outputs to a loudspeaker, wherein the improvement comprises: a) a connection type memory for storing a plurality of preferred connection types for wave segments, said connection types each representing a connection of an interpolated waveform for an end sampled value of a preceding wave segment of a particular type with an interpolated waveform for a lead sampled value of a following wave segment of a particular type, each of said preferred connection types determining a preferred sampling period for use during connection of said wave segments; b) means for assigning a connection type to a connection between a preceding wave segment and a following wave segment by interpolating a time axis zero cross point for said interpolated waveform for said end sampled value of said preceding wave segment and a time axis zero cross point for said interpolated waveform for said lead sampled value of said following wave segment; c) a wave segment connector providing connection of said wave segments using one of said preferred sampling portions as determined by the connection type assigned to the connection between said wave segments to provide a synthesized voice output independent of any distortion in the pitch wave rise; and d) means for electrically interconnecting said connection type memory and said wave segment connector with the control read only memory, the input/output interface, the central processing unit, the data read only memory, and the digital to analog convertor.
10. A method of smoothly connecting wave segments for use in creating a synthesized voice free of distortion in a pitch wave rise, comprising the steps of: a) interpolating between sampled values to determine interpolated values to produce an interpolated waveform; b) identifying a time axis zero cross point for an interpolated waveform of an end sampled value of a preceding wave segment; c) determining a time axis zero cross point for an interpolated waveform of a lead sampled value of a following wave segment; d) classifying the time axis zero cross point of the preceding wave segment and the following wave segment with a connection type memory to select a preferred wave segment connection type; e) selecting a preferred wave segment connection type and a preferred sampling period from a plurality of connection types and sampling periods as determined by said wave types; and f) connecting said preceding wave segment with said following wave segment using said selected preferred wave segment connection type and said selected preferred sampling period to provide a synthesized voice independent of distortion in the pitch wave rise.
11. A method of smoothly connecting wave segments which can be used for creating a synthesized voice free of distortion in the pitch wave rise according to claim 10, wherein the step of selecting a preferred wave segment connection type and a preferred sampling period comprises the steps of: a) categorizing the time axis zero cross points of each of the interpolated waveforms for the preceding wave segment and the following wave segment by determining which memory waveforms stored in a wave segment connection type memory are most similar to said interpolated waveforms; and b) interpolating between said end sampled value and said lead sampled value with the preferred sampling period corresponding to the preferred wave connection type, the sampling period selected from a group comprising a predetermined sampling time, three-halves a predetermined sampling time, and one half a predetermined sampling time.
Patent Citations (9)
Non-Patent Literature (1)
- Yato et al., "Speech Synthesis by the Compililation of Speech Segments (in Japanese)", presented Dec. 18, 1973, at a laboratory of Kokusai Denshin Denwa. This paper contains an English summary.