JP2001056695A - Speech synthesis method and storage medium storing speech synthesis program - Google Patents

Abstract

(57) [Problem] To provide a speech synthesis method that is suitable for reproducing a continuous speech waveform accompanied by natural sound and that can easily perform an operation for imparting a portamento effect. SOLUTION: A voice waveform of each voice unit is stored in a representative waveform storage device 16, a voice unit sequence is generated based on musical score information and lyric information, and voice corresponding to each voice unit of the generated voice unit sequence. The waveform is read from the representative waveform storage device 16, and the read voice waveform is synthesized to generate a continuous voice waveform. Then, in the steady period in which the pitch frequency is constant in the continuous voice waveform, the pitch frequency continuously changes to a pitch frequency different from this pitch frequency, and then the pitch frequency continuously changes to the original pitch frequency. Thus, a portamento effect is provided.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and a storage medium for generating a continuous voice waveform by synthesizing a voice waveform based on musical score information and lyric information, and more particularly to a method for generating a continuous voice waveform with a natural sound. The present invention relates to a speech synthesis method and a storage medium storing a speech synthesis program that are suitable for reproduction and that can easily perform an operation for providing a portamento effect.

[0002]

2. Description of the Related Art Conventionally, a continuous speech waveform is synthesized by synthesizing a speech waveform.
For example, as a method of generating
And Japanese Patent Application Laid-Open No.
You. First, the speech synthesis disclosed in Japanese Patent No. 2709198
The method is as follows. Original phoneme waveform sequence P₁~ P
_{n + 1}Each waveform P including three strong peaks₁, P_m, P_n
Is stored in the storage device as a representative waveform.
The representative waveform is stored for. Next, the input synthesis text
The phoneme sequence is obtained by analyzing the strike, and each phoneme of the phoneme sequence is obtained.
Read the representative waveform corresponding to
Representative waveform P ₁, P_m, P_nThe adjacent waveform P₁, P
_mWith the given synthetic pitch period T_pAnd connection time L
Waveform P interpolated by_1m(0), P_1m(1), ..., P
_1m(k) is generated.

Here, P _1m (i) (i = 1, 2,... K)
Is P ₁ × α (i) + P _m × β (i), and α (i),
beta (i) is a weighting factor for the waveform P _1, P _m respectively, α (i) = 0.5 × [1 + cos {π × (L -T
i) / L｝], β (i) = 1−α (i). Ti
Is the time interval from P _1m (0) to P _1m (i). On the other hand, the singing voice synthesizing apparatus disclosed in Japanese Patent Application Laid-Open No. Hei 7-46695 is as follows. The musical score / lyric information inputted by the musical score / lyrics input means, information on note length, pitch and volume, and time information of phonetic symbols such as consonants and vowels are divided into parts by the part dividing means. Also,
A singing voice having a different note length, pitch, and volume for each part based on the note length information, pitch information, and volume information changed by the note length information changing means, pitch information changing means, and volume information changing means. The signals are synthesized by the singing signal synthesizing means. A chorus singing voice signal is generated from the synthesized singing voice signal of each part by the chorus signal generation means, and the chorus singing voice is output from the singing voice output means.

[0004]

However, in the conventional speech synthesis technology, emphasis is placed on the quality of synthesized speech, the clarity of phonemes, and the reproducibility of personal information contained in the speech from which the synthesized speech was generated. And the style of vocalization was rarely considered. Here, the mode of occurrence indicates the voltament applied at the time of vocalization.

[0005] Natural singers, such as folk music, express ethnicity and originality by changing the manner of applying voltament in various ways, and portamento is an essential element for contemporary music. . For this reason, when the above-described conventional voice synthesis method is applied to synthesize a voice waveform based on musical score information and lyric information to generate a continuous voice waveform, the continuous voice waveform is not given a portamento effect. There is a problem that the singing voice due to the continuous voice waveform sounds unnatural and tends to have a mechanical sound that is far from a warm real singing voice.

In the above-mentioned conventional singing voice synthesizing method, although the portamento effect is added to the continuous voice waveform, in the section where the pitch frequency changes in the continuous voice waveform, the pitch frequency after the change is changed from the pitch frequency before the change. Since the portamento effect is simply applied so that the pitch frequency continuously changes up to the pitch frequency of, it is insufficient to reproduce a continuous sound waveform with a natural sound.

Also, depending on the genre of music, the manner of applying voltament may be different. Therefore, irrespective of the genre of music, if the portamento effect is applied to the continuous voice waveform uniformly, depending on the genre of music, The singing voice due to the waveform may sound unnatural.
On the other hand, in order to apply the portamento effect to the continuous sound waveform, knowledge for controlling the fundamental frequency is required, and it is assumed that a great burden is imposed on the user. In order to easily apply various voltament effects, it is desirable to minimize the operation related to the fundamental frequency so that the user can perform only the operation necessary for providing the portamento effect.

Therefore, the present invention has been made in view of such unresolved problems of the prior art, and is suitable for reproducing a continuous sound waveform with a natural sound. It is an object of the present invention to provide a speech synthesis method and a storage medium storing a speech synthesis program that can easily perform an operation for imparting a portamento effect.

[0009]

As a result of intensive studies, the present inventors have found that a continuous voice waveform of a singing voice such as ethnic music is included in a section of the continuous voice waveform in which the pitch frequency changes, and before the change. Not only the portamento effect is added so that the pitch frequency continuously changes from the pitch frequency of the change to the pitch frequency after the change, even in a steady period where the pitch frequency is constant in the continuous sound waveform,
It has been found that it has a certain portamento effect. We have also found that the portamento effect in this steady section differs depending on the music genre.
These motivations led to the completion of the present invention.

[0010] Furthermore, a method of modeling the shape of a fundamental frequency pattern with voltamentation from a continuous voice waveform of a singing voice such as ethnic music using as few parameters as possible and providing the parameters to the user has been considered.
From this conclusion, in order to achieve the above object, the speech synthesis method according to claim 1 of the present invention is a method for generating a continuous speech waveform by synthesizing a speech waveform, wherein In a steady period where the frequency is constant, a portamento effect is applied so that the pitch frequency continuously changes to a pitch frequency different from the pitch frequency, and then continuously changes to the original pitch frequency. .

Here, the condition for providing the portamento effect may be any condition, but from the viewpoint of reproducing a continuous sound waveform accompanied by a more natural sound, the condition for providing the portamento effect is as follows. Preferably, the length is not more than half of the length of the steady section. For example, 8
A portamento effect is applied to a half note so that the half note is shorter than a sixteenth note. Less than,
The same applies to the speech synthesis method according to the second aspect.

It is preferable that the amount of change in the pitch frequency be 0.4 times or less of the pitch frequency in the stationary section. For example, if the pitch frequency in the stationary section is 250 [Hz], and if the portamento effect is applied in the direction of increasing the pitch frequency, the pitch frequency at the maximum change (the above different pitch frequency) is 350 [Hz]. If the portamento effect is applied so as to be less than or equal to and the portamento effect is applied in the direction of decreasing the pitch frequency, the portamento effect is applied so that the pitch frequency at the maximum change becomes 150 [Hz] or more. I do. Hereinafter, the same applies to the voice synthesizing method according to the second aspect.

Further, according to a second aspect of the present invention, in the voice synthesizing method, a voice waveform of each voice unit is stored in a storage unit, and a voice unit sequence is generated based on musical score information and lyric information. A sound waveform corresponding to each sound unit of the read sound unit sequence from the storage means, and combining the read sound waveforms to generate a continuous sound waveform, wherein a pitch frequency of the continuous sound waveform is constant. In a certain stationary section, the pitch frequency continuously changes to a pitch frequency different from the pitch frequency, and then the pitch frequency continuously changes to the original pitch frequency,
Provides portamento effect.

Here, the speech waveform in units of speech refers to a speech waveform in which speech is defined as one unit, and includes, for example, a speech waveform in which phoneme is defined as one unit. The audio unit sequence refers to a sequence composed of a set of audio units. Further, in the voice synthesis method according to a third aspect of the present invention, in the voice synthesis method according to the second aspect, the different pitch frequencies are set based on a genre of music played according to the musical score information.

Further, in the voice synthesis method according to a fourth aspect of the present invention, in the voice synthesis method according to the third aspect, the different pitch frequencies are set based on a genre of music played in accordance with the score information. The pitch frequency is set to one of a higher pitch frequency and a lower pitch frequency than the original pitch frequency. Furthermore, a voice synthesis method according to a fifth aspect of the present invention is the voice synthesis method according to any one of the first to fourth aspects, wherein the change amount indicating a change amount from the original pitch frequency to the different pitch frequency. The portamento effect is provided based on a parameter and a time parameter indicating a time from the start to the end of the change of the pitch frequency.

Further, according to a sixth aspect of the present invention, in the voice synthesis method according to any one of the first to fifth aspects, the voice synthesis method includes the step of changing the pitch of the continuous voice waveform in a section where the pitch frequency changes. In giving the portamento effect so that the pitch frequency continuously changes from the previous pitch frequency to the changed pitch frequency, the change amount parameter indicating the change amount of the pitch frequency before and after the change and the start of the change of the pitch frequency The portamento effect is provided based on a time parameter indicating a time until the end of the change.

On the other hand, in order to achieve the above object, a storage medium storing a speech synthesis program according to claim 7 of the present invention generates a speech unit sequence based on musical score information and lyric information, and generates the generated speech. A storage medium storing a program for reading a sound waveform corresponding to each sound unit of the unit series from the storage means and synthesizing the read sound waveforms to generate a continuous sound waveform; In a steady period where the frequency is constant, a portamento effect is applied so that the pitch frequency continuously changes to a pitch frequency different from the pitch frequency, and then continuously changes to the original pitch frequency. A computer readable program storing a program for causing a computer to execute the portamento effect imparting step That.

With this configuration, when the information stored in the storage medium is read by the computer and the computer is executed based on the read information, the information is equivalent to the speech synthesis method according to claim 2. Is obtained. In the above, the speech synthesizing method and the storage medium storing the speech synthesizing program for achieving the above object have been proposed. However, the present invention is not limited to this. A synthesizer can also be proposed.

The first speech synthesizer is a device for synthesizing a speech waveform to generate a continuous speech waveform, and is different from the pitch frequency in a steady section of the continuous speech waveform where the pitch frequency is constant. The portamento effect is applied so that the pitch frequency continuously changes up to the pitch frequency, and then the pitch frequency continuously changes up to the original pitch frequency.

With such a configuration, a continuous speech waveform is generated by synthesizing the speech waveform, and the pitch frequency continuously changes to a pitch frequency different from this pitch frequency in a steady section of the continuous speech waveform. Thereafter, a portamento effect is applied so that the pitch frequency continuously changes to the original pitch frequency. Further, the second speech synthesizer includes storage means for storing a speech waveform of each speech unit, generates a speech unit sequence based on musical score information and lyrics information, and corresponds to each speech unit of the generated speech unit sequence. An audio waveform to be read from the storage means, and synthesizes the read audio waveform to generate a continuous audio waveform,
A stationary section detecting means for detecting a stationary section in which the pitch frequency is constant in the continuous sound waveform; and a steady section detected by the steady section detecting means, wherein the pitch frequency is different from the pitch frequency in the steady section. A portamento effect imparting means for imparting a portamento effect so that the pitch frequency continuously changes and thereafter the pitch frequency continuously changes to the original pitch frequency.

With such a configuration, a voice unit sequence is generated based on musical score information and lyrics information, and a voice waveform corresponding to each voice unit of the generated voice unit sequence is read from the storage means. The output speech waveforms are combined to generate a continuous speech waveform. Then, a steady section of the continuous voice waveform is detected by the steady section detecting means, and the pitch frequency is continuously changed to a pitch frequency different from the pitch frequency in the steady section in the detected steady section by the portamento effect imparting means. Change, then
A portamento effect is provided so that the pitch frequency continuously changes to the original pitch frequency.

Further, the third speech synthesizing device is provided with the second speech synthesizing device.
Wherein the portamento effect imparting means sets the different pitch frequency based on a genre of music played according to the musical score information. With such a configuration, different pitch frequencies are set by the portamento effect imparting means based on the genre of the music played in accordance with the musical score information.

Further, the fourth speech synthesizing device is provided with the third speech synthesizing device.
In the voice synthesizing apparatus, the portamento effect imparting means may change the different pitch frequency based on a genre of music played according to the score information, by using any one of a higher pitch frequency and a lower pitch frequency than the original pitch frequency. Is set to. With such a configuration, the different pitch frequency is set to one of a higher pitch frequency and a lower pitch frequency than the original pitch frequency based on the genre of the music played according to the score information by the portamento effect imparting means. Is done.

Further, the fifth speech synthesizing device is provided with the first speech synthesizing device.
In the speech synthesizer according to any one of the fourth to fourth aspects, the portamento effect imparting means may include a change amount parameter indicating a change amount from the original pitch frequency to the different pitch frequency, and a time from the start to the end of the change of the pitch frequency. The portamento effect is provided based on a time parameter indicating

With such a configuration, the portamento effect is given by the portamento effect giving means based on the change amount parameter and the time parameter. Further, a sixth speech synthesizer according to any one of the first to fifth speech synthesizers, wherein the section detection means for detecting a section of the continuous speech waveform in which a pitch frequency changes, and the section detection means. A second portamento effect imparting means for imparting a portamento effect so that the pitch frequency continuously changes from the pitch frequency before the change to the pitch frequency after the change in the detected section;
The portamento effect applying means is configured to apply the portamento effect based on a change amount parameter indicating a change amount of the pitch frequency before and after the change and a time parameter indicating a time from the start to the end of the change of the pitch frequency. ing.

With such a configuration, the section in which the pitch frequency changes in the continuous voice waveform is detected by the section detecting means, and the second portamento effect applying means detects the section in which the pitch frequency changes.
Based on the change amount parameter and the time parameter, the portamento effect is applied so that the pitch frequency continuously changes from the pitch frequency before the change to the pitch frequency after the change in the detected section.

[0027]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. 1 to 3 are diagrams showing an embodiment of a speech synthesis method according to the present invention.
First, a configuration of a system for implementing a speech synthesis method according to the present invention will be described with reference to FIGS.
FIG. 1 is a functional block diagram showing a configuration of a system for implementing a speech synthesis method according to the present invention, and FIG. 2 is a diagram showing examples of an original speech waveform, a representative waveform, and an interpolated speech waveform.

This system comprises an analyzing section 11 for analyzing and storing an audio waveform, and a synthesizing section 2 for synthesizing an audio waveform.
And 1. The analysis unit 11 includes an optimum waveform peak position detection circuit 14 for detecting an optimum audio waveform peak position, a peak vicinity waveform extraction circuit 15 for extracting an audio waveform near a waveform peak, and a representative memory for storing a representative waveform and labeling information. And a waveform storage device 16.

The optimum waveform peak position detecting circuit 14 receives the original speech waveform from the speech input terminal 12 and the phoneme labeling information from the labeling information input terminal 13 and classifies the inputted original speech waveform into phonemes by inspection. , Phoneme information and position information are added. The near-peak waveform extracting circuit 15 includes an optimum waveform peak position detecting circuit 1.
4, a speech waveform is input, and based on the phoneme information and position information added to the speech waveform, a peak position in the vicinity of about three representative waveforms of the speech waveform is detected for each phoneme. Then, a waveform centered on the detected peak position is cut out, and this is stored as a representative waveform in the representative waveform storage device 16 together with its position information and phoneme information. For example, as shown in FIG. 2A, when an original speech waveform sequence P ₁ , P ₂ , P ₃ ,..., P _m ,. Are extracted from the three representative waveforms P ₁ , P _m , and P _n .

The synthesizing section 21 generates a continuous audio waveform by connecting and synthesizing the audio waveform, and a waveform readout circuit 24 for reading the audio waveform from the representative waveform storage device 16, a pitch synchronous interpolation circuit 25 for interpolating the audio waveform. Synthesis circuit 27
And an analysis circuit 23 for analyzing lyrics information and score information
And a musical score information storage device 28 storing musical score information, a prosody information generating circuit 26, and a portamento effect imparting circuit 29 for imparting a portamento effect to the continuous voice waveform.

The analysis circuit 23 inputs the lyrics information from the lyrics information input terminal 22, reads the music score information from the music score information storage device 28, analyzes the input lyrics information and the read music score information, and analyzes the music data from the set of phoneme units. A phoneme unit sequence is generated, and the generated phoneme unit sequence is output to the waveform readout circuit 24 and the prosody information generation circuit 26. How to analyze lyrics information and score information
For example, a method disclosed in JP-A-7-146695 can be used.

The waveform reading circuit 24 reads a representative waveform corresponding to each phoneme unit of the phoneme unit sequence from the analysis circuit 23 from the representative waveform storage device 16 and outputs the read representative waveform to the pitch synchronous interpolation circuit 25. Has become. The prosody information generation circuit 26 calculates a synthetic pitch cycle and a connection time when synthesizing a representative waveform corresponding to each phoneme unit of the phoneme unit sequence based on the phoneme unit sequence from the analysis circuit 23, and calculates the calculated synthetic pitch. The period and the connection time are output to the pitch synchronous interpolation circuit 25.

The pitch synchronous interpolation circuit 25 interpolates the voice waveform between the representative waveforms from the waveform readout circuit 24 in synchronization with the synthesized pitch cycle based on the synthesized pitch cycle and the connection time from the prosody information generating circuit 26. , Obtained by superposing every pitch. At this time, a trapezoidal window, a cosine function window, or the like can be used as the overlapping window shape for each pitch.

For example, when the representative waveforms P ₁ and P _m shown in FIG. 2B are input and the synthetic pitch period T _p and the duration L are input, the time between the representative waveforms P ₁ and P _m is set to L. , Between the two waveforms P ₁ and P _m for each synthesized pitch period T _p by the following equation (1), and the synthesized waveforms P _1m (0), P _1m (1), P
_1m (2)... A composite waveform of P _1m (k) is obtained. P _1m (i) = P ₁ × α (i) + P _m × β (i) (1) where P _1m (0) = P ₁ , P _1m (k) = P _m and α (i)
And β (i) are weighting factors for the representative waveforms P ₁ and P _m and are represented by the following equations (2) and (3), respectively.

Α (i) = 0.5 × [1 + cos {π × (L−Ti) / L}] (2) β (i) = 1−α (i) (3) where i is a composite Waveform number and Ti are _calculated from P _1m (0) to P
Time interval _{1 m} (i), L indicates the time interval of the synthesized waveform P ₁ and P _m. In this way, the interpolated voice waveforms in which the representative waveforms are interpolated are output to the synthesis circuit 27.

The synthesizing circuit 27 includes a pitch synchronous interpolation circuit 25
A continuous speech waveform is generated by sequentially connecting and synthesizing the interpolated speech waveforms for each phoneme input from, and the generated continuous speech waveform is output to the portamento effect imparting circuit 29. Next, the configuration of the portamento effect imparting circuit 29 will be described with reference to FIGS.
FIG. 3 is a flowchart illustrating processing executed by the portamento effect imparting circuit 29, and is a diagram illustrating an example of a continuous sound waveform processed by the portamento effect imparting circuit 29.

The portamento effect applying circuit 29 receives the continuous sound waveform from the synthesizing circuit 27 and applies the portamento effect to the input continuous sound waveform by the following two methods. First, in a continuous section of a continuous voice waveform in which the pitch frequency is constant, the pitch frequency continuously changes to a pitch frequency different from this pitch frequency, and then the pitch frequency continuously changes to the original pitch frequency. Gives a portamento effect so that it changes. At this time, the application of the portamento effect is performed based on the change amount parameter ΔD2 and the time parameter T2.

Here, the change amount parameter ΔD2 is calculated as shown in FIG.
As shown in (current note portion), in the continuous voice waveform, the original pitch frequency is D, and the different pitch frequency is D.
If it is 2, it is indicated by the amount of change from the pitch frequency D to the pitch frequency D2. The time parameter T2 is indicated by the time required for the pitch frequency to change from the pitch frequency D to the pitch frequency D2. The time required for the pitch frequency to change from the pitch frequency D2 to the pitch frequency D is the pitch frequency D to the pitch frequency D2.
Up to the time required for the pitch frequency to change.

Second, a portamento effect is provided so that the pitch frequency changes continuously from the pitch frequency before the change to the pitch frequency after the change in the section of the continuous voice waveform where the pitch frequency changes. At this time, when the pitch frequency before the change is higher than the pitch frequency after the change, the change amount parameter ΔD
1 and the time parameter T1, and when the pitch frequency before the change is smaller than the pitch frequency after the change, the process is performed based on the change amount parameter ΔD3 and the time parameter T3.

Here, the change amount parameter ΔD1 is calculated as shown in FIG.
In the continuous voice waveform, the pitch frequency before the change is represented by D
1. Assuming that the pitch frequency after the change is D, the pitch frequency is represented by the amount of change from the pitch frequency D1 to the pitch frequency D. The time parameter T1 is indicated by the time required for the pitch frequency to change from the pitch frequency D1 to the pitch frequency D.

Further, as shown in FIG. 4 (the portion where the current note changes to a subsequent note), the change amount parameter ΔD3 represents the pitch frequency before the change and the pitch frequency after the change in the continuous voice waveform. Assuming that D3, the change amount from the pitch frequency D to the pitch frequency D3 is indicated. Further, the time parameter T3 is indicated by a time required for the pitch frequency to change from the pitch frequency D to the pitch frequency D3.

In order to provide the portamento effect described above, the portamento effect application circuit 29 is specifically shown in FIG.
The processing shown in the flowchart of FIG. The process will be described. First, the process proceeds to step S100, in which it is determined whether or not a steady section in which the pitch frequency is constant is detected in the continuous voice waveform, and when it is determined that the steady section has been detected (Yes ) Shifts to step S102 to input the change amount parameter ΔD2 and the time parameter T2. Here, the change amount parameter ΔD2 and the time parameter T2 may be input as appropriate values while the user observes the continuous sound waveform, or the change amount parameter ΔD2 and the time parameter T2 are read from a table or the like registered in advance. May be input.
Hereinafter, the same applies to steps S114 and S120.

Next, the flow shifts to step S104, where the genre of the music to be performed according to the musical score information is input.
Here, the music genre may be input directly by the user, or may be input by reading from the musical score information storage device 28. Next, the process proceeds to step S106,
Based on the input music genre, the change amount parameter ΔD2 is set to one of a higher pitch frequency and a lower pitch frequency than the pitch frequency in the steady section, and the process proceeds to step S108 to input the change amount parameter ΔD2 and On the basis of the time parameter T2, the first portamento effect is given in the steady section of the continuous sound waveform, and the flow shifts to step S110.

In step S110, it is determined whether or not a section in which the pitch frequency changes in the continuous voice waveform is detected. If it is determined that the section in which the pitch frequency changes is detected (Yes), the process proceeds to step S112. Then, it is determined whether or not the pitch frequency before the change is higher than the pitch frequency after the change. When it is determined that the pitch frequency before the change is higher (Yes), the process proceeds to step S114.

In step S114, a change amount parameter ΔD1 and a time parameter T1 are input, and step S1 is executed.
The process proceeds to S16, where the second portamento effect is provided in a section where the pitch frequency changes in the continuous speech waveform based on the input change amount parameter ΔD1 and time parameter T1, and the process proceeds to S118. In step S118, it is determined whether or not the processing has been completed for all the sections of the continuous sound waveform. When it is determined that the processing has been completed for all the sections (Yes), a series of processing is completed. If it is determined that it is not (No), the process proceeds to step S100.

On the other hand, if it is determined in step S112 that the pitch frequency before the change is smaller (No), the flow shifts to step S120 to input the change amount parameter ΔD3 and the time parameter T3, and shift to step S122. Then, based on the input change amount parameter ΔD3 and time parameter T3, the second portament effect is applied in a section where the pitch frequency changes in the continuous sound waveform, and the process proceeds to step S118.

On the other hand, when it is determined in step S110 that the section in which the pitch frequency changes is not detected (No),
The process moves to step S118. On the other hand, step S100
When it is determined that the stationary section is not detected (No), the process proceeds to step S110. Next, the operation of the above embodiment will be described.

First, the case of analyzing and storing a speech waveform will be described. When analyzing and storing the voice waveform, the user inputs the original voice waveform from the voice input terminal 12 and the phoneme labeling information from the labeling information input terminal 13 to the optimum waveform peak position detection circuit 14.
In the optimum waveform peak position detection circuit 14, when the original speech waveform and the phoneme labeling information are input, the original speech waveform is segmented for each phoneme by inspection, the phoneme information and the position information are added, and the waveform near the peak is cut out. Output to the circuit 15. The near-peak waveform extracting circuit 15 detects a peak position in the vicinity of the representative waveform for each phoneme based on the phoneme information and the position information added to the speech waveform, and detects the detected peak position. Is extracted and stored as a representative waveform in the representative waveform storage device 16 together with its position information and phoneme information.

Next, the case of synthesizing a speech waveform will be described. When analyzing and storing the voice waveform, the user inputs the lyric information from the lyric information input terminal 22 and the score information from the score information storage device 28 to the analysis circuit 23 by instructing the reading of the score information. I do. Analysis circuit 2
In 3, when the lyric information and the musical score information are input, the lyric information and the musical score information are analyzed to generate a phoneme unit sequence, and the generated phoneme unit sequence is output to the waveform reading circuit 24 and the prosody information generating circuit 26. You.

In the waveform reading circuit 24, when a phoneme unit sequence is input, a representative waveform corresponding to each phoneme unit of the phoneme unit sequence is read from the representative waveform storage device 16, and the read representative waveform is pitch-synchronized. Output to the interpolation circuit 25.
On the other hand, in the prosody information generation circuit 26, when the phoneme unit sequence is input, the synthesized pitch period and connection time are calculated based on the phoneme unit sequence, and the calculated synthesized pitch period and connection time are calculated by the pitch synchronous interpolation circuit 25. Is output to

In the pitch synchronous interpolation circuit 25, a representative waveform,
When the synthetic pitch period and the connection time are input, the interpolated voice is interpolated in synchronism with the synthetic pitch period based on the synthetic pitch period and the connection time, based on the synthetic pitch period and the connection time, and superimposed for each pitch. Waveform is synthesized circuit 27
Is output to In the synthesis circuit 27, when the interpolated voice waveform is input, the interpolated voice waveforms are sequentially connected and synthesized to generate a continuous voice waveform, and the generated continuous voice waveform is output to the portamento effect imparting circuit 29. You.

In the portamento effect imparting circuit 29, for example, when a continuous sound waveform as shown in FIG. 4 is input, first, through steps S110 to S116, a section in which the pitch frequency changes from the pitch frequency D1 to the pitch frequency D is performed. Is detected, and the second portamento effect is applied in the detected section based on the change amount parameter ΔD1 and the time parameter T1. As a result, as shown on the right side of FIG. 4, the pitch frequency continuously and smoothly changes from the pitch frequency D1 to the pitch frequency D over time T1. This continuous curve is approximated by, for example, a sine waveform.

Then, through steps S100 to S108, a stationary section in which the pitch frequency becomes D is detected, and the first portamento effect is applied in the detected stationary section based on the variation parameter ΔD2 and the time parameter T2. Is done. As a result, as shown in the center of FIG. 4, the pitch frequency continuously changes smoothly from the pitch frequency D to the pitch frequency D2 over the time T2, and thereafter, the pitch frequency changes from the pitch frequency D2 to the pitch frequency D at the time T2. And change continuously and smoothly. This continuous curve is approximated by, for example, a sine waveform. FIG.
, The continuous curve is convex, but depending on the genre of music, this may be concave.

Finally, steps S110 and S11
2, S120 and S122, a section in which the pitch frequency changes from the pitch frequency D to the pitch frequency D3 is detected, and the second portamental effect is detected in the detected section based on the change amount parameter ΔD3 and the time parameter T3. Granted. As a result, as shown on the left side of FIG. 4, the pitch frequency continuously and smoothly changes from the pitch frequency D to the pitch frequency D3 over time T3. This continuous curve is approximated by, for example, a sine waveform.

As described above, in the present embodiment, in the steady period where the pitch frequency is constant in the continuous speech waveform,
The portamento effect is applied so that the pitch frequency changes continuously to a pitch frequency different from the pitch frequency, and then changes continuously to the original pitch frequency. As a result, the singing voice of the continuous voice waveform sounds relatively natural and resembles a warm real singing voice. Therefore, a continuous sound waveform with a natural sound can be reproduced as compared with the related art.

Further, in the present embodiment, different pitch frequencies are set to one of a higher pitch frequency and a lower pitch frequency than the original pitch frequency based on the genre of music played according to the musical score information. did. As a result, depending on the music genre, it is possible to correct to some extent the disadvantage that the singing voice due to the continuous voice waveform sounds unnatural and has a mechanical sound that is far from the warm real singing voice. Therefore, a continuous sound waveform with a natural sound can be reproduced irrespective of the music genre as compared with the related art.

Further, in this embodiment, the first parameter is determined based on the variation parameter D2 and the time parameter T2.
Now has a portamento effect. Accordingly, when the user directly applies the portamento effect to the continuous sound waveform, it is only necessary to input two parameters, the change amount parameter D2 and the time parameter T2.
It is not necessary to perform the operation related to the fundamental frequency. Therefore, an operation for imparting the portamento effect can be easily performed as compared with the related art.

Further, in the present embodiment, based on the change amount parameter D1 and the time parameter T1, or the change amount parameter D3 and the time parameter T3, the second
Now has a portamento effect. Thus, when the user directly applies the portamento effect to the continuous sound waveform, it is only necessary to input two parameters, the change amount parameter D1 and the time parameter T1, or the change amount parameter D3 and the time parameter T3.
It is not necessary to perform the operation related to the fundamental frequency. Therefore, the operation for providing the portamento effect can be performed more easily.

Further, in the present embodiment, from a plurality of representative waveforms extracted from different points in the audio waveform, the audio waveform between the representative waveforms is interpolated according to a given pitch period and duration. To synthesize continuous speech. As a result, since the audio waveform information can be efficiently compressed and reproduced by using the representative waveform, the storage capacity can be significantly reduced in the waveform editing type speech synthesis method.
In addition, high-efficiency encoding can be performed by transmission / storage, and a high-quality continuous speech waveform can be obtained. Also,
By performing the interpolation processing of the audio waveform in synchronization with the pitch cycle and selecting the connection time, it is possible to control the utterance speed and the level of the audio without deteriorating the quality of the original audio waveform.

In the above embodiment, the change amount parameter ΔD2 is set to one of a higher pitch frequency and a lower pitch frequency than the pitch frequency in the stationary section based on the music genre. However, the present invention is not limited to this. For example, the value of the change amount parameter ΔD2 may be set according to the genre of music.

In the above-described embodiment, the weighting coefficient α (i) is configured using the one shown in the above equation (2). However, the weighting coefficient α (i) is not limited to this, and the weighting coefficient α (i) is used in the following equation (4) ) Can be used. α (i) = (L−Ti) / L (4) In the above embodiment, the speech synthesis method using phonemes as a unit has been described as an example, but speech synthesis based on other synthesis units such as syllables is also possible. Obviously it is available. Further, the present invention can be applied to speech synthesis in high-efficiency speech coding. In other words, only the representative waveform in FIG. 2B is encoded and transmitted or stored, the information is compressed in the transmission path or the storage device, and the synthesized voice is analyzed on the receiving side or the storage and reading side according to the above-described method of the present invention. Is obtained. Furthermore, in the case of waveform editing speech synthesis, several similar representative waveforms are further represented by one waveform by using a clustering technique such as a vector quantization method for the entire representative waveform obtained for all synthesis units. Also, the information compression ratio can be increased.

Further, in the above embodiment, the case where the speech synthesizing method according to the present invention is applied to the system including the analyzing unit 11 and the synthesizing unit 21 has been described. The present invention can be applied to other things without departing from the scope. Further, in the above embodiment, the configuration of the portamento effect imparting circuit 29 is not particularly shown.
9 may be configured by connecting a CPU, a ROM, a RAM, and the like via a bus. In this case, a control program for executing the processing shown in the flowchart of FIG. 3 is stored in the ROM, and when a continuous sound waveform is input from the synthesis circuit 27, the CPU stores the control program in a predetermined area of the ROM. The control program is read and activated, and the processing shown in the flowchart of FIG. 3 is executed. The present invention is not limited to this, and when executing the processing shown in the flowchart of FIG. 3, the program may be read into a RAM from a storage medium storing a program indicating these procedures and executed.

Here, the storage medium includes a semiconductor storage medium such as a RAM and a ROM, a magnetic storage type storage medium such as an FD and an HD, an optical read type storage medium such as a CD, CDV, LD, and DVD, and an MO such as an MO. A magnetic storage type / optical readout type storage medium includes any storage medium that can be read by a computer regardless of an electronic, magnetic, optical, or other read method.

[0064]

As described above, according to the voice synthesizing method according to the first to sixth aspects of the present invention, the singing voice by the continuous voice waveform is heard relatively naturally, and the singing voice is close to the warm real singing voice. And therefore, compared to the past,
An effect is obtained that a continuous sound waveform with a natural sound can be reproduced.

Further, according to the voice synthesizing method according to the third or fourth aspect of the present invention, depending on the genre of music, the singing voice of the continuous voice waveform sounds unnatural, and is far from the warm real singing voice. The problem of mechanical reverberation can be corrected to some extent, and, as compared with the conventional art, an effect that a continuous sound waveform with a natural reverberation can be reproduced regardless of the genre of music. Can be

Further, according to the voice synthesizing method according to the fifth aspect of the present invention, when the user directly applies the portamento effect to the continuous voice waveform, the user inputs two parameters, the change amount parameter and the time parameter. It is not necessary to perform the operation related to the fundamental frequency, and therefore, an effect that the operation for imparting the portamento effect can be easily performed as compared with the related art is obtained.

Further, according to the speech synthesizing method according to the sixth aspect of the present invention, there is obtained an effect that an operation for imparting a portamento effect can be performed more easily. On the other hand, according to the storage medium storing the voice synthesizing program according to claim 7 of the present invention, the singing voice of the continuous voice waveform sounds relatively naturally, and sounds like a warm real singing voice. In comparison, an effect that a continuous sound waveform with a natural sound can be reproduced can be obtained.

[Brief description of the drawings]

FIG. 1 is a functional block diagram showing a configuration of a system for implementing a speech synthesis method according to the present invention.

FIG. 2 is a diagram showing an example of an original audio waveform, a representative waveform, and an interpolated audio waveform.

FIG. 3 is a flowchart illustrating a process executed by a portamento effect imparting circuit 29;

FIG. 4 is a diagram showing an example of a continuous sound waveform processed by a portamento effect imparting circuit 29;

[Explanation of symbols]

Reference Signs List 11 analysis unit 14 optimal waveform peak position detection circuit 15 peak vicinity waveform extraction circuit 16 representative waveform storage device 21 synthesis unit 23 analysis circuit 24 waveform readout circuit 25 pitch synchronous interpolation circuit 26 prosody information generation circuit 27 synthesis circuit 28 score information storage device 29 Portamento effect imparting circuit

 ────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Yuji Aono 2-3-1 Otemachi, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corporation F-term (reference) 5D045 AA09 5D378 FF12 FF22 KK02

Claims (7)

[Claims] 1. A method for generating a continuous sound waveform by synthesizing a sound waveform, wherein a pitch frequency of the continuous sound waveform is constant up to a pitch frequency different from the pitch frequency in a steady period in which a pitch frequency is constant. A speech synthesis method characterized by adding a portamento effect so that the pitch frequency changes continuously and thereafter to the original pitch frequency. 2. A voice waveform for each voice unit is stored in storage means, a voice unit sequence is generated based on musical score information and lyric information, and a voice waveform corresponding to each voice unit of the generated voice unit sequence is generated. A method of generating a continuous voice waveform by reading from the storage means and synthesizing the read voice waveform, wherein the continuous voice waveform has a constant pitch frequency in a stationary section, and up to a pitch frequency different from the pitch frequency. A speech synthesis method characterized by adding a portamento effect so that a pitch frequency changes continuously and thereafter changes continuously to an original pitch frequency. 3. The speech synthesis method according to claim 2, wherein the different pitch frequencies are set based on a genre of music played according to the musical score information. 4. The pitch frequency according to claim 3, wherein the different pitch frequency is set to one of a higher pitch frequency and a lower pitch frequency than the original pitch frequency based on a genre of music played according to the musical score information. A speech synthesis method characterized in that: 5. A change parameter indicating a change amount from the original pitch frequency to the different pitch frequency, and a time parameter indicating a time from a start to a change end of the pitch frequency. A speech synthesis method, wherein the portamento effect is provided based on 6. The method according to claim 1, wherein the pitch frequency continuously changes from a pitch frequency before the change to a pitch frequency after the change in a section of the continuous voice waveform where the pitch frequency changes. In providing the portamento effect, the portamento effect is provided based on a change amount parameter indicating a change amount of the pitch frequency before and after the change and a time parameter indicating a time from the start to the end of the change of the pitch frequency. A speech synthesis method characterized by the following. 7. A voice unit sequence is generated based on musical score information and lyrics information, a voice waveform corresponding to each voice unit of the generated voice unit sequence is read from a storage unit, and the read voice waveform is synthesized to generate a continuous voice. A storage medium storing a program for generating a waveform, wherein in a continuous section in which the pitch frequency is constant in the continuous sound waveform, the pitch frequency continuously changes to a pitch frequency different from the pitch frequency, Thereafter, a computer-readable storage medium storing a program for causing a computer to execute a portamento effect applying step of applying a portamento effect so that the pitch frequency continuously changes to the original pitch frequency.