CN112786008B - Speech synthesis method and device, readable medium and electronic equipment - Google Patents

Abstract

The present disclosure relates to a speech synthesis method, device, readable medium and electronic device, and relates to the field of electronic information processing technology. The method includes: obtaining a text to be synthesized and a specified acoustic feature, wherein the specified acoustic feature is used to indicate the rhythmic feature of the audio, extracting a phoneme sequence corresponding to the text to be synthesized, expanding the specified acoustic feature according to the phoneme sequence, obtaining an acoustic feature sequence, and inputting the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model to obtain a target audio corresponding to the text to be synthesized output by the speech synthesis model, wherein the acoustic feature of the target audio matches the specified acoustic feature. The present disclosure controls the speech synthesis of the text by specifying the acoustic feature, so that the target audio output by the speech synthesis model can correspond to the specified acoustic feature, and can realize the explicit control of the acoustic feature in the speech synthesis process, thereby improving the expressiveness of the target audio.

Description

Speech synthesis method, device, readable medium and electronic device

Technical Field

The present disclosure relates to the technical field of electronic information processing, and in particular, to a speech synthesis method, device, readable medium and electronic device.

Background technique

With the continuous development of electronic information processing technology, voice, as an important carrier for people to obtain information, has been widely used in daily life and work. Application scenarios involving voice usually include speech synthesis processing, which refers to synthesizing user-specified text into audio. In the process of speech synthesis, if it is necessary to synthesize a target speech that meets certain acoustic characteristics based on the specified text, it is necessary to use a pre-prepared reference speech that meets the acoustic characteristics. However, the duration of the reference speech is often quite different from that of the target speech, which will lead to unstable synthesis results. In addition, it is difficult to prepare reference speech corresponding to various acoustic features in advance to meet the diverse needs of users. Therefore, in the process of speech synthesis, effective control of acoustic features cannot be achieved.

Summary of the invention

This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

In a first aspect, the present disclosure provides a speech synthesis method, the method comprising:

Acquire a text to be synthesized and a specified acoustic feature, where the specified acoustic feature is used to indicate a prosodic feature of the audio;

Extracting a phoneme sequence corresponding to the text to be synthesized;

Expanding the specified acoustic feature according to the phoneme sequence to obtain an acoustic feature sequence;

The phoneme sequence and the acoustic feature sequence are input into a pre-trained speech synthesis model to obtain a target audio corresponding to the text to be synthesized output by the speech synthesis model, wherein the acoustic features of the target audio match the specified acoustic features.

In a second aspect, the present disclosure provides a speech synthesis device, the device comprising:

An acquisition module, used for acquiring the text to be synthesized and a specified acoustic feature, wherein the specified acoustic feature is used for indicating the prosodic feature of the audio;

An extraction module, used for extracting a phoneme sequence corresponding to the text to be synthesized;

An expansion module, used for expanding the specified acoustic feature according to the phoneme sequence to obtain an acoustic feature sequence;

A synthesis module is used to input the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model to obtain the target audio corresponding to the text to be synthesized output by the speech synthesis model, and the acoustic features of the target audio match the specified acoustic features.

In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect of the present disclosure.

In a fourth aspect, the present disclosure provides an electronic device, including:

a storage device having a computer program stored thereon;

A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect of the present disclosure.

Through the above technical solution, the present disclosure first obtains the text to be synthesized and the specified acoustic features used to indicate the rhythmic features of the audio, then extracts the corresponding phoneme sequence from the text to be synthesized, and then expands the specified acoustic features according to the phoneme sequence to obtain the acoustic feature sequence, and finally inputs the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model, thereby obtaining the target audio that matches the specified acoustic features and corresponds to the text to be synthesized output by the speech synthesis model. The present disclosure controls the speech synthesis of the text by specifying the acoustic features, so that the target audio output by the speech synthesis model can correspond to the specified acoustic features, and can realize the explicit control of the acoustic features in the speech synthesis process, thereby improving the expressiveness of the target audio.

Other features and advantages of the present disclosure will be described in detail in the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

FIG1 is a flow chart of a speech synthesis method according to an exemplary embodiment;

FIG2 is a flow chart of another speech synthesis method according to an exemplary embodiment;

FIG3 is a processing flow chart of a speech synthesis model according to an exemplary embodiment;

FIG4 is a block diagram of a speech synthesis model according to an exemplary embodiment;

FIG5 is a flow chart of a training speech synthesis model according to an exemplary embodiment;

FIG6 is a flow chart of another method for training a speech synthesis model according to an exemplary embodiment;

FIG7 is a flow chart of another method for training a speech synthesis model according to an exemplary embodiment;

FIG8 is a block diagram of a speech synthesis device according to an exemplary embodiment;

FIG9 is a block diagram of another speech synthesis device according to an exemplary embodiment;

Fig. 10 is a block diagram of an electronic device according to an exemplary embodiment.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and/or in parallel. In addition, the method embodiments may include additional steps and/or omit the steps shown. The scope of the present disclosure is not limited in this respect.

The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

FIG. 1 is a flow chart of a speech synthesis method according to an exemplary embodiment. As shown in FIG. 1 , the method may include the following steps:

Step 101, obtaining the text to be synthesized and the specified acoustic features, where the specified acoustic features are used to indicate the prosodic features of the audio.

For example, first obtain the text to be synthesized that needs to be synthesized. The text to be synthesized can be, for example, one or more sentences in a text file specified by the user, or one or more paragraphs, one or more chapters in a text file, or one or more words in a text file. The text file can be, for example, an e-book, or other types of files, such as news, public account articles, blogs, etc. At the same time, the specified acoustic features can also be obtained. The specified acoustic features can be understood as specified by the user, and it is expected that the text to be synthesized will be synthesized into audio that meets the specified acoustic features (i.e., the target audio mentioned later). The specified acoustic features can include multiple dimensions, for example, it can include one or more of the fundamental frequency (English: Pitch), volume (English: Energy), and speech speed (English: Duration), and can also include: noise level, pitch, timbre, loudness, etc. Among them, the noise level can be understood as a feature that can reflect the size of the noise in the audio.

Step 102: extract the phoneme sequence corresponding to the text to be synthesized.

For example, the text to be synthesized can be input into a pre-trained recognition model to obtain the phoneme sequence corresponding to the text to be synthesized output by the recognition model. It is also possible to search for the phonemes corresponding to each character in the text to be synthesized in a pre-established dictionary, and then combine the phonemes corresponding to each character into a phoneme sequence corresponding to the text to be synthesized. Among them, phonemes can be understood as speech units divided according to the pronunciation of each character, and can also be understood as vowels and consonants in the pinyin corresponding to each character. The phoneme sequence includes the phonemes corresponding to each character in the text to be synthesized (one character can correspond to one or more phonemes). For example, if the text to be synthesized is "Today's weather is very good", the phonemes corresponding to each character can be searched in the dictionary in turn, so as to determine that the phoneme sequence is "jintiantianqihenhao".

Step 103: Expand the designated acoustic feature according to the phoneme sequence to obtain an acoustic feature sequence.

Step 104, input the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model to obtain the target audio corresponding to the text to be synthesized output by the speech synthesis model, and the acoustic features of the target audio match the specified acoustic features.

For example, after obtaining the phoneme sequence, the specified acoustic feature can be expanded according to the phoneme sequence to obtain an acoustic feature sequence, and the acoustic feature sequence includes the acoustic feature corresponding to each phoneme in the phoneme sequence. In one implementation, the acoustic feature sequence can be generated according to the length of the phoneme sequence (i.e., the number of phonemes included in the phoneme sequence), wherein the acoustic feature corresponding to each phoneme is the specified acoustic feature. In another implementation, the specified acoustic feature can also be used as the average value (or standard deviation) to generate the acoustic feature corresponding to each phoneme according to a preset distribution (e.g., Gaussian distribution or uniform distribution).

Afterwards, the phoneme sequence and the acoustic feature sequence can be used as the input of a pre-trained speech synthesis model, and the output of the speech synthesis model is the target audio corresponding to the text to be synthesized and matching the specified acoustic features. Among them, the speech synthesis model can be pre-trained, which can be understood as a TTS (English: Text To Speech, Chinese: from text to speech) model, which can generate the target audio corresponding to the text to be synthesized and matching the specified acoustic features according to the text to be synthesized and the specified acoustic features. Specifically, the speech synthesis model can be trained based on the Tacotron model, Deepvoice 3 model, Tacotron2 model, Wavenet model, etc., and the present disclosure does not make specific limitations on this. In this way, in the process of speech synthesis for the text to be synthesized, in addition to the semantics included in the text to be synthesized, the specified acoustic features are also taken into consideration, so that the target audio can have the specified acoustic features, thereby realizing the explicit control of the acoustic features in the process of speech synthesis, without spending a lot of time and manpower costs to pre-create reference speech corresponding to various acoustic features. At the same time, it also avoids the instability problem caused by the large difference in duration between the reference speech and the target audio, improves the expressiveness of the target audio, and also improves the user's auditory experience.

In summary, the present disclosure first obtains the text to be synthesized and the specified acoustic features for indicating the rhythmic features of the audio, then extracts the corresponding phoneme sequence from the text to be synthesized, and then expands the specified acoustic features according to the phoneme sequence to obtain the acoustic feature sequence, and finally inputs the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model, thereby obtaining the target audio that matches the specified acoustic features and corresponds to the text to be synthesized output by the speech synthesis model. The present disclosure controls the speech synthesis of text by specifying acoustic features, so that the target audio output by the speech synthesis model can correspond to the specified acoustic features, and can realize the explicit control of the acoustic features in the speech synthesis process, thereby improving the expressiveness of the target audio.

FIG. 2 is a flow chart of another speech synthesis method according to an exemplary embodiment. As shown in FIG. 2 , the implementation of step 103 may include:

Step 1031: Determine the acoustic feature corresponding to each phoneme in the phoneme sequence according to the specified acoustic feature.

Step 1032: The acoustic features corresponding to each phoneme are combined into an acoustic feature sequence.

For example, in one implementation, the length of the phoneme sequence, that is, the number of phonemes included in the phoneme sequence, can be determined first. Then the specified acoustic feature is copied to obtain an acoustic feature sequence with the same length as the phoneme sequence, wherein each acoustic feature is the same as the specified acoustic feature, that is, the acoustic feature corresponding to each phoneme in the acoustic feature sequence is the specified acoustic feature. For example, if the length of the phoneme sequence is 100 (i.e., it includes 100 phonemes), then the acoustic feature corresponding to each phoneme can be determined as the specified acoustic feature, and then the acoustic features corresponding to the 100 phonemes can be composed into an acoustic feature sequence. Taking the specified acoustic feature as a 1*5-dimensional vector as an example, the acoustic feature sequence includes 100 1*5-dimensional vectors, which can form a 100*5-dimensional vector.

FIG. 3 is a processing flow chart of a speech synthesis model according to an exemplary embodiment. As shown in FIG. 3 , the speech synthesis model can be used to perform the following steps:

Step A: determining a text feature sequence corresponding to the text to be synthesized according to the phoneme sequence, wherein the text feature sequence includes a text feature corresponding to each phoneme in the phoneme sequence.

Step B: Generate target audio based on the text feature sequence and the acoustic feature sequence.

For example, the specific process of synthesizing the target audio by the speech synthesis model can first extract the text feature sequence (i.e., Text Embedding) corresponding to the text to be synthesized based on the phoneme sequence. The text feature sequence includes the text features corresponding to each phoneme in the phoneme sequence. The text feature can be understood as a text vector that can represent the phoneme. For example, the phoneme sequence includes 100 phonemes, and the text vector corresponding to each phoneme is a 1*80-dimensional vector, so the text feature sequence can be a 100*80-dimensional vector.

After obtaining the text feature sequence, the text feature sequence can be combined with the acoustic feature sequence to generate a target audio that matches the specified acoustic feature. For example, the text feature sequence and the acoustic feature sequence can be concatenated to obtain a combined sequence, and then the target audio is generated based on the combined sequence. For example, the phoneme sequence includes 100 phonemes, the text feature sequence can be a 100*80-dimensional vector, and the corresponding acoustic feature sequence is a 100*5-dimensional vector, then the combined sequence can be a 100*85-dimensional vector. The target audio can be generated based on this 100*85-dimensional vector.

Taking the speech synthesis model shown in FIG4 as an example, the speech synthesis model is a Tacotron model, which includes: an encoder (i.e., Encoder), an attention network (i.e., Attention), a decoder (i.e., Decoder) and a post-processing network (i.e., Post-processing). The encoder may include an embedding layer (i.e., Character Embedding layer), a pre-processing network (Pre-net) sub-model and a CBHG (English: Convolution Bank + Highway network + bidirectional Gated Recurrent Unit, Chinese: Convolution layer + high-speed network + bidirectional recurrent neural network) sub-model. The phoneme sequence can be input into the encoder. First, the phoneme sequence is converted into a word vector through the embedding layer, and then the word vector is input into the Pre-net sub-model to perform a nonlinear transformation on the word vector, thereby improving the convergence and generalization ability of the speech synthesis model. Finally, the text feature sequence that can represent the text to be synthesized is obtained according to the word vector after nonlinear transformation through the CBHG sub-model.

Afterwards, the acoustic feature sequence and the text feature sequence output by the encoder can be concatenated to obtain a combined sequence, and then the combined sequence is input into the attention network, and the attention network can add an attention weight to each element in the combined sequence. Specifically, the attention network can be a location sensitive attention (English: Location Sensitive Attention) network, or a GMM (English: Gaussian Mixture Model, abbreviated GMM) attention network, or a Multi-Head Attention network, which is not specifically limited in the present disclosure.

The output of the attention network is then used as the input of the decoder. The decoder may include a preprocessing network submodel (which may be the same as the preprocessing network submodel included in the encoder), Attention-RNN, and Decoder-RNN. The preprocessing network submodel is used to perform nonlinear transformation on the input. The structure of Attention-RNN is a one-layer unidirectional, zoneout-based LSTM (English: Long Short-Term Memory, Chinese: Long Short-Term Memory Network), which can take the output of the preprocessing network submodel as input and output it to the Decoder-RNN after passing through the LSTM unit. The Decoder-RNN is a two-layer unidirectional, zoneout-based LSTM, which outputs Mel spectrum information through the LSTM unit, and the Mel spectrum information may include one or more Mel spectrum features. Finally, the Mel spectrum information is input into the post-processing network, which may include a vocoder (for example, Wavenet vocoder, Griffin-Lim vocoder, etc.) for converting the Mel spectrum feature information to obtain the target audio.

FIG5 is a flow chart showing a method of training a speech synthesis model according to an exemplary embodiment. As shown in FIG5 , the speech synthesis model is obtained by training in the following manner:

Step C, extracting the real acoustic features of the training audio corresponding to the training text, where the real acoustic features are used to indicate the prosodic features of the training audio.

Step D: Expand the real acoustic features according to the training phoneme sequence corresponding to the training text to obtain a training acoustic feature sequence.

Step E: input the training phoneme sequence and the training acoustic feature sequence into the speech synthesis model, and train the speech synthesis model according to the output of the speech synthesis model and the training audio.

To train the speech synthesis model, it is necessary to first obtain the training text and the training audio corresponding to the training text. There can be multiple training texts, and correspondingly, there are also multiple training audios. For example, a large amount of text can be captured on the Internet as training text, and then the audio corresponding to the training text is used as training audio. For the training audio, the real acoustic features corresponding to the training audio can be extracted. For example, the real acoustic features corresponding to the training audio can be obtained by signal processing, annotation, etc., wherein the real acoustic features are used to indicate the rhythmic features of the training audio, and may include: at least one of the fundamental frequency, volume, and speech rate of the training audio, and may also include: noise level, pitch, timbre, loudness, etc. At the same time, the training phoneme sequence corresponding to the training text can also be extracted. The training phoneme sequence can include the training phonemes corresponding to each word in the training text (one word can correspond to one or more training phonemes).

Afterwards, the real acoustic features are expanded according to the training phoneme sequence corresponding to the training text to obtain a training acoustic feature sequence. The training acoustic feature sequence includes the training acoustic features corresponding to each training phoneme. For example, the training acoustic feature sequence can be generated according to the number of training phonemes included in the training phoneme sequence, wherein the training acoustic features corresponding to each training phoneme are all real acoustic features.

Finally, the training phoneme sequence and the training acoustic feature sequence are used as the input of the speech synthesis model, and the speech synthesis model is trained according to the output of the speech synthesis model and the training audio. For example, the difference (or mean square error) between the output of the speech synthesis model and the training audio can be used as the loss function of the speech synthesis model. With the goal of reducing the loss function, the back propagation algorithm is used to correct the parameters of the neurons in the speech synthesis model. The parameters of the neurons can be, for example, the weights (English: Weight) and biases (English: Bias) of the neurons. Repeat the above steps until the loss function meets the preset conditions, such as the loss function is less than the preset loss threshold.

FIG6 is a flowchart of another method for training a speech synthesis model according to an exemplary embodiment. As shown in FIG6 , the real acoustic feature includes at least one of fundamental frequency, volume, and speech rate. Accordingly, the implementation of step C may include:

Step C1: if the real acoustic feature includes speech speed, determine the duration corresponding to each training phoneme in the training phoneme sequence according to the training audio and the training phoneme sequence to determine the speech speed of the training audio.

Specific implementation methods may include:

First, the duration corresponding to each training phoneme is determined according to the training audio and the training phoneme sequence. For example, HTS (English: HMM-based Speech Synthesis System) can be used to divide the training audio according to the training phonemes included in the training phoneme sequence to obtain the duration corresponding to each training phoneme, which can be expressed as duration _i , indicating the duration corresponding to the i-th training phoneme.

Afterwards, a logarithmic operation is performed on the duration corresponding to each training phoneme to obtain the logarithmic duration corresponding to each training phoneme. By logarithmic operation, the variation range of the duration can be compressed, thereby amplifying the variation degree of the duration. For example, it can be expressed as log_duration _i , which represents the logarithmic duration corresponding to the i-th training phoneme.

Finally, the statistical value of the logarithmic duration corresponding to each training phoneme in the training phoneme sequence is used as the speaking speed of the training audio. For example, the average value (or standard deviation, extreme value, etc.) of the logarithmic duration corresponding to each training phoneme can be used as the speaking speed of the training audio, which can be expressed as log_duration_mean.

Step C2: if the real acoustic feature includes the fundamental frequency, extract the fundamental frequency of each audio frame included in the training audio to determine the fundamental frequency of the training audio.

Specific implementation methods may include:

First, audio processing tools such as sox, librosa, and straight can be used to process the training audio to obtain the fundamental frequency of each audio in the training audio, and then logarithmic operation is performed on the fundamental frequency corresponding to each audio frame to obtain the logarithmic fundamental frequency corresponding to each audio frame. Through logarithmic operation, the variation range of the fundamental frequency can be compressed, thereby amplifying the degree of variation of the fundamental frequency. For example, the fundamental frequency corresponding to each audio frame can be expressed as pitch _j , which represents the fundamental frequency corresponding to the jth audio frame. Correspondingly, the logarithmic fundamental frequency corresponding to the jth audio frame can be expressed as log_pitch _j .

Then, the statistical value of the logarithmic fundamental frequency corresponding to each audio frame in the training audio is used as the fundamental frequency of the training audio. For example, the mean value and standard deviation of the logarithmic fundamental frequency corresponding to each audio frame can be used as the fundamental frequency of the training audio, which can be expressed as log_pitch_mean and log_pitch_std, where log_pitch_mean can reflect the magnitude of the fundamental frequency of the training audio as a whole, and log_pitch_std can reflect the variation range of the fundamental frequency of the training audio.

Step C3: if the real acoustic feature includes volume, extract the volume of each audio frame included in the training audio to determine the volume of the training audio.

Specific implementation methods may include:

First, you can use audio processing tools such as sox, librosa, and straight to process the training audio to obtain the volume of each audio in the training audio, and then perform logarithmic operations on the volume corresponding to each audio frame to obtain the logarithmic volume corresponding to each audio frame. Through logarithmic operations, the range of volume changes can be compressed, thereby amplifying the degree of volume change. For example, the volume corresponding to each audio frame can be expressed as energy _j , which represents the volume corresponding to the jth audio frame. Correspondingly, the logarithmic volume corresponding to the jth audio frame can be expressed as log_energy _j .

Then, the statistical value of the logarithmic volume corresponding to each audio frame is used as the volume of the training audio. For example, the average value of the logarithmic volume corresponding to each audio frame can be used as the volume of the training audio, which can be expressed as log_energy_mean.

Furthermore, if the real acoustic feature also includes a noise level, step C may further include: determining the noise level corresponding to the training audio according to a linear prediction coefficient corresponding to the training audio.

For example, the LPC coefficient (Linear Prediction Coefficient) of the training audio can be determined, and then the first dimension of the LPC coefficient of the training audio is logarithmically operated, and the result of the logarithmic operation is used as the noise level corresponding to the training audio. Through the logarithmic operation, the range of noise level variation can be compressed, thereby amplifying the degree of noise level variation. The noise level can be expressed as log_spectral_tilt, for example.

In the case where the fundamental frequency, volume, speaking rate and noise level are included in the real acoustic features, the fundamental frequency, volume, speaking rate and noise level of the training audio can be used to form the real acoustic features of the training audio. For example, the real acoustic feature can be a 1*5-dimensional vector: {fundamental frequency: (log_pitch_mean, log_pitch_std), volume: log_energy_mean, speaking rate: log_duration_mean, noise level: log_spectral_tilt}. Accordingly, in the case where the specified acoustic features include the fundamental frequency, volume, speaking rate and noise level, the specified acoustic features obtained in step 101 can also include the above 5 dimensions.

FIG. 7 is a flowchart of another method for training a speech synthesis model according to an exemplary embodiment. As shown in FIG. 7 , the speech synthesis model may also be obtained by training in the following manner:

Step F: determining the statistical acoustic features of the training set according to the real acoustic features of a plurality of training audios included in the preset training set.

Step G, normalizing the true acoustic features of each training audio according to the statistical acoustic features.

Accordingly, step D may be implemented as follows:

The normalized real acoustic features are expanded according to the training phoneme sequence to obtain a training acoustic feature sequence.

For example, before the real acoustic features are expanded, the real acoustic features can also be normalized. For example, the training set includes multiple training texts, each training text corresponds to a training audio. The real acoustic features of each training audio can be determined in the manner of steps C1 to C4, and the statistical acoustic features of the training set can be determined. The statistical acoustic features can be, for example, the mean value, standard deviation, variance or extreme value of the real acoustic features. Then the real acoustic features of each training audio are normalized according to the statistical acoustic features. For example, the mean value μ and standard deviation σ of the real acoustic features can be used as statistical acoustic features, and then the real acoustic features between [μ-3σ, μ+3σ] are mapped to [-1,1], and the real acoustic features outside [μ-3σ, μ+3σ] can be truncated to -1 or 1. The mean value and standard deviation of each dimension in the real acoustic features can also be obtained respectively, and each dimension in the real acoustic features can be normalized. For example, taking the log_pitch_mean in the real acoustic feature as an example, the average pitch_μ and standard deviation pitch_σ of the log_pitch_mean of each training audio can be obtained, and then the log_pitch_mean between [pitch_μ-3pitch_σ, pitch_μ+3pitch_σ] is mapped to [-1,1], and the log_pitch_mean outside [pitch_μ-3pitch_σ, pitch_μ+3pitch_σ] is truncated to -1 or 1 to normalize the log_pitch_mean.

Furthermore, the normalized real acoustic features can be expanded according to the training phoneme sequence to obtain a training acoustic feature sequence. For example, the training acoustic feature sequence can be generated according to the number of training phonemes included in the training phoneme sequence, wherein the training acoustic features corresponding to each training phoneme are all normalized real acoustic features.

It should be noted that the specified acoustic features obtained in step 101 may also include the above-mentioned 5 dimensions that have been normalized. The specified acoustic features after normalization are more explanatory. Taking the specified acoustic features as {-1, 1, 0, 1, 0} as an example, where the value of log_pitch_mean is -1, it means that the target audio generated by the speech synthesis model and meeting the specified acoustic features is characterized by being low. The value of log_pitch_std is 1, indicating that the fundamental frequency of the target audio varies greatly. The value of log_energy_mean is 0, indicating that the target audio is of normal volume. The corresponding value of log_duration_mean is 1, indicating that the speaking speed of the target audio is slow (that is, the average duration corresponding to the phonemes is long). The corresponding value of log_spectral_tilt is 0, indicating that the noise level of the target audio is normal.

In summary, the present disclosure first obtains the text to be synthesized and the specified acoustic features for indicating the rhythmic features of the audio, then extracts the corresponding phoneme sequence from the text to be synthesized, and then expands the specified acoustic features according to the phoneme sequence to obtain the acoustic feature sequence, and finally inputs the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model, thereby obtaining the target audio that matches the specified acoustic features and corresponds to the text to be synthesized output by the speech synthesis model. The present disclosure controls the speech synthesis of text by specifying acoustic features, so that the target audio output by the speech synthesis model can correspond to the specified acoustic features, and can realize the explicit control of the acoustic features in the speech synthesis process, thereby improving the expressiveness of the target audio.

FIG8 is a block diagram of a speech synthesis device according to an exemplary embodiment. As shown in FIG8 , the device 200 may include the following modules:

The acquisition module 201 is used to acquire the text to be synthesized and the specified acoustic features, where the specified acoustic features are used to indicate the prosodic features of the audio.

The extraction module 202 is used to extract the phoneme sequence corresponding to the text to be synthesized.

The expansion module 203 is used to expand the specified acoustic feature according to the phoneme sequence to obtain an acoustic feature sequence.

The synthesis module 204 is used to input the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model to obtain the target audio corresponding to the text to be synthesized output by the speech synthesis model, and the acoustic features of the target audio match the specified acoustic features.

FIG. 9 is a block diagram of another speech synthesis device according to an exemplary embodiment. As shown in FIG. 9 , the expansion module 203 may include:

The determination submodule 2031 is used to determine the acoustic feature corresponding to each phoneme in the phoneme sequence according to the specified acoustic feature.

The expansion submodule 2032 is used to combine the acoustic features corresponding to each phoneme into an acoustic feature sequence.

In one application scenario, the speech synthesis model in the above embodiment can be used to perform the following steps:

Step A: determining a text feature sequence corresponding to the text to be synthesized according to the phoneme sequence, wherein the text feature sequence includes a text feature corresponding to each phoneme in the phoneme sequence.

Step B: Generate target audio based on the text feature sequence and the acoustic feature sequence.

In one application scenario, the specified acoustic feature includes at least one of fundamental frequency, volume, and speech rate.

In another application scenario, the speech synthesis model is trained in the following way:

Step C, extracting the real acoustic features of the training audio corresponding to the training text, where the real acoustic features are used to indicate the prosodic features of the training audio.

Step D: Expand the real acoustic features according to the training phoneme sequence corresponding to the training text to obtain a training acoustic feature sequence.

Step E: input the training phoneme sequence and the training acoustic feature sequence into the speech synthesis model, and train the speech synthesis model according to the output of the speech synthesis model and the training audio.

Specifically, the real acoustic feature includes: at least one of fundamental frequency, volume, and speech rate. The implementation of step C may include:

Step C1: if the real acoustic feature includes speech speed, determine the duration corresponding to each training phoneme in the training phoneme sequence according to the training audio and the training phoneme sequence to determine the speech speed of the training audio.

The specific implementation method may include: first, determining the duration corresponding to each training phoneme according to the training audio and the training phoneme sequence. Then, performing a logarithmic operation on the duration corresponding to each training phoneme to obtain the logarithmic duration corresponding to each training phoneme. Finally, taking the statistical value of the logarithmic duration corresponding to each training phoneme in the training phoneme sequence as the speech speed of the training audio.

Step C2: if the real acoustic feature includes the fundamental frequency, extract the fundamental frequency of each audio frame included in the training audio to determine the fundamental frequency of the training audio.

The specific implementation method may include: first, performing a logarithmic operation on the fundamental frequency corresponding to each audio frame to obtain the logarithmic fundamental frequency corresponding to each audio frame, and then using the statistical value of the logarithmic fundamental frequency corresponding to each audio frame in the training audio as the fundamental frequency of the training audio.

Step C3: if the real acoustic feature includes volume, extract the volume of each audio frame included in the training audio to determine the volume of the training audio.

The specific implementation method may include: first, performing a logarithmic operation on the volume corresponding to each audio frame to obtain the logarithmic volume corresponding to each audio frame, and then using the statistical value of the logarithmic volume corresponding to each audio frame in the training audio as the volume of the training audio.

In another application scenario, the speech synthesis model can also be trained in the following way:

Step F: determining the statistical acoustic features of the training set according to the real acoustic features of a plurality of training audios included in the preset training set.

Step G, normalizing the true acoustic features of each training audio according to the statistical acoustic features.

Accordingly, step D may be implemented as follows:

The normalized real acoustic features are expanded according to the training phoneme sequence to obtain a training acoustic feature sequence.

Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

In summary, the present disclosure first obtains the text to be synthesized and the specified acoustic features for indicating the rhythmic features of the audio, then extracts the corresponding phoneme sequence from the text to be synthesized, and then expands the specified acoustic features according to the phoneme sequence to obtain the acoustic feature sequence, and finally inputs the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model, thereby obtaining the target audio that matches the specified acoustic features and corresponds to the text to be synthesized output by the speech synthesis model. The present disclosure controls the speech synthesis of text by specifying acoustic features, so that the target audio output by the speech synthesis model can correspond to the specified acoustic features, and can realize the explicit control of the acoustic features in the speech synthesis process, thereby improving the expressiveness of the target audio.

Referring to FIG. 10 below, it shows a schematic diagram of the structure of an electronic device (i.e., the execution subject of the above-mentioned speech synthesis method) 300 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG. 10 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.

As shown in FIG10 , the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to the bus 304.

Typically, the following devices may be connected to the I/O interface 305: input devices 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication device 309 may allow the electronic device 300 to communicate wirelessly or wired with other devices to exchange data. Although FIG. 10 shows an electronic device 300 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the terminal devices and servers may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the text to be synthesized and the specified acoustic features, wherein the specified acoustic features are used to indicate the rhythmic features of the audio; extracts the phoneme sequence corresponding to the text to be synthesized; expands the specified acoustic features according to the phoneme sequence to obtain an acoustic feature sequence; inputs the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model to obtain the target audio corresponding to the text to be synthesized output by the speech synthesis model, and the acoustic features of the target audio match the specified acoustic features.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, an acquisition module may also be described as a "module for acquiring text to be synthesized and specifying acoustic features".

The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, Example 1 provides a speech synthesis method, including: obtaining a text to be synthesized and a specified acoustic feature, wherein the specified acoustic feature is used to indicate the rhythmic feature of the audio; extracting a phoneme sequence corresponding to the text to be synthesized; expanding the specified acoustic feature according to the phoneme sequence to obtain an acoustic feature sequence; inputting the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model to obtain a target audio corresponding to the text to be synthesized output by the speech synthesis model, wherein the acoustic feature of the target audio matches the specified acoustic feature.

According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, which expands the specified acoustic features according to the phoneme sequence to obtain an acoustic feature sequence, including: determining the acoustic features corresponding to each phoneme in the phoneme sequence according to the specified acoustic features; and composing the acoustic features corresponding to each phoneme into the acoustic feature sequence.

According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 1, wherein the speech synthesis model is used to: determine a text feature sequence corresponding to the text to be synthesized based on the phoneme sequence, wherein the text feature sequence includes text features corresponding to each phoneme in the phoneme sequence; and generate the target audio based on the text feature sequence and the acoustic feature sequence.

According to one or more embodiments of the present disclosure, Example 4 provides the method of Examples 1 to 3, wherein the specified acoustic feature includes at least one of fundamental frequency, volume, and speech rate.

According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 1, wherein the speech synthesis model is obtained by training in the following manner: extracting real acoustic features of training audio corresponding to a training text, wherein the real acoustic features are used to indicate the rhythmic features of the training audio; expanding the real acoustic features according to a training phoneme sequence corresponding to the training text to obtain a training acoustic feature sequence; inputting the training phoneme sequence and the training acoustic feature sequence into the speech synthesis model, and training the speech synthesis model based on the output of the speech synthesis model and the training audio.

According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 5, wherein the real acoustic features include: at least one of fundamental frequency, volume, and speaking rate; the extracting the real acoustic features of the training audio corresponding to the training text includes: if the real acoustic features include speaking rate, determining the duration corresponding to each training phoneme in the training phoneme sequence according to the training audio and the training phoneme sequence to determine the speaking rate of the training audio; if the real acoustic features include fundamental frequency, extracting the fundamental frequency of each audio frame included in the training audio to determine the fundamental frequency of the training audio; if the real acoustic features include volume, extracting the volume of each audio frame included in the training audio to determine the volume of the training audio.

According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 6, wherein the method of determining the duration corresponding to each training phoneme in the training phoneme sequence according to the training audio and the training phoneme sequence to determine the speaking speed of the training audio comprises: determining the duration corresponding to each training phoneme according to the training audio and the training phoneme sequence; performing a logarithmic operation on the duration corresponding to each training phoneme to obtain the logarithmic duration corresponding to each training phoneme; using the statistical value of the logarithmic duration corresponding to each training phoneme in the training phoneme sequence as the speaking speed of the training audio; extracting the training audio comprises The method comprises: performing a logarithmic operation on the fundamental frequency corresponding to each of the audio frames to obtain the logarithmic fundamental frequency corresponding to each of the audio frames; using the statistical value of the logarithmic fundamental frequency corresponding to each of the audio frames in the training audio as the fundamental frequency of the training audio; extracting the volume of each audio frame included in the training audio to determine the volume of the training audio comprises: performing a logarithmic operation on the volume corresponding to each of the audio frames to obtain the logarithmic volume corresponding to each of the audio frames; using the statistical value of the logarithmic volume corresponding to each of the audio frames in the training audio as the volume of the training audio.

According to one or more embodiments of the present disclosure, Example 8 provides the methods of Examples 5 to 7, and the speech synthesis model is also trained in the following manner: according to the real acoustic features of multiple training audios included in a preset training set, the statistical acoustic features of the training set are determined; according to the statistical acoustic features, the real acoustic features of each training audio are normalized; and the real acoustic features are expanded according to the training phoneme sequence corresponding to the training text to obtain the training acoustic feature sequence, including: the normalized real acoustic features are expanded according to the training phoneme sequence to obtain the training acoustic feature sequence.

According to one or more embodiments of the present disclosure, Example 9 provides a speech synthesis device, including: an acquisition module, used to acquire a text to be synthesized and a specified acoustic feature, wherein the specified acoustic feature is used to indicate the rhythmic feature of the audio; an extraction module, used to extract a phoneme sequence corresponding to the text to be synthesized; an expansion module, used to expand the specified acoustic feature according to the phoneme sequence to obtain an acoustic feature sequence; and a synthesis module, used to input the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model to obtain a target audio corresponding to the text to be synthesized output by the speech synthesis model, wherein the acoustic feature of the target audio matches the specified acoustic feature.

According to one or more embodiments of the present disclosure, Example 10 provides a computer-readable medium having a computer program stored thereon, which implements the steps of the methods described in Examples 1 to 8 when executed by a processing device.

According to one or more embodiments of the present disclosure, Example 11 provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of the method described in Examples 1 to 8.

The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

Although the subject matter has been described in language specific to structural features and/or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims (9)

1. A speech synthesis method, characterized in that the method comprises: Acquire a text to be synthesized and a specified acoustic feature, where the specified acoustic feature is used to indicate a prosodic feature of the audio; Extracting a phoneme sequence corresponding to the text to be synthesized; Expanding the specified acoustic feature according to the phoneme sequence to obtain an acoustic feature sequence; Inputting the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model to obtain a target audio corresponding to the text to be synthesized output by the speech synthesis model, wherein the acoustic feature of the target audio matches the specified acoustic feature; The speech synthesis model is obtained by training in the following way: Extracting real acoustic features of the training audio corresponding to the training text, wherein the real acoustic features are used to indicate the prosodic features of the training audio; Expanding the real acoustic features according to the training phoneme sequence corresponding to the training text to obtain a training acoustic feature sequence; Inputting the training phoneme sequence and the training acoustic feature sequence into the speech synthesis model, and training the speech synthesis model according to the output of the speech synthesis model and the training audio; The speech synthesis model is also trained in the following way: Determine the statistical acoustic features of the training set according to the real acoustic features of the plurality of training audios in the preset training set; Normalizing the real acoustic features of each of the training audios according to the statistical acoustic features; The step of expanding the real acoustic features according to the training phoneme sequence corresponding to the training text to obtain a training acoustic feature sequence includes: The normalized real acoustic features are expanded according to the training phoneme sequence to obtain the training acoustic feature sequence. 2. The method according to claim 1, characterized in that the step of expanding the specified acoustic feature according to the phoneme sequence to obtain the acoustic feature sequence comprises: Determining, according to the specified acoustic feature, an acoustic feature corresponding to each phoneme in the phoneme sequence; The acoustic features corresponding to each of the phonemes are combined into the acoustic feature sequence. 3. The method according to claim 1, wherein the speech synthesis model is used for: Determine a text feature sequence corresponding to the to-be-synthesized text according to the phoneme sequence, wherein the text feature sequence includes a text feature corresponding to each phoneme in the phoneme sequence; The target audio is generated according to the text feature sequence and the acoustic feature sequence. 4. The method according to any one of claims 1-3 is characterized in that the specified acoustic features include: at least one of fundamental frequency, volume, and speaking rate. 5. The method according to claim 1, characterized in that the real acoustic features include: at least one of fundamental frequency, volume, and speech rate; the real acoustic features of the training audio corresponding to the training text are extracted, comprising: If the real acoustic feature includes speech speed, determining the duration corresponding to each training phoneme in the training phoneme sequence according to the training audio and the training phoneme sequence, so as to determine the speech speed of the training audio; If the real acoustic feature includes a fundamental frequency, extracting the fundamental frequency of each audio frame included in the training audio to determine the fundamental frequency of the training audio; If the real acoustic feature includes volume, the volume of each audio frame included in the training audio is extracted to determine the volume of the training audio. 6. The method according to claim 5, characterized in that the step of determining, based on the training audio and the training phoneme sequence, the duration corresponding to each training phoneme in the training phoneme sequence to determine the speaking speed of the training audio comprises: Determine the duration corresponding to each training phoneme according to the training audio and the training phoneme sequence; Performing a logarithmic operation on the duration corresponding to each of the training phonemes to obtain the logarithmic duration corresponding to each of the training phonemes; Taking the statistical value of the logarithmic duration corresponding to each training phoneme in the training phoneme sequence as the speaking speed of the training audio; The extracting the fundamental frequency of each audio frame included in the training audio to determine the fundamental frequency of the training audio comprises: Performing a logarithmic operation on the fundamental frequency corresponding to each of the audio frames to obtain the logarithmic fundamental frequency corresponding to each of the audio frames; Using the statistical value of the logarithmic fundamental frequency corresponding to each of the audio frames in the training audio as the fundamental frequency of the training audio; The extracting the volume of each audio frame included in the training audio to determine the volume of the training audio includes: Performing a logarithmic operation on the volume corresponding to each of the audio frames to obtain a logarithmic volume corresponding to each of the audio frames; The statistical value of the logarithmic volume corresponding to each of the audio frames in the training audio is used as the volume of the training audio. 7. A speech synthesis device, characterized in that the device comprises: An acquisition module, used for acquiring the text to be synthesized and a specified acoustic feature, wherein the specified acoustic feature is used for indicating the prosodic feature of the audio; An extraction module, used for extracting a phoneme sequence corresponding to the text to be synthesized; An expansion module, used for expanding the specified acoustic feature according to the phoneme sequence to obtain an acoustic feature sequence; A synthesis module, used for inputting the phoneme sequence and the acoustic feature sequence into a pre-trained speech synthesis model to obtain a target audio corresponding to the text to be synthesized output by the speech synthesis model, wherein the acoustic feature of the target audio matches the specified acoustic feature; The speech synthesis model is obtained by training in the following way: Extracting real acoustic features of the training audio corresponding to the training text, wherein the real acoustic features are used to indicate the prosodic features of the training audio; Expanding the real acoustic features according to the training phoneme sequence corresponding to the training text to obtain a training acoustic feature sequence; Inputting the training phoneme sequence and the training acoustic feature sequence into the speech synthesis model, and training the speech synthesis model according to the output of the speech synthesis model and the training audio; The speech synthesis model is also trained in the following way: Determine the statistical acoustic features of the training set according to the real acoustic features of the plurality of training audios in the preset training set; Normalizing the real acoustic features of each of the training audios according to the statistical acoustic features; The step of expanding the real acoustic features according to the training phoneme sequence corresponding to the training text to obtain a training acoustic feature sequence includes: The normalized real acoustic features are expanded according to the training phoneme sequence to obtain the training acoustic feature sequence. 8. A computer-readable medium having a computer program stored thereon, characterized in that when the program is executed by a processing device, the steps of the method according to any one of claims 1 to 6 are implemented. 9. An electronic device, comprising: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 6.