CN110910904A

CN110910904A - Method for establishing voice emotion recognition model and voice emotion recognition method

Info

Publication number: CN110910904A
Application number: CN201911355782.XA
Authority: CN
Inventors: 王磊
Original assignee: Zhejiang Baiying Technology Co Ltd
Current assignee: Zhejiang Baiying Technology Co Ltd
Priority date: 2019-12-25
Filing date: 2019-12-25
Publication date: 2020-03-24

Abstract

The invention discloses a method for establishing a speech emotion recognition model, which comprises the following steps: extracting 1584-dimensional emotional acoustic features by using opensimile; generating a UBM general background model by using emotional acoustic characteristics as voice training of a natural emotional state; generating a GMM (Gaussian mixture model) corresponding to each type of natural emotion state on the basis of the UBM general background model aiming at each type of natural emotion state; and taking the GMM corresponding to each type of natural emotion state as a speech emotion recognition model.

Description

Method for establishing voice emotion recognition model and voice emotion recognition method

Technical Field

The invention relates to the field of voice signal processing, in particular to a method for establishing a voice emotion recognition model and a voice emotion recognition method.

Background

With the development of artificial intelligence technology, computers become more and more intelligent, and people like classmates, apple siri and more intelligent dialogue systems enter the lives of people, and people can communicate and interact with various machines through the intelligent dialogue systems, but almost all the intelligent dialogue systems cannot identify the emotion of people and make feedback with enough intelligence. The emotion information in the voice is a very important behavior signal reflecting human emotion, and meanwhile, the recognition of the emotion information contained in the voice is an important part for realizing natural human-computer interaction. The existing face recognition technology can judge and analyze human expressions, but cannot be applied to intelligent outbound scenes, and emotion recognition in the existing intelligent outbound field is also a blank field.

Disclosure of Invention

The invention aims to solve the technical problem of providing a method for establishing a speech emotion recognition model and a speech emotion recognition method, so as to realize the recognition of emotion carried by speech in the field of intelligent outbound and improve the semantic understanding precision.

In a first aspect, the present invention provides a method for establishing a speech emotion recognition model, the method comprising:

extracting 1584-dimensional emotional acoustic features by using opensimile;

generating a UBM general background model by using the emotional acoustic characteristics as the voice training of the natural emotional state;

generating a GMM (Gaussian mixture model) corresponding to each type of natural emotion state on the basis of the UBM general background model aiming at each type of natural emotion state;

and taking the GMM corresponding to each type of natural emotion state as a speech emotion recognition model.

In the above scheme, the natural emotional state includes happiness, sadness, anger, and neutrality.

In a second aspect, the present invention provides a speech emotion recognition method based on any one of the first aspect, including:

acquiring a voice file, performing VAD (voice activity detection) pretreatment on the voice file, dividing the voice file into at least one audio according to a mute part, and simultaneously converting the at least one audio into acoustic characteristics;

inputting the acoustic features into a speech emotion recognition model to obtain GMM super vectors;

the GMM supervectors are used as embedding of the at least one audio and input into an XGboost model to obtain a label of the at least one audio;

voting is carried out on the tags of the at least one audio by adopting a Bagging algorithm to obtain a voting result of the tags of the at least one audio;

and taking the voting result occupying most of the numbers as a final output result, and marking the emotion of the voice file.

In the above aspect, the method further includes: and cutting off the mute part of the voice file on a time frequency domain to obtain at least one piece of audio after segmentation.

The invention has the beneficial effects that: based on the technical scheme, the embodiment of the invention provides a method for establishing a speech emotion recognition model and a speech emotion recognition method, wherein a speech emotion recognition model based on GMM-UBM can be trained only by a small amount of linguistic data, so that correct recognition of speech emotion of a user is realized; meanwhile, the model can realize real-time identification, is used on line and can realize millisecond response; secondly, the model is low in maintenance cost, and recognition of various speech emotions can be achieved only by collecting badcase retraining models.

Drawings

FIG. 1 is a schematic flow chart of a method for establishing a speech emotion recognition model according to the present invention;

FIG. 2 is a schematic flow chart of a speech emotion recognition method provided by the present invention.

Detailed Description

The technical solutions of the present invention are further described in detail below with reference to specific embodiments, and it is obvious that the described embodiments are only a part of embodiments of the present invention, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The embodiment of the invention specifically describes a method for establishing a speech emotion recognition model and a speech emotion recognition method provided by the invention.

Referring to fig. 1, as shown in fig. 1, a flow chart of a method for establishing a speech emotion recognition model provided by the present invention is schematically shown, including:

s101, extracting 1584-dimensional emotional acoustic features by using opensimile;

opensmile (open source media large feature space extraction toolkit) is a modular and flexible feature extractor, an emotional feature set extracted based on Opensmile comprises 1582-dimensional emotional acoustic features, 34 low-level descriptors (LLDs) and 34 corresponding differences are used as 68 LLD contour values, 21 functions are applied on the basis to obtain 1428 emotional acoustic features, in addition, 19 functions are applied on 4 pitch-based LLDs and 4 delta coefficients thereof to obtain 152 emotional acoustic features, and finally, the number of additional pitches (pseudo syllables) and the duration of total input (2 features) are added.

Specifically, the extracted Low Level Descriptor (LLD) includes:

fundamental frequency characteristics including mean, variance, difference, smooth profile curve and the like; root mean square signal energy characteristics and their smooth contours; mel-frequency cepstral coefficients MFCC; linear Prediction Coefficients (LPC); differential frame-to-frame jitter (jitter).

It can be understood that 1584-dimensional emotional acoustic features are extracted as training data based on Opensmile (open source media large feature space extraction toolkit) to cover voices of various environments to train the UBM general background model.

S102, generating a UBM general background model by using the emotional acoustic characteristics as the voice training of the natural emotional state;

it can be understood that, in the case of lacking emotion corpus, the data cannot effectively describe the probability distribution of the speaker emotion feature space, so a UBM general background model is introduced to adaptively obtain an emotion recognition model corresponding to each emotion, and the training data of the UBM general background model is 1584-dimensional emotion acoustic features extracted based on Opensmile (open source media large feature space extraction toolkit).

In one example, an EM algorithm is used to train a UBM general background model, where the UBM general background model is a gaussian mixture model with a mixture order of M, and the UBM general background model uses a model parameter λ ═ { ω ═ ω_i，μ_i，∑_iI is 1,2 … m.

S103, generating a GMM (Gaussian mixture model) corresponding to each type of natural emotion state on the basis of the UBM general background model aiming at each type of natural emotion state;

it will be appreciated that each of the above categories of natural emotional states includes happiness, sadness, anger, and neutrality, or further includes either or both of fear and surprise.

In one example, according to a given UBM general background model and the training vectors of the four natural emotion states, performing parameter fine adjustment on the UBM general background model to determine a GMM model, wherein the probability distribution of the training vectors in the UBM general background model is calculated; fully counting by utilizing the probability distribution and the mixed weighted value of the training vector, the average vector and the variance; and finally, using the new sufficient statistics to update the sufficient statistics of the UBM general background model to obtain the GMM model.

And S104, taking the GMM corresponding to each generated natural emotion state as a speech emotion recognition model.

Referring to fig. 2, as shown in fig. 2, a flow chart of a speech emotion recognition method based on a method for establishing a speech emotion recognition model provided by the present invention is schematically shown, and the flow chart includes:

s201, obtaining a voice file, performing VAD preprocessing on the voice file to obtain at least one audio and converting the audio into acoustic characteristics;

with respect to step S201, the voice file is divided into at least one audio according to the mute section, and at the same time, the at least one audio is converted into an acoustic feature.

In one example, in a time-frequency domain, a mute part of a voice file is cut off, and at least one piece of audio after cutting is obtained.

The time domain of the voice file is used as an X axis, the sound pressure of the voice file is used as a Y axis to establish a two-dimensional coordinate system, because the voice signal is a continuously fluctuating signal, the representation form of a section of voice signal in the two-dimensional coordinate system is that at least one audio frequency and more than or equal to 0 mute sections exist, the more than or equal to 0 mute sections in the voice file are cut off to obtain at least one audio frequency, and the at least one audio frequency is converted into acoustic characteristics.

S202, inputting the acoustic features into a speech emotion recognition model to obtain a GMM super vector;

s203, inputting the GMM super vector serving as the embedding of the at least one audio into an XGboost model to obtain a label of the at least one audio;

specifically, the eXtreme Gradient Boosting (Xgboost) model is a machine learning model for classification and regression, and the main idea is to integrate more weak classifiers (such as decision trees) to realize the function of a strong classifier. That is, the Xgboost model is composed of a plurality of weak classifiers, and a plurality of output results are obtained by inputting one input data to the plurality of weak classifiers, respectively, and the plurality of data results are superimposed to obtain the final output data.

In one example, the GMM supervector is input into the XGBoost model as embedding (mapping of vector space) of at least one piece of audio to obtain a label of the at least one piece of audio.

S204, voting is carried out on the at least one audio label by adopting a Bagging algorithm to obtain a voting result of the at least one audio label;

with respect to step S204, the voting result is that the natural emotional state of the voice includes any one of happiness, sadness, anger and neutrality.

And S205, taking the voting result occupying most of the numbers as a final output result, and marking the emotion of the voice file.

For S205, for example, the happy vote number is 1, the sad vote number is 0, the angry vote number is 0, the neutral vote number is 0, and the happy vote occupying the majority vote number is used as the emotion recognition result of the final voice file, and the voice file is subjected to happy emotion marking.

Based on the technical scheme, the method for establishing the speech emotion recognition model based on the GMM-UBM and the speech emotion recognition method only need a small amount of emotion linguistic data to train the UBM general background model, then the GMM model is generated in a self-adaptive mode based on the UBM general background model to serve as the speech emotion recognition model, and meanwhile real-time speech emotion recognition is achieved based on the speech emotion recognition model.

The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, which are within the protection scope of the present invention.

Claims

1. A method for establishing a speech emotion recognition model is characterized by comprising the following steps:

extracting 1584-dimensional emotional acoustic features by using opensimile;

and taking the GMM model corresponding to each type of natural emotion state generated by self-adaptation as a speech emotion recognition model.

2. The method of claim 1, wherein the natural emotional states include happy, sad, angry, and neutral.

3. A speech emotion recognition method based on the method for creating a speech emotion recognition model according to any one of claims 1 to 2, characterized in that the method comprises:

4. The speech emotion recognition method of claim 2, wherein the method further comprises: and cutting off the mute part of the voice file on a time frequency domain to obtain at least one piece of audio after segmentation.