CN112349267B - Synthesized voice detection method based on attention mechanism characteristics - Google Patents

Abstract

The invention relates to the field of synthesized voice and converted voice detection, aiming at improving the robustness of an ASVspoof system for speaker authentication fraud voice detection, strengthening an unvoiced part in voice and weakening a voiced part in the voice characteristic extraction process so as to improve the robustness of the characteristic in fraud attack detection of an automatic speaker system, the invention discloses a synthesized voice detection method based on attention mechanism characteristics, a corresponding weight matrix is trained for each test voice through an attention mechanism algorithm, and whether framed data are unvoiced and voiced can be screened out through the ratio of short-time capacity and zero-crossing rate; then, scoring is carried out on the screened unvoiced data and the original data, so that the data of unvoiced parts are highlighted, and the data of voiced parts are weakened; and respectively training a real Gaussian mixture model and a cheating voice Gaussian mixture model according to the subsequent characteristics, and grading and confirming. The invention is mainly applied to the occasions of synthesizing voice and converting voice.

Description

Synthetic Speech Detection Method Based on Attention Mechanism Features

technical field

The invention relates to the field of synthetic speech and converted speech detection. Based on the different importance of unvoiced and voiced sounds in the task of synthetic speech detection, a designed feature based on attention mechanism can be used to distinguish real speech from fraudulent speech (synthesized speech) and converted speech) show good robustness.

Background technique

In recent years, with the increasing maturity of speaker recognition technology, automatic speaker verification (ASV) systems have been widely used in various scenarios, such as voice assistants, online banking, etc. However, many studies have revealed the vulnerability of speaker verification systems to different spoofing attacks. These spoofing attacks include: synthesized speech and converted speech.

With the development and maturity of speech synthesis and speech conversion technology, the synthesized and converted speech becomes more and more real, and it is even difficult for the human ear to distinguish the authenticity of the speech. This brings great security risks to the speaker recognition system. In order to prevent criminals from committing illegal and criminal acts through speech synthesis or speech conversion technology, the Interspeech competition held three speaker authentication fraudulent speech detection (ASVspoof) challenges from 2015 to 2019, so this field has received more and more attention. People's attention, and thus also put forward a lot of solutions.

This field mainly improves the robustness of speaker authentication fraud speech detection system (ASVspoof) by improving the front-end feature extraction and back-end classifier. At the level of feature extraction, predecessors used Mel-frequency cepstral coefficients (MFCC), linear-frequency cepstral coefficients (LFCC), and constant-Q cepstral coefficients ( CQCC) and so on, while the group delay phase characteristic Modified Group Delay (MGD) is also widely used. The current experimental results show that the phase feature is more robust. In terms of classifiers, early Gaussian MixtureModel (GMM) and Support Vector Machine (SVM) are widely used. Currently the best classifier is the Residual Convolutional Neural Network Model (ResNet) model.

In addition to features and classifiers, many researchers have analyzed the impact of noise and reverberation on synthetic speech detection, the impact of different phonemes on synthetic speech detection, and the research and analysis of speakers and their speech content, etc.

Contents of the invention

In order to overcome the deficiencies in the prior art, the purpose of the present invention is to improve the robustness of the speaker authentication fraud speech detection ASVspoof system by proposing a novel feature. An attention mechanism proposed by the present invention, which strengthens the unvoiced part of the speech and weakens the voiced part of the speech in the process of speech feature extraction, thereby improving the robustness of this feature in the fraud attack detection of automatic speaker systems sex. For this reason, the technical solution that the present invention takes is, based on the synthetic speech detection method of attention mechanism feature, trains the corresponding weight matrix for each trial speech by attention mechanism algorithm, can both be able to by the ratio of short-term ability and zero-crossing rate Filter out whether the framed data is unvoiced or not; then score the filtered unvoiced data and the original data, so as to highlight the data of the unvoiced part and weaken the data of the voiced part; then train the real Gaussian mixture of the subsequent features Model and fraudulent speech Gaussian mixture model, and confirm the score.

Specific steps are as follows:

Step 1, data preparation:

First, divide the data in the corpus into training set, validation set, and test set. The training set is used to train the model, the validation set is used to detect whether the model is trained well, and the test set is used to verify the robustness of the model;

Step 2, speech signal processing:

Pre-emphasize, frame, and window the speech signal, and obtain the spectrogram information of the speech signal through Fourier transform. The pre-emphasis of the speech signal is to emphasize the high-frequency part of the speech and enhance the resolution of the high-frequency part; The purpose of framing is to obtain a stable signal, so as to meet the requirements of Fourier transform;

Step 3, weight matrix training:

Firstly, the unvoiced frame in the speech signal is extracted by short-term energy and zero-crossing rate, and then the Euclidean distance is used to calculate the similarity between each frame in the speech signal and the extracted unvoiced frame. If the unvoiced frame in the original speech is calculated and extracted The similarity of the unvoiced frame extracted, the value of the corresponding weight matrix will be relatively large, if the similarity between the voiced frame in the original speech and the extracted unvoiced frame is calculated, the value of the corresponding weight matrix will be relatively small; finally The final attention feature is obtained by dot multiplying the obtained weight matrix with the matrix represented by the spectrogram of the original speech;

Step 4, classification model training

The spectral features of the real data and the spectral features of the fraudulent data in the training set are used as the input data of the Gaussian mixture model GMM model, and the GMM model of the real speech and the GMM model of the fraudulent speech are trained respectively.

Step 5, authenticity identification and scoring

Input the spectral features extracted from the development set data into the trained model, and use the maximum likelihood ratio to score to judge the authenticity of the speech signal.

The detailed steps of weight matrix training are as follows:

1) Extract unvoiced frames: Use the ratio of short-term energy and zero-crossing rate to judge unvoiced frames and voiced frames.

The short-term energy calculation formula of each frame of speech signal is as follows:

Among them, x(m) is the framed signal, w(m) is the window function, and the zero-crossing rate of each frame is calculated as follows:

Among them, sgn(n) is a symbolic function:

Calculate the ratio EZR of short-term energy and zero-crossing rate. When EZR is greater than 0.02, the frame signal is a voiced signal. When EZR is greater than 0.002 and less than 0.02, the frame signal is an unvoiced signal, otherwise it is mute;

2) Calculate the weight matrix:

By calculating the similarity between each frame of the original speech signal and the unvoiced frame signal extracted in the previous step, the corresponding weight in the weight matrix is obtained. The calculation formula is as follows:

Wherein S represents the original speech frame signal, U represents the unvoiced frame signal extracted from the original speech, m is the number of original speech signal frames, and n represents the number of unvoiced segment frames;

3) Dynamic feature extraction:

On the basis of the static features, the delta method is used to extract the features of the first-order dynamics and the second-order dynamics. The formula is as follows, where p=2.

Features and beneficial effects of the present invention are:

The present invention uses the attention mechanism to strengthen the unvoiced part of the speech signal and weaken the voiced part, thereby extracting the features based on the unvoiced and voiced-oriented attention mechanism, which is based on the attention mechanism feature (LFCC-USE) compared to the baseline system Features—Linear cepstral coefficient feature LFCC (baseline) EER increased by 41.4%, and t-DCF increased by 40.2%.

The results show that the method achieves a good voiceprint recognition effect. The equal error rate and tandem evaluation of the detection cost function are somewhat reduced. This suggests that features based on the unvoiced-directed attention mechanism can make the final trained model more robust.

Description of drawings:

Fig. 1 is a flow chart of extracting unvoiced segments in speech signals.

Figure 2 Schematic diagram of attention weight matrix extraction.

Detailed ways

The purpose of the present invention is to improve the robustness of the ASVspoof system by proposing a novel feature. GajanSuthokumar et al. found that in the ASVspoof2017 corpus, unvoiced sounds carry more discriminative information than voiced sounds. Therefore, this paper proposes an attention mechanism, which strengthens the unvoiced part of the speech and weakens the voiced part of the speech during the process of speech feature extraction, thereby improving the robustness of this feature in automatic speaker system fraud attack detection. Stickiness.

The technical solution that realizes the object of the present invention is:

Feature extraction method based on attention mechanism. Through the attention mechanism algorithm, the corresponding weight matrix is trained for each voice in the ASVspoof2019 corpus. Since unvoiced sounds have low short-term energy and high zero-crossing rate, and voiced sounds have high short-term energy and low zero-crossing rate, the ratio of short-term power and zero-crossing rate can be used to screen whether the framed data is unvoiced or voiced. Then, the screened unvoiced data and the original data are scored, so as to highlight the data of the unvoiced part and weaken the data of the voiced part. Then train the real Gaussian mixture model and the Gaussian mixture model such as fraudulent voice with the following features, and perform scoring confirmation. This experiment mainly includes the following five parts: data preparation, unvoiced sound feature extraction, feature extraction, classification model training, authenticity identification and scoring.

The present invention proposes a feature extraction method based on attention mechanism, comprising the following steps:

Step 1, data preparation:

First, the data in the ASVspoof2019 corpus is divided into training set, verification set, and test set. The training set is used to train the model, the verification set is used to check the quality of the model training, and the test set is used to verify the robustness of the model.

Step 2, speech signal processing:

Perform pre-emphasis, framing, windowing and other processing on the speech signal, and obtain the spectrogram information of the speech signal through Fourier transform. Speech signal pre-emphasis is to emphasize the high-frequency part of the speech and enhance the resolution of the high-frequency part. The purpose of framing is to obtain a stable signal, so as to meet the requirements of Fourier transform.

Step 3, weight matrix training:

Firstly, the unvoiced frame in the speech signal is extracted by short-term energy and zero-crossing rate, and then the Euclidean distance is used to calculate the similarity between each frame in the speech signal and the extracted unvoiced frame. If the unvoiced frame in the original speech is calculated and extracted The value of the corresponding weight matrix will be relatively large for the similarity of the extracted unvoiced frames. If the similarity between the voiced frames in the original speech and the extracted unvoiced frames is calculated, the value of the corresponding weight matrix will be relatively small. Finally, the obtained weight matrix is multiplied by the matrix represented by the spectrogram of the original speech to obtain the final attention feature.

Step 4, classification model training

The spectral features of the real data and the spectral features of the fraudulent data in the training set are used as the input data of the GMM model, and the GMM model of the real voice and the GMM model of the fraudulent voice are trained respectively.

Step 5, authenticity identification and scoring

Input the spectral features extracted from the development set data into the trained model, and use the maximum likelihood ratio to score to judge the authenticity of the speech signal

Describe below in conjunction with accompanying drawing the synthetic speech detection method based on the unvoiced and voiced sound-guided attention mechanism that the present invention implements, mainly comprise the following steps:

Step 1, data preparation:

The data used in this invention is the Logical Access (LA) database in the ASVspoof2019 challenge. The LA database contains real and fake speech data generated using 17 different Synthesized Speech (TTS) and Converted Speech (VC) systems. These 17 different TTS and VC speech data are from the VCTK database, but there is no overlap with the data in the 2019 database. Six of them were designated as known attacks and the remaining 11 were designated as unknown attacks. The LA database divides all data into training set, test set and evaluation set. The training and development sets contain only known attacks, while the evaluation set contains 2 known attacks and 11 unknown spoofing attacks. Among the 6 known attacks, there are 2 types of VC voice data and 4 types of TTS voice data.

Step 2, Speech Signal Processing

After the data is ready, the speech data is preprocessed first. The purpose of preprocessing is to eliminate problems such as human vocal organs or high-frequency leakage caused by voice collection equipment, which will affect the quality of voice signals. After preprocessing, the signal is more uniform and smooth, and the quality of speech processing is improved.

Preprocessing mainly includes pre-emphasis, framing and windowing. The window function we use here is the Hamming window.

Step 3, weight matrix training

As shown in Figure 1 based on the schematic diagram of judging unvoiced and voiced sounds, it contains three main steps

2) Extracting unvoiced frames: According to the performance characteristics of short-term energy and zero-crossing rate of unvoiced and voiced sounds, unvoiced sounds have low short-term energy and high zero-crossing rate, while voiced sounds have high short-term energy and low zero-crossing rate. Use the ratio of short-term energy and zero-crossing rate to judge unvoiced frames and voiced frames.

The short-term energy calculation formula of each frame of speech signal is as follows:

Among them, x(m) is a framed signal, and w(m) is a window function. The calculation formula for the zero-crossing rate of each frame is as follows:

Among them, sgn(n) is a symbolic function:

Calculate the ratio of short-term energy to zero-crossing rate (EZR). When EZR is greater than 0.02, the frame signal is a voiced signal. When EZR is greater than 0.002 and less than 0.02, the frame signal is an unvoiced signal, otherwise it is silent. The extraction process is shown in Figure 1.

2) Calculate the weight matrix:

The calculation method of the weight matrix draws on the weight matrix calculation method of the self-attention mechanism commonly used in the field of natural speech processing. By calculating the similarity between each frame of the original speech signal and the unvoiced frame signal extracted in the previous step, the corresponding weights in the weight matrix are obtained. The extraction process is shown in Figure 2, and the calculation formula is as follows:

Among them, S represents the original speech frame signal, U represents the unvoiced frame signal extracted from the original speech, m is the number of original speech signal frames, and n represents the number of unvoiced segment frames.

The resulting weight matrix will be

3) Dynamic feature extraction:

On the basis of the static features, the delta method is used to extract the features of the first-order dynamics and the second-order dynamics. The formula is as follows, where p=2.

Step 4, classification model training:

The features extracted in the three steps are sent to the GMM model for training. Since the real speech and the fraudulent speech have been marked in the training set, a GMM model of the real speech and a GMM model of the fraudulent speech can be obtained.

Step 5, authenticity identification and scoring:

After feature extraction, the evaluation set data is scored with the model coefficients obtained from GMM model training. Here, the maximum likelihood scoring is used. When the score is greater than 0, it is judged as real speech, and when the score is less than 0, it is judged as fraudulent speech.

The results of the experiments were evaluated using the equal error rate (EER) and the detection cost function (t-DCF) evaluated in tandem.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (2)

1. A synthetic voice detection method based on attention mechanism features is characterized in that a corresponding weight matrix is trained for each test voice through an attention mechanism algorithm, and whether framed data are unvoiced and voiced is screened out through the ratio of short-time energy to zero-crossing rate; then, scoring is carried out on the screened unvoiced data and the original data, so that the data of unvoiced parts are highlighted, and the data of voiced parts are weakened; respectively training a real Gaussian mixture model and a fraud voice Gaussian mixture model for the subsequent characteristics, and performing scoring confirmation, wherein a corresponding weight matrix is trained: firstly, extracting unvoiced frames in a voice signal through short-time energy and a zero crossing rate, secondly, calculating the similarity between each frame in the voice signal and the extracted unvoiced frame by using Euclidean distance, wherein if the similarity between the unvoiced frame in the original voice and the extracted unvoiced frame is calculated, the value of a corresponding weight matrix is larger, and if the similarity between the voiced frame in the original voice and the extracted unvoiced frame is calculated, the value of the corresponding weight matrix is smaller; finally, performing dot product operation on the obtained weight matrix and a matrix represented by a spectrogram of the original voice to obtain the final attention feature;

the detailed steps of the weight matrix training are as follows:

1) And (3) extracting an unvoiced frame: the ratio of the short-time energy to the zero crossing rate is used for judging the unvoiced frame and the voiced frame, and the short-time energy calculation formula of each frame of voice signal is as follows:

wherein x (m) is a framing signal, w (m) is a window function, and the zero-crossing rate of each frame is calculated according to the following formula:

wherein sgn (n) is a sign function:

calculating the ratio EZR of the short-time energy to the zero crossing rate, wherein when the EZR is larger than 0.02, the frame signal is a voiced signal, when the EZR is larger than 0.002 and smaller than 0.02, the frame signal is an unvoiced signal, and otherwise, the frame signal is muted;

2) Calculating a weight matrix:

the similarity between each frame of voice signal of the original voice and the unvoiced frame signal extracted in the previous step is calculated to obtain a corresponding weight in the weight matrix, and the calculation formula is as follows:

wherein S represents an original speech frame signal, U represents an unvoiced sound frame signal extracted from an original speech, m is the number of original speech signal frames, and n represents the number of unvoiced sound segment frames;

3) Dynamic feature extraction:

and performing feature extraction on the first-order dynamic state and the second-order dynamic state by adopting a delta method on the basis of the static features.

2. The attention mechanism feature-based synthesized speech detection method according to claim 1, comprising the steps of:

step one, data preparation

Firstly, dividing data in a corpus into a training set, a verification set and a test set, wherein the training set is used for training a model, the verification set is used for detecting the quality of model training, and the test set is used for verifying the robustness of the model;

step two, processing the voice signal

Carrying out pre-emphasis, framing and windowing processing on the voice signal, and obtaining spectrogram information of the voice signal through Fourier transformation;

step three, training the weight matrix

Step four, training classification models

Respectively training the speech spectrum characteristics of the real data and the speech spectrum characteristics of the fraud data in the training set as input data of a Gaussian Mixture Model (GMM), so as to obtain a GMM of real speech and a GMM of fraud speech;

step five, identifying and scoring true and false

And inputting the speech spectrum characteristics obtained by extracting the development set data into a trained model, and scoring by using the maximum likelihood ratio to judge the authenticity of the speech signal.

Images