KR100784456B1 - Voice Enhancement System using GMM - Google Patents

Abstract

Conventional single channel speech enhancement methods assume the input signal of a certain period as the noise section and estimate the frequency characteristics of the noise signal from it. Then, by subtracting the frequency energy value of the noise signal from the input speech signal, an improved output signal of sound quality is obtained. This single channel sound quality improvement method is superior in stationary noise environment where the frequency characteristic of noise does not change with time, but its performance is improved in dynamic noise environment where the frequency characteristic of noise varies with time. It has a disadvantage that is not excellent.

The sound quality improvement system of the present invention estimates the frequency characteristics of dynamic noise using a Gaussian Mixture Model (GMM) and applies it to a single channel speech enhancement method to improve single channel sound quality in a dynamic noise environment. Improve the performance of the method.

Dynamic Noise Cancellation, Single Channel Sound Quality Enhancement

Description

Sound Enhancement System using GMM

1 is a block diagram illustrating a single channel sound quality enhancement system according to the prior art.

Figure 2 is a block diagram showing a single channel sound quality improvement system according to an embodiment of the present invention.

* Explanation of symbols for main parts of the drawings

210: frequency spectrum estimation unit 220: feature extraction unit

230: static noise estimator 240: dynamic noise estimator

250: noise characteristic estimation unit 260: filter frequency characteristic control unit

270: sound quality improvement filter unit 280: dynamic noise GMM model

290: Voice GMM Model

The present invention relates to a single channel sound quality improvement system for removing noise added to an input signal to improve the sound quality of the input signal.

Conventional single channel sound quality enhancement techniques assume an additional noise signal during an initial predetermined period of an input signal, estimate a frequency spectrum characteristic of the additional noise signal, and remove the additional noise from the speech signal to obtain a voice signal with improved sound quality.

상기 수학식 1은 음성 신호가 존재하지 않을때 잡음 스팩트럼상 콤포넌트(Yk)의 확률을 나타내며, 상기 수학식 2는 음성 신호가 존재할 때 잡음 스팩트럼상 콤포넌트(Yk)의 확률을 나타낸다. 즉, 상기 두 수식은 각각 음성이 존재하지 않는 경우 혹은 존재하는 경우에 대한 입력 신호의 확률분포를 나타낸다.
k번째 스펙트럼빈(spectral bin)의 개연성비(likelihood ratio, LR)은 상기 두 수식의 확률분포들로부터 하기 수학식 3과 같이 정의 된다.

여기서,

와

는 a posteriori와 a prior를 각각 나타내며 하기 수학식 4 및 5와 같다.

n 번째 프레임의 a prior(SNR)는 다음과 같은 디시젼-다이렉티드(decision-directed, DD) 방법에 의해 구할 수 있다.

여기서 잡음이 부가되지 않은 음성신호의 주파수 크기(

)는 단기-진폭-평가기(the short-time amplitude estimator)의 최소 민-스퀘어 에러(the minimum mean square error, MSE) 방법에 의해 추정할 수 있다. 이렇게 구한 개연성비(LR)로부터 평탄화한 개연성비(smoothed likelihood ratio, SLR)를 다음 수학식 7과 같이 구할 수 있다.

이렇게 구한 SLR로부터 Bayes’rule을 이용하여 풀면 음성이 존재하지 않을 확률을 구할 수 있고 이는 다음 수학식 8과 같으며, 음성/비음성 구간 추정부(130)에 의해 수행된다..

이러한 과정을 통하여 구한 음성이 존재하지 않을 확률을 이용하여 다음 수학식 9와 같이 잡음의 주파수 파워를 추정할 수 있다.

위에서 예상되는 잡음 전력 스펙트럼은 다음 수학식 10과 같이 추정된다.

In FIG. 1, the voice / non-voice section estimator 130 defines an additional noise signal during an initial predetermined period of the input signal, and the voice / non-voice estimator 130 estimates the characteristics of the additional noise signal over time. In the case of non-voice interval, the frequency characteristic of the additional noise over time is estimated by adding the frequency characteristic of the additional noise of the current interval at a constant ratio. Thereafter, the filter frequency characteristic controller 160 controls the filtering characteristic of the sound quality improvement filter 170 by using the estimated frequency characteristic of the additional noise.
Prior to explaining the present invention, the frequency spectrum estimation method of noise in the conventional single channel sound quality enhancement method as shown in FIG. 1 will be described. Assuming that the frequency and noise components of noise and speech have a statistical complex Gaussian distribution, the noise is additive and does not intersect with the frequency components of the speech signal, the following equations (1) and (2) The same relationship holds.

Equation 1 represents the probability of the component on the noise spectrum (Yk) in the absence of the speech signal, Equation 2 represents the probability of the component on the noise spectrum (Yk) in the presence of the speech signal. In other words, the above two equations represent the probability distribution of the input signal when there is no voice or when there is a voice.
The likelihood ratio (LR) of the k th spectral bin is defined by Equation 3 below from the probability distributions of the two equations.

here,

Wow

Represents a posteriori and a prior, respectively, and are represented by Equations 4 and 5 below.

The a prior (SNR) of the nth frame can be obtained by the following decision-directed (DD) method.

Where the magnitude of the frequency of the speech signal

) Can be estimated by the minimum mean square error (MSE) method of the short-time amplitude estimator. The smoothed likelihood ratio (SLR) flattened from the probability probability ratio LR thus obtained can be obtained as in Equation 7 below.

Solving using Bayes'rule from the SLR thus obtained, it is possible to obtain the probability that there is no voice, which is expressed by Equation 8 below, and is performed by the voice / non-voice interval estimation unit 130.

The frequency power of the noise can be estimated using Equation 9 by using the probability that the speech obtained through this process does not exist.

The expected noise power spectrum is estimated as in Equation 10 below.

The noise characteristic estimator 150 calculates the frequency power of the input signal without noise by using the MMSE method or the Wiener filtering method using the frequency power of the noise and the frequency power of the input signal estimated through the above process. The frequency characteristics of the sound quality improvement filter 170 may be obtained by using the MMSE or Wiener filtering method using the frequency power of the input signal and the noise power of the noise obtained without the noise. Applying such a sound quality enhancement filter 170 to the input signal it is possible to obtain a voice signal with improved sound quality.
The frequency spectrum estimation of additional noise is superior to the static noise where the characteristics of the additional noise do not change over time, but in the case of dynamic noise where the characteristics of the frequency spectrum change rapidly over time, It has the disadvantage of not being able to track frequency spectrum characteristics.

Such static noise includes refrigerator fan sound, PC fan sound, laptop fan sound, white noise, etc. Dynamic noise includes music from the radio, news, phone ringing, doorbell melody, etc. Additional noises you may encounter. Dynamic noise as well as static noise added to the input signal act as a cause of degrading the sound quality of the input signal.

An object of the present invention is to provide a sound quality improvement system capable of removing dynamic noise, which has been devised to solve the above problems, and which acts as a cause of degrading sound quality of an input signal.

To this end, the present invention estimates the presence or absence of dynamic noise using a Gaussian Mixture Model (GMM), and uses this information to estimate the frequency spectrum characteristics of the dynamic noise to remove it from the input signal, thereby improving sound quality. To provide a further purpose.

The sound quality improvement system of the present invention for achieving the above object, the sound quality enhancement filter unit for adjusting the filtering characteristics in real time to improve the sound quality of the input signal; A frequency spectrum estimator for analyzing an input signal on a frequency band; A static noise estimator for calculating a degree of static noise in an output signal of the frequency spectrum estimator according to a frequency characteristic of a preset static noise; A dynamic noise estimator for calculating a degree of dynamic noise in an output signal of the frequency spectrum estimator according to a dynamic noise GMM model; A noise characteristic estimator for estimating a noise characteristic of an input signal from information on the static noise level and the dynamic noise level; And a filter frequency characteristic control unit for adjusting the filtering characteristic of the sound quality enhancement filter unit according to the noise characteristic.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

(Example)

The conventional single channel enhancement method works well for static noise where the characteristics of the added noise do not change with time, but in the case of dynamic noise where the characteristics of the added noise vary with time, It has the disadvantage of not eliminating dynamic noise. To remove dynamic noise whose characteristics change over time, we need a methodology on how to estimate the dynamic noise and how to estimate the frequency power of the dynamic noise.

As shown in FIG. 2, the sound quality enhancement system of the present embodiment includes a sound quality enhancement filter 270 in which filtering characteristics are adjusted in real time to improve sound quality of an input signal; A frequency spectrum estimator 210 for analyzing the input signal on a frequency band; A static noise estimator 230 for calculating a degree of static noise in the output signal of the frequency spectrum estimator 210 according to a preset frequency characteristic of the static noise; A dynamic noise estimator 240 for calculating a degree of dynamic noise in an output signal of the frequency spectrum estimator according to a dynamic noise GMM model; A noise characteristic estimator 250 for estimating a noise characteristic of an input signal from information about the static noise degree and the dynamic noise degree; And a filter frequency characteristic controller 260 for adjusting the filtering characteristic of the sound quality improving filter 270 according to the noise characteristic. Each component shown may be a hardware module or a software module.

The dynamic noise GMM model 280 may further include a dynamic noise GMM model 280 for the dynamic noise estimation of the dynamic noise estimator 240, and a voice GMM model 290 may be used to estimate the dynamic noise GMM model 280. ) Can be provided together. The apparatus may further include a feature extractor 220 for extracting reference features for comparison with the dynamic noise GMM model in a signal converted by the frequency spectrum estimator 210 into a frequency axis signal. The feature extractor 220 may extract a feature by applying feature vectors which are generally used for speech recognition. For example, the feature extractor 220 may use a frequency gradient feature.

First, each component and signals will be described. The input signal of the figure is a signal for inputting a single channel sound quality enhancement method, and the frequency spectrum estimator 210 converts an input signal using Fourier Transform Power into a signal on a frequency axis. The feature extractor 220 detects an inclination of the frequency axis between the outputs of the mel filter bank constituting the frequency spectrum estimator 210. Here, the mel filter bank refers to a filter bank composed of a plurality of filters of a frequency band obtained by transforming a linear frequency band into a mel frequency band.

In this embodiment, the slope of the frequency power is used as the output of the Mel filter bank used for noise detection. However, the present invention is not limited thereto. In some embodiments, the Mel filter bank 럼 strum (MFCC) and LPC (Linear Prediction Coefficient) Various feature vectors such as strum, perceptually based linear prediction (PLP) columns, RASTA PLP columns, mel filterbank energies, etc. and feature vectors representing changes over time, such as their delta or delta-delta, can be used. Can be.

Dynamic noise GMM model 280 records GMM models for dynamic noise, and voice GMM model 290 records GMM models for noise-free general speech signals.

The corrected noise estimator 230 estimates the speech or non-voice interval from the input signal, and the dynamic noise estimator 240 uses the GMM model of the speech and the GMM model of the dynamic noise to determine whether the characteristic of the input signal is the dynamic noise. In this case, the log likelihood ratio (LLR) value described below is used.

The noise characteristic estimator 250 estimates the spectral power of the static or dynamic noise signal using the speech / non-voice interval estimation information, the dynamic noise interval estimation information, and the frequency spectrum information of the input signal. In this case, the spectral power of the dynamic noise signal is updated only when the voice / non-speech section estimation information by the static noise estimator 230 is voice, and the estimation method will be described later.

The filter frequency characteristic controller 260 adjusts the frequency characteristics of the sound quality enhancement filter by using the frequency spectrum information of the input signal, the frequency spectrum of the static noise signal, and the frequency spectrum of the dynamic noise signal.

The sound quality improvement filter unit 270 filters the input signal using the frequency characteristic determined by the filter frequency characteristic control unit 260 to generate an output signal having improved sound quality.

Hereinafter, a method of determining whether an input signal, which is a feature according to the spirit of the present invention, is dynamic noise, and a method of estimating frequency power of dynamic noise when it is determined as dynamic noise will be described in detail. Both methods are performed by the dynamic noise estimator 240 of FIG.

Assuming that the observation vector of the speech signal with noise is X and H ₀ is the absence of speech, H ₁ represents the presence of speech, the probability of the speech signal observation vector in the absence of speech is expressed by Equation 11 below.

Here, the definition of each variable value is as follows.

μ _{0, i} : Mean of the i-th mixture

Σ _{0, i} : covariance of the i mixture

ω _{o, i} : weight of the ith mixture

N: total number of mixture

On the contrary, in the presence of speech, the probability of the observation vector of the speech signal is expressed by Equation 12 below.

The GMM models used herein are obtained in advance through training from observation vectors of speech signals of a training corpus and observation vectors of noise signals.

The likelihood ratio (LR) Λ between the probability of the observation vector in the presence of voice and the probability of the observation vector in the absence of voice is expressed by Equation 13.

Although it may be implemented to immediately determine whether the input signal is dynamic noise using the probability probability Λ value obtained according to the above equation, it is preferable to use the log probability probability rather than the probability probability Λ value that is too changeable. The log likelihood ratio LLR between the probability of the observation vector in the presence of speech and the probability of the observation vector in the absence of speech is expressed by Equation 14 below.

The LLR value thus obtained is used as a criterion for determining whether or not the input signal is a dynamic noise signal. That is, when the LLR value is lower than the specific threshold, it is determined that the input signal is a dynamic noise signal.

According to the above process, the method of estimating the frequency power of the dynamic noise when it is determined as the dynamic noise is as follows. When the observation vector X of the input signal is observed, the probability that the voice does not exist is expressed by Equation 15 below.

Here, q can be calculated | required from following formula (16).

Here, p (H ₁ ) represents the probability that a frame is negative in the absence of other additional conditions, and p (H ₀ ) represents the probability that it is not negative.

In the case of dynamic noise, the smoothing ratio of the likelihood ratio (LR) between the probability of the observation vector in the presence of speech and the probability of the observation vector in the absence of speech does not change too quickly. ) Process is required. Therefore, the smoothed likelihood ratio (SLR) is obtained by the following equation (17).

(k represents a forgetting factor and can be a value between 0 and 1.)

When the observation vector Χ of the input signal obtained using the SLR obtained here is observed, the probability that the voice does not exist is expressed by Equation 18 below.

The dynamic noise estimator 240 of FIG. 2 includes information for determining whether or not dynamic noise is determined according to the probability probability Λ value of Equation 13, and a value of Ψ ^{(n) according} to Equation 17. 250).

The noise characteristic estimator 250 performs a function of estimating a frequency spectrum of noise power indicating a power characteristic in frequency of noise included in an input signal irrespective of static / dynamic noise types.

The filter frequency characteristic controller 260 receives the frequency spectrum of the noise power and obtains the frequency power of the noise for performing auto-regressive dynamic noise estimation.

In the case of the static noise, as described above, the noise power spectrum is estimated by Equation 10, and the noise frequency power is estimated by Equation 9 applied thereto. On the other hand, in the case of dynamic noise according to the present invention, the noise frequency power is estimated by Equation 19, and the noise power spectrum substituted in this equation is estimated by Equation 20 below.

The expected noise power spectrum is estimated by the following equation (20).

The noise characteristic estimator 250 calculates the frequency power of the input signal without noise by using the MMSE method or the Wiener filtering method using the frequency power of the noise and the frequency power of the input signal estimated through the above process. Using the frequency power of the input signal and the noise power of the noise obtained as described above, the filter frequency characteristic control unit 260 may obtain the frequency characteristic of the sound quality enhancement filter unit 270 using MMSE or Wiener filtering. have. By applying the sound quality enhancement filter unit 270 in which the frequency specification is adjusted to the input signal, it is possible to obtain an audio signal having improved sound quality.

As shown in FIG. 2, the dynamic noise estimator 240 performs dynamic noise estimation, using a dynamic noise GMM model, and can be broadly divided into two methods of modeling various dynamic noises using a GMM. . The first is to model each dynamic noise into several GMMs, and the second is to model several dynamic noises into one GMM.

In the case of modeling by the first method, noise power spectrum estimation, noise frequency power estimation, and dynamic noise suppression evaluation according to Equations 19 and 20 are performed. This uses a method of estimating each noise component for a specific dynamic noise and uses the estimated specific dynamic noise component when a specific dynamic noise occurs.

On the other hand, in the case of modeling in the second method, dynamic noise suppression evaluation is performed by using a moving average as shown in Equation 21 below.

At this time, as shown in Equation 22, a moving average is obtained by excluding frames that are not dynamic noise sections.

The formula is β correction factor. It is also related to the characteristics of dynamic noise obtained and added experimentally. The method of 2 uses a method of estimating the temporal change characteristic of dynamic noise using a moving average.

The present invention has been described in detail with reference to preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications may be made by those skilled in the art within the scope of the technical idea of the present invention. It is possible.

According to the present invention, the dynamic noise cancellation performance, which is a disadvantage of the conventional single channel sound quality enhancement method, is greatly improved, thereby improving the sound quality of the input voice signal in a dynamic noise environment as well as a static noise environment.

Claims (7)

A sound quality improving filter unit configured to adjust filtering characteristics in real time to improve sound quality of an input signal; A frequency spectrum estimator for analyzing an input signal on a frequency band; A static noise estimator for calculating a degree of static noise in an output signal of the frequency spectrum estimator according to a frequency characteristic of a preset static noise; A dynamic noise estimator for calculating a degree of dynamic noise in an output signal of the frequency spectrum estimator according to a dynamic noise GMM model; A noise characteristic estimator for estimating a noise characteristic of an input signal from information on the static noise level and the dynamic noise level; And Filter frequency characteristic control unit for adjusting the filtering characteristics of the sound quality enhancement filter according to the noise characteristics Sound quality enhancement system comprising a. The method of claim 1, The noise characteristic estimating unit estimates a frequency spectrum of noise power according to the following equation.

The method of claim 2, The filter frequency characteristic control unit calculates the frequency power of the noise according to the following equation.

The system of claim 2, wherein the probability values of the equations are calculated according to the following equations.

The system of claim 2, wherein the probability values of the equations are calculated according to the following equations.

The method of claim 1, wherein the dynamic noise estimator, Sound quality improvement system, characterized in that for determining whether the input signal is a dynamic noise from the probability probability (Λ) obtained according to the following equation.

The method of claim 6, wherein the dynamic noise estimator, Sound quality improvement system, characterized in that for determining whether the input signal is a dynamic noise from the log probability probability (Λ) obtained by the following equation.

Images