CN101388215A - A method and device for noise shaping - Google Patents
A method and device for noise shaping Download PDFInfo
- Publication number
- CN101388215A CN101388215A CNA2007101539531A CN200710153953A CN101388215A CN 101388215 A CN101388215 A CN 101388215A CN A2007101539531 A CNA2007101539531 A CN A2007101539531A CN 200710153953 A CN200710153953 A CN 200710153953A CN 101388215 A CN101388215 A CN 101388215A
- Authority
- CN
- China
- Prior art keywords
- signal
- noise shaping
- unit
- speech
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
本发明提供了一种噪声整形的方法和装置,对语音信号s(n)进行自适应的噪声整形处理,得到信号s1(n);对所述信号s1(n)进行固定的噪声整形处理,得到信号u(n);对信号u(n)进行语音编码,形成编码端的输出信号。还提供了一种噪声整形的方法和装置,将语音信号s(n)进行固定的噪声整形处理,得到信号s1′(n);对所述信号s1′(n)进行自适应的噪声整形处理,得到信号u′(n);对所述信号u′(n)进行语音编码,形成编码端的输出信号。由于固定的噪声整形处理和自适应的噪声整形处理均是对语音信号本身所做的噪声整形处理,而不涉及语音信号与预测滤波器的残差信号,所以,在解码端不需要做任何改变即可解码还原出经噪声整形的语音信号。
The present invention provides a method and device for noise shaping, which performs adaptive noise shaping processing on speech signal s(n) to obtain signal s 1 (n); performs fixed noise shaping on said signal s 1 (n) Processing to obtain the signal u(n); voice encoding is performed on the signal u(n) to form an output signal at the encoding end. A noise shaping method and device are also provided, wherein the speech signal s(n) is subjected to fixed noise shaping processing to obtain a signal s 1 ′(n); adaptive noise shaping is performed on the signal s 1 ′(n) Reshaping processing to obtain signal u'(n); voice encoding is performed on said signal u'(n) to form an output signal of the encoding end. Since both the fixed noise shaping process and the adaptive noise shaping process are noise shaping processes on the speech signal itself, and do not involve the speech signal and the residual signal of the prediction filter, there is no need to make any changes at the decoding end The noise-shaped speech signal can be decoded and restored.
Description
技术领域 technical field
本发明涉及信号处理技术,特别涉及一种噪声整形的方法和装置。The invention relates to signal processing technology, in particular to a noise shaping method and device.
背景技术 Background technique
噪声整形是一种利用人耳听觉掩蔽原理对编解码器所产生的量化噪声谱进行整形的技术,即把信号中的噪声谱调整为近似语音谱的形状,利用人耳听觉掩蔽效应,使得信号中的噪声不容易被察觉。通常在进行语音信号传输时,为了减少语音信号传输时占用的带宽,一般将语音信号进行编码后进行传输,然后在接收端进行解码得到该语音信号,为了提高解码后语音信号的主观听觉质量,常常在编码端进行噪声整形处理,然后再将噪声整形处理和编码后的语音信号发送出去。Noise shaping is a technology that uses the principle of human auditory masking to shape the quantized noise spectrum generated by the codec, that is, adjusts the noise spectrum in the signal to approximate the shape of the speech spectrum, and uses the human ear masking effect to make the signal The noise in is not easy to detect. Usually, in the transmission of voice signals, in order to reduce the bandwidth occupied by the voice signal transmission, the voice signal is generally encoded and then transmitted, and then decoded at the receiving end to obtain the voice signal. In order to improve the subjective auditory quality of the decoded voice signal, The noise shaping process is often performed at the encoding end, and then the noise shaping process and the encoded speech signal are sent out.
图1为现有技术中的噪声整形设备的组成结构图,如图1所示,在编码端的处理过程为:首先计算语音信号与其经过预测滤波器后得到预测信号的误差,得到预测残差信号d(n);d(n)与反馈信号fq(n)进行组合计算后得到量化器的输入信号u(n);u(n)经过量化器进行编码处理得到编码后的信号uq(n);进行反馈时,量化器的输入信号u(n)与量化器的输出信号uq(n)进行差值计算,得到差值信号q(n);该差值信号q(n)经过噪声整形滤波器的整形滤波处理后得到反馈信号fq(n)。量化器编码后的输出信号uq(n)经传输后到达解码端,在解码端的处理过程为:由于在编码端输出的信号uq(n)实质上是语音信号与其经过预测滤波器后得到预测信号的误差,经处理后得到的一种残差信号,所以,在解码端需要将接收到的信号uq(n)加上其经预测滤波器得到的预测信号,得到最终还原出的语音信号,该还原出的语音信号便是经过噪声整形后的语音信号。Fig. 1 is the composition structure diagram of the noise shaping equipment in the prior art, as shown in Fig. 1, the processing process at the encoding end is: first calculate the error of the speech signal and its prediction signal obtained after passing through the prediction filter, and obtain the prediction residual signal d(n); d(n) is combined with the feedback signal fq(n) to obtain the input signal u(n) of the quantizer; u(n) is encoded by the quantizer to obtain the encoded signal uq(n) ; When performing feedback, the difference between the input signal u(n) of the quantizer and the output signal uq(n) of the quantizer is calculated to obtain the difference signal q(n); the difference signal q(n) is filtered by noise shaping The feedback signal fq(n) is obtained after shaping and filtering by the filter. The output signal uq(n) encoded by the quantizer reaches the decoding end after transmission, and the processing process at the decoding end is as follows: since the signal uq(n) output at the encoding end is essentially a speech signal and the prediction signal obtained after passing through the prediction filter The error is a residual signal obtained after processing. Therefore, at the decoding end, it is necessary to add the received signal uq(n) to the predicted signal obtained by the predictive filter to obtain the final restored speech signal. The restored speech signal is the speech signal after noise shaping.
但是,现有技术中的这种噪声整形的方法,其在编码端的输出信号并不是语音信号经噪声整形后的编码信号,而是对语音信号和其经预测滤波器处理后的信号相减得到的残差信号进行噪声整形滤波处理后的编码信号,所以,在解码端也必须加入预测滤波器对残差信号进行补偿才能还原出语音信号,也就是说,采用这种噪声整形方法时,同时也需要对原有解码端进行更改。However, in this noise shaping method in the prior art, the output signal at the encoding end is not the encoded signal of the speech signal after noise shaping, but the subtraction of the speech signal and the signal processed by the predictive filter to obtain The coded signal after the residual signal is processed by noise shaping and filtering, so the prediction filter must be added at the decoding end to compensate the residual signal to restore the speech signal, that is to say, when using this noise shaping method, at the same time It is also necessary to modify the original decoder.
发明内容 Contents of the invention
本发明实施例提供了一种噪声整形的方法和装置,使得对语音信号进行噪声整形处理时不需要对原有解码端进行更改。Embodiments of the present invention provide a noise shaping method and device, so that the original decoding end does not need to be modified when performing noise shaping processing on speech signals.
一种噪声整形的方法,该方法包括:A method of noise shaping, the method comprising:
对语音信号s(n)进行自适应的噪声整形处理,得到信号s1(n);Perform adaptive noise shaping processing on the speech signal s(n) to obtain the signal s 1 (n);
对所述信号s1(n)进行固定的噪声整形处理,得到信号u(n);performing fixed noise shaping processing on the signal s 1 (n) to obtain a signal u(n);
对信号u(n)进行语音编码,形成编码端的输出信号。Speech coding is performed on the signal u(n) to form an output signal at the coding end.
一种噪声整形的方法,该方法包括:A method of noise shaping, the method comprising:
将语音信号s(n)进行固定的噪声整形处理,得到信号s1′(n);Perform fixed noise shaping processing on the speech signal s(n) to obtain the signal s 1 '(n);
对所述信号s1′(n)进行自适应的噪声整形处理,得到信号u′(n);performing adaptive noise shaping processing on the signal s 1 '(n) to obtain the signal u'(n);
对所述信号u′(n)进行语音编码,形成编码端的输出信号。Speech coding is performed on the signal u'(n) to form an output signal at the coding end.
一种噪声整形的装置,该装置包括:A noise shaping device, the device comprising:
自适应噪声整形单元,用于对语音信号s(n)进行自适应噪声整形处理,得到信号s1(n);An adaptive noise shaping unit is used to perform adaptive noise shaping processing on the speech signal s(n) to obtain a signal s 1 (n);
固定噪声整形单元,用于对所述自适应噪声整形单元进行自适应噪声整形处理后得到的信号s1(n)进行固定噪声整形处理,得到信号u(n);A fixed noise shaping unit, configured to perform fixed noise shaping processing on the signal s 1 (n) obtained after the adaptive noise shaping unit performs adaptive noise shaping processing, to obtain a signal u(n);
语音编码单元,用于对所述固定噪声整形单元处理后得到的信号u(n)进行语音编码,得到编码输出端的输出信号。The speech encoding unit is configured to perform speech encoding on the signal u(n) obtained after processing by the fixed noise shaping unit, to obtain an output signal at the encoding output terminal.
一种噪声整形的装置,该装置包括:A noise shaping device, the device comprising:
固定噪声整形单元,用于对语音信号s(n)进行固定噪声整形处理,得到信号s1′(n);The fixed noise shaping unit is used to perform fixed noise shaping processing on the speech signal s(n) to obtain the signal s 1 '(n);
自适应噪声整形单元,用于对所述固定噪声整形单元得到的信号s1′(n)进行自适应噪声整形处理,得到信号u′(n);An adaptive noise shaping unit, configured to perform adaptive noise shaping processing on the signal s 1 '(n) obtained by the fixed noise shaping unit to obtain a signal u'(n);
语音编码单元,用于将所述自适应噪声整形单元得到的信号u′(n)进行语音编码,得到编码端的输出信号。The speech encoding unit is configured to perform speech encoding on the signal u′(n) obtained by the adaptive noise shaping unit, to obtain an output signal of the encoding end.
由以上技术方案可以看出,在本发明实施例提供的方法和装置中,对语音信号先进行自适应的噪声整形处理,再进行固定的噪声整形处理,对进行固定噪声整形处理后的信号进行语音编码,形成编码端的输出信号;或者,对语音信号先进行固定的噪声整形处理,再进行自适应的噪声整形处理,对进行自适应噪声整形处理后的信号进行语音编码,形成编码端的输出信号。由于固定的噪声整形处理和自适应的噪声整形处理均是对语音信号本身所做的噪声整形处理,而不涉及语音信号与预测滤波器的残差信号,所以,在解码端不需要做任何改变即可解码还原出经噪声整形的语音信号。It can be seen from the above technical solutions that in the method and device provided by the embodiments of the present invention, the voice signal is first subjected to adaptive noise shaping processing, and then fixed noise shaping processing is performed, and the signal after fixed noise shaping processing is performed Speech coding to form the output signal of the coding end; or, the speech signal is firstly subjected to fixed noise shaping processing, and then adaptive noise shaping processing is performed, and speech coding is performed on the signal after adaptive noise shaping processing to form the output signal of the coding end . Since both the fixed noise shaping process and the adaptive noise shaping process are noise shaping processes on the speech signal itself, and do not involve the speech signal and the residual signal of the prediction filter, there is no need to make any changes at the decoding end The noise-shaped speech signal can be decoded and restored.
附图说明 Description of drawings
图1为现有技术中的噪声整形设备的组成结构图;FIG. 1 is a structural diagram of a noise shaping device in the prior art;
图2为本发明实施例提供的噪声整形装置的第一种组成结构图;FIG. 2 is a structural diagram of the first type of noise shaping device provided by an embodiment of the present invention;
图3为本发明实施例提供的噪声整形装置的第二种组成结构图;Fig. 3 is a second composition structure diagram of the noise shaping device provided by the embodiment of the present invention;
图4为本发明实施例提供的一个具体噪声整形装置的结构示意图;FIG. 4 is a schematic structural diagram of a specific noise shaping device provided by an embodiment of the present invention;
图5为本发明实施例提供的另一个具体噪声整形装置的结构示意图。Fig. 5 is a schematic structural diagram of another specific noise shaping device provided by an embodiment of the present invention.
具体实施方式 Detailed ways
为了使本发明的目的、技术方案和优点更加清楚,下面结合附图和具体实施例对本发明进行详细描述。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.
本发明实施例提供的噪声整形的方法包括:对语音信号s(n)进行固定的噪声整形以及根据该语音信号s(n)进行自适应的噪声整形处理后,形成信号u(n);将该信号u(n)进行语音编码形成编码端的输出信号。The noise shaping method provided by the embodiment of the present invention includes: performing fixed noise shaping on the speech signal s(n) and performing adaptive noise shaping processing according to the speech signal s(n) to form a signal u(n); The signal u(n) is speech coded to form an output signal at the coding end.
其中,对语音信号s(n)进行固定的噪声整形以及根据该语音信号s(n)进行自适应的噪声整形处理的顺序,可以是先进行固定的噪声整形处理,再根据该语音信号s(n)进行自适应的噪声整形处理;也可以是先根据该语音信号s(n)进行自适应的噪声整形处理,再进行固定的噪声整形处理。Wherein, the sequence of performing fixed noise shaping on the speech signal s(n) and performing adaptive noise shaping processing according to the speech signal s(n) may be to perform fixed noise shaping processing first, and then according to the speech signal s(n) n) performing adaptive noise shaping processing; or first performing adaptive noise shaping processing according to the speech signal s(n), and then performing fixed noise shaping processing.
先根据语音信号s(n)进行自适应的噪声整形处理,再进行固定的噪声整形处理的过程可以包括以下过程:The process of first performing adaptive noise shaping processing according to the speech signal s(n), and then performing fixed noise shaping processing may include the following processes:
将语音信号s(n)与第二反馈信号f2q2(n)进行组合后得到信号s1(n),将信号s1(n)与第一反馈信号f1q1(n)进行组合后得到信号u(n);对该信号u(n)进行语音编码后得到编码端的输出信号;将编码端的输出信号经解码后得到的信号uq(n)与信号s1(n)组合后得到信号q1(n),信号q1(n)经第一噪声整形滤波处理后得到反馈信号f1q1(n);将信号uq(n)与语音信号s(n)组合后得到信号q2(n),根据语音信号s(n)对信号q2(n)进行第二噪声整形滤波处理后得到反馈信号f2q2(n)。Combine the voice signal s(n) with the second feedback signal f 2 q 2 (n) to obtain the signal s 1 (n), and combine the signal s 1 (n) with the first feedback signal f 1 q 1 (n) The signal u(n) is obtained after combination; the output signal of the encoding end is obtained after speech encoding the signal u(n); the signal uq(n) obtained after decoding the output signal of the encoding end is combined with the signal s 1 (n) The signal q 1 (n) is obtained, and the signal q 1 (n) is processed by the first noise shaping filter to obtain the feedback signal f 1 q 1 (n); the signal uq(n) is combined with the speech signal s(n) to obtain the signal q 2 (n), the feedback signal f 2 q 2 (n) is obtained after the second noise shaping and filtering process is performed on the signal q 2 (n) according to the voice signal s(n).
在该过程中,根据语音信号s(n)对信号q2(n)进行第二噪声整形滤波处理后得到的反馈信号f2q2(n)与语音信号s(n)进行组合得到信号s1(n),实现了对语音信号进行的自适应噪声整形处理。进一步将信号q1(n)经第一噪声整形滤波处理后得到反馈信号f1q1(n)与s1(n)进行组合得到信号u(n)实现了对语音信号进行的固定噪声整形处理。In this process, the feedback signal f 2 q 2 (n) obtained after performing the second noise shaping filtering process on the signal q 2 (n) according to the speech signal s(n) is combined with the speech signal s(n) to obtain the signal s 1 (n), realizes the adaptive noise shaping processing of the speech signal. Further combine the signal q 1 (n) with the first noise shaping filter to obtain the feedback signal f 1 q 1 (n) and s 1 (n) to obtain the signal u(n) to realize the fixed noise shaping of the speech signal deal with.
先对语音信号进行固定的噪声整形处理,再根据该语音信号s(n)进行自适应的噪声整形处理的过程可以包括以下过程:The speech signal is first subjected to fixed noise shaping processing, and then the process of performing adaptive noise shaping processing according to the speech signal s(n) may include the following processes:
将语音信号s(n)与第一反馈信号f1q1′(n)进行组合后得到信号s1′(n),将信号s1′(n)与第二反馈信号f2q2′(n)进行组合后得到信号u′(n);对该信号u′(n)进行语音编码后得到编码端的输出信号;将编码端的输出信号经解码后得到的信号uq′(n)与信号s1′(n)组合后得到信号q2′(n),根据语音信号s1′(n)对信号q2′(n)进行第二噪声整形滤波处理后得到反馈信号f2q2′(n);将信号uq′(n)与语音信号s(n)组合后得到信号q1′(n),信号q1′(n)进行第二噪声整形滤波处理后得到反馈信号f1q1′(n)。The speech signal s(n) is combined with the first feedback signal f 1 q 1 ′(n) to obtain the signal s 1 ′(n), and the signal s 1 ′(n) is combined with the second feedback signal f 2 q 2 ′ (n) After the combination, the signal u'(n) is obtained; the output signal of the encoding end is obtained after the speech encoding of the signal u'(n); the signal uq'(n) obtained after decoding the output signal of the encoding end is combined with the signal After combining s 1 ′(n), the signal q 2 ′(n) is obtained. According to the voice signal s 1 ′(n), the signal q 2 ′(n) is processed by the second noise shaping filter to obtain the feedback signal f 2 q 2 ′ (n); signal q 1 ′(n) is obtained after combining signal uq′(n) and speech signal s(n), and feedback signal f 1 q is obtained after signal q 1 ′(n) is subjected to second noise shaping filter processing 1 '(n).
在该过程中,将信号q1′(n)经第一噪声整形滤波处理后得到反馈信号f1q1′(n)与语音信号s(n)进行组合得到信号s1′(n)实现了对语音信号进行的固定噪声整形处理。进一步根据语音信号s1′(n)对信号q2′(n)进行第二噪声整形滤波处理后得到反馈信号f2q2′(n)与信号s1′(n)进行组合,实现了对语音信号进行的自适应噪声整形处理。In this process, the signal q 1 ′(n) is processed by the first noise shaping filter to obtain the feedback signal f 1 q 1 ′(n) and the speech signal s(n) to obtain the signal s 1 ′(n) to achieve Fixed noise shaping processing for speech signals. Further, according to the voice signal s 1 ′(n), the signal q 2 ′(n) is subjected to the second noise shaping and filtering process, and the feedback signal f 2 q 2 ′(n) is combined with the signal s 1 ′(n) to realize Adaptive noise shaping processing for speech signals.
上述过程中,进行固定噪声整形处理是对信号频谱包络进行的粗略噪声整形处理,根据语音信号进行的自适应噪声整形处理是对信号频谱精细结构进行的噪声整形处理。In the above process, the fixed noise shaping process is a rough noise shaping process on the signal spectrum envelope, and the adaptive noise shaping process based on the speech signal is a noise shaping process on the signal spectrum fine structure.
图2为本发明实施例提供的噪声整形装置的第一种组成结构图,如图2所示,该装置包括:自适应噪声整形单元210、固定噪声整形单元220和语音编码单元230。FIG. 2 is a structural diagram of the first type of noise shaping device provided by the embodiment of the present invention. As shown in FIG. 2 , the device includes: an adaptive
自适应噪声整形单元210,用于对语音信号s(n)进行自适应噪声整形处理,得到信号s1(n)。The adaptive
固定噪声整形单元220,用于对自适应噪声整形单元210进行自适应噪声整形处理后得到的信号s1(n)进行固定噪声整形处理,得到信号u(n)。The fixed
语音编码单元230,用于对固定噪声整形单元220处理后得到的信号u(n)进行语音编码,得到编码端的输出信号。The
其中,自适应噪声整形单元210可以包括:第一组合单元211、第二语音解码单元212、第四组合单元213、线性预测分析单元214和第二噪声整形滤波单元215。Wherein, the adaptive
第一组合单元211,用于将语音信号s(n)与第二噪声整形滤波单元215得到的反馈信号f2q2(n)进行组合后得到信号s1(n)。The first combining
第二语音解码单元212,用于对语音编码单元230编码后得到的输出信号进行解码,得到信号uq(n)。The second
第四组合单元213,用于将语音信号s(n)与第二语音解码单元212得到的信号uq(n)进行组合,得到信号q2(n)。The
线性预测分析单元214,用于对语音信号s(n)进行线性预测分析。A linear
第二噪声整形滤波单元215,用于根据线性预测分析单元214得到的线性预测分析结果,对第四组合单元213得到的信号q2(n)进行噪声整形滤波处理,得到反馈信号f2q2(n)。The second noise shaping and
自适应噪声整形单元210还可以包括:预加重单元216,用于对语音信号s(n)进行预加重处理;The adaptive
第二噪声整形滤波单元215,还可以根据预加重单元216的处理结果,执行所述对第四组合单元213得到的信号q2(n)进行噪声整形滤波处理的操作。The second noise shaping and
线性预测分析单元214和第二噪声整形滤波单元215可以设置在同一个单元中。The linear
固定噪声整形单元220可以包括:第二组合单元221、第一语音解码单元222、第三组合单元223和第一噪声整形滤波单元224。The fixed
第二组合单元221,用于将自适应噪声整形单元210得到的信号s1(n)与第一噪声整形单元得到的反馈信号f1q1(n)进行组合,得到信号u(n)。The
第一语音解码单元222,用于对语音编码单元230编码后得到的输出信号进行解码,得到信号uq(n)。The first
第三组合单元223,用于将自适应噪声整形单元210得到的信号s1(n)与第一语音解码单元222得到的信号uq(n)进行组合,得到信号q1(n)。The
第一噪声整形滤波单元224,用于对第三组合单元223得到的信号q1(n)进行噪声整形滤波处理,得到反馈信号f1q1(n)。The first noise shaping and
另外,也可以将第一语音解码单元222和第二语音解码单元212设置为一个单元。In addition, the first
图3为本发明实施例提供的噪声整形装置的第二种组成结构图,如图3所示,该装置包括:固定噪声整形单元310、自适应噪声整形单元320和语音编码单元330。FIG. 3 is a second structure diagram of a noise shaping device provided by an embodiment of the present invention. As shown in FIG. 3 , the device includes: a fixed noise shaping unit 310 , an adaptive noise shaping unit 320 and a speech encoding unit 330 .
固定噪声整形单元310,用于对语音信号s(n)进行固定噪声整形处理,得到信号s1′(n)。The fixed noise shaping unit 310 is configured to perform fixed noise shaping processing on the speech signal s(n) to obtain a signal s 1 '(n).
自适应噪声整形单元320,用于对固定噪声整形单元310得到的信号s1′(n)进行自适应噪声整形处理,得到信号u′(n)。The adaptive noise shaping unit 320 is configured to perform adaptive noise shaping processing on the signal s 1 ′(n) obtained by the fixed noise shaping unit 310 to obtain a signal u′(n).
语音编码单元330,用于将自适应噪声整形单元320处理后得到的信号u′(n)进行语音编码,得到编码端的输出信号。The speech encoding unit 330 is configured to perform speech encoding on the signal u'(n) obtained after processing by the adaptive noise shaping unit 320, to obtain an output signal at the encoding end.
其中,固定噪声整形单元310可以包括:第一组合单元311、第一语音解码单元312、第四组合单元313和第一噪声整形滤波单元314。Wherein, the fixed noise shaping unit 310 may include: a first combining unit 311 , a first speech decoding unit 312 , a fourth combining unit 313 and a first noise shaping and filtering unit 314 .
第一组合单元311,用于将语音信号s(n)与第一噪声整形滤波单元得到的反馈信号f1q1′(n)进行组合后得到信号s1′(n)。The first combining unit 311 is configured to combine the speech signal s(n) and the feedback signal f 1 q 1 ′(n) obtained by the first noise shaping and filtering unit to obtain a signal s 1 ′(n).
第一语音解码单元312,用于对语音编码单元330编码后得到的输出信号进行解码,得到信号uq′(n)。The first speech decoding unit 312 is configured to decode the output signal encoded by the speech encoding unit 330 to obtain a signal uq'(n).
第四组合单元313,用于将语音信号s(n)与第一语音解码单元312得到的信号uq′(n)进行组合,得到信号q1′(n)。The fourth combination unit 313 is configured to combine the speech signal s(n) with the signal uq'(n) obtained by the first speech decoding unit 312 to obtain a signal q 1 '(n).
第一噪声整形滤波单元314,用于对第四组合单元313得到的信号q1′(n)进行噪声整形滤波处理,得到反馈信号f1q1′(n)。The first noise shaping and filtering unit 314 is configured to perform noise shaping and filtering processing on the signal q 1 ′(n) obtained by the fourth combining unit 313 to obtain a feedback signal f 1 q 1 ′(n).
自适应噪声整形单元320可以包括:第二组合单元321、第二语音解码单元322、第三组合单元323、线性预测分析单元324和第二噪声整形滤波单元325。The adaptive noise shaping unit 320 may include: a second combination unit 321 , a second speech decoding unit 322 , a third combination unit 323 , a linear prediction analysis unit 324 and a second noise shaping and filtering unit 325 .
第二组合单元321,用于将固定噪声整形单元310得到的信号s1′(n)与第二噪声整形滤波单元325得到的反馈信号f2q2(n)进行组合后得到信号u1(n)。The second combination unit 321 is configured to combine the signal s 1 '(n) obtained by the fixed noise shaping unit 310 and the feedback signal f 2 q 2 (n) obtained by the second noise shaping and filtering unit 325 to obtain a signal u 1 ( n).
第二语音解码单元322,用于对语音编码单元230编码后得到的输出信号进行解码,得到信号uq′(n)。The second speech decoding unit 322 is configured to decode the output signal encoded by the
第三组合单元323,用于将固定噪声整形单元310得到的信号s1′(n)与第二语音解码单元322得到的信号uq′(n)进行组合,得到信号q2′(n)。The third combining unit 323 is configured to combine the signal s 1 '(n) obtained by the fixed noise shaping unit 310 and the signal uq'(n) obtained by the second speech decoding unit 322 to obtain a signal q 2 '(n).
线性预测分析单元324,用于对固定噪声整形单元310得到的信号s1′(n)进行线性预测分析。A linear predictive analysis unit 324, configured to perform linear predictive analysis on the signal s 1 ′(n) obtained by the fixed noise shaping unit 310.
第二噪声整形滤波单元325,用于根据线性预测分析单元324得到的线性预测分析结果,对第三组合单元323得到的信号q2′(n)进行噪声整形滤波处理,得到反馈信号f2q2′(n)。The second noise shaping and filtering unit 325 is used to perform noise shaping and filtering processing on the signal q 2 '(n) obtained by the third combination unit 323 according to the linear prediction analysis result obtained by the linear prediction analysis unit 324, to obtain the feedback signal f 2 q 2 '(n).
自适应噪声整形单元320还可以包括:预加重单元326,用于对信号s1′(n)进行预加重处理;The adaptive noise shaping unit 320 may also include: a pre-emphasis unit 326, configured to perform pre-emphasis processing on the signal s 1 '(n);
第二噪声整形滤波单元325,还可以根据预加重单元216的处理结果,执行所述对第三组合单元323得到的信号q2′(n)进行噪声整形滤波处理的操作。The second noise shaping and filtering unit 325 may also perform the operation of performing noise shaping and filtering on the signal q 2 ′(n) obtained by the third combination unit 323 according to the processing result of the
线性预测分析单元324和预加重单元326可以设置在同一个单元中实现。The linear prediction analysis unit 324 and the pre-emphasis unit 326 may be implemented in the same unit.
第一语音解码单元312和第二语音解码单元322也可以设置为一个单元。The first speech decoding unit 312 and the second speech decoding unit 322 may also be configured as one unit.
图2所示的噪声整形装置的组成结构图是与先根据语音信号进行自适应噪声整形处理,再进行固定噪声整形处理对应的,图3所示的噪声整形装置的组成结构图是与先进行固定噪声整形处理,再进行自适应噪声整形处理相对应的。The structure diagram of the noise shaping device shown in Fig. 2 is corresponding to performing adaptive noise shaping processing according to the speech signal first, and then performing fixed noise shaping processing, and the composition structure diagram of the noise shaping device shown in Fig. 3 is the same as first performing The fixed noise shaping process is corresponding to the adaptive noise shaping process.
为了使本发明更加的清楚,下面具一个具体实施例对图2所示结构的噪声整形装置以及其对应的方法进行详细描述。图4为本发明实施例提供的一个具体噪声整形装置的结构示意图。图4中,将第一语音解码单元和第二语音解码单元设置为一个本地语音解码器,将预加重单元和线性预测分析单元设置为一个预加重和线性预测分析器。下面是利用该具体装置实现噪声整形的反馈处理过程:In order to make the present invention more clear, the noise shaping device with the structure shown in FIG. 2 and its corresponding method will be described in detail below with a specific embodiment. Fig. 4 is a schematic structural diagram of a specific noise shaping device provided by an embodiment of the present invention. In Fig. 4, the first speech decoding unit and the second speech decoding unit are set as a local speech decoder, and the pre-emphasis unit and linear prediction analysis unit are set as a pre-emphasis and linear prediction analyzer. The following is the feedback process for implementing noise shaping using this specific device:
1)将语音信号s(n)与第二反馈信号f2q2(n)都输入第一加法器,进行组合后输出信号s1(n)。1) Both the speech signal s(n) and the second feedback signal f 2 q 2 (n) are input into the first adder, and combined to output the signal s 1 (n).
利用该第一加法器进行的信号组合可以是将语音信号s(n)与第二反馈信号f2q2(n)相加,也可以是相减。在取相加或相减时会影响到第二噪声整形滤波器的噪声整形滤波方式的选取不同。The signal combination performed by the first adder may be the addition or subtraction of the speech signal s(n) and the second feedback signal f 2 q 2 (n). When adding or subtracting, it will affect the selection of the noise shaping filter mode of the second noise shaping filter.
2)将第一加法器输出的信号s1(n)与第一反馈信号f1q1(n)都输入第二加法器,进行组合后输出信号u(n)。2) Both the signal s 1 (n) output by the first adder and the first feedback signal f 1 q 1 (n) are input into the second adder, and combined to output a signal u(n).
利用该第二加法器进行的信号组合也可以是将语音信号s1(n)与第一反馈信号f1q1(n)相加,也可以是相减。在取相加或相减时会影响到第一噪声整形滤波器的噪声整形滤波方式的选取不同。The signal combination performed by the second adder may also be the addition or subtraction of the speech signal s 1 (n) and the first feedback signal f 1 q 1 (n). When adding or subtracting, it will affect the selection of the noise shaping filter mode of the first noise shaping filter.
3)将第二加法器输出的信号u(n)输入语音编码器进行语音编码后,得到编码端的输出信号。该输出信号即为需要进行传输最终达到解码端的信号。在解码端对该信号进行解码便可还原出语音信号。该还原出的语音信号便是进行了噪声整形后的语音信号,提高了该语音信号的主观听觉质量。3) After inputting the signal u(n) output by the second adder into the speech coder for speech coding, the output signal of the coding end is obtained. The output signal is the signal that needs to be transmitted and finally reaches the decoding end. The speech signal can be restored by decoding the signal at the decoding end. The restored speech signal is the speech signal after noise shaping, which improves the subjective auditory quality of the speech signal.
4)在编码端将语音编码器输出的信号经本地语音解码器解码,输出信号uq(n)。4) At the encoding end, the signal output by the speech encoder is decoded by the local speech decoder to output the signal uq(n).
5)将本地语音解码器输出的信号uq(n)与信号第一加法器输出的信号s1(n)输入第三加法器,进行组合后输出信号q1(n)。5) Input the signal uq(n) output by the local speech decoder and the signal s 1 (n) output by the first signal adder into the third adder, and output signal q 1 (n) after combining.
第三加法器对信号uq(n)和信号s1(n)的组合为将两信号进行相减,对相减后绝对值符号的选取也会影响到第一噪声整形滤波器的噪声整形滤波方式的选取。The combination of signal uq(n) and signal s 1 (n) by the third adder is to subtract the two signals, and the selection of the sign of the absolute value after subtraction will also affect the noise shaping filtering of the first noise shaping filter way of selection.
6)将第三加法器输出的信号q1(n)输入第一噪声整形滤波器进行处理,输出反馈信号f1q1(n)。6) Input the signal q 1 (n) output by the third adder into the first noise shaping filter for processing, and output the feedback signal f 1 q 1 (n).
该第一噪声整形滤波器的处理方法是预先设定的,也就是说,它是独立于语音信号。一旦设定完毕后,对于不同的语音信号,其噪声整形方式F1是固定不变的,其主要功能是对语音信号的频谱包络进行整形,即对量化噪声的平坦谱进行粗略的噪声整形,抑制高频噪声。The processing method of the first noise shaping filter is preset, that is, it is independent of the speech signal. Once the setting is completed, the noise shaping method F1 is fixed for different speech signals, and its main function is to shape the spectrum envelope of the speech signal, that is, to perform rough noise shaping on the flat spectrum of the quantization noise , to suppress high-frequency noise.
当第二加法器的组合方式为相加,第三加法器的组合方式为s1(n)减去uq(n)时,设语音编码器以及本地语音解码器产生的量化噪声为:△(n)=u(n)-uq(n),经过第一噪声整形滤波器后的量化噪声为q1(n)=s1(n)-uq(n)。对上述两个式子进行z变换后,可以得出经过第一噪声整形滤波器后的量化噪声与语音编码器以及本地语音解码器产生的量化噪声在频谱上的关系为:
在预先设定第一噪声整形滤波器的处理方式时,可以利用(1)式,并根据想要得到的信号的性能,对处理方式的表达式F1进行选取。例如,想要降低高频噪声能量,可以根据(1)式进行选择,可以令F1=-0.8z-1,便可以得到不错的效果。当然,选取的F1并不是只有唯一的表达式,可以利用(1)式,根据实际的需求进行选取,也就是说,该第一噪声整形滤波器可以有多种设计,零极点、全零点、全极点滤波器中能够得到想要的信号性能的滤波器均可以选取。When presetting the processing mode of the first noise shaping filter, formula (1) can be used, and the expression F1 of the processing mode can be selected according to the performance of the signal to be obtained. For example, if you want to reduce the high-frequency noise energy, you can choose according to formula (1), and you can set F 1 =-0.8z -1 , and you can get good results. Of course, the selected F 1 is not only a unique expression, but can use formula (1) to select according to actual needs, that is to say, the first noise shaping filter can have multiple designs, zero-pole, all-zero , all-pole filters that can obtain the desired signal performance can be selected.
当第二加法器和/或第三加法器的组合方式发生变化时,可以采用同样的方式得出不同的经过第一噪声整形滤波器后的量化噪声与语音编码器以及本地语音解码器产生的量化噪声在频谱上的关系,即第二加法器和/或第三加法器采用不同的组合方式,(1)式中的符号略有变化,例如,当第二加法器的组合为相减,而第三加法器的组合为uq(n)减去s1(n)时,(1)式变为:
7)将本地语音解码器输出的信号uq(n)与语音信号s(n)输入第四加法器,进行组合后输出信号q2(n)。7) Input the signal uq(n) output by the local speech decoder and the speech signal s(n) into the fourth adder, combine them and output the signal q 2 (n).
第四加法器对信号uq(n)和语音信号s(n)的组合方式为将两信号相减,相减后所得数据绝对值符号的选取也会影响到第二噪声整形滤波器的噪声整形滤波方式的选取。The combination method of the fourth adder to the signal uq(n) and the voice signal s(n) is to subtract the two signals, and the selection of the absolute value sign of the data obtained after the subtraction will also affect the noise shaping of the second noise shaping filter Selection of filtering method.
8)将语音信号输入预加重和线性预测分析器,进行预加重和线性预测分析处理,第二噪声整形滤波器利用预加重和线性预测分析处理器输出的处理结果,确定自身的自适应噪声整形滤波处理方法。8) The voice signal is input into the pre-emphasis and linear predictive analyzer for pre-emphasis and linear predictive analysis processing, and the second noise shaping filter utilizes the processing results output by the pre-emphasis and linear predictive analysis processor to determine its own adaptive noise shaping filtering method.
该自适应噪声整形滤波处理方法是随着语音信号的频谱做动态变化的。The adaptive noise shaping and filtering processing method is dynamically changed along with the frequency spectrum of the speech signal.
9)将第四加法器输出的信号q2(n)输入第二噪声整形滤波器,第二噪声整形滤波利用自身的自适应噪声整形滤波处理方法,对信号q2(n)进行处理,输出反馈信号f2q2(n)。9) Input the signal q 2 (n) output by the fourth adder into the second noise shaping filter, and the second noise shaping filter uses its own adaptive noise shaping filter processing method to process the signal q 2 (n), and output Feedback signal f 2 q 2 (n).
当第一加法器的组合方式为相加,第四加法器的组合方式为与语音信号s(n)减去信号uq(n)时,仍然设语音编码器以及本地语音解码器产生的量化噪声为:△(n)=u(n)-uq(n),经过第二噪声整形滤波器后的量化噪声为q2(n)=s(n)-uq(n),又由于q2(n)=q1(n)-f2q2(n-1),对上述两个式子进行z变换后,可以得出经过第二噪声整形滤波器后的量化噪声与语音编码器以及本地语音解码器产生的量化噪声在频谱上的关系为:When the combination mode of the first adder is addition, and the combination mode of the fourth adder is to subtract the signal uq(n) from the speech signal s(n), the quantization noise produced by the speech encoder and the local speech decoder is still assumed is: △(n)=u(n)-uq(n), the quantization noise after the second noise shaping filter is q 2 (n)=s(n)-uq(n), and because q 2 ( n)=q 1 (n)-f 2 q 2 (n-1), after carrying out z transformation to above-mentioned two formulas, can draw the quantization noise after the second noise shaping filter and speech coder and local The quantization noise produced by the speech decoder is related on the frequency spectrum as:
第二噪声整形滤波器的整形滤波方式的选取是根据语音信号的变化而进行自适应变化的,它的整形滤波方式的表达式F2是根据预加重和线性预测分析后的结果选取的。其主要功能是对语音信号进行频谱精细结构的噪声整形,也就是在不改变之前噪声整形频谱大致形状的情况下,对谐波之间的频谱进行更为精细的噪声整形,从而进一步提高主观听觉质量。The selection of the shaping filtering mode of the second noise shaping filter is adaptively changed according to the change of the speech signal, and the expression F2 of its shaping filtering mode is selected according to the results of pre-emphasis and linear prediction analysis. Its main function is to perform noise shaping on the fine spectrum structure of the speech signal, that is, to perform finer noise shaping on the spectrum between harmonics without changing the general shape of the previous noise shaping spectrum, thereby further improving subjective hearing quality.
由于噪声功率谱随频率的增加而减小,其大部分能量集中在低频范围内,为了抑制语音信号的低频噪声,采用了预加重的方法。预加重处理是利用语音信号特性和噪声特性的差别来有效地对信号进行处理,通过加重语音信号的高频分量,来对低频噪声进行抑制,具体预加重的方法是现有技术中已有的方法,在此不再赘述。Because the noise power spectrum decreases with the increase of frequency, most of its energy is concentrated in the low frequency range. In order to suppress the low frequency noise of the speech signal, a pre-emphasis method is adopted. Pre-emphasis processing is to use the difference between the characteristics of the speech signal and the noise to effectively process the signal, and suppress the low-frequency noise by emphasizing the high-frequency component of the speech signal. The specific pre-emphasis method is existing in the prior art. method, which will not be repeated here.
为了使第二噪声整形滤波器的处理方式更加接近语音信号的频谱图,本发明实施例采用线性预测分析的方法使得语音信号的每个取样值能够用过去若干各取样值的线性组合来逼近,具体为通过实际语音信号取样值和线性预测样值之间的均方误差最小,来确定唯一的一组线性组合中所用的加权系数作为线性预测结果。通过线性预测分析的方法可以实现对语音信号中的中高频信号谐波之间的噪声进行抑制。这也是现有技术中的已有的方法,在此不再赘述。In order to make the processing mode of the second noise shaping filter closer to the spectrogram of the speech signal, the embodiment of the present invention adopts the method of linear predictive analysis so that each sampling value of the speech signal can be approximated by a linear combination of several sampling values in the past, Specifically, a unique set of weighting coefficients used in the linear combination is determined as the linear prediction result by minimizing the mean square error between the actual speech signal sample value and the linear prediction sample value. The noise between the middle and high frequency signal harmonics in the speech signal can be suppressed by the method of linear predictive analysis. This is also an existing method in the prior art, and will not be repeated here.
另外,线性预测分析可以在对语音信号进行预加重处理后的信号上进行性,根据线性预测分析得到的预测结果确定第二噪声整形滤波的处理方式的表达式F2。也可以只对语音信号进行线性预测分析,根据线性预测分析的预测结果确定第二噪声整形滤波的处理方式的表达式F2。线性预测分析后得出m个加权系数αi,i=1,...,m。再由式(2)以及想要得到的信号的性能,确定表达式F2。例如,可以令
图5为本发明实施例提供的另一个具体噪声整形装置的结构示意图。图5中的具体结构是图3所示的噪声整形装置以及对应的方法的一个具体实例,该结构的实现方式与图4的不同之处在于,首先进行自适应噪声整形处理,再进行固定噪声整形处理,具体处理过程不再赘述。Fig. 5 is a schematic structural diagram of another specific noise shaping device provided by an embodiment of the present invention. The specific structure in Fig. 5 is a specific example of the noise shaping device and the corresponding method shown in Fig. 3. The difference between the implementation of this structure and Fig. 4 is that the adaptive noise shaping process is performed first, and then the fixed noise Plastic processing, the specific processing process will not be repeated.
由以上描述可以看出,在本发明实施例提供的方法和装置中,对语音信号先进行自适应的噪声整形处理,再进行固定的噪声整形处理,对进行固定噪声整形处理后的信号进行语音编码,形成编码端的输出信号;或者,对语音信号先进行固定的噪声整形处理,再进行自适应的噪声整形处理,对进行自适应噪声整形处理后的信号进行语音编码,形成编码端的输出信号。由于固定的噪声整形处理和自适应的噪声整形处理均是对语音信号本身所做的噪声整形处理,而不涉及语音信号与预测滤波器的残差信号,所以,在解码端不需要做任何改变即可解码还原出经噪声整形的语音信号。It can be seen from the above description that in the method and device provided by the embodiments of the present invention, the speech signal is first subjected to adaptive noise shaping processing, and then fixed noise shaping processing is performed, and the speech signal after the fixed noise shaping processing is performed. Encoding to form an output signal at the encoding end; or, perform fixed noise shaping processing on the speech signal first, and then perform adaptive noise shaping processing, and perform speech encoding on the signal after adaptive noise shaping processing to form an output signal at the encoding end. Since both the fixed noise shaping process and the adaptive noise shaping process are noise shaping processes on the speech signal itself, and do not involve the speech signal and the residual signal of the prediction filter, there is no need to make any changes at the decoding end The noise-shaped speech signal can be decoded and restored.
并且,本发明实施例给出了具体的固定滤波整形和自适应滤波整形相结合的二层结构的整形滤波装置和方法,通过固定滤波整形对语音信号进行频谱包络的整形,实现对噪声的平坦谱进行粗略的噪声整形,通过对语音信号进行预加重和线性预测分析来确定自适应滤波整形的方式,根据语音信号的变化实时地进行自适应滤波整形,从而在不改变语音信号频谱大致形状的条件下,对语音信号频谱的精细结构进行整形,从而进一步提高主观听觉质量。同时,采用预加重和线性预测分析来确定自适应滤波整形方式,能够很好的对语音信号的低频噪声进行抑制,同时也对语音信号的中高频噪声进行抑制,使语音信号的听觉质量得到进一步地提高。Moreover, the embodiment of the present invention provides a specific two-layer structure shaping filter device and method that combines fixed filter shaping and adaptive filter shaping. Perform rough noise shaping on the flat spectrum, determine the way of adaptive filtering and shaping by pre-emphasizing and linear predictive analysis on the speech signal, and perform adaptive filtering and shaping in real time according to changes in the speech signal, so as not to change the general shape of the speech signal spectrum Under the condition of , the fine structure of the speech signal spectrum is shaped, so as to further improve the subjective auditory quality. At the same time, pre-emphasis and linear predictive analysis are used to determine the adaptive filter shaping method, which can well suppress the low-frequency noise of the speech signal, and at the same time suppress the middle and high-frequency noise of the speech signal, so that the auditory quality of the speech signal is further improved. improved.
以上所述仅为本发明的较佳实施例而已,并不用以限制本发明,凡在本发明的精神和原则之内,所做的任何修改、等同替换、改进等,均应包含在本发明保护的范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection.
Claims (20)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007101539531A CN101388215B (en) | 2007-09-15 | 2007-09-15 | A method and device for noise shaping |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007101539531A CN101388215B (en) | 2007-09-15 | 2007-09-15 | A method and device for noise shaping |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101388215A true CN101388215A (en) | 2009-03-18 |
| CN101388215B CN101388215B (en) | 2011-01-12 |
Family
ID=40477585
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007101539531A Expired - Fee Related CN101388215B (en) | 2007-09-15 | 2007-09-15 | A method and device for noise shaping |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101388215B (en) |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6356872B1 (en) * | 1996-09-25 | 2002-03-12 | Crystal Semiconductor Corporation | Method and apparatus for storing digital audio and playback thereof |
| KR100477699B1 (en) * | 2003-01-15 | 2005-03-18 | 삼성전자주식회사 | Quantization noise shaping method and apparatus |
-
2007
- 2007-09-15 CN CN2007101539531A patent/CN101388215B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN101388215B (en) | 2011-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101508819B1 (en) | Multi-mode audio codec and celp coding adapted therefore | |
| RU2585990C2 (en) | Device and method for encoding by huffman method | |
| US20230410822A1 (en) | Filling of Non-Coded Sub-Vectors in Transform Coded Audio Signals | |
| EP2791937B1 (en) | Generation of a high band extension of a bandwidth extended audio signal | |
| US9082398B2 (en) | System and method for post excitation enhancement for low bit rate speech coding | |
| US8175145B2 (en) | Post-processing for reducing quantization noise of an encoder during decoding | |
| CN1922656B (en) | Apparatus and method for determining quantizer step size | |
| CN101377925B (en) | Self-adaptation adjusting method for improving apperceive quality of g.711 | |
| CN102844810B (en) | Flexible and scalable combined innovation codebook for use in celp coder and decoder | |
| KR100668319B1 (en) | Method and apparatus for converting audio signals and encoding method and apparatus adapted to audio signals, inverse transform method and apparatus for audio signals and apparatus and apparatus for adaptive decoding for audio signals | |
| CN103370740A (en) | Improved encoding of an improvement stage in a hierarchical encoder | |
| CN100487789C (en) | Perception weighting filtering wave method and perception weighting filter thererof | |
| CN101320565B (en) | Perception weighting filtering wave method and perception weighting filter thererof | |
| CN101388215B (en) | A method and device for noise shaping | |
| EP3455854B1 (en) | Adaptive audio codec method and apparatus | |
| CN101741504A (en) | A method and device for determining the order of signal linear predictive coding | |
| CN101533639A (en) | Voice signal processing method and device | |
| JP4516345B2 (en) | Speech coding information processing apparatus and speech coding information processing program | |
| CN102930872A (en) | Method and device for postprocessing pitch enhancement in broadband speech decoding | |
| CN101887727B (en) | Speech coding data conversion system and method from HELP coding to MELP coding | |
| KR100221186B1 (en) | Voice coding and decoding device and method thereof | |
| Li et al. | Non-linear techniques for pitch and waveform enhancement in PWI coders | |
| CN101388211A (en) | A method and device for noise shaping in the process of speech encoding and decoding | |
| de Lamare | e-mails: delamare (a) infolink. com. br, alcaim (a) cetucpuc-rio. br |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110112 Termination date: 20160915 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |