CN114974278A - Voice processing method, device, equipment and storage medium - Google Patents

Abstract

The invention discloses a voice processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: decomposing the voice section to be processed to obtain a multi-frame decomposed voice section; adjusting the decomposed voice segments of the multiframes based on an adjustment algorithm to obtain multiframes of voice segments to be superposed; calculating and obtaining an amplitude gain factor of each frame of the voice section to be superposed; adjusting the amplitude of the voice segment to be superposed in each frame based on the amplitude gain factor to determine a target amplitude; and acquiring a target voice section based on the target amplitude. According to the technical scheme, the amplitude gain factor is increased to strictly control the amplitude of the voice section to be superposed, so that the problem that the amplitude of the synthesized voice is too large to cause large distortion in the real-time communication recovery process is solved, the situations of too large amplitude of the tail part of the voice waveform and the like are avoided, and the recovered audio is closer to the original state.

Description

Voice processing method, device, equipment and storage medium

technical field

The present invention belongs to the technical field of audio processing, and in particular relates to a voice processing method, device, equipment and storage medium.

Background technique

In duplex calls, echo problems often occur, and the speaker can hear what he has just said from the receiver; as shown in Figure 1, paths 1-7 indicate the sound propagation path of A when speaking: when the far-end A's When a person speaks a sentence, the microphone of far-end A collects the sound and generates a voice signal, which is transmitted to the speaker of near-end B for broadcasting; The voice signal is transmitted to the loudspeaker of far-end A again for broadcast, that is, far-end A said a word, and after a while, he heard what he just said from the loudspeaker; conversely, if near-end B speaks, the same is true. Therefore, in order to prevent the speaker from hearing what he just said from the receiver, the echo cancellation needs to be performed. For example, if an echo canceller is added to the near-end B to cancel the echo, the far-end A will not hear what it just said.

At present, the commonly used echo cancellation algorithms include Least Mean Square (LMS) and Normalized Least Mean Square (NLMS) algorithms. The algorithm requires the input of the microphone signal and the The reference signal (the near-end speaker signal is actually the far-end microphone signal, also known as the reference signal), when only the far-end speaks, the filter targets the microphone signal, making the reference signal as close as possible to the microphone signal to simulate echo path.

However, due to network fluctuations and hardware usage, the reference signal may be out of sync with the microphone signal, that is, the interval between the audio signal collected by the near-end microphone and the interval played by the near-end speaker may deviate, which will lead to echoes. There is a delay between the reference signal and the microphone signal during cancellation. If the long-term network is not good, the delay will become larger and larger, which will lead to the failure to use the reference signal to refer to the microphone signal to cancel the echo; although the echo cancellation algorithm generally There is a delay alignment method between the reference signal and the microphone signal, but this can only be performed within a small delay value to align the two audio frequencies. When the delay is too large and the delay is unstable, it is difficult to estimate the quasi-delay value.

SUMMARY OF THE INVENTION

The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, an object of the present invention is to provide a voice processing method, apparatus, device and storage medium.

In order to solve the above-mentioned technical problems, the embodiments of the present invention provide the following technical solutions:

A speech processing method, comprising:

Perform decomposition processing on the speech segment to be processed to obtain multi-frame decomposed speech segments;

Adjusting the decomposed speech segments of multiple frames based on an adjustment algorithm to obtain multiple frames of speech segments to be superimposed;

Calculate the amplitude gain factor of each frame of the to-be-superimposed speech segment;

Based on the amplitude gain factor, the amplitude of the speech segment to be superimposed in each frame is adjusted to determine the target amplitude;

Based on the target amplitude, a target speech segment is obtained.

Optionally, the adjustment of the decomposed speech segments of multiple frames based on the adjustment algorithm to obtain the speech segments to be superimposed in multiple frames includes:

If the decomposed speech segment has frame missing or frame redundancy, determining the decomposed speech segment to be adjusted based on the decomposed speech segment;

The decomposed speech segment to be adjusted is adjusted to obtain the speech segment to be superimposed.

Optionally, if the decomposed speech segment has frame missing or frame redundancy, determining the decomposed speech segment to be adjusted based on the decomposed speech segment, including:

If the decomposed speech segment has frame missing;

then determining at least one frame of the decomposed speech segment to be adjusted based on the decomposed speech segment;

The to-be-adjusted and decomposed speech segment is stretched to obtain the to-be-superimposed speech segment.

Optionally, the calculating to obtain the amplitude gain factor of the speech segment to be superimposed in each frame, including:

Obtain the length L of the speech segment to be superimposed in each frame;

Based on the length, the speech segment to be superimposed in each frame is divided into L samples; wherein, L is a positive integer;

Obtain the amplitude value corresponding to the sample point in each frame;

Obtain the target superposition position of the sample points in each frame and the original amplitude value of the target superposition position in each frame;

Based on the amplitude value and the original amplitude value, the amplitude gain factor of the speech segment to be superimposed in each frame is obtained by calculation.

Optionally, calculating and obtaining the amplitude gain factor of the speech segment to be superimposed in each frame based on the amplitude value and the original amplitude value, including:

Summing up the L amplitude values of the speech segments to be superimposed in each frame to obtain a first value M; wherein, M≥0;

Summing up the L original amplitude values of the speech segments to be superimposed in each frame to obtain a second value N; wherein, N≥0;

Based on the first value M and the second value N, the amplitude gain factor is obtained.

Optionally, the obtaining the amplitude gain factor based on the first value M and the second value N includes:

Obtain the number Q of zero values in the L said amplitude values; wherein, Q≥0, and Q is an integer;

Based on the length L and the number Q of zeros in the L amplitude values, a reference coefficient λ of the amplitude gain factor is obtained; wherein, λ≥1;

Based on the reference coefficient λ, the first value M and the second value N, the amplitude gain factor is obtained.

Optionally, the amplitude gain factor is calculated and obtained based on the following formula:

Among them, β _k is the kth amplitude gain factor; k is a positive integer.

Optionally, the reference coefficient is obtained based on the following calculation formula:

Optionally, the target speech segment is calculated and obtained based on the following calculation formula:

S _k =P _k *β _k

Wherein, _Sk is the target speech segment of the Kth frame; _Pk is the to-be-superimposed speech segment of the Kth frame.

An embodiment of the present invention also provides a voice processing device, comprising:

The decomposition module is used to decompose the speech segment to be processed, and obtain multi-frame decomposed speech segments;

an adjustment module, configured to adjust the decomposed speech segments of the multiple frames based on an adjustment algorithm, and obtain the speech segments to be superimposed in the multiple frames;

A calculation module, for calculating and obtaining the amplitude gain factor of the speech segment to be superimposed in each frame;

A determination module, configured to adjust the amplitude of the speech segment to be superimposed in each frame based on the amplitude gain factor, and determine the target amplitude;

an obtaining module, configured to obtain a target speech segment based on the target amplitude.

Embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, where the processor implements the above when executing the computer program the method described.

Embodiments of the present invention also provide a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program, wherein when the computer program runs, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned method described.

The embodiment of the present invention has the following technical effects:

The above technical solutions of the present invention, 1) In the embodiment of the present invention, the voice signal sent by the network is buffered based on the buffer to form a buffer queue. Based on the buffer queue, the voice signal can be transmitted to the speaker at a uniform speed and stability, and the time delay is solved. If the delay is too large and the delay is unstable, it is possible to accurately estimate the delay during the transmission of the voice signal, thereby improving the accuracy of echo cancellation.

2) Using the WSOLA algorithm to leave an overlapped part between the two frames, at the same time, windowing is performed on each frame to reduce the impact of the discontinuous waveform; Directly find a frame of decomposed speech segment for direct splicing, but find a decomposed speech segment closest to the ideal frame in an interval and superimpose it, thereby reducing the distortion problem caused by phase jumps.

3) Strictly control the amplitude of the speech segment to be superimposed by increasing the amplitude gain factor, which solves the problem that large distortion may occur in the recovery process of real-time communication caused by the excessively large amplitude of the synthesized speech and the lack of frames due to network delay is just right. Occurred at the end of a voice band, there is a considerable difference between the adjusted voice and the original voice, which avoids the situation that the tail of the voice waveform is too large, and makes the restored audio closer to the original state.

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the invention.

Description of drawings

1 is a schematic diagram of an existing sound propagation path;

2 is a schematic structural diagram of a speech processing system provided by an embodiment of the present invention;

3 is a schematic flowchart of a speech processing method provided by an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of a voice processing apparatus provided by an embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention and should not be construed as limiting the present invention.

First, in order to facilitate the understanding of the embodiments by those skilled in the art, some terms are explained:

(1) WebRTC: Web Real-Time Communication, web real-time communication.

(2) AEC: Acoustic Echo Cancellation, echo cancellation.

(3) API: Application Programming Interface, application program interface.

(4) OLA: Overlap-and-Add, overlapping stacking algorithm.

(5) WSOLA: Waveform similarity Overlap-Add, waveform similarity overlay algorithm; and the basic principle is to intercept a frame in the original speech signal, add a window, and select the second frame within a range (for example, within a certain frame). The frame's phase parameter should be phase-aligned with the first frame, look for the third frame in another range, which should be most similar to the second frame, and then add them together.

Secondly, at present, in real-time voice communication, human voice is composed of countless relatively short voice bands. Therefore, in real-time environment, the situation of voice does not fully meet the requirements of the standard waveform similarity superposition algorithm, and it is difficult to have a better recovery effect. Moreover, since the standard waveform similarity superposition algorithm focuses on maintaining the pitch frequency of the speech, the amplitude control during the speech recovery process is not strictly controlled, which may lead to the problem of excessively large synthesized speech amplitude in some cases. During the recovery process of real-time communication, large distortion may occur, which will affect the effect of real-time voice communication.

In addition, for typical human speech, a piece of speech is not absolutely continuous, it usually consists of many short continuous speeches, and we call these very short continuous speeches as speech bands. Between two adjacent speech bands, there is often a silent band; since there are many such gaps in actual human speech, and if the missing frame caused by network delay happens at the end of a speech band, then according to the WSOLA algorithm The waveform at the missing frame will be filled. In this way, a normal speech band cannot end in time, so there will be a considerable difference between the adjusted speech and the original speech.

In order to solve the above-mentioned technical problems, the present invention provides the following technical solutions:

As shown in FIG. 2, an embodiment of the present invention provides a speech processing system, including:

Speakers, microphones, echo cancellers and buffers;

Wherein, the buffer is arranged between the speaker and the network, and is respectively connected with the speaker and the network; the buffer is used for receiving the voice signal sent by the network and sending it to the speaker;

The loudspeaker, the microphone and the buffer are all connected with the echo canceller, and are used for realizing echo cancellation during the transmission of the voice signal based on the echo canceller.

In practical application scenarios, when the network speed fluctuates, the network may transmit multiple frames of voice signals in sequence, or the network may not send any frame of voice signals for a period of time.

Therefore, in order to realize the uniform and stable transmission of voice signals to the speaker based on the network, the embodiment of the present invention sets a buffer for buffering the voice signals sent by the network, and forms a buffer queue; The data transmission rate of the near-end, as far as the near-end is concerned, it is assumed that the sampling rate of the audio is 8000 Hz (or other preset values, the embodiment of the present invention takes 8000 Hz as an example), that is to say, the near-end speaker can only broadcast 8000 Hz per second. Only 8,000 points can be received by the microphone; that is, 8,000 points are provided to the speaker at a constant rate per second based on the network.

Further, when the network is slow for a long time and the voice segment in the buffer queue may have frame missing, the buffer queue in the buffer is interpolated to maintain the forwarding rate of 8000 points per second;

Further, when the network is fast for a long time, there may be frame redundancy in the voice segment in the buffer queue, that is, the buffer area of the buffer may be full, but the network continues to send voice to the buffer. Therefore, in order to ensure the forwarding rate of 8000 points per second, the embodiment of the present invention performs shortening processing on some speech segments.

For delay estimation, in practical application scenarios, a method of large-scale rough estimation and small-scale accurate calculation can be used. For example, in the AEC module of WebRTC, the API interface obtains a millisecond passed in by the caller. The echo delay estimation value, because the embodiment of the present invention sets up a buffer and processes the speech segments in the buffer queue, which solves the problem that the reference signal cannot be used to refer to the microphone signal normally, and then the AEC internally stores this value in this value. On the basis of the estimated value, an accurate estimation of the delay in a small range is carried out.

In the embodiment of the present invention, the voice signal sent by the network is buffered based on the buffer to form a buffer queue. Based on the buffer queue, the voice signal can be transmitted to the speaker at a constant speed and stability, and the problems of excessive delay and unstable delay can be solved. , so as to accurately estimate the time delay in the process of voice signal transmission, thereby improving the accuracy of echo cancellation.

As shown in FIG. 3, an embodiment of the present invention provides a speech processing method, which is applied to the above-mentioned system, including:

Step S1: decompose the speech segment to be processed, and obtain multi-frame decomposed speech segments;

Specifically, before processing the voice signal in the buffer queue, the voice signal in the buffer queue is firstly decomposed and divided into different decomposed voice segments.

For example, the duration of each decomposed speech segment may be between 50ms and 100ms.

Step S2: adjusting the decomposed speech segments of multiple frames based on an adjustment algorithm, to obtain multiple frames of speech segments to be superimposed;

Specifically, the adjustment of the decomposed speech segments of multiple frames based on the adjustment algorithm to obtain the speech segments to be superimposed in multiple frames includes:

If the decomposed speech segment has frame missing or frame redundancy, determining the decomposed speech segment to be adjusted based on the decomposed speech segment;

The decomposed speech segment is adjusted based on the decomposed speech segment to be adjusted to obtain the speech segment to be superimposed.

Further, if the decomposed speech segment has frame missing or frame redundancy, determining the decomposed speech segment to be adjusted based on the decomposed speech segment, including:

If the decomposed speech segment has frame missing;

then determining at least one frame of the decomposed speech segment to be adjusted based on the decomposed speech segment;

The to-be-adjusted and decomposed speech segment is stretched to obtain the to-be-superimposed speech segment.

Specifically, when a decomposed speech segment has frame missing or insufficient frames, the start time of the decomposed speech segment is obtained, several decomposed speech segments before the start time are searched based on the start time, and the The decomposed speech segments are determined to be decomposed speech segments to be adjusted, and then the decomposed speech segments to be adjusted are elongated to cover the position where the frame is missing.

Among them, the decomposed speech segments that are most similar to the decomposed speech segments are determined based on the decomposed speech segments with frame missing based on the WSOLA algorithm, and the decomposed speech segments are elongated. The specific elongated value can be determined based on the WSOLA algorithm. Sure.

Further, if the decomposed speech segment has frame missing or frame redundancy, determining the decomposed speech segment to be adjusted based on the decomposed speech segment, including:

if there is frame redundancy in the decomposed speech segment;

then determining at least one frame of the decomposed speech segment to be adjusted based on the decomposed speech segment;

The to-be-adjusted and decomposed speech segment is shortened to obtain the to-be-superimposed speech segment.

Specifically, when frame redundancy occurs in a decomposed speech segment, the start time of the decomposed speech segment is obtained, several decomposed speech segments before the start time are searched based on the start time, and these decomposed speech segments are decomposed. The speech segment is determined as the speech segment to be adjusted and decomposed, and then the speech segments to be adjusted and decomposed are shortened.

Among them, based on the WSOLA algorithm, the decomposed speech segments that are most similar to the decomposed speech segments are determined according to the decomposed speech segments with frame redundancy, and the decomposed speech segments are shortened. The specific shortening value can be determined based on the WSOLA algorithm. .

In practical application scenarios, in order to change the transmission rate of the speech signal, the prior art performs stretching or compression processing on the acquired multi-frame decomposed speech segments, so as to realize the change of the speech signal transmission by changing the duration of each decomposed speech segment. effect of speed.

For example, based on the OLA algorithm, after obtaining the multi-frame decomposed speech segment, directly based on OLA, the first part of the multi-frame decomposed speech segment is spliced together; when the speech speed of a segment needs to be slowed down, it will be based on the segment. Each obtained decomposed speech segment is stretched, that is, the duration of each decomposed speech segment is extended.

However, the method directly based on the first splicing method to superimpose multiple frames is prone to the problem of discontinuous waveform at the splicing point, that is, the pitch in the speech signal is broken; During the cropping process, there is no way to ensure that each frame can cover the full cycle and ensure its phase alignment, which leads to phase jump distortion.

In the embodiment of the present invention, the WSOLA algorithm is used to leave an overlapped part between the two frames, and at the same time, each frame is subjected to windowing processing, thereby reducing the influence caused by the discontinuity of the waveform;

In addition, based on the WSOLA algorithm, the embodiment of the present invention does not directly find a frame of decomposed speech segments for direct splicing, but finds a decomposed speech segment that is closest to the ideal frame in an interval and superimposes it, thereby reducing phase jumps caused by phase jumps. Distortion problem caused by change.

Step S3: calculating and obtaining the amplitude gain factor of the speech segment to be superimposed in each frame;

Specifically, the calculation to obtain the amplitude gain factor of the speech segment to be superimposed in each frame includes:

Obtain the length L of the speech segment to be superimposed in each frame;

Based on the length, the speech segment to be superimposed in each frame is divided into L samples; wherein, L is a positive integer;

Obtain the amplitude value corresponding to the sample point in each frame;

Obtain the target superposition position of the sample points in each frame and the original amplitude value of the target superposition position in each frame;

Based on the amplitude value and the original amplitude value, the amplitude gain factor of the speech segment to be superimposed in each frame is obtained by calculation.

In the embodiment of the present invention, the length of each speech segment to be superimposed is obtained by calculating, for example, when the length of the obtained speech segment to be superimposed is 50ms, then each speech segment to be superimposed is divided into 50 samples;

Then obtain the amplitude value corresponding to each sample point in the 50 sample points;

Determine the target stacking position of each sample point based on the WSOLA algorithm, that is, correspondingly, obtain 50 target stacking positions, and obtain the original amplitude values corresponding to the 50 target stacking positions;

The 50 amplitude values and the 50 original amplitude values are calculated and processed to obtain the amplitude gain factor of each speech segment to be superimposed.

Further, calculating and obtaining the amplitude gain factor of the speech segment to be superimposed in each frame based on the amplitude value and the original amplitude value includes:

Summing up the L amplitude values of the speech segments to be superimposed in each frame to obtain a first value M; wherein, M≥0;

Summing up the L original amplitude values of the speech segments to be superimposed in each frame to obtain a second value N; wherein, N≥0;

Based on the first value M and the second value N, the amplitude gain factor is obtained.

Further, obtaining the amplitude gain factor based on the first value M and the second value N includes:

Obtain the number Q of zero values in the L said amplitude values; wherein, Q≥0, and Q is an integer;

Based on the length L and the number Q of zeros in the L amplitude values, a reference coefficient λ of the amplitude gain factor is obtained; wherein, λ≥1;

Based on the reference coefficient λ, the first value M and the second value N, the amplitude gain factor is obtained.

In the embodiment of the present invention, without considering the reference coefficient λ, if the amplitude value of the position where a speech segment to be superimposed is to be superimposed is larger than its original position, that is, when N is greater than M, a The amplitude gain factor is greater than 1, otherwise it will reach an amplitude gain factor less than 1; the amplitude gain factor enables each frame to be superimposed to obtain a more reasonable target amplitude value, which is used to ensure the recovered speech waveform. Has the greatest similarity in amplitude to the original waveform.

However, when the number of zero-valued samples in the speech segment to be superimposed is large, the calculated value of the amplitude gain factor will be too small, which will seriously affect the recovery effect of speech; in order to solve this problem, the present invention In the embodiment, a reference coefficient λ is added, which is used to balance the situation of zero-valued large-area sample points caused by frame deletion, thereby achieving a better recovery effect.

Further, the amplitude gain factor is calculated and obtained based on the following formula:

Among them, β _k is the kth amplitude gain factor; k is a positive integer.

Specifically, the decomposed speech segments of n frames obtained by decomposing the same segment of speech segments to be processed are numbered, then K≤n;

Taking the speech segment to be superimposed in the K-th frame as an example, β _k obtained is the k-th amplitude gain factor.

In the embodiment of the present invention, although the lengths of the multi-frame speech segments to be superimposed divided based on a segment of speech segments to be processed are the same, the sum of the amplitude values of different speech segments to be superimposed or the sum of the corresponding original amplitude values It may not be the same. Therefore, the amplitude gain factor of each voice segment to be superimposed obtained from the same audio signal obtained by the above algorithm according to the embodiment of the present invention may also be different, thereby realizing the amplitude gain of different voice segments to be superimposed. adjustment effect.

Further, the reference coefficient is obtained based on the following calculation formula:

Step S4: Based on the amplitude gain factor, the amplitude of the to-be-superimposed speech segment in each frame is adjusted to determine the target amplitude;

Specifically, the target speech segment is calculated and obtained based on the following calculation formula:

S _k =P _k *β _k

Wherein, _Sk is the target speech segment of the Kth frame; _Pk is the speech segment to be superimposed in the Kth frame.

In the embodiment of the present invention, during the superposition process, the WSLOA algorithm is improved, and the amplitude gain factor is calculated and obtained, which is used to control the target amplitude value of the speech segment to be superimposed based on the amplitude gain factor.

Step S5: Obtain a target speech segment based on the target amplitude.

Specifically, multi-frame target speech segments are superimposed based on the WSOLA algorithm, and the restored audio waveform is output.

In the embodiment of the present invention, the amplitude of the speech segment to be superimposed is strictly controlled by increasing the amplitude gain factor, thereby solving the problem that large distortion may occur in the recovery process of real-time communication caused by the excessively large amplitude of the synthesized speech and the problems caused by network delays The missing frame happens at the end of a voice band, and there is a considerable difference between the adjusted voice and the original voice, which avoids the situation that the tail of the voice waveform is too large, and makes the restored audio closer to the original state. .

An embodiment of the present invention further provides a voice processing apparatus 400, including:

The decomposition module 401 is used to decompose the speech segment to be processed, and obtain multi-frame decomposed speech segments;

An adjustment module 402, configured to adjust the decomposed speech segments of multiple frames based on an adjustment algorithm, and obtain multiple frames of speech segments to be superimposed;

The calculation module 403 is used to calculate and obtain the amplitude gain factor of the speech segment to be superimposed in each frame;

A determination module 404, configured to adjust the amplitude of the speech segment to be superimposed in each frame based on the amplitude gain factor, and determine the target amplitude;

The obtaining module 405 is configured to obtain the target speech segment based on the target amplitude.

Optionally, the adjustment of the decomposed speech segments of multiple frames based on the adjustment algorithm to obtain the speech segments to be superimposed in multiple frames includes:

If the decomposed speech segment has frame missing or frame redundancy, determining the decomposed speech segment to be adjusted based on the decomposed speech segment;

The decomposed speech segment to be adjusted is adjusted to obtain the speech segment to be superimposed.

Optionally, if the decomposed speech segment has frame missing or frame redundancy, determining the decomposed speech segment to be adjusted based on the decomposed speech segment, including:

If the decomposed speech segment has frame missing;

then determining at least one frame of the decomposed speech segment to be adjusted based on the decomposed speech segment;

The to-be-adjusted and decomposed speech segment is stretched to obtain the to-be-superimposed speech segment.

Optionally, the calculating to obtain the amplitude gain factor of the speech segment to be superimposed in each frame, including:

Obtain the length L of the speech segment to be superimposed in each frame;

Based on the length, the speech segment to be superimposed in each frame is divided into L samples; wherein, L is a positive integer;

Obtain the amplitude value corresponding to the sample point in each frame;

Obtain the target superposition position of the sample points in each frame and the original amplitude value of the target superposition position in each frame;

Based on the amplitude value and the original amplitude value, the amplitude gain factor of the speech segment to be superimposed in each frame is obtained by calculation.

Optionally, calculating and obtaining the amplitude gain factor of the speech segment to be superimposed in each frame based on the amplitude value and the original amplitude value, including:

Summing up the L amplitude values of the speech segments to be superimposed in each frame to obtain a first value M; wherein, M≥0;

Summing up the L original amplitude values of the speech segments to be superimposed in each frame to obtain a second value N; wherein, N≥0;

Based on the first value M and the second value N, the amplitude gain factor is obtained.

Optionally, obtaining the amplitude gain factor based on the first value M and the second value N, including:

Obtain the number Q of zero values in the L said amplitude values; wherein, Q≥0, and Q is an integer;

Based on the length L and the number Q of zeros in the L amplitude values, a reference coefficient λ of the amplitude gain factor is obtained; wherein, λ≥1;

Based on the reference coefficient λ, the first value M and the second value N, the amplitude gain factor is obtained.

Optionally, the amplitude gain factor is calculated and obtained based on the following formula:

Among them, β _k is the kth amplitude gain factor; k is a positive integer.

Optionally, the reference coefficient is obtained based on the following calculation formula:

Optionally, the target speech segment is calculated and obtained based on the following calculation formula:

S _k =P _k *β _k

Wherein, _Sk is the target speech segment of the Kth frame; _Pk is the to-be-superimposed speech segment of the Kth frame. include

Embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, where the processor implements the above when executing the computer program the method described.

Embodiments of the present invention also provide a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program, wherein when the computer program runs, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned method described.

In addition, other structures and functions of the apparatus in the embodiment of the present invention are known to those skilled in the art, and in order to reduce redundancy, details are not described here.

It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered as an ordered listing of executable instructions for implementing the logical functions, and may be embodied in any computer readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in combination with these Used in order to execute a system, apparatus or device. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) with one or more frames of wiring, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, the multi-frame steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one frame of embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more of the frame embodiments or examples.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", " Rear, Left, Right, Vertical, Horizontal, Top, Bottom, Inner, Outer, Clockwise, Counterclockwise, Axial, The orientations or positional relationships indicated by "radial direction", "circumferential direction", etc. are based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying the indicated devices or elements. It must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be construed as a limitation of the present invention.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature defined with "first" and "second" may expressly or implicitly include at least one frame of the feature. In the description of the present invention, "multi-frame" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.

In the present invention, unless otherwise expressly specified and limited, the terms "installed", "connected", "connected", "fixed" and other terms should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection , or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between the two elements, unless otherwise specified limit. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

In the present invention, unless otherwise expressly specified and limited, a first feature "on" or "under" a second feature may be in direct contact between the first and second features, or the first and second features indirectly through an intermediary touch. Also, the first feature being "above", "over" and "above" the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is level higher than the second feature. The first feature being "below", "below" and "below" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature has a lower level than the second feature.

Although the embodiments of the present invention have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (12)

1. a speech processing method, is characterized in that, comprises: Perform decomposition processing on the speech segment to be processed to obtain multi-frame decomposed speech segments; Adjusting the decomposed speech segments of multiple frames based on an adjustment algorithm to obtain multiple frames of speech segments to be superimposed; Calculate the amplitude gain factor of each frame of the to-be-superimposed speech segment; Based on the amplitude gain factor, the amplitude of the speech segment to be superimposed in each frame is adjusted to determine the target amplitude; Based on the target amplitude, a target speech segment is obtained. 2. The method according to claim 1, wherein the adjusting the decomposed speech segments of multiple frames based on an adjustment algorithm to obtain the speech segments to be superimposed in multiple frames, comprising: If the decomposed speech segment has frame missing or frame redundancy, determining the decomposed speech segment to be adjusted based on the decomposed speech segment; The decomposed speech segment to be adjusted is adjusted to obtain the speech segment to be superimposed. 3. The method according to claim 2, wherein if the decomposed speech segment has frame missing or frame redundancy, then determining the decomposed speech segment to be adjusted based on the decomposed speech segment, comprising: If the decomposed speech segment has frame missing; then determining at least one frame of the decomposed speech segment to be adjusted based on the decomposed speech segment; The to-be-adjusted and decomposed speech segment is stretched to obtain the to-be-superimposed speech segment. 4. The method according to claim 1, wherein the calculating to obtain the amplitude gain factor of the speech segment to be superimposed in each frame, comprising: Obtain the length L of the speech segment to be superimposed in each frame; Based on the length, the speech segment to be superimposed in each frame is divided into L samples; wherein, L is a positive integer; Obtain the amplitude value corresponding to the sample point in each frame; Obtain the target superposition position of the sample points in each frame and the original amplitude value of the target superposition position in each frame; Based on the amplitude value and the original amplitude value, the amplitude gain factor of the speech segment to be superimposed in each frame is obtained by calculation. 5. The method according to claim 4, wherein the calculating and obtaining the amplitude gain factor of the speech segment to be superimposed in each frame based on the amplitude value and the original amplitude value, comprising: Summing up the L amplitude values of the speech segments to be superimposed in each frame to obtain a first value M; wherein, M≥0; Summing up the L original amplitude values of the speech segments to be superimposed in each frame to obtain a second value N; wherein, N≥0; Based on the first value M and the second value N, the amplitude gain factor is obtained. 6. The method according to claim 5, wherein the obtaining the amplitude gain factor based on the first value M and the second value N comprises: Obtain the number Q of zero values in the L said amplitude values; wherein, Q≥0, and Q is an integer; Based on the length L and the number Q of zeros in the L amplitude values, a reference coefficient λ of the amplitude gain factor is obtained; wherein, λ≥1; Based on the reference coefficient λ, the first value M and the second value N, the amplitude gain factor is obtained. 7. The method according to claim 5, wherein the amplitude gain factor is calculated and obtained based on the following formula:

Among them, β _k is the kth amplitude gain factor; k is a positive integer. 8. The method according to claim 6, wherein the reference coefficient is obtained based on the following calculation formula:

9. The method according to claim 7, wherein the target speech segment is calculated and obtained based on the following calculation formula: S _k =P _k *β _k Wherein, _Sk is the target speech segment of the Kth frame; _Pk is the to-be-superimposed speech segment of the Kth frame. 10. A voice processing device, comprising: The decomposition module is used to decompose the speech segment to be processed, and obtain multi-frame decomposed speech segments; an adjustment module, configured to adjust the decomposed speech segments of the multiple frames based on an adjustment algorithm, and obtain the speech segments to be superimposed in the multiple frames; A calculation module, for calculating and obtaining the amplitude gain factor of the speech segment to be superimposed in each frame; A determination module, configured to adjust the amplitude of the speech segment to be superimposed in each frame based on the amplitude gain factor, and determine the target amplitude; an obtaining module, configured to obtain a target speech segment based on the target amplitude. 11. An electronic device, characterized by comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, when the processor executes the computer program, the computer program as claimed in the claim is implemented The method of any one of claims 1 to 9. 12. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored computer program, wherein, when the computer program is run, the device where the computer-readable storage medium is located is controlled to perform as claimed in the claims The method of any one of 1 to 9.

Images