CN116364115A - Sound breaking detection method and device, electronic equipment and storage medium - Google Patents

Abstract

Embodiments of the present disclosure provide a broken sound detection method and device, electronic equipment, and a storage medium, which belong to the technical field of signal processing. The method comprises: dividing the audio signal to be detected into N time-domain signal frames; Confidence is used to distinguish between normal audio signals and clipped audio signals; according to the first clipping confidence of consecutive M time domain signal frames, determine the second clipping confidence corresponding to the detection window formed by consecutive M time domain signal frames degree; in response to the second clipping confidence degree being greater than the preset first threshold, it is determined that there is a broken sound within the corresponding detection window. The disclosure solves the problems of high false detection rate and low accuracy rate of broken sound detection caused by the existing fixed clipping detection threshold.

Description

Breaking sound detection method and device, electronic equipment, storage medium

technical field

The present disclosure relates to the technical field of signal processing, and more specifically, the embodiments of the present disclosure relate to a broken sound detection method and device, electronic equipment, and a storage medium.

Background technique

This section is intended to provide a background or context for the stated implementations of the disclosure. The descriptions herein are not admitted to be prior art by inclusion in this section.

Cracking sound, also known as popping sound, is a very harsh sound. In the presence of cracking sound, the original sound will become ambiguous. When there is a broken sound in the audio signal, the audio quality will be seriously affected. Therefore, it is necessary to detect the broken sound on the audio signal.

Since most broken sounds are caused by signal clipping, in related technologies, clipping is usually detected by a clipping threshold, and then broken sounds are judged by clipping. But in fact, clipping does not necessarily produce aurally broken sound, which is prone to false detection; and for scenes with large signal fluctuations, the accuracy of detection using a fixed clipping threshold is low.

Contents of the invention

Embodiments of the present disclosure provide a broken sound detection method and device, electronic equipment, and a storage medium.

In the first aspect of the embodiments of the present disclosure, a broken sound detection method is provided. The method includes: dividing the audio signal to be detected into N time-domain signal frames, where N is a positive integer; The statistical distribution of the signal amplitude in the frame determines the first clipping confidence level corresponding to the frame, and the first clipping confidence level is used to distinguish between normal audio signals and clipping audio signals; according to M consecutive time-domain signal frames The first clipping confidence level is determined to determine the second clipping confidence level corresponding to the detection window formed by consecutive M time-domain signal frames; the M is a positive integer smaller than the N; in response to the second clipping If the confidence level is greater than the preset first threshold, it is determined that there is a broken sound within the corresponding detection window.

Optionally, the determining the first clipping confidence level according to the statistical distribution of the signal amplitude in each frame of the time-domain signal includes: for each frame of the time-domain signal, counting the number of signals in each amplitude range , to construct a statistical histogram; the amplitude section is obtained by dividing the amplitude interval formed by the maximum amplitude and minimum amplitude of the time-domain signal frame; find the target block in the statistical histogram , the target block is a block whose both ends are higher than the middle of the block, and the back end of the block is higher than the front end of the block, and the front end of the block and the back end of the block are based on the search Determined in sequence; determine the lateral distance at both ends of the block in each of the target blocks; determine the first clipping according to the ratio of the maximum value in each of the lateral distances to the total number of the amplitude segments Confidence.

Optionally, the searching for the target block includes: respectively starting from both ends of the statistical histogram and moving to the middle in order to search for the target block.

Optionally, the method further includes: for each time-domain signal frame, determining the first clipping ratio of the frame according to the proportion of the target signal in the time-domain signal frame, the target signal being the amplitude The signal corresponding to the maximum value and the minimum value of the amplitude; according to the first clipping ratio of consecutive M time-domain signal frames, determine the second clipping ratio corresponding to the detection window; the determination that there is a broken sound in the detection window includes: a response When the second clipping confidence level is greater than a preset first threshold and the second clipping ratio is larger than a preset second threshold, it is determined that there is a broken sound within the detection window.

Optionally, the method further includes: determining the frequency-domain energy feature of the time-domain signal frame in the detection window; the determining that there is a broken sound in the detection window includes: responding to the signal frame in the detection window not satisfying at least one of the following : The second clipping confidence level is greater than a preset first threshold, and the second clipping ratio is larger than a preset second threshold, then according to the frequency domain energy feature, determine whether there is a broken sound in the detection window.

Optionally, the frequency-domain energy feature includes a cutoff frequency value, and the determining the frequency-domain energy feature of the time-domain signal frame in the detection window includes: performing time-frequency transformation on each time-domain signal frame in the detection window , to obtain a corresponding frequency domain signal; determining the energy center of gravity of the frequency domain signal as the cutoff frequency value of the signal frame.

Optionally, the determining whether there is a broken sound in the detection window according to the frequency domain energy feature includes: determining that there is clipping in the signal frame in response to the cutoff frequency value of the signal frame being greater than a frequency threshold, the The frequency threshold is determined based on the maximum frequency value of the frequency domain signal; in response to at least a preset number of signal frames with clipping within the detection window, it is determined that there is a broken sound in the detection window.

Optionally, the determining the corresponding second clipping confidence level according to the first clipping confidence level of consecutive M time-domain signal frames includes: the first clipping confidence level of the M time-domain signal frames in the detection window Perform the first weighting process on the wave confidence degree to obtain the second clipping confidence degree; said determining the second clipping ratio corresponding to the detection window according to the first clipping ratio of consecutive M time-domain signal frames includes: A second weighting process is performed on the first clipping ratios of the M time-domain signal frames in the detection window to obtain the second clipping ratio.

Optionally, the M is determined based on the minimum duration of continuous clipping to form a broken sound and the length of a time-domain signal frame.

In the second aspect of the embodiment of the present invention, there is provided a broken sound detection device, which is characterized in that the device includes: a signal division module configured to divide the audio signal to be detected into N time-domain signal frames, the N is a positive integer; the first determination module is configured to determine the first clipping confidence corresponding to the frame according to the statistical distribution of the signal amplitude in each frame of the time-domain signal, and the first clipping confidence is used For distinguishing between normal audio signals and clipped audio signals; the second determination module is configured to determine the detection window corresponding to the detection window formed by consecutive M time-domain signal frames according to the first clipping confidence of consecutive M time-domain signal frames The second clipping confidence degree; the M is a positive integer smaller than the N; the broken sound determination module is configured to determine the corresponding detection window memory in response to the second clipping confidence degree being greater than a preset first threshold In broken sound.

Optionally, the first determination module includes: a histogram construction module configured to, for each time-domain signal frame, count the number of signals in each amplitude range to construct a statistical histogram; the amplitude range The segment is obtained by dividing the amplitude interval formed by the maximum amplitude and the minimum amplitude of the time-domain signal frame; the search module is configured to search for the target block in the statistical histogram, and the target block Both ends of the block are higher than the middle of the block, and the back end of the block is higher than the front end of the block. The front end of the block and the back end of the block are determined according to the sequence of the search; the distance The determination module is configured to determine the lateral distance between both ends of the block in each of the target blocks; the confidence determination module is configured to determine the maximum value of each of the lateral distances according to the total number of the amplitude segments. ratio to determine the first clipping confidence.

Optionally, the search module is further configured to: respectively start from both ends of the statistical histogram and move to the middle in order to search for the target block.

Optionally, the device further includes a clipping ratio determining module, and the clipping ratio determining module is configured to: for each time-domain signal frame, determine the frame according to the proportion of the target signal in the time-domain signal frame The first clipping ratio of , the target signal is the signal corresponding to the maximum amplitude value and the minimum amplitude value; according to the first clipping ratio of consecutive M time-domain signal frames, determine the second clipping ratio corresponding to the detection window The broken sound determination module is also configured to: in response to the second clipping confidence being greater than a preset first threshold, and the second clipping ratio is greater than a preset second threshold, determine that there is a Breaking sound.

Optionally, the device further includes: a frequency domain feature determination module configured to determine the frequency domain energy feature of the time domain signal frame within the detection window; the broken sound determination module is further configured to: respond to If the signal frame does not meet at least one of the following: the second clipping confidence is greater than a preset first threshold, and the second clipping ratio is greater than a preset second threshold, then the detection is determined according to the frequency domain energy feature Whether there is a broken sound in the window.

Optionally, the frequency-domain energy feature includes a cutoff frequency value, and the frequency-domain feature determination module is further configured to: perform time-frequency transformation on each time-domain signal frame within the detection window to obtain a corresponding frequency-domain signal ; Determine the energy center of gravity of the frequency domain signal as the cutoff frequency value of the signal frame.

Optionally, the broken sound determination module is further configured to: determine that there is clipping in the signal frame in response to the cutoff frequency value of the signal frame being greater than a frequency threshold, and the frequency threshold is a maximum value based on the frequency domain signal The frequency value is determined; in response to at least a preset number of signal frames with clipping within the detection window, it is determined that there is a broken sound in the detection window.

Optionally, the second determination module is further configured to: perform a first weighting process on the first clipping confidences of the M time-domain signal frames within the detection window to obtain the second clipping confidences; The clipping ratio determining module is further configured to: perform a second weighting process on the first clipping ratios of the M time-domain signal frames within the detection window to obtain the second clipping ratio.

Optionally, the M is determined based on the minimum duration of continuous clipping to form a broken sound and the length of a time-domain signal frame.

In a third aspect of the embodiments of the present invention, a storage medium is provided, on which a program is stored, and when the program is executed by a processor, the methods in the above-mentioned embodiments are implemented.

In the fourth aspect of the embodiments of the present invention, an electronic device is provided, including: a processor and a memory, the memory stores executable instructions, and the processor is used to call the executable instructions stored in the memory to execute the method in the above-mentioned embodiments .

According to the broken sound detection method provided by the embodiments of the present disclosure, on the one hand, according to the statistical distribution of the signal amplitude in each frame of the time-domain signal frame, the first clipping confidence corresponding to the frame can be determined to distinguish normal audio signals And clipped audio signals, that is, the clipping threshold can be dynamically adjusted based on the statistical distribution of the signal, which solves the problem of clipping detection accuracy caused by the existing fixed clipping detection threshold, thereby ensuring the accuracy of broken sound detection . On the other hand, the second clipping confidence level corresponding to the detection window is determined by the first clipping confidence level of M consecutive time-domain signal frames, and the broken sound is detected with the detection window as the detection unit, so as to ensure that the detected broken sound is actually It avoids the false detection of short-term clipping without hearing the broken sound, and reduces the false detection rate of the broken sound. In addition, the invention detects broken sounds in the time domain, which has low computational complexity and high detection efficiency.

Description of drawings

The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the present disclosure are shown by way of illustration and not limitation, in which:

Fig. 1 schematically shows a flowchart of a method for detecting broken sounds according to an embodiment of the present disclosure.

Fig. 2 schematically shows a schematic diagram of a statistical histogram according to an embodiment of the present disclosure.

Fig. 3 schematically shows a flowchart of determining a first clipping confidence level of a signal frame based on a statistical histogram according to an embodiment of the present disclosure.

Fig. 4 schematically shows a flowchart of a broken sound detection process according to an embodiment of the present disclosure.

Fig. 5 schematically shows a diagram of a broken sound detection result of an audio signal according to an embodiment of the present disclosure.

Fig. 6 schematically shows a structural block diagram of a broken sound detection device according to an embodiment of the present disclosure.

Fig. 7 schematically shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present invention.

In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

Detailed ways

The principle and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and implement the present disclosure, rather than to limit the scope of the present disclosure in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Those skilled in the art know that the embodiments of the present disclosure may be implemented as a system, device, device, method or computer program product. Therefore, the present disclosure may be embodied in the form of complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

According to the embodiments of the present disclosure, a broken sound detection method and device, electronic equipment, and a storage medium are proposed.

Summary of the invention

Broken sound: Also known as popping sound, it is a phenomenon that the sound quality is obviously damaged due to the input audio signal exceeding the maximum range of the current digital representation of the audio signal. It is manifested as harsh sound, making the original sound unclear. From the spectrogram, the energy of each frequency band is very high.

Clipping: Also known as clipping, is a form of audio distortion that limits a signal above a certain threshold. From the waveform, the peaks and troughs are flattened, and the value becomes the threshold. Generally, recording equipment will set the threshold to 1 to prevent excessive power. Reaching a certain degree of clipping will lead to a broken sound in the sense of hearing.

Fast Fourier Transform (FFT): It is a fast algorithm of discrete Fourier transform, which is used to transform the time domain signal into the frequency domain. IFFT is the inverse transform of FFT, that is, the frequency domain signal is changed back to the time domain.

Mel Spectrum (Mel Spectrum): Based on the short-time Fourier transform, a long signal is divided into frames, windowed, and then FFT is performed on each frame, and finally the results of each frame are stacked along another dimension The resulting two-dimensional signal form (spectrogram).

The reason for the broken sound may be that the volume of the audio signal exceeds the upper limit of the analog-to-digital converter of the recording device during the process of collecting audio by the microphone. The reason for the broken sound may also be that the audio signal is amplified during the audio signal processing process, and the amplified result exceeds the maximum digital expression of the sound bit depth. In the sense of hearing, broken sound is manifested as hoarseness or noise.

Broken sound detection is to detect whether there is a broken sound in the audio signal and the specific location of the broken sound. Through the broken sound detection, the recording parameters or audio signal processing parameters can be adjusted in real time to avoid the broken sound; it can also be used to assist the broken sound repair algorithm The position where the broken sound is detected is targeted for repair.

In related technologies, the domain of analysis can be divided into time-domain broken sound detection and frequency-domain broken sound detection. The time domain mainly counts the values of each audio sampling point, and the frequency domain analyzes and judges by calculating the spectrum extraction features. But regardless of the time-domain or frequency-domain method, the broken sound is detected by detecting whether there is a clipping segment and the position of the clipping segment in the audio. In fact, short-term clipping does not cause audible cracks, that is, audible cracks are not completely equivalent to clipping.

In view of the above problems, the present invention is based on the idea of: determining the first clipping confidence level of the frame through the statistical distribution of signals in the time domain signal frame, so as to perform clipping detection of the frame, so that different frames can be realized. The dynamic clipping detection threshold can improve the accuracy of clipping detection; a broken sound detection window formed by continuous M time-domain signal frames is also designed, and the broken sound detection is performed with the detection window as a unit, reducing the short-term clipping as a broken sound. Sound false detection rate. Combining the intra-frame features and the continuous frame features of continuous multi-frames, the accurate detection of auditory cracks is realized.

The broken sound detection method of the present application can be used in the audio processing process of various application programs. For example, social application programs, music application programs, video application programs, game application programs, and other application programs with audio processing functions. For social applications, the broken sound detection method provided by this application can be used to detect in real time whether there is a broken sound and the location of the broken sound during a social call, so as to adjust device parameters or network status in time. For music applications, the broken sound detection method provided by this application can be used to detect in real time whether there is broken sound and the location of the broken sound part in playing music, so as to adjust device parameters or switch other audio to ensure user experience.

The methods and devices in the embodiments of the present disclosure can be applied to at least one of electronic devices including but not limited to servers and terminals that can be configured to execute the methods provided in the embodiments of the present application. In other words, the broken sound detection method can be executed by software or hardware installed on the terminal device or/and server device, and the software can be a block chain platform. The server includes but is not limited to: single server, server cluster, cloud server or cloud server cluster, etc. The present disclosure takes an electronic device as an example for description.

exemplary method

The preferred embodiments of the present disclosure will be described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention. The embodiments and the features in the embodiments can be combined with each other.

The following describes a broken sound detection method according to an exemplary embodiment of the present disclosure with reference to FIG. 1 , which may include steps S110-S140.

Step S110, dividing the audio signal to be detected into N time-domain signal frames, where N is a positive integer.

Step S120, according to the statistical distribution of signal amplitude in each frame of the time-domain signal, determine the first clipping confidence corresponding to the frame, and the first clipping confidence is used to distinguish normal audio signals from clipped audio signals.

Step S130, according to the first clipping confidence of M consecutive time-domain signal frames, determine the second clipping confidence corresponding to the detection window formed by M consecutive time-domain signal frames; M is a positive integer smaller than N.

Step S140, in response to the second clipping confidence being greater than the preset first threshold, it is determined that there is a broken sound within the corresponding detection window.

The broken sound detection method provided by the embodiments of the present disclosure, on the one hand, can determine the first clipping confidence corresponding to the frame according to the statistical distribution of the signal amplitude in each frame of the time-domain signal, so as to distinguish between normal audio signals and Clipping audio signals means that the clipping threshold can be dynamically adjusted based on the statistical distribution of the signal, which solves the problem of clipping detection accuracy caused by the existing fixed clipping detection threshold, thereby ensuring the accuracy of broken sound detection. On the other hand, the second clipping confidence level corresponding to the detection window is determined by the first clipping confidence level of M consecutive time-domain signal frames, and the broken sound is detected with the detection window as the detection unit, so as to ensure that the detected broken sound is actually It avoids the false detection of short-term clipping without hearing the broken sound, and reduces the false detection rate of the broken sound. In addition, the invention detects broken sounds in the time domain, which has low computational complexity and high detection efficiency.

The above steps will be described in detail below.

In step S110, the audio signal to be detected is divided into N time-domain signal frames.

In this example embodiment, the audio signal to be detected may be a time-domain audio signal, and its source may be recorded audio, real-time call audio, or music audio to be played, etc., which is not limited in this example. The audio signal to be detected can be divided into frames according to a fixed duration, such as 5 milliseconds, 10 milliseconds, 15 milliseconds, etc., or the signal frame can be divided according to a set frame length and overlap rate, which is not limited in this example. The overlap rate refers to the degree of signal overlap. Signal overlap means that there is a certain length of overlap between two adjacent frames of signals. Through signal overlap, part of the real signal consumed by window function weighting can be eliminated. N is the number of divided signal frames, and the value may be an integer greater than 1. It is also possible to record the time stamp of each signal frame to distinguish the signal frame and its sequence.

In step S120, the first clipping confidence level corresponding to the frame is determined according to the statistical distribution of the signal amplitude in each frame of the time-domain signal.

In this example implementation, for each time-domain signal frame, statistics may be performed on the distribution of signal amplitudes, and the first clipping confidence level may be determined based on the statistical results. The normal audio signal is distinguished from the clipped audio signal using the first clipping confidence level.

Exemplarily, the first clipping confidence level may be determined through the following steps.

In the first step, for each time-domain signal frame, the number of signals in each amplitude range is counted to construct a statistical histogram.

In this exemplary embodiment, the amplitude interval formed by the maximum amplitude and the minimum amplitude of the time-domain signal frame may be segmented, for example, the amplitude interval is equally divided into K (K is a positive integer) segments, The number of signals in each amplitude segment is counted, and a statistical histogram is constructed. As shown in Figure 2, the abscissa of the histogram is the number of the amplitude segment (such as 1, 2, ... K), and the ordinate is the number of signals.

The second step is to find the target block in the statistical histogram.

In this exemplary embodiment, the target block refers to a block whose both ends of the block in the statistical histogram are higher than the middle of the block, and the back end of the block is higher than the front end of the block, and the front end of the block and the back end of the block are Determined according to the order of searching and traversing statistical histograms, the histogram traversed first is the front end of the block, and the histogram traversed later is the back end of the block. The target block is shown in the dotted box in Figure 2. The target block can be found by cyclic traversal. For example, search from the left end to the right end of the statistical histogram, find a block whose both ends are higher than the middle of the block, and the right end of the block is higher than the left end of the block as the target block. It is also possible to search from the right end to the left end of the statistical histogram to find a block whose both ends of the block are higher than the middle of the block and whose left end is higher than the right end of the block as the target block. Exemplarily, it is also possible to start from both ends (ie, the left end and the right end) of the statistical histogram at the same time, and move to the middle in order to search for the target block, so as to speed up the search.

The third step is to determine the lateral distance between the two ends of the block in each target block.

In this exemplary implementation, the difference between the number at the right end of the block and the number at the left end of the block can be used to obtain the horizontal distance, as shown in FIG. 2 , the horizontal distance is k2-k1, 1≤k1<k2≤K.

Step 4: Determine the first clipping confidence level according to the ratio of the maximum value in each lateral distance to the total number of amplitude segments.

In this example implementation, when the number of target blocks is 1, the first clipping confidence level may be determined by making a business between the lateral distance of the target block and the amplitude range. When the number of target blocks is greater than 1, the target block with the largest lateral distance may be selected to determine the first clipping confidence level.

For example, it is assumed that for a k-th amplitude segment of the K amplitude segments, the number of signals falling within this segment is H(k). As shown in Fig. 3, the first clipping confidence can be determined through the following steps.

Step S301, initialize the left histogram quantity Yl0=H(kl), the right histogram quantity Yr0=H(kr), the left distance dl (the horizontal distance from the left pointer to the left end of the histogram) and the right distance dr (the right pointer to the histogram The lateral distance at the right end) is 0, that is, dl=dr=0, the left pointer kl=1, the right pointer kr=K; the maximum lateral distance parameter Dmax=0.

In step S302, the left pointer kl is increased by 1, and the left distance dl is increased by 1.

Step S303, determine in the statistical histogram whether the left histogram quantity H(kl) corresponding to the kl pointer is greater than Yl0, if yes, go to S304, otherwise go to S308.

Step S304, update Yl0=H(kl) and reset dl to 0, otherwise, go to step S340. This step is a traversal process from left to middle.

In step S305, the right pointer kr is decreased by 1, and the right distance dr is increased by 1.

Step S306, determine in the statistical histogram whether the number of right histograms H(kr) corresponding to the kr pointer is greater than Yr0, if yes, go to S307, otherwise go to S308.

Step S307, update Yr0=H(kr), and reset dr to 0, otherwise, go to step S340. This step is a traversal process from right to middle.

Step S308, updating Dmax to be the maximum value among Dmax, dl and dr at this time.

Step S309, judge whether kr is smaller than kl, if so, go to step S310, otherwise go to steps S302 and S305, and enter the next cycle.

Step S310, determine the first clipping confidence Rcl=Dmax/K of the time frame.

In the above steps, S302-S303 and S305-306 can be performed synchronously, and the first clipping confidence level of each signal frame can be determined and recorded according to the above process.

In step S130, according to the first clipping confidence levels of the consecutive M time-domain signal frames, a second clipping confidence level corresponding to the detection window formed by the M consecutive time-domain signal frames is determined.

In this example implementation, M is a positive integer less than N, M can be determined based on the minimum duration of continuous clipping to form a broken sound and the length of the time-domain signal frame, and the minimum duration of continuous clipping to form a broken sound can be based on experience and reality The situation is determined. For example, the minimum hour length can be set to 0.4 seconds. The ratio between the minimum hour length and the signal frame length can be rounded up and taken as M, or the ratio can be properly enlarged and rounded up as M. In this example, This is not limited. M consecutive time-domain signal frames may be formed into a detection window for broken sounds, that is, sound broken detection is performed on each detection window.

In this exemplary embodiment, a correlation operation may be performed on the first clipping confidence levels of M consecutive time-domain signal frames to determine the second clipping confidence level. For example, the correlation operation may be an averaging operation, a weighting operation or aggregation processing, which is not limited in this example.

Exemplarily, a first weighting process may be performed on the first clipping confidences of the M time-domain signal frames within the detection window to obtain the second clipping confidences.

In this example embodiment, the first weighting process refers to weighted summation processing, and the sum of weights corresponding to M signal frames may be 1, for example, Gaussian weighting processing may be performed on the first clipping confidence of each M signal frame , to obtain the second clipping confidence level, and the Gaussian weighting process means that when calculating the average value, the numerical calculation weights of different positions are different. Gaussian weighting means that the weight value follows a Gaussian distribution, with small ends and a large middle.

In step S140, in response to the second clipping confidence being greater than the preset first threshold, it is determined that there is a broken sound within the corresponding detection window.

In this example embodiment, the first threshold refers to the broken sound threshold of the detection window, which can be set according to actual conditions. In a case where the second clipping confidence level of the detection window is greater than the first threshold, it may be considered that there is a broken sound in the detection window.

In some embodiments, in addition to the clipping confidence, a clipping ratio may also be added as a crack detection feature, and the method may further include the following steps.

For each time-domain signal frame, the first clipping ratio of the frame is determined according to the proportion of the target signal in the time-domain signal frame. A second clipping ratio corresponding to the detection window is determined according to the first clipping ratios of M consecutive time-domain signal frames.

In this exemplary embodiment, the target signal is a signal corresponding to a maximum amplitude value and a minimum amplitude value. Since the essence of clipping is that when the absolute value of the signal is higher than the threshold value, it will be cut to the threshold value, so there must be multiple sampling points whose values are equal to the threshold value in the clipping segment, and the first clipping ratio is determined based on this. For each time-domain signal frame, first determine the maximum amplitude value and the minimum amplitude value, and count the number n1 of sampling points of the maximum amplitude value and minimum amplitude value in the frame, then the first clipping ratio of the frame is n1 /n2, n2 is the total number of sampling points of the frame.

In this exemplary embodiment, a correlation operation may be performed on the first clipping ratios of M consecutive time-domain signal frames to determine the second clipping ratio. For example, the correlation operation may be an averaging operation, a weighting operation or aggregation processing, which is not limited in this example. Exemplarily, a second weighting process may be performed on the first clipping ratios of the M time-domain signal frames within the detection window to obtain the second clipping ratio. The second weighting process may be Gaussian weighting process, and the second weighting process may be the same as or different from the first weighting process, which is not limited in this example.

The broken sound detection strategy based on the two features of clipping confidence and clipping ratio is: in response to the second clipping confidence being greater than the preset first threshold, and the second clipping ratio being larger than the preset second threshold, determine the There is a broken sound in the detection window.

In this example implementation, the second threshold is the threshold of the broken sound in the clipping ratio dimension, which can be set according to actual conditions. By using the features of the two dimensions of clipping confidence and clipping ratio to determine whether there is a broken sound in the detection window, the detection accuracy of the broken sound can be improved.

In practice, in addition to obvious clipping segments, there are also some scenes where the signal value is close to the clipping threshold but the actual hearing is broken. For this scene, the present invention is based on the characteristics of clipping, that is, clipping will cause high-frequency energy to be higher than normal. Audio is higher and detected by adding frequency domain features.

In some embodiments, the method further comprises: determining a frequency domain energy signature of the time domain signal frame within the detection window.

In this example embodiment, the frequency domain energy feature may include various frequency domain energy indicators, such as power value, energy maximum value, energy center of gravity, cutoff frequency value, etc., which is not limited in this example.

Exemplarily, the frequency-domain energy feature includes a cut-off frequency value, and time-frequency transformation can be performed on each frame of the time-domain signal frame in the detection window to obtain the corresponding frequency-domain signal; and then the energy center of gravity of the frequency-domain signal is determined as the signal The cutoff frequency value for frames.

In this exemplary embodiment, windowing processing and FFT may be performed sequentially on each signal frame to obtain a corresponding Mel spectrum, and then the energy distribution center of gravity of the Mel spectrum, that is, the frequency Fc, is determined. The center of gravity of the energy distribution can be determined by the ratio of the sum of the energy of each frequency position (the product of the amplitude value of the position and the frequency value) to the total number of frequency positions.

In a case where the signal frame in a detection window does not satisfy: the second clipping confidence is greater than the preset first threshold or/and the second clipping ratio is greater than the preset second threshold, the detection window may be determined based on the cutoff frequency value Whether there is a broken sound in it.

Exemplarily, by comparing the cutoff frequency value of the signal frame with the frequency threshold, it is determined that there is clipping in the signal frame if the cutoff frequency value is greater than the frequency threshold. The frequency threshold may be determined based on the maximum frequency value of the frequency domain signal, for example, the frequency threshold may be set to 1/3˜1/4 of the maximum frequency value. It is then determined whether the number of signal frames with clipping within the detection window reaches a preset number (such as 2), and if the number reaches the preset number, it is determined that there is a broken sound in the detection window. The frequency-domain energy feature avoids the missed detection of the special case of no clipping but hearing-breaking sound, and improves the detection accuracy.

As shown in FIG. 4 , the specific flow of a broken sound detection method according to the embodiment of the present application will be introduced below.

Step S401, divide the audio signal to be detected into N time-domain signal frames.

Step S402: Determine the first clipping confidence level corresponding to the frame according to the statistical distribution of the signal amplitude in each frame of the time-domain signal.

Step S403, performing a first weighting process on the first clipping confidence levels of the M time-domain signal frames within the detection window to obtain a second clipping confidence level.

Step S404, for each time-domain signal frame, determine the first clipping ratio of the frame according to the proportion of the target signal in the time-domain signal frame, and the target signal is corresponding to the maximum amplitude value and the minimum amplitude value Signal.

Step S405, performing a second weighting process on the first clipping ratios of the M time-domain signal frames within the detection window to obtain a second clipping ratio.

Step S406, performing time-frequency transformation on each frame of the time-domain signal within the detection window to obtain a corresponding frequency-domain signal.

Step S407, determining the energy center of gravity of the frequency domain signal as the cutoff frequency value of the signal frame.

Step S408, judging whether the second clipping confidence of the current detection window is greater than the first threshold, and whether the second clipping ratio is greater than the second threshold, if yes, go to step S409, otherwise go to step S410.

In step S409, it is determined that there is a broken sound in the current detection window.

Step S410, judging whether the number of target signal frames in the current detection window is greater than the preset number, if yes, go to S409, otherwise, go to step S411. The target signal frame is a signal frame with a cutoff frequency value greater than a frequency threshold.

Step S411, slide forward by one step (for example, one time frame), go to step S402, and perform broken sound detection on the next detection window.

In the above-mentioned embodiment, the determination process of breaking sound judgment features: clipping confidence (S402, S403), clipping ratio (S404, S405) and frequency domain energy features (S406, S407) can be carried out simultaneously to speed up the detection efficiency .

The specific details of the above embodiment have been described in detail in the foregoing broken sound detection method, so details are not repeated here.

Experimental verification

A section of audio signal is input to the electronic equipment with the broken sound detection method proposed by the present invention, and the broken sound detection is carried out. The detection result is as shown in Figure 5, and the dotted line frame selection part in the figure is the broken sound part detected, in the audio signal Both ends at the beginning and the end, indicating that this part has a sense of hearing breaking. At the same time, it can be seen that the detected broken sound part has clipping for a long time, but the middle part of the audio signal also has clipping phenomenon, but the broken sound is not detected by the method of the present invention, and this part belongs to clipping. The sound is wave but there is no broken sound part. However, if the existing method of detecting broken sound based on clipping is directly adopted, it is obvious that there is broken sound in the middle part. Experiments show that the detection accuracy of broken sound by the method of the present invention is higher.

Although the vast majority of broken sounds are caused by signal clipping, clipping is not equivalent to broken sounds, because only clipping that lasts for a certain period of time can form aurally broken sounds. Based on this, on the one hand, the present invention extracts multiple groups of clipping features such as clipping confidence and clipping ratio in the detection window, and performs combined analysis on multiple groups of clipping features of signal frames (detection windows) that last for a certain period of time, effectively avoiding False detection of clipped but not broken audio clips. On the other hand, by adding frequency-domain energy features to deal with the scene where the signal is not clipped but the energy is large and the sound is broken, the missed detection of the scene is effectively avoided. Specifically, considering that a signal frame with a large energy center of gravity is a common feature of clipping and the special scene, the special scene is detected based on the energy center of gravity to reduce the missed detection rate.

The present invention also determines the clipping confidence level by statistical signal distribution, cleverly avoids the setting of the clipping threshold, and avoids the problem of clipping detection failure caused by unreasonable selection of the threshold value, and the clipping confidence level is determined by the time-domain signal frame Compared with the frequency domain determination process, the computational complexity is greatly reduced. At the same time, the determined clipping confidence can be continuously distributed between 0 and 1, which can provide greater flexibility for the subsequent sound break detection strategy compared with the traditional dichotomous situation.

Exemplary device

It should be noted that, the broken sound detection method provided by the embodiment of the present disclosure may be executed by a corresponding device. Next, a broken sound detection device according to an exemplary embodiment of the present disclosure will be described first with reference to FIG. 6 .

Fig. 6 schematically shows a block diagram of a broken sound detection device according to an embodiment of the present invention.

Referring to Fig. 6, according to an embodiment of the broken sound detection device 600 of the present invention, the device 600 may include: a signal division module 610, a first determination module 620, a second determination module 630 and a broken sound determination module 640, wherein: The signal division module 610 may be configured to divide the audio signal to be detected into N time-domain signal frames, where N is a positive integer; the first determination module 620 may be configured according to the signal amplitude in each frame of the time-domain signal Statistical distribution, determine the first clipping confidence corresponding to the frame, the first clipping confidence is used to distinguish normal audio signals and clipping audio signals; the second determination module 630 can be configured to The first clipping confidence level of the signal frame determines the second clipping confidence level corresponding to the detection window formed by consecutive M time-domain signal frames; M is a positive integer less than N; the broken sound determination module 640 can be configured as In response to the second clipping confidence being greater than the preset first threshold, it is determined that a broken sound exists within the corresponding detection window.

In some embodiments of the present disclosure, based on the foregoing solution, the first determination module 620 includes: a histogram construction module configured to, for each time-domain signal frame, count the number of signals in each amplitude segment to construct statistics Histogram; the amplitude section is obtained by dividing the amplitude interval formed by the maximum amplitude and minimum amplitude of the time domain signal frame; the search module is configured to search for the target block in the statistical histogram, the target A block is a block in which both ends of the block are higher than the middle of the block, and the back end of the block is higher than the front end of the block. The front end of the block and the back end of the block are determined according to the order of search; the distance determination module is It is configured to determine the lateral distance between both ends of the block in each target block; the confidence degree determining module is configured to determine the first clipping confidence degree according to the ratio of the maximum value in each lateral distance to the total number of amplitude segments.

In some embodiments of the present disclosure, based on the aforementioned solutions, the search module is further configured to: start from both ends of the statistical histogram respectively, move to the middle in sequence, and search for the target block.

In some embodiments of the present disclosure, based on the foregoing solution, the apparatus 600 further includes a clipping ratio determination module, and the clipping ratio determination module is configured to: for each time-domain signal frame, according to the target signal in the frame time-domain signal frame Determine the first clipping ratio of the frame, the target signal is the signal corresponding to the maximum amplitude value and the minimum amplitude value; according to the first clipping ratio of consecutive M time-domain signal frames, determine the corresponding detection window The second clipping ratio; the broken sound determination module 640 is also configured to: in response to the second clipping confidence being greater than the preset first threshold, and the second clipping ratio is greater than the preset second threshold, determine the detection window memory In broken sound.

In some embodiments of the present disclosure, based on the foregoing solution, the device 600 further includes: a frequency domain feature determination module configured to determine the frequency domain energy feature of the time domain signal frame within the detection window; the broken sound determination module 640 is also configured It is: in response to the signal frame in the detection window not satisfying at least one of the following: the second clipping confidence is greater than the preset first threshold, and the second clipping ratio is greater than the preset second threshold, then according to the frequency domain energy feature, determine the Check whether there is a broken sound in the window.

In some embodiments of the present disclosure, based on the foregoing solution, the frequency-domain energy feature includes a cutoff frequency value, and the frequency-domain feature determination module is further configured to: perform time-frequency transformation on each frame of the time-domain signal frame within the detection window to obtain Corresponding frequency domain signal; determine the energy center of gravity of the frequency domain signal as the cutoff frequency value of the signal frame.

In some embodiments of the present disclosure, based on the foregoing solution, the broken sound determination module 640 is further configured to: determine that there is clipping in the signal frame in response to the cutoff frequency value of the signal frame being greater than a frequency threshold, and the frequency threshold is based on the frequency domain signal Determined by the maximum frequency value of ; in response to at least a preset number of signal frames with clipping within the detection window, it is determined that there is a broken sound in the detection window.

In some embodiments of the present disclosure, based on the foregoing solution, the second determination module 630 is further configured to: perform a first weighting process on the first clipping confidences of the M time-domain signal frames within the detection window to obtain the first Two clipping confidence levels; the clipping ratio determination module is further configured to: perform a second weighting process on the first clipping ratios of the M time-domain signal frames within the detection window to obtain a second clipping ratio.

In some embodiments of the present disclosure, based on the foregoing solution, M is determined based on the minimum duration of continuous clipping to form a broken sound and the length of a time-domain signal frame.

The specific details of each module or unit in the above broken sound detection device have been described in detail in the corresponding broken sound detection method, so details will not be repeated here.

Exemplary medium

After introducing the method of the exemplary embodiment of the present invention, next, the medium of the exemplary embodiment of the present invention will be described.

In some possible implementations, various aspects of the present invention can also be implemented as a storage medium, on which program code is stored, and when the program code is executed by the processor of the device, it is used to implement the above-mentioned "exemplary method" in this specification The steps in the crack detection method according to various exemplary embodiments of the present invention are described in the section.

Specifically, the processor of the device is used to implement the following steps when executing the program code:

The audio signal to be detected is divided into N time-domain signal frames, and N is a positive integer; according to the statistical distribution of the signal amplitude in each frame of the time-domain signal frame, the first clipping confidence corresponding to the frame is determined, and the first clipping Wave confidence is used to distinguish between normal audio signals and clipped audio signals; according to the first clipping confidence of consecutive M time-domain signal frames, determine the second clipping corresponding to the detection window formed by consecutive M time-domain signal frames Confidence: M is a positive integer smaller than N; in response to the second clipping confidence being greater than the preset first threshold, it is determined that there is a broken sound in the corresponding detection window.

The foregoing is a schematic solution of a computer-readable storage medium in this embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the above-mentioned broken sound detection method belong to the same concept, and the details of the technical solution of the storage medium that are not described in detail can be found in the description of the technical solution of the above-mentioned broken sound detection method .

It should be noted that: the above-mentioned storage medium may be a readable storage medium. The readable storage medium may be, for example but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

The program code contained on the readable storage medium can be transmitted by any appropriate medium, including but not limited to: wireless, cable, optical cable, RF, etc., or any suitable combination of the above.

Program codes for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., as well as conventional procedural programming Language - such as "C" or similar programming language. The program code may execute entirely on the user electronic device, partly on the user electronic device and partly on the remote electronic device, or entirely on the remote electronic device or server. In cases involving a remote electronic device, the remote electronic device may be connected to the user electronic device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it may be connected to an external electronic device (such as by using an Internet service Provider via Internet connection).

Exemplary electronic device

After introducing the method, medium, and apparatus of the exemplary embodiment of the present disclosure, next, an electronic device according to another exemplary embodiment of the present disclosure is introduced.

Those skilled in the art can understand that various aspects of the present invention can be implemented as systems, methods or program products. Therefore, various aspects of the present invention can be embodied in the following forms, that is: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which can be collectively referred to herein as "circuit", "module" or "system".

An electronic device 700 according to this embodiment of the present invention is described below with reference to FIG. 7 . The electronic device 700 shown in FIG. 7 is only an example, and should not limit the functions and application scope of the embodiments of the present invention.

As shown in FIG. 7, an electronic device 700 is represented in the form of a general electronic device. Components of the electronic device 700 may include but not limited to: at least one processing unit 710 , at least one storage unit 720 , and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710 ).

Wherein, the storage unit stores program codes, and the program codes can be executed by the processing unit 710, so that the processing unit 710 executes the steps according to various exemplary embodiments of the present invention described in the "Exemplary Methods" section of this specification.

The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and/or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .

The storage unit 720 may also include a program/utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, Implementations of networked environments may be included in each or some combination of these examples.

Bus 730 may represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local area using any of a variety of bus structures. bus.

The electronic device 700 can also communicate with one or more external devices (such as keyboards, pointing devices, Bluetooth devices, etc.), and can also communicate with one or more devices that enable the user to interact with the electronic device 700, and/or communicate with the device that enables the user to interact with the electronic device 700. The electronic device 700 is capable of communicating with any device (eg, router, modem, etc.) that communicates with one or more other electronic devices. Such communication may be performed through the display unit 740 and an input/output (I/O) interface 750 connected to the display unit 740 . Moreover, the electronic device 700 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and/or a public network such as the Internet) through the network adapter 760 . As shown, the network adapter 760 communicates with other modules of the electronic device 700 through the bus 730 . It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

Through the description of the above implementations, those skilled in the art can easily understand that the example implementations described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of software products, and the software products can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make an electronic device (which may be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

The foregoing is a schematic solution of an electronic device 700 in this embodiment. It should be noted that the technical solution of the electronic device 700 belongs to the same concept as the technical solution of the above-mentioned broken sound detection method, and details that are not described in detail in the technical solution of the electronic device can be referred to in the technical solution of the above-mentioned broken sound detection method. describe.

It should be noted that although several modules or sub-modules of the crack detection device are mentioned in the above detailed description, this division is only exemplary and not mandatory. Actually, according to the embodiment of the present invention, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided to be embodied by a plurality of modules or units.

In addition, while operations of the methods of the present invention are depicted in the figures in a particular order, there is no requirement or implication that these operations must be performed in that particular order, or that all illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution.

Although the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the specific embodiments of the invention, nor does division of aspects imply that features in these aspects cannot be combined to achieve Benefit, this division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims (10)

1. a broken sound detection method, is characterized in that, described method comprises: Dividing the audio signal to be detected into N time-domain signal frames, where N is a positive integer; According to the statistical distribution of the signal amplitude in each frame of the time-domain signal, determine the first clipping confidence level corresponding to the frame, and the first clipping confidence level is used to distinguish between normal audio signals and clipping audio signals; According to the first clipping confidence of consecutive M time-domain signal frames, determine the second clipping confidence corresponding to the detection window formed by consecutive M time-domain signal frames; the M is a positive integer smaller than the N; In response to the second clipping confidence being greater than a preset first threshold, it is determined that there is a broken sound within the corresponding detection window. 2. The method according to claim 1, wherein, determining the first clipping confidence level according to the statistical distribution of the signal amplitude in each frame of the time-domain signal frame includes: For each time-domain signal frame, the number of signals in each amplitude section is counted to construct a statistical histogram; the amplitude section is the amplitude formed by the maximum amplitude and the minimum amplitude of the time-domain signal frame obtained by dividing the value range; Find the target block in the statistical histogram, the target block is a block whose both ends are higher than the middle of the block, and the back end of the block is higher than the front end of the block, the front end of the block and the back end of the block The terminal is determined according to the sequence of the search; determining the lateral distance between the two ends of the blocks in each of the target blocks; The first clipping confidence is determined according to a ratio of a maximum value in each of the lateral distances to the total number of amplitude segments. 3. The method according to claim 2, wherein the searching for the target block comprises: Starting from both ends of the statistical histogram respectively, moving to the middle in order to search for the target block. 4. The method according to claim 1, wherein the method further comprises: For each time-domain signal frame, according to the proportion of the target signal in the frame of the time-domain signal frame, determine the first clipping ratio of the frame, and the target signal is a signal corresponding to the maximum amplitude value and the minimum amplitude value ; Determine a second clipping ratio corresponding to the detection window according to the first clipping ratio of consecutive M time-domain signal frames; The determination that there is a broken sound in the detection window includes: In response to the second clipping confidence level being greater than a preset first threshold and the second clipping ratio being larger than a preset second threshold, it is determined that there is a broken sound within the detection window. 5. method according to claim 4, is characterized in that, described method also comprises: determining the frequency-domain energy characteristics of the time-domain signal frame within the detection window; The determination that there is a broken sound in the detection window includes: In response to the signal frame within the detection window not satisfying at least one of the following: The second clipping confidence level is greater than a preset first threshold, The second clipping ratio is greater than a preset second threshold, Then, according to the frequency-domain energy feature, it is determined whether there is a broken sound within the detection window. 6. The method according to claim 5, wherein the frequency-domain energy feature comprises a cutoff frequency value, and the determination of the frequency-domain energy feature of the time-domain signal frame in the detection window comprises: performing time-frequency transformation on each frame of the time-domain signal frame in the detection window to obtain a corresponding frequency-domain signal; Determine the energy center of gravity of the frequency domain signal as the cutoff frequency value of the signal frame. 7. The method according to claim 6, wherein said determining whether there is a broken sound in the detection window according to the frequency-domain energy feature comprises: determining that there is clipping in the signal frame in response to the cutoff frequency value of the signal frame being greater than a frequency threshold, the frequency threshold being determined based on the maximum frequency value of the frequency domain signal; In response to there being at least a preset number of signal frames with clipping within the detection window, it is determined that a broken sound exists in the detection window. 8. The method according to claim 4, wherein said determining the corresponding second clipping confidence level according to the first clipping confidence level of consecutive M time-domain signal frames comprises: performing a first weighting process on the first clipping confidences of the M time-domain signal frames within the detection window to obtain the second clipping confidences; The determining the second clipping ratio corresponding to the detection window according to the first clipping ratio of consecutive M time-domain signal frames includes: Performing a second weighting process on the first clipping ratios of the M time-domain signal frames within the detection window to obtain the second clipping ratio. 9. A broken sound detection device, characterized in that the device comprises: A signal division module configured to divide the audio signal to be detected into N time-domain signal frames, where N is a positive integer; The first determination module is configured to determine the first clipping confidence level corresponding to the frame according to the statistical distribution of the signal amplitude in each frame of the time domain signal, and the first clipping confidence level is used to distinguish normal audio signals and clip audio signals; The second determination module is configured to determine the second clipping confidence corresponding to the detection window formed by the continuous M time-domain signal frames according to the first clipping confidence of consecutive M time-domain signal frames; the M is a positive integer less than said N; The broken sound determining module is configured to determine that there is broken sound within the corresponding detection window in response to the second clipping confidence being greater than a preset first threshold. 10. An electronic device, comprising: a processor and a memory, the memory stores executable instructions, and the processor is used to call the executable instructions stored in the memory to execute as described in any one of claims 1 to 8 Methods.

Images