CN116962955A - Multi-channel sound mixing method, equipment and medium - Google Patents

Abstract

This application relates to the field of audio processing technology, and specifically to a multi-channel mixing method, equipment and medium. The method includes: acquiring first multi-channel audio data, the first multi-channel audio data including audio data of M channels to be mixed; determining that there is energy in the first multi-channel audio data that meets a preset energy threshold Audio data, and perform energy reduction processing on audio data whose energy is greater than the preset energy threshold in the first multi-channel audio data; obtain the second multi-channel audio data according to the energy reduction processing result; perform energy reduction processing on the second multi-channel audio data Perform downmixing to obtain mixing output data with N mixing channels, where M>N, and N≥1. The multi-channel mixing method provided in the embodiment of the present application can solve the problem of sound breakage caused by excessive energy of some audio frames during channel downmixing, obtain a more ideal channel downmixing result, and improve the user's experience. Auditory experience.

Description

Multi-channel mixing method, device and medium

Technical Field

The present application relates to the field of audio processing technology, and in particular to a multi-channel mixing method, device and medium.

Background Art

With the rapid development of modern technology, in various scenarios where audio playback is required, due to the mismatch between audio data and the number of channels of the audio output device, it is often necessary to complete real-time multi-channel mixing when outputting audio, generally converting multi-channel audio data into audio data with fewer channels, that is, channel downmixing. For example, when playing AiMax sources on a large screen, there will be multi-channel audio data such as 3.1, 5.1, and 7.1, but when the large-screen output device is switched to a digital audio interface (Sony/Philips Digital Interface, spdif)/audio return channel (Audio Return Channel, ARC)/Bluetooth output, there is a situation where only two channels are output. In order to retain as much audio stream information as possible, it is necessary to downmix the data of multiple channels to generate two-channel data.

Currently, the following two solutions are generally used for downmixing of multi-channel audio to two channels:

1) Use the first two channels of multi-channel data as output, and discard the center channel, surround channel, bass channel, etc. When outputting, this solution will cause the loss of human voice because some human voice audio data appears in the discarded channels. At the same time, since only two channels are used as output, the user's listening experience will be reduced.

2) Dolby downmixing scheme is used to perform weighted summation of the data related to the left and right channels in the multi-channel audio data to obtain the two-channel audio data output. However, for audio data that does not meet Dolby specifications, for example, for audio data with high bass audio data energy, when the Dolby downmixing scheme is used for channel downmixing, the sound will be broken, resulting in a poor listening experience for the user.

Summary of the invention

The embodiments of the present application provide a multi-channel mixing method, device and medium, which solve the problem of distortion of audio data after downmixing in the current channel downmixing solution, affecting the user's auditory experience.

In a first aspect, an embodiment of the present application provides a multi-channel mixing method, which is applied to an electronic device, including: obtaining first multi-channel audio data, the first multi-channel audio data including audio data of M channels to be mixed; determining that there is audio data in the first multi-channel audio data whose energy meets a preset energy threshold, and performing energy reduction processing on the audio data in the first multi-channel audio data whose energy is greater than the preset energy threshold; obtaining second multi-channel audio data based on the energy reduction processing result; down-mixing the second multi-channel audio data to obtain mixed output data with N mixed channels, where M>N and N≥1.

It can be understood that the first multi-channel audio data is input data when the channels are down-mixed, and the second multi-channel audio data is output data after the channels are down-mixed.

In some embodiments, the preset energy threshold is a pre-set minimum energy value that may cause the audio to break after mixing. In some embodiments, the preset energy threshold is a pre-set other energy value that may cause the audio to break after mixing and affect the user's auditory experience. This application is not limited to this.

In some embodiments, the first multi-channel audio data can be 2.1-channel, 3.1-channel, 5.1-channel, 7.1-channel or other multi-channel audio data, and the mixed output data can be mono audio data, two-channel audio data, or other multi-channel audio data not exceeding the first multi-channel audio data.

It can be understood that the multi-channel mixing method in the present application is to mix multi-channel audio data (i.e., the first multi-channel audio data) into audio data with fewer channels, i.e., mixed output data. Before down-mixing the audio data of each channel, the audio data exceeding the preset energy threshold is determined by energy tracking of the audio data of each channel, and energy suppression is performed to obtain the second multi-channel audio data after energy suppression, and the second multi-channel audio data is down-mixed. The multi-channel mixing method of the embodiment of the present application can fully adapt to and support the down-mixing of channels of various multi-channel audio data, can solve the problem of broken sound caused by excessive energy of some audio frames during down-mixing, obtain a more ideal down-mixing result of the channel, and enhance the user's auditory experience.

In a possible implementation of the first aspect above, determining that there is audio data in the first multi-channel audio data whose energy is greater than a preset energy threshold includes: performing frame processing on the first multi-channel audio data to obtain multiple audio frames, and determining frame energy of the multiple audio frames; determining that there is a high-energy audio frame in the first multi-channel audio data whose frame energy is greater than a preset energy threshold.

It can be understood that, in some embodiments, audio frames whose frame energy in the first multi-channel audio data does not exceed a preset energy threshold are low-energy audio frames, and energy reduction processing may not be performed on the low-energy audio frames.

In a possible implementation of the first aspect above, energy reduction processing is performed on audio data in the first multi-channel audio data whose energy is greater than a preset energy threshold to obtain second multi-channel audio data, including: determining a target gain of a high-energy audio frame, and determining a frame gain of the high-energy audio frame based on the target gain; and determining a target audio frame corresponding to the high-energy audio frame after the energy reduction processing based on the frame gain of the high-energy audio frame.

It can be understood that the target gain is an energy suppression factor when performing energy reduction processing on a high-energy audio frame, and the energy reduction of the high-energy audio frame can be achieved by using the energy suppression factor.

In some embodiments, a target gain may also be provided for a low-energy audio frame, and the target gain of the low-energy audio frame is 1, that is, no energy reduction is performed on the low-energy audio frame.

In a possible implementation of the first aspect, the frame energy of the high-energy audio frame is determined by the following formula: The high-energy audio frame includes L sampling points; β represents a frame energy smoothing coefficient; x _i (n)(k) represents the audio data of the kth sampling point in the nth audio frame of the i-th channel to be mixed among the M channels to be mixed; represents the energy of the kth sampling point in the nth audio frame of the ith channel to be mixed among the M channels to be mixed; Indicates the frame energy of the nth audio frame of the ith channel to be mixed among the M channels to be mixed.

In some embodiments, each audio frame may include L=512 sampling points, that is, the frame length of the audio frame is 512. In other embodiments, L may also be other values, which is not limited in the present application.

In a possible implementation of the first aspect above, the preset energy threshold includes a first threshold and/or a second threshold; the high-energy audio frame includes at least one of the following: among multiple audio frames of M channels to be mixed, an audio frame whose average frame energy of at least one audio frame with the same index corresponding to the same mixed channel is greater than the first threshold is a high-energy audio frame; among each audio frame of the same channel to be mixed, an audio frame whose maximum frame energy of at least two audio frames consecutive to the corresponding audio frame is greater than the second threshold is a high-energy audio frame.

It can be understood that the index of the audio frame is the serial number corresponding to an audio frame in any one of the M channels to be mixed. For example, for the nth audio frame of the ith channel to be mixed among the M channels to be mixed, its index is n.

In a possible implementation of the first aspect, the maximum frame energy of each audio frame of the M to-be-mixed channels is determined according to the frame energy of an audio frame having the maximum frame energy among audio frames corresponding to the same mixing channel and having the same index as the audio frames.

In a possible implementation of the first aspect above, the target gain of the high-energy audio frame is determined according to a preset energy threshold and a maximum frame energy of at least two audio frames consecutive to each high-energy audio frame.

In a possible implementation of the first aspect, the frame gain is determined by the following formula: Where α represents the frame gain smoothing coefficient; represents the target gain of the nth audio frame of the ith channel to be mixed among the M channels to be mixed; represents the frame gain of the n-1th audio frame of the ith channel to be mixed among the M channels to be mixed; Indicates the frame gain of the nth audio frame of the ith channel to be mixed among the M channels to be mixed.

In some embodiments, a low-energy audio frame whose frame energy in the first multi-channel audio data does not exceed a preset energy threshold may also use the above formula to calculate its frame gain, wherein the target gain of the low-energy audio frame is 1.

In a possible implementation of the first aspect above, determining a target audio frame corresponding to the high-energy audio frame after energy reduction processing according to the frame gain of the high-energy audio frame includes: determining a sampling point gain of each sampling point in the high-energy audio frame according to the frame gain of the high-energy audio frame; performing energy reduction processing on audio data of each sampling point in the high-energy audio frame according to each sampling point gain to obtain audio data of each sampling point in the target audio frame; and generating a target audio frame according to the audio data of each sampling point in the target audio frame.

In a possible implementation of the first aspect, the gain of each sampling point is determined by the following formula:

Wherein, FrameLen represents the frame length of the target audio frame; FrameGain _xi(n-1) represents the frame gain of the n-1th audio frame of the ith channel to be mixed among the M channels to be mixed; FrameGain _xi(n) represents the frame gain of the nth audio frame of the ith channel to be mixed among the M channels to be mixed; GainBuGainBuff[i][k] _xi(n-1) represents the sampling point gain of the kth sampling point of the n-1th audio frame of the ith channel to be mixed among the M channels to be mixed; GainBuGainBuff[i][k] _xi(n) represents the sampling point gain of the kth sampling point of the nth audio frame of the ith channel to be mixed among the M channels to be mixed.

In some embodiments, a low-energy audio frame in the first multi-channel audio data whose frame energy does not exceed a preset energy threshold can also use the above formula to calculate its sampling point gain, where the frame gain of the low-energy audio frame is calculated based on the target gain of the low-energy audio frame being 1.

In a possible implementation of the first aspect, audio data of each sampling point in the target audio frame is determined by audio data of each sampling point of a high-energy audio frame corresponding to the target audio frame and a corresponding sampling point gain.

In a possible implementation of the first aspect above, obtaining second multi-channel audio data according to the energy reduction processing result includes: generating second multi-channel audio data according to the target audio frame and a low-energy audio frame in the first multi-channel audio data whose energy is not greater than a preset energy threshold.

In a possible implementation of the first aspect, downmixing the second multi-channel audio data to obtain mixed output data having N second channels includes: performing weighted summation on target audio frames and low-energy audio frames corresponding to the same second channel in the second multi-channel audio data to obtain the mixed output data.

It can be understood that the process of obtaining the mixed output data is to downmix the second multi-channel audio data using the Dolby downmix method, wherein the weight coefficient for weighted summation is a preset parameter.

In a second aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; one or more memories; one or more memories storing one or more programs, and when one or more programs are executed by one or more processors, the electronic device executes the above-mentioned multi-channel mixing method.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the above-mentioned multi-channel mixing method.

In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program/instruction, which implements the above-mentioned multi-channel mixing method when executed by a processor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a scenario of a multi-channel mixing method provided in an embodiment of the present application;

FIG2 is a schematic flow chart of a mixing method for downmixing six channels into two channels provided in an embodiment of the present application;

FIG3 is a schematic diagram showing a flow chart of a multi-channel mixing method provided in an embodiment of the present application;

FIG4 is a schematic flow chart of another multi-channel mixing method provided in an embodiment of the present application;

FIG5 is a schematic diagram showing a flow chart of an energy suppression method provided in an embodiment of the present application;

FIG6 is a schematic diagram of a code stream waveform of multi-channel audio data provided in an embodiment of the present application;

FIG7 is a schematic diagram of a code stream waveform and an energy spectrum of a mixed channel after channel downmixing;

FIG8 is a schematic diagram showing the hardware structure of a mobile phone provided in an embodiment of the present application.

DETAILED DESCRIPTION

As mentioned above, for audio data that does not meet Dolby specifications, the audio data obtained by downmixing the channels will have a problem of distortion. Specifically, for audio data that does not meet Dolby specifications, since the energy of the bass channel audio data is too high, when the weighted sum is performed to obtain the mixed audio data, the energy of the corresponding audio data will also be too high. As a result, when the corresponding audio data is output, the user hears the audio distortion, and the user's auditory experience is not good.

In order to solve the problem of audio data being broken after downmixing and affecting the user's auditory experience in the above-mentioned channel downmixing solution, the present application proposes a multi-channel mixing method. The method includes: an electronic device determines the audio data in each channel of the multi-channel audio data that exceeds a preset energy threshold, and performs energy suppression (i.e., energy reduction) on it, and then calculates the suppressed audio data of each channel. Then, based on the preset channel downmixing algorithm, the suppressed multi-channel audio data can be weighted summed to obtain the audio data after channel downmixing.

It can be understood that in some embodiments, energy suppression can be performed by calculating the frame gain of each audio frame in each channel and reducing the frame gain by a corresponding suppression factor to obtain a suppressed audio frame, thereby obtaining energy suppressed audio data.

It can be understood that the preset channel downmixing algorithm is the corresponding relationship between each channel before and after the channel downmixing, and the weight coefficient of the weighted sum. For example, in a channel downmixing scheme in which six channels are downmixed to two channels, where the six channels are the left channel, the left surround channel, the bass channel, the center channel, the right channel, and the right surround channel, the preset channel downmixing algorithm is: the left channel, the left surround channel, the bass channel, and the center channel are downmixed to the left channel, and the right channel, the right surround channel, the bass channel, and the center channel are downmixed to the right channel, and the weight coefficient during the weighted summation is the default weight coefficient in the Dolby downmixing scheme.

It can be understood that in some embodiments, multi-channel audio data can be divided into multiple audio frames, and then energy tracking and energy suppression can be performed based on the audio frames. Among them, the audio frame is to divide the audio data into multiple audio data segments through framing processing, each segment is an audio frame, there is overlap between the audio frames, and the audio frames of each channel are divided in the same way. In other embodiments, energy suppression can be performed on audio data whose energy exceeds a preset energy threshold in other ways, and this application does not limit this.

The multi-channel mixing method provided in the embodiment of the present application, before performing weighted summation on the audio data of each channel, tracks the energy of the audio frames of each channel, determines the audio frames that exceed a preset energy threshold, and performs energy suppression. This method can fully adapt to a variety of audio data and support channel downmixing of a variety of multi-channel audio data. It can solve the problem of distortion caused by excessive energy of some audio frames during channel downmixing, obtain a more ideal channel downmixing result, and enhance the user's auditory experience.

It can be understood that the electronic devices in the embodiments of the present application include, but are not limited to, mobile phones (including foldable screen mobile phones), tablet computers, laptop computers, desktop computers, servers, wearable devices, head-mounted displays, mobile email devices, car equipment, portable game consoles, portable music players, reader devices, televisions embedded with or coupled with one or more processors, and other electronic devices. For the convenience of explanation, the following takes the electronic device as an example to introduce the present application.

1 and 2 , taking the output of audio data from the mobile phone 100 to the Bluetooth headset 200 as an example, the application scenario in the embodiment of the present application is introduced.

FIG. 1 is a schematic diagram showing an application scenario of a multi-channel mixing method.

As shown in Fig. 1, the scene includes a mobile phone 100 and a Bluetooth headset 200. The mobile phone 100 and the Bluetooth headset are wirelessly connected via Bluetooth.

When the user wears the Bluetooth headset 200 and plays the multi-channel audio data in the mobile phone 100, the mobile phone 100 needs to down-mix the multi-channel audio data, convert it into two-channel audio data including a left channel and a right channel, and send the audio data to the Bluetooth headset 200 via Bluetooth. After receiving the audio data, the Bluetooth headset 200 will play it.

FIG. 2 is a schematic flow chart of a mixing method for downmixing six channels into two channels.

Specifically, as shown in FIG2 , taking six-channel audio data as an example, the six channels include a left channel A1, a left surround channel A2, a bass channel A3, a center channel A4, a right channel A5, and a right surround channel A6. Before the mobile phone 100 performs channel downmixing of the audio data, it can first perform frame processing on the six-channel audio data and determine the frame energy of each audio frame. When the frame energy of the audio frame of any channel is greater than the preset energy threshold, the energy suppression factor of the audio frame is determined, the energy of the audio frame is suppressed, and the six-channel audio data after the energy suppression is calculated according to each suppressed audio frame. Then, the six-channel audio data after energy suppression is downmixed using a preset channel downmixing algorithm to obtain two-channel audio data, and the two-channel audio data includes a left channel a1 and a right channel a2. Then the mobile phone 100 can send the audio data of the left channel a1 to the left earphone of the Bluetooth headset 200, and send the audio data of the right channel a2 to the right earphone of the Bluetooth headset 200.

It can be understood that the multi-channel mixing method in the present application can not only support the mixing of the above six channels into two channels, but also support the mixing of any M channels into N channels, where M>N.

The multi-channel mixing method in the embodiment of the present application is further introduced below with reference to the accompanying drawings.

FIG3 is a schematic flow chart of a multi-channel mixing method provided in an embodiment of the present application.

As shown in FIG3 , the multi-channel mixing method includes:

301: Obtain multi-channel audio data.

It can be understood that different channels in the multi-channel audio data (i.e., the first multi-channel audio data in the previous text) are different channel types, wherein each channel is a channel to be mixed, such as a left channel, a right channel, a left surround channel, a right surround channel, etc. The audio data of different channels can be output through different speakers, or can be output through channel downmixing and output through the same channel.

In some embodiments, the multi-channel audio data may be 3.1, 5.1, 7.1, or other multi-channel audio data. Among them, the 3.1 channel includes a left channel, a bass channel, a center channel, and a right channel; the 5.1 channel includes a left channel, a left rear surround channel, a bass channel, a center channel, a right channel, and a right surround channel; and the 7.1 channel includes a left channel, a left rear surround channel, a bass channel, a center channel, a right channel, a right surround channel, a left rear surround channel, and a right rear surround channel. In some embodiments, the multi-channel audio data may also include more or fewer channels than in the above examples, and the present application does not limit this.

It can be understood that in some embodiments, the acquired multi-channel audio data also includes a mask of the multi-channel audio data. The mask is used to cover a layer of mask on the multi-channel audio data to select or shield part of the audio data. According to the mask, the corresponding relationship between the channels before and after mixing the multi-channel audio data can also be determined, that is, the audio data of each mixed channel is mixed with the audio data of the channels before mixing.

In some embodiments, after obtaining the multi-channel audio data, the multi-channel audio data can also be initialized. Wherein, initialization can include determining a preset channel downmixing algorithm, which can specifically include: determining the correspondence between the channels before and after mixing, and performing weighted summation on the multi-channel audio data to obtain the weight coefficients of the audio data of each channel after the corresponding mixing. It can be understood that the audio data of at least one channel corresponding to the same mixed channel after the channel downmixing (i.e., the output channel after the channel downmixing) in the multi-channel audio data can be used as a joint detection channel. Furthermore, when performing energy judgment of the audio frame, the energy of the corresponding audio frame in the joint detection channel can be initially judged. For example, the average energy of the corresponding audio frame can be judged to determine whether it is greater than the set initial judgment energy threshold. In some embodiments, initialization can also include initializing the parameters of the formula in the preset channel downmixing algorithm or the energy suppression algorithm.

302: Divide the multi-channel audio data into frames to obtain multiple audio frames.

It can be understood that the characteristics of multi-channel audio data and the parameters characterizing its essential characteristics will change over time, that is, audio data has time-varying characteristics, so the energy tracking and energy suppression of multi-channel audio data below can be based on short-term analysis. Specifically, the multi-channel audio data can be divided into multiple segments, each segment is an audio frame. And the essential characteristics of the same audio frame remain unchanged or relatively stable.

Furthermore, in some embodiments, the multi-channel audio data may be directly divided into frames, for example, the frame length of each audio frame is 10-30 ms.

In some other embodiments, the multi-channel audio data may be sampled, the continuous multi-channel audio data may be converted into discrete multi-channel audio data, and the audio data of 512 consecutive sampling points may be combined into an audio frame.

303: If the frame energy of the mth audio frame is greater than the preset energy threshold. The preset energy threshold is pre-set. If the frame energy of the mth audio frame is greater than the preset energy threshold, it indicates that after the channel is down-mixed, part of the audio corresponding to the audio frame may be distorted, and energy suppression is required, that is, execute step 304. If the frame energy of the mth audio frame is less than or equal to the preset energy threshold, it indicates that after the channel is down-mixed, part of the audio corresponding to the audio frame will not be distorted, and energy suppression is not required, that is, execute step 305.

It can be understood that the index of the audio frame is the serial number corresponding to an audio frame in any one of the multiple channels of the multi-channel audio data. For example, for the mth audio frame of the ith channel to be mixed among the M channels to be mixed, its index is m.

In some embodiments, the preset energy threshold can be determined based on the minimum frame energy value that may cause audio data to break after channel downmixing. For example, the preset energy threshold is -6dB or -3dB. Specifically, it can be determined based on different electronic devices, audio output devices, and multi-channel audio data, and this application does not limit this.

In some embodiments, step 303 may determine whether the average frame energy of the mth audio frame is greater than a set energy threshold by calculating the average frame energy of the mth audio frame of each channel in each joint detection channel and judging the average frame energy.

In some embodiments, step 303 can determine whether the frame energy of the mth audio frame is greater than the set energy threshold by only calculating the frame energy of the mth audio frame of each channel respectively, and determining whether the frame energy of the audio frame is greater than the set energy threshold. In other embodiments, the maximum frame energy of the mth audio frame of each channel in each joint detection channel can be taken as the frame energy of the mth audio frame, and then determining whether the frame energy is greater than the set energy threshold to determine whether the frame energy of the mth audio frame is greater than the set energy threshold. Further, after determining the maximum frame energy of the mth audio frame of each channel in each joint detection channel as the frame energy of the mth audio frame, it can be determined whether the maximum frame energy of at least one audio frame near the mth audio frame is greater than the set energy threshold. For example, it is determined whether the maximum frame energy of the frame energies of the mth audio frame and the m-1th audio frame is greater than the set energy threshold to determine whether the frame energy of the mth audio frame is greater than the set energy threshold.

In some embodiments, the energy threshold is set to include a preliminary judgment energy threshold (i.e., the first threshold in the foregoing text) and a precise judgment energy threshold (i.e., the second threshold in the foregoing text). Furthermore, step 303 can make a preliminary judgment on the frame energy of the mth audio frame. Specifically, first calculate the average frame energy of the mth audio frame in each joint detection channel, and judge the average frame energy to determine whether it is greater than the preliminary judgment energy threshold. If it is greater, further precise judgment is performed. Specifically, the precise judgment can be performed by taking the maximum frame energy of the mth audio frame of each channel in each joint detection channel as the frame energy of the mth audio frame, and then it can be determined whether the maximum frame energy of at least two consecutive audio frames near the mth audio frame is greater than the precise judgment energy threshold. Step 303 will be further introduced below in conjunction with the formula.

In some embodiments, the frame energy of the audio frame can be obtained by performing Fourier transform on the multi-channel audio data to obtain the energy spectrum of the multi-channel audio data, and calculating the frame energy of the audio frame composed of multiple sampling points based on the energy values of multiple consecutive sampling points. The specific calculation method will be introduced in conjunction with the formula below.

304: Use an energy suppression algorithm to perform energy suppression on the mth audio frame to obtain the mth target audio frame.

It can be understood that the energy suppression algorithm is an algorithm for determining the target audio frame after energy suppression based on the frame energy of the mth audio frame. In some embodiments, the energy suppression factor can be calculated based on the calculated frame energy of the mth audio frame and a preset formula, and then the energy of the mth audio frame is suppressed based on the energy suppression factor to obtain the mth target audio frame. In some embodiments, the energy suppression factor is a target gain, and then the energy suppression can be to calculate the frame gain of the mth audio frame based on the target gain, and calculate the mth target audio frame based on the frame gain.

In some embodiments, when each audio frame includes multiple sampling points, when calculating the target audio frame, the frame gain of the mth audio frame can be calculated by using the energy suppression factor, and the gain of each sampling point in the mth audio frame (i.e., the sampling point gain) is calculated based on the frame gain of the mth audio frame, and then the audio data of each sampling point in the target audio frame is determined based on the gain of each sampling point, and signal reconstruction is performed to determine the audio data of the mth target audio frame.

305: Determine the mth audio frame as the mth target audio frame.

It can be understood that in some embodiments, if the frame energy of the mth audio frame does not exceed the preset energy threshold, then after channel downmixing, the audio data corresponding to the audio frame will not be distorted due to excessive energy after being output, and will not affect the user's auditory experience. Therefore, there is no need to suppress the energy of the audio frame, and the audio data of the original audio frame can be retained.

306: Based on a preset channel downmixing algorithm, perform channel downmixing on target audio frames of multiple channels to obtain mixed output data.

It can be understood that the preset channel downmixing rule includes the corresponding channels before and after the channel downmixing and the channel downmixing calculation formula. That is, the target audio frames of some channels in the multi-channels of the audio data of each mixed channel after the channel downmixing can be determined by the corresponding channels before and after the channel downmixing. Then, according to the determined speech frame corresponding to the same mixed channel, it is substituted into the corresponding channel downmixing formula to obtain the mixed output data of the mixed channel. Among them, the mixed output data is the audio data of one channel output to the Bluetooth headset 200 in the above text.

It can be understood that in some embodiments, after obtaining the target audio frame, in step 305 or 304, the Dolby downmix algorithm can be used to perform channel downmixing to obtain the mixed output data of each mixed channel. Specifically, the target audio frames of the corresponding indexes in each joint detection channel can be weighted summed to obtain the mixed audio frames in the mixed output data corresponding to the target audio frames, and then the mixed audio frames corresponding to the same mixed channel can be spliced to obtain the output data of the mixed channel.

The embodiment of the present application tracks the energy of the voice data in the multi-channel audio data through the above-mentioned multi-channel mixing method, and suppresses the energy of the audio data with higher energy, and then performs channel downmixing based on the multi-channel audio data after energy suppression. The multi-channel mixing method in the embodiment of the present application can be applied to both multi-channel audio data of Dolby specifications and multi-channel audio data of non-Dolby specifications, that is, the multi-channel mixing method in the embodiment of the present application can be applied to a variety of multi-channel channel downmixing, and can achieve adaptive channel downmixing, and will not cause distortion due to excessive energy of the audio data. At the same time, the multi-channel mixing method in the embodiment of the present application only suppresses the energy of some audio data with higher energy, and does not abandon the audio data. While solving the problem of distortion in downmixing, the audio data of each channel can also be retained, thereby improving the user's auditory experience.

4 , the multi-channel mixing method in the embodiment of the present application is further introduced by taking the mixing method of downmixing two channels into a single channel as an example.

FIG4 is a schematic flow chart of another multi-channel mixing method in an embodiment of the present application.

As shown in Figure 4, the multi-channel audio data includes a code stream of audio data of channel 1 and a code stream of audio data of channel 2. After acquiring the audio data of channel 1 and the audio data of channel 2, the electronic device performs frame processing on the audio data of the two channels, and each channel can obtain 6 audio data frames.

It can be seen from the code stream of the audio data of each channel after framing that the second and fourth audio frames in the audio frame of channel 1 are not very stable, and the second and third audio frames in the audio frame of channel 2 are not very stable. In order to avoid the situation of broken sound in the mixed data after downmixing, it is necessary to suppress the energy of the second and fourth audio frames in the audio frame of channel 1, and the second and third audio frames in the audio frame of channel 1, respectively, to obtain the code stream of the target audio frame after suppression of each channel. Furthermore, the Dolby downmix algorithm can be used to perform weighted summation on the corresponding target audio frames in the two channels after downmixing to obtain the corresponding mixed audio frames in the mixed channel, and 6 mixed audio frames constitute the mixed output data.

In conjunction with FIG5 , an energy suppression method in an embodiment of the present application is further introduced below.

FIG5 is a flow chart showing an energy suppression method in an embodiment of the present application.

As shown in FIG5 , the method includes:

501: Multi-channel data. The multi-channel data obtained in step 501 is the multi-channel audio data obtained in step 301. Step 501 is similar to step 301 and will not be described in detail in this application.

502: Generate an initial downmix algorithm according to the channel arrangement and determine the mixing channel.

It can be understood that the channel arrangement is the type and number of channels in the multi-channel data. The initialization downmixing algorithm is generated according to the channel arrangement, the correspondence between the front and rear channels of the channel downmixing and the mixed channels, and the formula algorithm for calculating the data of the mixed channels. Specifically, in some embodiments, generating the initialization downmixing algorithm may include determining the correspondence between M channels and N channels through the mask mask of the multi-channel data. For example, when the six channels are downmixed to two channels, the left channel, left surround, bass, and center are downmixed to the left channel, and the right channel, right surround, bass, and center are downmixed to the right channel. In some embodiments, generating the initialization downmixing algorithm also includes calculating the formula for each mixed output data in the mixed channel and the initialization of the parameters in the formula. Among them, when the channels are downmixed, each channel of the same mixed channel (i.e., mixed channel) can be expressed as a joint detection channel, that is, the data of each channel in the joint detection channel corresponding to the mixed channel is weighted and summed to obtain the data that the mixed channel (i.e., mixed channel) needs to output.

503: VAD detection result is 1.

The full name of VAD detection is Voice Activity Detection. The condition of VAD detection can be that the frame energy of the audio data is greater than the set energy threshold.

In some embodiments, the multi-channel data may be sampled and framed before VAD detection. The frame processing may be, after sampling the multi-channel data, dividing some continuous sampling points into an audio frame, for example, taking 512 sampling points as an audio frame. Furthermore, in some embodiments, the VAD detection condition may be to detect whether the frame energy of the audio frame is greater than a preset VAD detection threshold (i.e., the first threshold in the foregoing text), wherein the VAD detection threshold may be, for example, a frame energy greater than -6dB. When the VAD detection judgment result is true (true), that is, the VAD detection result is 1, it indicates that the frame energy of the audio frame exceeds the preset VAD detection threshold, and the audio frame may be broken after the channel downmixing. It can be understood that VAD detection is the initial judgment in the foregoing text, and further precise judgment can be performed based on the initial judgment result.

In some embodiments, it is possible to jointly detect whether the average frame energy of each audio frame in the channel is greater than a preset VAD detection threshold. If the average frame energy is too high, it is necessary to track the energy of each audio frame in each channel and suppress the energy of the audio frames that meet the energy suppression conditions.

It can be understood that when the VAD detection result is 1, it is necessary to track the energy of the audio frame. When the VAD detection result is false, that is, the VAD detection result is 0, the channel downmix will not cause distortion due to the excessive energy of the audio frame. Then, the VAD detection can be performed on the next audio frame.

504: Calculate frame energy, track the maximum energy of the joint detection channel and the previous and next frames.

It can be understood that if the VAD detection result is 1, it is necessary to perform energy suppression on the corresponding audio frame.

Specifically, taking L sampling points as a speech frame, the frame energy of the audio frame can be calculated by the following formula:

in, Represents the frame energy of the nth audio frame in the ith channel. β is the smoothing coefficient when calculating the frame energy, and x _i (n)(k) represents the input data of the kth sampling point of the nth audio frame in the ith channel, that is, part of the data in the acquired multi-channel data. Wherein i＝0,1,2,3,4,5·····, the value of i is related to the number of channels, k can be an integer between 0 and L, and the value range of n is the number of speech frames after framing the multi-channel data. In some embodiments, the smoothing coefficient β＝0.3.

It can be understood that in the above step 504, by calculating the frame energy The frame energy of the data of each channel can be tracked, and then when the frame energy of the audio frame is tracked and it is determined that the frame energy is greater than the set detection threshold, step 505 can be executed.

Specifically, the audio frame with the maximum energy in the corresponding audio frames in the joint detection channel may be calculated, and may be determined in the following manner.

in, Indicates the maximum frame energy value of the nth audio frame in each channel data of the joint detection channel.

In some embodiments, after the maximum frame energy value of the current n-th audio frame is determined by the above formula (2), the maximum energy value of the previous and next audio frames can be determined by the following formula:

in, Indicates the maximum energy of the audio frames before and after the nth audio frame. represents the maximum frame energy value of the n-1th audio frame in each channel data of the joint detection channel, Indicates the maximum frame energy value of the n+1th audio frame in each channel data of the joint detection channel.

In some embodiments, The maximum value of the energy maxima in the audio frames before and after the n-th and n-1-th audio frames is also determined and calculated.

505: Calculate the target gain and the frame gain, and finally calculate the gain of each sampling point, multiply it by the fixed gain, and output the result.

It can be understood that the output result is the input data when the channel downmixing is performed.

In some embodiments, the calculation is After that, it can be determined whether the energy of the current nth audio frame exceeds the set detection threshold (i.e., the second threshold mentioned above). If it exceeds, it indicates that the speech frame has the risk of distortion after channel downmixing and needs energy suppression. Then, the target gain and frame gain of the audio frame can be calculated to determine the energy suppression factor.

It can be understood that the determination of setting the detection threshold accurately determines the frame energy of each channel to determine whether it is necessary to suppress the audio frame with excessive energy.

It can be understood that the target gain can be understood as an energy suppression factor, which is used to reduce the frame gain, thereby achieving the purpose of reducing the energy of the audio frame.

In some embodiments, the target gain may be calculated by the following formula:

It can be understood that Threshold represents the set detection threshold, Threshold = -3dB. Indicates the target gain of the nth audio frame of the ith channel.

It can be seen from formula (4) that when the maximum frame energy value of the previous and next frames is less than the set detection threshold Threshold, it indicates that the frame energy of the audio frame is appropriate and there will be no downmixing distortion caused by excessive energy. When the maximum frame energy value of the previous and next frames is greater than or equal to the set detection threshold Threshold, it indicates that the frame energy of the audio frame is too high and there may be downmixing distortion caused by excessive energy. It is necessary to calculate the target gain and suppress the frame gain of the audio frame.

In some embodiments, after the target gain is calculated, the frame gain of the audio frame may be determined by the following formula:

in, represents the frame gain of the nth audio frame of the ith channel, represents the frame gain of the n-1th audio frame of the ith channel, and α represents a smoothing coefficient when calculating the frame gain. In some embodiments, the smoothing coefficient α=0.1.

In some embodiments, after the frame gain is calculated according to the above formula (5), the sampling point gain of each sampling point in the audio frame can be calculated using the following formula:

in, represents the sampling point gain of the kth sampling point in the nth audio frame of the ith channel, represents the sampling point gain of the kth sampling point in the n-1th audio frame of the ith channel, and FrameLen represents the frame length of the audio frame. For example, if 512 sampling points are used as an audio frame, the frame length of the audio frame is 512.

It can be seen from the above formula (6) that when calculating the sampling point gain, the sampling point gain of each sampling point is related to the frame gain of the corresponding speech frame and the sampling point gain of the corresponding index number of the previous speech frame. The index of each sampling point is the sequence number of each sampling point in the corresponding audio frame, for example, the index of the kth sampling point in an audio frame is k.

Further, in some embodiments, the calculation formula for obtaining the target audio frame according to the sampling point gain obtained in Formula 6 is:

Among them, α _i (n)[i][k] represents the audio data of the kth sampling point of the nth target audio frame of the ith channel, and _xi (n)[i][k] represents the audio data of the kth sampling point of the nth target audio frame of the ith channel, that is, _xi (n)(k) in Formula 1.

It can be understood that in some embodiments, audio data of continuous target audio frames can be obtained based on audio data of discrete sampling points in the target audio frame, and the audio data of each target audio frame can be used as input data for channel downmixing.

In some embodiments, after calculating each audio frame after energy suppression, channel downmixing can be performed using the following formula:

Among them, a _j (n) represents the output result of the nth audio frame of the jth mixing channel, w _i,j (n) represents the mixing weight of the nth audio frame of the ith channel in the jth mixing channel, and α _i (n) represents the input data of the nth audio frame of the ith channel when the channel is downmixed. When the channel is downmixed to two channels, j = 0, 1, when the channel is downmixed to three channels, j = 0, 1, 2, and so on. A = 0, 1, 2, 3, 4, 5······, respectively represent multiple channels before mixing. For example, A = 0, 1, 2, 3, 4, 5, respectively represent the left channel, left surround channel, bass channel, center channel, right channel, and right surround channel. J represents the number of channels before mixing corresponding to the jth mixing channel.

In order to more clearly illustrate the positive effects of the multi-channel mixing method in the embodiment of the present application, the multi-channel mixing method in the embodiment of the present application is simulated in combination with Figures 6 and 7. The simulation software is clion software. In addition, the multi-channel audio data is 5.1 channel audio data, the smoothing coefficient α=0.1, β=0.3, and the detection threshold Threshold=-3dB is set as the simulation condition for simulation.

FIG. 6 is a schematic diagram of a code stream waveform of multi-channel audio data in an embodiment of the present application.

FIG. 7 is a schematic diagram showing a code stream waveform and an energy spectrum of a mixed channel after channel downmixing.

As shown in FIG6, the six code stream waveforms in the figure represent the left channel, the right channel, the center channel, the bass channel, the left surround channel, and the right surround channel from top to bottom. The horizontal axis represents time, and the vertical axis represents the audio data of the audio data. In the process of code stream transparent transmission, each sampling point is represented by 16 bits. Therefore, in the case of a fixed coefficient downmixing scheme, if you want to ensure that the mixing result is not broken, the sum of each channel must be less than the range that can be represented by 16 bits, otherwise the data will overflow and wrap around, and then data jumps will occur, which will cause noise. As shown in FIG6, if the Dolby downmix coefficient is used for mixing, some of the data in the box in FIG6 has a large energy peak. If you want to ensure that the downmix is not broken, the coefficient of the channel downmix needs to become very small. At the same time, since the volume is positively correlated with energy, the volume of the corresponding mixed channel will become smaller, resulting in a smaller overall volume of the downmixed audio, affecting the user's auditory experience.

As shown in Figure 7, the first row of code stream waveforms is a code stream waveform of the audio data of the left channel mixing channel obtained by the multi-channel mixing method in this application, and the second row of code stream waveforms is a code stream waveform of the audio data of the right channel mixing channel obtained by the Dolby mixing method. In the first row of code stream waveforms and the second row of code stream waveforms, the horizontal axis represents time, and the vertical axis represents the audio data of the audio data. The third row corresponds to the audio data of the left channel mixing channel of the first row, which is the energy spectrum of the audio data of the left channel mixing channel, and the fourth row corresponds to the audio data of the right channel mixing channel of the second row, which is the energy spectrum of the audio data of the right channel mixing channel. In the third row of energy spectrum and the fourth row of energy spectrum, the horizontal axis represents time, and the vertical axis represents energy.

As can be seen from FIG. 7, the code stream waveform of the audio data of the right channel after being processed by the multi-channel mixing method in the embodiment of the present application is more stable in envelope changes, rarely showing large fluctuations, and its energy stripes are relatively stable, not too high and spread throughout the entire frequency domain. However, the code stream waveform of the audio data of the right channel mixed channel that has not been processed by the multi-channel mixing method in the present application has energy stripes throughout the entire frequency domain in its energy-rich part, such as the audio data framed by the box in FIG. 7, and there is an obvious problem of broken sound, and the noise is also relatively obvious.

It can be seen that the multi-channel mixing method in the embodiment of the present application performs energy tracking on the voice data of each channel, thereby suppressing the energy of audio data with excessively high energy. This reduces the risk of mixing distortion without losing audio data, thereby improving the user's auditory experience.

FIG8 shows a schematic diagram of the hardware structure of a mobile phone 100 according to an embodiment of the present application.

The mobile phone 100 can execute the display method provided in the embodiment of the present application. In FIG8 , similar components have the same reference numerals. As shown in FIG8 , the mobile phone 100 may include a processor 110, a power module 140, a memory 180, a camera 101, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, an interface module 160, and a display screen 102, etc.

It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the mobile phone 100. In other embodiments of the present application, the mobile phone 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, a central processing unit (CPU), a graphics processing unit (GPU), an image signal processor (ISP), a digital signal processor (DSP), a microprocessor (MCU), an artificial intelligence (AI) processor, or a processing module or processing circuit of a programmable logic device (FPGA). Among them, different processing units can be independent devices or integrated in one or more processors. For example, in some examples of the present application, the processor 110 can be used to determine whether the energy of the mth audio frame is greater than a set energy threshold and calculate an energy suppression factor. In some embodiments, the processor 110 can also be used to down-mix the channels of the obtained target audio frame to obtain mixed output data.

The memory 180 can be used to store data, software programs and modules, and can be a volatile memory (VolatileMemory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (FlashMemory), a hard disk (Hard Disk Drive, HDD) or a solid-state drive (SSD); or a combination of the above types of memory, or it can also be a removable storage medium, such as a secure digital (Secure Digital, SD) memory card. In some embodiments of the application, the memory 180 is used to store multi-channel audio data of the mobile phone 100 and a preset channel downmixing algorithm.

The power module 140 may include a power source, a power management component, etc. The power source may be a battery. The power management component is used to manage the charging of the power source and the power supply of the power source to other modules. The charging management module is used to receive charging input from the charger; the power management module is used to connect the power source, the charging management module and the processor 110.

The mobile communication module 130 may include, but is not limited to, an antenna, a power amplifier, a filter, a low noise amplifier (LNA), etc. The mobile communication module 130 may provide solutions for wireless communications including 2G/3G/4G/5G, etc., applied to the mobile phone 100. The mobile communication module 130 may receive electromagnetic waves by an antenna, and perform filtering, amplification, and other processing on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 130 may also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna. In some embodiments, at least some of the functional modules of the mobile communication module 130 may be arranged in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 130 may be arranged in the same device as at least some of the modules of the processor 110.

The wireless communication module 120 may include an antenna, and transmit and receive electromagnetic waves via the antenna. The wireless communication module 120 may provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc., which are applied to the mobile phone 100. The mobile phone 100 may communicate with the network and other devices through wireless communication technology.

In some embodiments, the mobile communication module 130 and the wireless communication module 120 of the mobile phone 100 may also be located in the same module.

The camera 101 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP (Image Signal Processor) to convert it into a digital image signal. The mobile phone 100 can implement the shooting function through the ISP, the camera 101, the video codec, the GPU (Graphic Processing Unit), the display screen 102, and the application processor. For example, in some embodiments of the present application, the camera 101 is used to collect facial images and QR code images, and the mobile phone 100 performs facial recognition, QR code recognition, etc.

The display screen 102 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a Micro LED, a Micro OLED, a quantum dot light-emitting diode (QLED), etc. For example, the display screen 102 is used to display various UI interfaces of the mobile phone 100 in the horizontal/vertical screen state, such as split screen, parallel horizon, and a single APP exclusive screen.

The sensor module 190 may include a proximity light sensor, a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

The audio module 150 can convert digital audio information into analog audio signal output, or convert analog audio input into digital audio signal. The audio module 150 can also be used to encode and decode audio signals. In some embodiments, the audio module 150 can be arranged in the processor 110, or some functional modules of the audio module 150 can be arranged in the processor 110.

The interface module 160 includes an external memory interface, a Universal Serial Bus (USB) interface, and a Subscriber Identification Module (SIM) card interface. The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external memory card communicates with the processor 110 via the external memory interface to implement a data storage function. The Universal Serial Bus interface is used for the mobile phone 100 to communicate with other mobile phones. The Subscriber Identification Module card interface is used to communicate with a SIM card installed in the mobile phone 100, for example, to read a phone number stored in the SIM card, or to write a phone number into the SIM card.

In some embodiments, the mobile phone 100 further includes buttons, motors, and indicators, etc. Among them, the buttons may include a volume button, a power on/off button, etc. The motor is used to make the mobile phone 100 produce a vibration effect. The indicator may include a laser indicator, a radio frequency indicator, an LED indicator, etc.

The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements), at least one input device and at least one output device.

Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

Program code can be implemented with high-level procedural language or object-oriented programming language to communicate with processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language. In some cases, the disclosed embodiments can be implemented with hardware, firmware, software or any combination thereof. The disclosed embodiments can also be implemented as instructions carried or stored thereon by one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, instructions can be distributed over a network or by other computer-readable media. Therefore, the machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to floppy disks, optical disks, optical discs, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROM), random access memories (RAM), erasable programmable read-only memories (EPROM), electrically erasable programmable read-only memories (EEPROM), magnetic or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic or other forms of propagation signals. Therefore, the machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

In addition, the technical solution of the present application also provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on the electronic device 100, the electronic device 100 executes the display method provided by the technical solution of the present application.

In addition, the technical solution of the present application also provides a computer program product, which includes instructions, and the instructions are used to implement the display method provided by the technical solution of the present application.

In addition, the technical solution of the present application also provides a chip device, which includes: a communication interface for inputting and/or outputting information; a processor for executing a computer executable program so that a device equipped with the chip device executes the display method provided by the technical solution of the present application.

In the accompanying drawings, some structural or method features may be shown in a specific arrangement and/or order. However, it should be understood that such a specific arrangement and/or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and/or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

It should be noted that the units/modules mentioned in the various device embodiments of the present application are all logical units/modules. Physically, a logical unit/module can be a physical unit/module, or a part of a physical unit/module, or can be implemented as a combination of multiple physical units/modules. The physical implementation method of these logical units/modules themselves is not the most important. The combination of functions implemented by these logical units/modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units/modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units/modules in the above-mentioned device embodiments.

It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims (16)

1. A multi-channel mixing method, applied to electronic equipment, characterized by including: Obtain first multi-channel audio data, where the first multi-channel audio data includes audio data of M channels to be mixed; Determine that there is audio data with an energy greater than a preset energy threshold in the first multi-channel audio data, and perform energy analysis on the audio data with an energy greater than the preset energy threshold in the first multi-channel audio data. Amplitude reduction processing; Obtain second multi-channel audio data according to the energy reduction processing result; The second multi-channel audio data is downmixed to obtain mixing output data having N mixing channels, where M>N, and N≥1. 2. The multi-channel mixing method according to claim 1, wherein determining that there is audio data with energy greater than a preset energy threshold in the first multi-channel audio data includes: Perform frame segmentation processing on the first multi-channel audio data to obtain multiple audio frames, and determine the frame energy of the multiple audio frames; It is determined that there is a high-energy audio frame whose frame energy is greater than a preset energy threshold in the first multi-channel audio data. 3. The multi-channel mixing method according to claim 2, wherein the frame energy of the high-energy audio frame is determined by the following formula: Wherein, the high-energy audio frame includes L sampling points; β represents the frame energy smoothing coefficient; x _i (n) (k) represents the audio data of the k-th sampling point in the n-th audio frame of the i-th channel to be mixed among the M channels to be mixed; Represents the energy of the k-th sampling point in the n-th audio frame of the i-th channel to be mixed among the M channels to be mixed; Indicates the frame energy of the n-th audio frame of the i-th channel to be mixed among the M channels to be mixed. 4. The multi-channel mixing method according to claim 2, wherein the preset energy threshold includes a first threshold and/or a second threshold; The high-energy sound includes at least one of the following: Among the multiple audio frames of the M to-be-mixed channels, the audio frame corresponding to at least one audio frame with the same index of the same mixing channel whose average frame energy is greater than the first threshold is the high energy audio frames; Among the audio frames of the same channel to be mixed, the audio frame in which the maximum frame energy of at least two audio frames that are continuous with the corresponding audio frame is greater than the second threshold is the high-energy audio frame. 5. The multi-channel mixing method according to claim 4, characterized in that the maximum frame energy of each audio frame of the M to be mixed channels is based on the value corresponding to each audio frame. The frame energy of the audio frame with the largest frame energy among audio frames with the same mixing channel and the same index is determined. 6. The multi-channel mixing method according to claim 2, characterized in that, energy reduction processing is performed on the audio data in the first multi-channel audio data whose energy is greater than the preset energy threshold, include: Determine a target gain of the high-energy audio frame, and determine a frame gain of the high-energy audio frame based on the target gain; According to the frame gain of the high-energy audio frame, a target audio frame corresponding to the high-energy audio frame after energy reduction processing is determined. 7. The multi-channel mixing method according to claim 6, wherein the target gain of the high-energy audio frame is based on the preset energy threshold and is related to each of the high-energy audio frames. Determined by the maximum frame energy of at least two consecutive audio frames. 8. The multi-channel mixing method according to claim 7, wherein the frame gain is determined by the following formula: in, α represents the frame gain smoothing coefficient; Represents the target gain of the n-th audio frame of the i-th channel to be mixed among the M channels to be mixed; Represents the frame gain of the n-1th audio frame of the i-th audio channel to be mixed among the M audio channels to be mixed; Represents the frame gain of the n-th audio frame of the i-th channel to be mixed among the M channels to be mixed. 9. The multi-channel mixing method according to claim 6, wherein the target audio frame corresponding to the high-energy audio frame after energy reduction processing is determined according to the frame gain of the high-energy audio frame. ,include: Determine the sampling point gain of each sampling point in the high-energy audio frame according to the frame gain of the high-energy audio frame; According to the gain of each sampling point, perform energy reduction processing on the audio data of each sampling point in the high-energy audio frame to obtain the audio data of each sampling point in the target audio frame; The target audio frame is generated according to the audio data of each sampling point of the target audio frame. 10. The multi-channel mixing method according to claim 9, characterized in that the gain of each sampling point is determined by the following formula: in, FrameLen represents the frame length of the target audio frame; Represents the frame gain of the n-1th audio frame of the i-th channel to be mixed among the M channels to be mixed; Represents the frame gain of the n-th audio frame of the i-th channel to be mixed among the M channels to be mixed; Represents the sampling point gain of the k-th sampling point of the n-1th audio frame of the i-th audio channel to be mixed among the M audio channels to be mixed; Represents the sampling point gain of the k-th sampling point of the n-th audio frame of the i-th channel to be mixed among the M channels to be mixed. 11. The multi-channel mixing method according to claim 9, characterized in that the audio data of each sampling point in the target audio frame is through each sampling point of the high-energy audio frame corresponding to the target audio frame. The audio data and the corresponding sampling point gain are determined. 12. The multi-channel mixing method according to claim 6, wherein the second multi-channel audio data obtained according to the energy reduction processing result includes: The second multi-channel audio data is generated according to the target audio frame and the low-energy audio frame in the first multi-channel audio data whose energy is not greater than a preset energy threshold. 13. The multi-channel mixing method according to claim 12, wherein the second multi-channel audio data is down-mixed to obtain mixing output data having N second channels. ,include: Perform a weighted sum of the target audio frame and the low-energy audio frame corresponding to the same second channel in the second multi-channel audio data to obtain the mixing output data. 14. An electronic device, characterized in that it includes: memory for storing instructions for execution by one or more processors of the electronic device, and The processor is one of the processors of the electronic device, and is used for controlling and executing the multi-channel mixing method according to any one of claims 1 to 13. 15. A computer-readable storage medium, characterized in that instructions are stored on the storage medium, and when the instructions are executed on a computer, they cause the computer to execute the multi-voice method according to any one of claims 1 to 13. way of mixing. 16. A computer program product, characterized in that the computer program product includes instructions, and the instructions are used to implement the multi-channel mixing method according to any one of claims 1 to 13.