WO2025091293A1 - Grouping method, encoder, decoder, and storage medium - Google Patents

Abstract

A grouping method and apparatus and a storage medium. The grouping method is executed by an encoder, and comprises: grouping a plurality of channels to obtain at least one channel group, wherein the at least one channel group includes N first channel groups, each first channel group comprises three channels, the at least one channel group includes M second channel groups, and each second channel group comprises two channels, wherein N is 1, and M is a non-negative integer. The method solves the problem that a single channel cannot be grouped during channel grouping. The present invention provides a method of dividing a channel group comprising three channels, ensuring that each channel can be grouped into a corresponding group during channel grouping, improving the accuracy of channel encoding.

Description

Grouping method, encoder, decoder and storage medium Technical Field

The present disclosure relates to the field of multimedia technology, and in particular to a grouping method, an encoder, a decoder, and a storage medium.

Background Art

With the rapid development of multimedia technology, audio can be applied in various fields. In addition, in order to improve the spatial sense and orientation sense of audio, the audio can be 3D encoded and 3D decoded to ensure that the audio heard by the user is indistinguishable from the audio heard in the actual environment.

Summary of the invention

The present disclosure solves the problem that a single channel cannot be grouped when grouping channels, and provides a method of dividing a channel group including three channels, thereby ensuring that each channel can have a corresponding group when grouping channels, ensuring the accuracy of channel grouping, and further ensuring the reduction of redundancy between channels, ensuring the accuracy of channel encoding, and further ensuring the accuracy of audio encoding.

The embodiments of the present disclosure provide a grouping method, a device, and a storage medium.

According to a first aspect of an embodiment of the present disclosure, a grouping method is proposed. The method is performed by an encoder, and the method includes:

A plurality of channels are grouped to obtain at least one channel group, wherein the at least one channel group includes N first channel groups, the first channel group includes three channels, and the at least one channel group includes M second channel groups, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer.

According to a second aspect of an embodiment of the present disclosure, a grouping method is proposed. The method is performed by a decoder, and the method includes:

The first information is decoded to obtain channel information of at least one channel grouping, where there are N first channel groups in the at least one channel grouping, the first channel grouping includes three channels, and there are M second channel groups in the at least one channel grouping, the second channel grouping includes two channels, where N is 1 and M is a non-negative integer.

According to a third aspect of an embodiment of the present disclosure, a grouping method is proposed, the method comprising:

The encoder groups the multiple channels to obtain at least one channel group, wherein the at least one channel group includes N first channel groups, the first channel group includes three channels, and the at least one channel group includes M second channel groups, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer;

The decoder decodes the first information to obtain channel information of at least one channel group.

According to a fourth aspect of an embodiment of the present disclosure, a coding and decoding device is proposed, including:

A processing module is used to group multiple channels to obtain at least one channel group, wherein there are N first channel groups in the at least one channel group, the first channel group includes three channels, and there are M second channel groups in the at least one channel group, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer.

According to a fifth aspect of an embodiment of the present disclosure, a coding and decoding device is proposed, including:

a processing module, configured to decode the first information to obtain channel information of at least one channel group, wherein the at least one channel group includes N first channel groups, the first channel group includes three channels, and the at least one channel group includes M The second channel grouping includes two channels, wherein N is 1 and M is a non-negative integer.

According to a sixth aspect of an embodiment of the present disclosure, a coding and decoding device is provided, including:

one or more processors;

The encoding and decoding device is used to execute any method described in the first aspect.

According to a seventh aspect of an embodiment of the present disclosure, a coding and decoding device is provided, including:

one or more processors;

Wherein, the encoding and decoding device is used to execute any method described in the second aspect.

According to an eighth aspect of the embodiments of the present disclosure, a coding and decoding system is proposed, including:

An encoder and a decoder, wherein the encoder is configured to implement the grouping method described in the first aspect, and the decoder is configured to implement the grouping method described in the second aspect.

According to a ninth aspect of an embodiment of the present disclosure, a storage medium is proposed, wherein the storage medium stores instructions, and when the instructions are executed on a communication device, the communication device executes a method as described in any one of the first aspect or the second aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the embodiments of the present disclosure and their descriptions are used to explain the embodiments of the present disclosure and do not constitute an improper limitation on the embodiments of the present disclosure. In the drawings:

FIG1 is a schematic diagram of the architecture of a coding and decoding system according to an embodiment of the present disclosure;

FIG2A is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

FIG2B is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

FIG2C is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

FIG2D is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

FIG2E is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure;

FIG3A is a schematic diagram of a flow chart of a grouping method according to an embodiment of the present disclosure;

FIG3B is a flow chart of a grouping method according to an embodiment of the present disclosure;

FIG4A is a schematic diagram of a flow chart of a grouping method according to an embodiment of the present disclosure;

FIG4B is a schematic diagram of a flow chart of a grouping method according to an embodiment of the present disclosure;

FIG5 is a flow chart of a grouping method according to an embodiment of the present disclosure;

FIG6 is a flow chart of a grouping method according to an embodiment of the present disclosure;

FIG7A is a schematic diagram of the structure of a coding and decoding device proposed in an embodiment of the present disclosure;

FIG7B is a schematic diagram of the structure of a coding and decoding device proposed in an embodiment of the present disclosure;

FIG8A is a schematic diagram of the structure of a communication device provided in an embodiment of the present disclosure;

FIG8B is a schematic diagram of the structure of a chip proposed in an embodiment of the present disclosure.

DETAILED DESCRIPTION

The present disclosure provides a grouping method, device and storage medium.

According to a first aspect of an embodiment of the present disclosure, a grouping method is proposed. The method is performed by an encoder, and the method includes:

A plurality of channels are grouped to obtain at least one channel group, wherein the at least one channel group includes N first channel groups, the first channel group includes three channels, and the at least one channel group includes M second channel groups, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer.

In the above embodiment, the problem that a single channel cannot be grouped when grouping channels is solved, and a method of dividing a channel group including three channels is provided to ensure that each channel can have a corresponding group when grouping channels, thereby ensuring the accuracy of channel grouping, thereby ensuring the reduction of redundancy between channels, ensuring the accuracy of channel encoding, and thereby ensuring the accuracy of audio encoding.

In conjunction with some embodiments of the first aspect, in some embodiments,

The step of grouping the plurality of channels to obtain at least one channel group includes:

Obtaining a similarity between any two channels of the multiple channels;

The plurality of channels are grouped based on the similarity between any two channels to obtain the at least one channel group.

In the above embodiment, the channels are grouped by the similarity between every two channels, so as to ensure that the similarity of the channels included in each channel group meets the requirements, ensure the accuracy of the channel grouping, and further ensure the reduction of redundancy between channels, ensure the accuracy of channel encoding, and further ensure the accuracy of audio encoding.

In combination with some embodiments of the first aspect, grouping the multiple channels based on the similarity between any two channels to obtain the at least one channel group includes:

For any two channels among the multiple channels, determine the two channels with the greatest similarity as a candidate channel group;

Excluding the other channels except the two channels with the greatest similarity, the two channels with the greatest similarity are determined as a candidate channel group until one independent channel remains or no channel remains;

The at least one channel grouping is determined based on the obtained candidate channel groupings and/or the independent channels.

In the above embodiment, the channels are divided into the same channel group in the order of similarity from large to small, ensuring that the similarities of the channels included in each channel group are channels with large correlation, ensuring the accuracy of channel grouping, and further ensuring the reduction of redundancy between channels, ensuring the accuracy of channel encoding, and further ensuring the accuracy of audio encoding.

In combination with some embodiments of the first aspect, the determining the at least one channel grouping based on the obtained candidate channel grouping and/or the independent channel includes:

Obtaining a similarity between the independent channel and each channel in each candidate channel group;

When the similarity between the independent channel and each channel in the first candidate channel group is greater than a similarity threshold, determining the independent channel and each channel in the first candidate channel group as a first channel group;

Each of the remaining candidate channel groups is determined as a second channel group.

In the above embodiment, the remaining single channels are also divided into a channel group to ensure that there is no single channel that is not divided into a channel group, thereby ensuring that the redundancy between channels is reduced, ensuring the accuracy of channel encoding, and thus ensuring the accuracy of audio encoding.

In combination with some embodiments of the first aspect, in some embodiments, grouping the multiple channels to obtain at least one channel group includes:

The multiple channels are grouped to directly obtain the N first channel groups and the M second channel groups.

In combination with some embodiments of the first aspect, in some embodiments, grouping the multiple channels to directly obtain the N first channel groups and the M second channel groups includes:

By performing a global search on the multiple channels, the multiple channels are grouped according to the similarity between any two channels to obtain N first channel groups and the M second channel groups.

In combination with some embodiments of the first aspect, in some embodiments, grouping the multiple channels to obtain at least one channel group includes:

Grouping the multiple channels to obtain multiple candidate channel groups including two channels and at least one independent channel;

An independent channel among the at least one independent channel is divided into one candidate channel group among the plurality of candidate channel groups to obtain the N first channel groups and the M second channel groups.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

Downmixing the audio signal in each of the channel groups to obtain a downmixed audio signal;

The downmixed audio signal is encoded to obtain an audio stream.

In the above embodiment, the audio signals of the channel groups are down-mixed and encoded to obtain an audio stream, which ensures that the encoded audio stream can be transmitted normally and also ensures the accuracy of the audio stream.

In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

First information is sent, where the first information is used to indicate channel information of the plurality of channel groups.

In the above embodiment, the channel information of each channel group is notified through the first information, thereby ensuring the reliability of transmitting the channel group.

In combination with some embodiments of the first aspect, in some embodiments, the channel information includes at least one of the following:

The number of channel groups including two channels;

channel identifiers of channels included in a channel group including two channels;

An energy parameter of a channel grouping comprising two channels;

The number of channel groups including three channels;

channel identifiers of channels included in a channel group including three channels;

An energy parameter of a channel grouping comprising three channels;

The energy parameter is used to adjust the energy of the channels in the channel group.

In the above embodiment, the channel information includes information of channel grouping of two channels or three channels, thereby ensuring the comprehensiveness of the channel information.

In a second aspect, an embodiment of the present disclosure provides a grouping method, which is performed by a decoder, and includes:

The first information is decoded to obtain channel information of at least one channel grouping, where there are N first channel groups in the at least one channel grouping, the first channel grouping includes three channels, and there are M second channel groups in the at least one channel grouping, the second channel grouping includes two channels, where N is 1 and M is a non-negative integer.

In conjunction with some embodiments of the second aspect, in some embodiments, the channel information includes at least one of the following:

The number of channel groups including two channels;

channel identifiers of channels included in a channel group including two channels;

An energy parameter of a channel grouping comprising two channels;

The number of channel groups including three channels;

channel identifiers of channels included in a channel group including three channels;

An energy parameter of a channel grouping comprising three channels;

The energy parameter is used to adjust the energy of the channels in the channel group.

In conjunction with some embodiments of the second aspect, in some embodiments, the method further includes:

When it is determined that the channel groups do not include a channel grouping for three channels, two-channel upmixing is performed on the audio stream.

In conjunction with some embodiments of the second aspect, in some embodiments, the method further includes:

When it is determined that the channel groups include a channel grouping with three channels, three-channel upmixing is performed on the audio stream.

In a third aspect, an embodiment of the present disclosure provides a grouping method, the method comprising:

The encoder groups the multiple channels to obtain at least one channel group, wherein the at least one channel group includes N first channel groups, the first channel group includes three channels, and the at least one channel group includes M second channel groups, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer;

The decoder decodes the first information to obtain channel information of the at least one channel grouping, wherein at least one channel grouping among the multiple channel groups includes three channels.

In a fourth aspect, an embodiment of the present disclosure provides a coding and decoding device, which includes at least one of a transceiver module and a processing module; wherein the encoder is used to execute the optional implementation methods of the first and third aspects.

In a fifth aspect, an embodiment of the present disclosure provides a coding and decoding device, which includes at least one of a transceiver module and a processing module; wherein the access network device is used to execute the optional implementation methods of the second and third aspects.

In a sixth aspect, an embodiment of the present disclosure provides a coding and decoding device, including:

one or more processors;

The encoding and decoding device is used to execute the method described in any one of the first aspect and the third aspect.

In a seventh aspect, an embodiment of the present disclosure provides a coding and decoding device, including:

one or more processors;

The encoding and decoding device is used to execute the method described in any one of the second aspect and the third aspect.

In an eighth aspect, an embodiment of the present disclosure provides a storage medium, wherein the storage medium stores first information, and when the first information is run on a communication device, the communication device executes a method as described in any one of the first aspect, the second aspect, and the third aspect.

In a ninth aspect, an embodiment of the present disclosure proposes a program product. When the program product is executed by a communication device, the communication device executes any one of the methods described in the first aspect, the second aspect and the third aspect.

In a tenth aspect, an embodiment of the present disclosure proposes a computer program, which, when executed on a communication device, enables the communication device to execute any one of the methods described in the first aspect, the second aspect, and the third aspect.

In an eleventh aspect, an embodiment of the present disclosure provides a chip or a chip system, wherein the chip or the chip system comprises a processing circuit configured to execute any one of the methods described in the first aspect, the second aspect, and the third aspect.

It can be understood that the above encoders, storage media, program products, computer programs, chips or chip systems are all used to implement the present disclosure. Therefore, the beneficial effects that can be achieved by the method proposed in the embodiment can refer to the beneficial effects in the corresponding method, which will not be repeated here.

The disclosed embodiments propose a grouping method, device, and storage medium. In some embodiments, the terms grouping method, information grouping method, grouping method, etc. can be replaced with each other, the terms encoding and decoding device, information processing device, indicating device, etc. can be replaced with each other, and the terms information processing system, encoding and decoding system, etc. can be replaced with each other.

The embodiments of the present disclosure are not exhaustive, but are only illustrative of some embodiments, and are not intended to be a specific limitation on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, the embodiments can be arbitrarily combined, for example, some or all of the steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

In each embodiment of the present disclosure, unless otherwise specified or there is a logical conflict, the terms and/or descriptions between the embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form a new embodiment based on their internal logical relationships.

The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular form, such as "a", "an", "the", "above", "said", "aforementioned", "this", etc., may mean "one and only one", or "one or more", "at least one", etc. For example, when using articles such as "a", "an", "the" in English in translation, the noun after the article may be understood as a singular expression or a plural expression.

In the embodiments of the present disclosure, “plurality” refers to two or more.

In some embodiments, the terms "at least one of", "one or more", "a plurality of", "multiple", etc. can be used interchangeably.

In some embodiments, "at least one of A and B", "A and/or B", "A in one case, B in another case", "in response to one case A, in response to another case B", etc., may include the following technical solutions according to the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed); in some embodiments, A and B (both A and B are executed). When there are more branches such as A, B, C, etc., the above is also similar.

In some embodiments, the recording method of "A or B" may include the following technical solutions according to the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed). When there are more branches such as A, B, C, etc., the above is also similar.

The prefixes such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different description objects, and do not constitute any restrictions on the position, order, priority, quantity or content of the description objects. For the statement of the description object, please refer to the description in the context of the claims or embodiments, and no unnecessary restrictions should be imposed due to the use of prefixes. For example, if the description object is a "field", the ordinal number before the "field" in the "first field" and the "second field" does not limit the position or order between the "fields". "First" and "second" do not limit whether the "fields" they modify are in the same message, nor do they limit the order of the "first field" and the "second field". For another example, if the description object is a "level", the ordinal number before the "level" in the "first level" and the "second level" does not limit the priority between the "levels". For another example, the number of description objects is not limited by ordinal numbers and can be one or more. Taking "first device" as an example, the number of "devices" can be one or more. In addition, different prefixes modify The objects may be the same or different. For example, if the description object is “device”, then the “first device” and the “second device” may be the same device or different devices, and their types may be the same or different. For another example, if the description object is “information”, then the “first information” and the “second information” may be the same information or different information, and their contents may be the same or different.

In some embodiments, “including A”, “comprising A”, “used to indicate A”, and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

In some embodiments, terms such as "time/frequency", "time/frequency domain", etc. refer to the time domain and/or the frequency domain.

In some embodiments, terms such as "in response to ...", "in response to determining ...", "in the case of ...", "at the time of ...", "when ...", "if ...", "if ...", etc. can be used interchangeably.

In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not lower than", and "above" can be replaced with each other, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "no more than", "lower than", "lower than or equal to", "not higher than", and "below" can be replaced with each other.

In some embodiments, devices and equipment may be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. In some cases, they may also be understood as "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", "subject", etc.

In some embodiments, "network" can be interpreted as devices included in the network, such as access network equipment, core network equipment, etc.

In some embodiments, "access network device (AN device)" may also be referred to as "radio access network device (RAN device)", "base station (BS)", "radio base station (radio base station)", "fixed station" and in some embodiments may also be understood as "node", "access point (access point)", "transmission point (TP)", "reception point (RP)", "transmission and/or reception point (transmission/reception point, TRP)", "panel", "antenna panel", "antenna array", "cell", "macro cell", "small cell", "femto cell", "pico cell", "sector", "cell group", "serving cell", "carrier", "component carrier", "bandwidth part (bandwidth part, BWP)", etc.

In some embodiments, "encoder (terminal)" or "encoder device (terminal device)" can be referred to as "user equipment (encoder)", "user encoder (user terminal)", "mobile station (MS)", "mobile encoder (mobile terminal, MT)", subscriber station (subscriber station), mobile unit (mobile unit), subscriber unit (subscriber unit), wireless unit (wireless unit), remote unit (remote unit), mobile device (mobile device), wireless device (wireless device), wireless communication device (wireless communication device), remote device (remote device), mobile subscriber station (mobile subscriber station), access terminal, mobile encoder (mobile terminal), wireless encoder (wireless terminal), remote encoder (remote terminal), handheld device (handset), user agent (user agent), mobile client (mobile client), client (client), etc.

In some embodiments, acquisition of data, information, etc. may comply with the laws and regulations of the country where the data is obtained.

In some embodiments, data, information, etc. may be obtained with the user's consent.

In addition, each element, each row, or each column in the table of the embodiment of the present disclosure can be implemented as an independent embodiment. Any combination of pixels, any rows, and any columns can also be implemented as independent embodiments.

FIG1 is a schematic diagram of the architecture of a codec system according to an embodiment of the present disclosure. As shown in FIG1 , the method provided by the embodiment of the present disclosure can be applied to a codec system 100, and the codec system can include an encoder 101 and a decoder 102. It should be noted that the codec system 100 can also include other devices, and the present disclosure does not limit the devices included in the codec system 100.

In some embodiments, the encoder 101 and the decoder 102 are both arranged in the terminal. In some embodiments, the terminal can be various devices. For example, it includes at least one of a mobile phone, a wearable device, an Internet of Things device, a car with communication function, a smart car, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) encoder device, an augmented reality (AR) encoder device, a wireless encoder device in industrial control, a wireless encoder device in self-driving, a wireless encoder device in remote medical surgery, a wireless encoder device in smart grid, a wireless encoder device in transportation safety, a wireless encoder device in smart city, and a wireless encoder device in smart home, but is not limited thereto.

It can be understood that the encoding and decoding system described in the embodiment of the present disclosure is for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not constitute a limitation on the technical solution proposed in the embodiment of the present disclosure. A person of ordinary skill in the art can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution proposed in the embodiment of the present disclosure is also applicable to similar technical problems.

The following embodiments of the present disclosure may be applied to the codec system 100 shown in FIG1 , or part of the subject, but are not limited thereto. The subjects shown in FIG1 are examples, and the codec system may include all or part of the subjects in FIG1 , or may include other subjects other than FIG1 , and the number and form of the subjects are arbitrary, and the subjects may be physical or virtual, and the connection relationship between the subjects is an example, and the subjects may be connected or disconnected, and the connection may be in any manner, and may be a direct connection or an indirect connection, and may be a wired connection or a wireless connection.

The embodiments of the present disclosure may be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, the fourth generation mobile communication system (4G), the fifth generation mobile communication system (5G), 5G new radio (NR), Future Radio Access (FRA), New-Radio Access Technology (RAT), New Radio (NR), New radio access (NX), Future generation radio access ...-Radio The present invention relates to wireless communication networks such as LTE, LTE-A, LTE-X, Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra Wide Band (UWB), Bluetooth (registered trademark)), Public Land Mobile Network (PLMN) network, Device to Device (D2D) system, Machine to Machine to Machine (M2M) system, Internet of Things (IoT) system, Vehicle to Everything (V2X), systems using other packet methods, and next-generation systems extended based on them. In addition, a combination of multiple systems (for example, a combination of LTE or LTE-A with 5G, etc.) may also be applied.

In some embodiments, the present disclosure is used for three-dimensional sound audio coding. Among them, the three-dimensional sound audio coding is one of the key technologies of immersive audio technology. Three-dimensional sound increases the sense of space and orientation compared to traditional sound, allowing listeners to reproduce the sound heard in the real world, thereby meeting people's needs for highly restored sound and highly immersive experience, while providing personalized selection and interactive experience.

In some embodiments, in order to reproduce the spatial sense and orientation sense of sound, the technology can rely on a channel-based approach, an object-based approach, a sound field-based approach, and a combination of the above three forms, among which: channel-based audio is a group of interrelated channels, common ones are 5.1 channels, 7.1 channels, 5.1.4 channels, 7.1.4 channels, etc. Each format corresponds to a speaker layout, and the best playback effect can be obtained under the corresponding speaker layout.

In some embodiments, object-based audio is a collection of a series of mono audio elements and corresponding metadata. The metadata represents information such as the location, intensity, size, etc. of the object. During playback, the object is mapped to one or more speakers or binaurally rendered to headphones based on the metadata information to achieve the desired spatial audio effect.

In some embodiments, sound field-based audio is a 3D sound field modeling format defined on the surface of a sphere. The principle is that sound is transmitted as a pressure wave. For a sound scene at a given time, each point needs to be reflected by several pressure functions. If the pressure value of each point in the space is known, the sound in the space can be reconstructed. There is a certain relationship between the pressure of each point in the space and its neighboring points. In order to give full play to the advantages of scene-based audio production, it is necessary to accurately obtain the coefficients and improve the encoding quality of the sound field space coefficients. The collected sound field signal is called Higher Order Ambisonics (HOA). The performance of the HOA system increases with the increase of the HOA order, but the number of HOA signals also increases.

FIG2 is an interactive schematic diagram of a grouping method according to an embodiment of the present disclosure. As shown in FIG2 , an embodiment of the present disclosure relates to a grouping method, and the method includes:

Step S2101: the encoder groups multiple channels to obtain at least one channel group.

In some embodiments, a channel refers to independent audio signals collected or played back at different spatial locations when recording or playing sound. A channel refers to an element of audio, and different types of audio have different numbers of channels. For example, a HOA3 signal has 16 channels, and a MC22.2 signal has 24 channels.

In some embodiments, the audio signal comprises audio data and side information.

In some embodiments, a channel group refers to a group including at least two channels. The encoder encodes the channels in units of channel groups to obtain an audio stream.

In some embodiments, the name of the channel group is not limited, and may be, for example, a channel group, a channel group category, etc.

In some embodiments, there are N first channel groups in at least one channel group, the first channel group includes three channels, and M is a non-negative integer.

In some embodiments, there are M second channel groups in the at least one channel group, the second channel group includes two channels, where N is 1.

In some embodiments, N may also be 0, that is, when a plurality of channels are grouped, no channel group including three channels is obtained, that is, 0 first channel groups.

Optionally, for the value of M, M should not be greater than half of the number of channels. Optionally, if the number of channels for grouping channels is P, then the value of M is M greater than or equal to 0 and less than or equal to P/2, where P is a positive integer.

In some embodiments, before the channels are grouped, it is necessary to pre-process the signals included in the channels, and then group the channels after the pre-processing is completed.

Optionally, the method of preprocessing the signal included in the channel includes at least one of the following: transient detection, window type judgment, time-frequency Transform, frequency domain noise shaping, time domain noise shaping, and band extension coding.

In some embodiments, grouping the multiple channels to obtain at least one channel group includes: obtaining similarity between any two channels in the multiple channels, and grouping the multiple channels based on the similarity between any two channels to obtain at least one channel group.

In the embodiment of the present disclosure, the encoder may obtain the similarity between every two channels, and then group the multiple channels according to the magnitude relationship of the similarity between every two channels to obtain at least one channel group.

In some embodiments, a plurality of channels are grouped based on similarity between any two channels to obtain at least one channel grouping, including: for any two channels among the plurality of channels, two channels with the greatest similarity are determined as a candidate channel grouping, and among channels other than the two channels with the greatest similarity, the two channels with the greatest similarity are determined as a candidate channel grouping until one independent channel remains or no channels remain, and at least one channel grouping is determined based on the obtained candidate channel grouping and/or independent channels.

For example, the multiple channels are channel 1, channel 2, channel 3, channel 4 and channel 5, then the similarity between every two channels is obtained respectively. For example, the similarity between channel 1 and channel 2 is the highest, then channel 1 and channel 2 are grouped as a channel, and then channel 1 and channel 2 are excluded, and the channels with the greatest similarity among channel 3, channel 4 and channel 5 are grouped as a channel. At this time, one channel 5 remains, and channel 5 is an independent channel.

In some embodiments, at least one channel grouping is determined based on the obtained candidate channel groups and/or independent channels, including: obtaining the similarity between the independent channel and each channel in each candidate channel group, and when the similarity between the independent channel and each channel in the first candidate channel group is greater than a similarity threshold, determining the independent channel and each channel in the first candidate channel group as a first channel group, and determining each remaining candidate channel group as a second channel group.

In the embodiment of the present disclosure, when the similarity between an independent channel and each channel in the first candidate channel group is greater than the similarity threshold, it means that the independent channel is similar to both channels in the first candidate channel group. In this case, the independent channel can be divided into the first candidate channel group. At this time, the first candidate channel group includes three channels, that is, the first channel group, and the remaining channel group still includes two channels, that is, the second channel group.

In some embodiments, when a plurality of channels are grouped based on the similarity between any two channels, when it is determined that an odd number of channels satisfy the grouping condition and then grouped to obtain at least one channel group, the similarity between the independent channel and each channel in each candidate channel group is obtained, and when the similarity between the independent channel and each channel in the first candidate channel group is greater than a similarity threshold, the independent channel and each channel in the first candidate channel group are determined as a first channel group, and each of the remaining candidate channel groups are respectively determined as a second channel group.

It should be noted that the embodiment of the present disclosure is described by taking the example of obtaining independent channels. In another embodiment, for any two channels among the multiple channels, the two channels with the greatest similarity are determined as a candidate channel group, and among the channels other than the two channels with the greatest similarity, the two channels with the greatest similarity are determined as a candidate channel group until no channels remain, and at least one channel group is determined based on the obtained candidate channel group.

It should be noted that, for any two channels among the multiple channels, if the similarity between any two channels is not greater than the similarity threshold, the channels are not grouped.

It should be noted that in the embodiment of the present disclosure, the similarity between an independent channel and two channels in multiple candidate channel groups may be greater than the similarity threshold. In this case, it is necessary to additionally determine into which candidate channel group the independent channel is to be divided.

In some embodiments, the sum of similarities between the independent channel and two channels in each candidate channel group is obtained, the largest sum is selected from the sums corresponding to each candidate channel group, and the independent channel is divided into the channel group corresponding to the largest sum.

For example, the independent channel is channel 5, the first candidate channel group includes channel 1 and channel 2, and the second candidate channel group includes channel 3 and channel 4. If the sum of the similarities between channel 5 and channel 1 and channel 2 is 1.8, and the sum of the similarities between channel 5 and channel 3 and channel 4 is 1.9, channel 5 is divided into the second candidate channel group.

In some embodiments, among the similarities between the independent channel and the two channels in each candidate channel group, the candidate channel group corresponding to the greatest similarity is determined, and the independent channel is divided into the channel group corresponding to the greatest similarity.

For example, the independent channel is channel 5, the first candidate channel group includes channel 1 and channel 2, and the second candidate channel group includes channel 3 and channel 4. If the similarity between channel 5 and channel 1 is 0.7, the similarity between channel 5 and channel 2 is 0.9, the similarity between channel 5 and channel 3 is 0.8, and the similarity between channel 5 and channel 4 is 0.6, then channel 5 is divided into the first candidate channel group.

It should be noted that, in the above embodiment, if there is an independent channel that is the same as the sum or maximum value of the channels in at least two candidate channel groups, one of the candidate channel groups is randomly selected and the independent channel is divided into the selected channel group.

It should be noted that the grouping in the embodiment of the present disclosure includes two grouping schemes. Each grouping scheme is described below.

In some embodiments, multiple channels are grouped to directly obtain N first channel groups and M second channel groups. Optionally, in the embodiments of the present disclosure, when multiple channels are grouped, the N first channel groups and M second channel groups obtained by grouping are directly output, and no other grouping results are output during the process.

Optionally, grouping the multiple channels to directly obtain N first channel groups and M second channel groups includes: performing a global search on the multiple channels and grouping the multiple channels according to the similarity between any two channels to obtain N first channel groups and M second channel groups.

In a possible implementation, the similarity between any two of every three channels in the multiple channels is obtained, and if the similarity between any two of every three channels is greater than a similarity threshold, the three channels are divided into a candidate channel group, and if a candidate channel group is finally obtained, the candidate channel group is determined as the first channel group. If multiple candidate channel groups are finally obtained, the first channel group is determined from the multiple candidate channel groups according to the similarity relationship of each candidate channel group.

Optionally, if multiple candidate channel groups are finally obtained, then determining a first channel group from the multiple candidate channel groups according to the similarity relationship of each candidate channel group includes: obtaining the sum of the similarities between every two channels in each candidate channel group, selecting the largest sum from the sums corresponding to each candidate channel group, and determining the channel group corresponding to the largest sum as the first channel group.

For example, the first candidate channel grouping includes channel 1, channel 2, and channel 3, and the second candidate channel grouping includes channel 4, channel 5, and channel 6. If the sum of the similarities between each two channels in channel 1, channel 2, and channel 3 is 2.7, and the sum of the similarities between channel 4, channel 5, and channel 6 is 2.6, then the first candidate channel grouping is determined to be the first channel grouping.

Optionally, if multiple candidate channel groups are finally obtained, then according to the similarity relationship between each candidate channel group, a first channel group is determined from the multiple candidate channel groups, including: determining a candidate channel group corresponding to the greatest similarity between the independent channel and every two channels in each candidate channel group, and determining the candidate channel group corresponding to the greatest similarity as the first channel group.

In some embodiments, the process of grouping multiple channels is similar to the process of grouping based on similarity in the above embodiments, and will not be described in detail here.

In some embodiments, multiple channels are grouped to obtain multiple candidate channel groups including two channels and at least one independent channel, and an independent channel of the at least one independent channel is divided into one candidate channel group in the multiple candidate channel groups to obtain N first channel groups and M second channel groups.

It should be noted that the embodiment of the present disclosure is described by taking grouping of multiple channels as an example. In another embodiment, after the multiple channels are grouped, channel information of each channel group is also generated.

In some embodiments, the channel information includes at least one of the following:

(1) The number of channel groups including two channels;

In some embodiments, the number of channel groups including two channels may also be referred to as the number of second channel groups, or the number of second channel groups, or the number of channel group pairs, or the number of groups, etc., which is not limited in the embodiments of the present disclosure.

In some embodiments, the number of channel groups including two channels is used to represent the number of second channel group pairs of the current frame.

(2) channel identifiers of channels included in a channel group including two channels;

In some embodiments, the channel identifiers of the channels included in the channel group including two channels are used to indicate the index of the channel pair, and the index values of the two channels in the current channel pair can be obtained by parsing.

(3) Energy parameters of a channel grouping including two channels;

(4) The number of channel groups including three channels;

In some embodiments, the number of channel groups including three channels may also be referred to as the number of first channel groups, or the number of first channel groups, or the number of channel group pairs, or the number of groups, etc., which is not limited in the embodiments of the present disclosure.

In some embodiments, the number of channel groups including three channels is used to represent the number of first channel group pairs of the current frame.

(5) channel identifiers of channels included in a channel group including three channels;

In some embodiments, the channel identifiers of the channels included in the channel group including three channels are used to indicate the index of the channel pair, and the index values of the three channels in the current channel pair can be obtained by parsing.

(6) Energy parameters of a channel grouping including three channels;

The energy parameter is used to adjust the energy of the channels in the channel grouping.

In some embodiments, the energy parameter is used to quantize an index of an inter-channel amplitude difference ILD parameter between a first channel and a second channel in a current channel pair for inter-channel energy/amplitude adjustment.

In some embodiments, a plurality of channels are grouped to directly obtain N first channel groups and M second channel groups, and channel information of the first channel groups and the second channel groups is generated.

In some embodiments, a plurality of channels are grouped to obtain a plurality of candidate channel groups including two channels and at least one independent channel, channel information of the plurality of candidate channel groups and the at least one independent channel is generated, an independent channel of the at least one independent channel is divided into one candidate channel group of the plurality of candidate channel groups to obtain N first channel groups and M second channel groups, and the channel information of the plurality of candidate channel groups and the at least one independent channel is rewritten to obtain channel information of the first channel group and the second channel group.

Optionally, multiple channels are grouped to obtain M+1 candidate channel groups including two channels and at least one independent channel, channel information of the multiple candidate channel groups and the at least one independent channel is generated, and an independent channel of the at least one independent channel is divided into one candidate channel group in the multiple candidate channel groups to obtain N first channel groups and M second channel groups.

It should be noted that the method of grouping multiple channels to obtain the first channel grouping can be called 3-channel sum and difference coding. The method of grouping multiple channels to obtain the second channel grouping can be called 2-channel sum and difference coding.

It should be noted that the method executed in the embodiment of the present disclosure is executed by the judgment module, or by other modules, and the embodiment of the present disclosure is not limited.

Step S2102: the encoder downmixes the audio signal in each channel group to obtain a downmixed audio signal.

In some embodiments, downmixing refers to mixing the grouped channels using an orthogonal normalization matrix to obtain a mixed channel for each channel.

Optionally, the orthogonal normalized matrix is preset, and the embodiment of the present disclosure does not limit the orthogonal normalized matrix.

For example, for a channel group including two channels, the orthogonal normalization matrix used is Among them, the first row is the sum vector and the second row is the difference vector.

For a channel group including three channels, the orthogonal normalization matrix used is Among them, the first row is the sum vector, and the second and third rows are the difference vectors.

In some embodiments, M _{3ch ms} I _{3ch ms} = O _{3ch ms} , M _{2ch ms} I _{2ch ms} = O _{2ch ms} . Wherein, I _{3ch ms} is a 1*3 column vector, and the column vector refers to the audio data in the channel group including three channels. _{I 2ch ms} is a 1*2 column vector, and the column vector refers to the audio data in the channel group including two channels.

It should be noted that the steps performed in the embodiment of the present disclosure are performed by a downmixing module, or are performed in other ways, which is not limited in the embodiment of the present disclosure.

Step S2103: the encoder encodes the downmixed audio signal to obtain an audio stream.

In some embodiments, encoding includes bit allocation quantization entropy coding and code stream multiplexing.

The following is an example of how to execute the above steps.

For example, referring to FIG. 2B , the decision module (step S2101) is executed to determine which sum and difference coding method or a combination of the two is used. The channels include an L channel, an R channel, a C channel, an LS channel and an RS channel, wherein the similarity between any two channels is shown in Table 1.

Table 1

The downmix threshold is 0.5. The decision module obtains the sum-difference coding method to be used through a certain algorithm.

Among them, 3CH M/S refers to a module in which three channels are divided into the first channel grouping, and 2CH M/S refers to a module in which two channels are divided into the second channel grouping.

1. Use maximum similarity iteration to select the first channel pair L channel and R channel (maximum similarity cox_L_R=0.85), and the second channel pair LS channel and RS channel (maximum similarity cox_LS_RS=0.66 among the remaining un-downmixed channels).

2. Calculate the similarity between all channels that are not downmixed (C channel) and the two channels of the first channel pair and the second channel pair.

The similarity between the C channel and the L channel cox_C_L=0.74 is greater than the downmix threshold, and the similarity between the C channel and the R channel cox_C_R=0.78 is greater than the downmix threshold.

3. Output the judgment results of 3CH M/S and 2CH M/S. In this embodiment, the judgment result of 3CH M/S outputs L channel, R channel, and C channel. The judgment result of 2CH M/S outputs LS channel and RS channel.

Subsequently, the downmixing module is executed.
M _{2ch ms} I _{2ch ms} ＝O _{2ch ms}
M _{3ch ms} I _{3ch ms} ＝O _{3ch ms}

The two-channel downmix matrix M _{2ch ms} is a 2x2 orthogonal normalized matrix, in which the first row is a sum vector and the second row is a difference vector. The three-channel downmix matrix M _{3ch ms} is a 3x3 orthogonal normalized matrix, in which the first row is a sum vector and the second and third rows are difference vectors.

I _{2ch ms} is a 1x2 column vector, and I _{3ch ms} is a 1x3 column vector. The data contained in the vector is the pre-processed audio data, in units of sampling points or frequency points.

Generate channel information for each channel group.

For another example, referring to FIG. 2C , two-channel pairing decision is performed to obtain two-channel pairing information, including the number of pairs and the channel pair index.

The channels include an L channel, an R channel, a C channel, an LS channel and an RS channel, wherein the similarity between any two channels is as shown in Table 1, and the downmix threshold is 0.5.

The two-channel pair decision uses maximum similarity iteration to select the first channel pair L channel and R channel (maximum similarity cox_L_R=0.85), and the second channel pair LS channel and RS channel (maximum similarity cox_LS_RS among the remaining un-downmixed channels=0.66).

The number of the second channel groups finally obtained is 2, including the first channel group L channel and R channel, and the second channel group LS channel and RS channel.

Secondly, a three-channel pairing judgment is executed. If a three-channel pairing is generated, the above channel grouping result will be changed.

First, the similarity between all channels that are not downmixed (C channel) and the two channels of the first channel pair and the second channel pair is calculated.

The similarity between the C channel and the L channel cox_C_L=0.74 is greater than the downmix threshold, and the similarity between the C channel and the R channel cox_C_R=0.78 is greater than the downmix threshold.

Output the judgment results of 3CH M/S and 2CH M/S. In this embodiment, the judgment result of 3CH M/S outputs L channel, R channel, and C channel. After the judgment result of 2CH M/S is changed, 1 channel group is output, that is, LS channel and RS channel are output.

Generates two-channel and three-channel channel information.

Implements two-channel and three-channel downmixing modules.
M _{2ch ms} I _{2ch ms} ＝O _{2ch ms}
M _{3ch ms} I _{3ch ms} ＝O _{3ch ms}

The two-channel downmix matrix M _{2ch ms} is a 2x2 orthogonal normalized matrix, in which the first row is a sum vector and the second row is a difference vector. The three-channel downmix matrix M _{3ch ms} is a 3x3 orthogonal normalized matrix, in which the first row is a sum vector and the second and third rows are difference vectors.

The above channel information is rewritten to obtain new channel information.

In some embodiments, the encoder obtains an audio stream by executing the above steps. Referring to FIG2D , after obtaining the channel signal, the channel signal is preprocessed, and then the preprocessed channel signal is down-mixed between channels, and then bit allocation quantization entropy coding is performed, and finally bit stream multiplexing is performed to obtain an audio stream.

Among them, the preprocessing includes transient detection window type judgment, time-frequency transformation, frequency domain noise shaping, time domain noise shaping, band extension coding and other processes. Inter-channel group downmixing includes three-channel group downmixing of L channel, R channel and C channel to obtain M1 channel, S11 channel and S12 channel; and two-channel group downmixing of LS channel and RS channel to obtain M2 channel and S2 channel, and LFE channel is not processed.

Step S2104: the encoder sends the first information and the audio stream.

In some embodiments, a decoder receives first information and an audio stream.

In some embodiments, the encoder may send the first information and the audio stream separately. For example, the encoder sends the first information first and then sends the audio stream. Alternatively, the encoder sends the audio stream first and then sends the first information. In some embodiments, the encoder may send the first information and the audio stream simultaneously.

In some embodiments, the channel information includes at least one of the following:

The number of channel groups including two channels;

channel identifiers of channels included in a channel group including two channels;

An energy parameter of a channel grouping comprising two channels;

The number of channel groups including three channels;

channel identifiers of channels included in a channel group including three channels;

An energy parameter of a channel grouping comprising three channels;

The energy parameter is used to adjust the energy of the channels in the channel grouping.

Step S2105: The decoder decodes the first information to obtain channel information of at least one channel group.

Step S2106: When the decoder determines that the channel grouping does not include a channel grouping with three channels, the decoder performs two-channel upmixing on the audio stream; when the decoder determines that the channel grouping includes a channel grouping with three channels, the decoder performs three-channel upmixing on the audio stream.

In some embodiments, the decoder determines whether the encoder has performed a two-channel downmix and a three-channel downmix. If the encoder has performed a two-channel downmix, the decoder performs a two-channel upmix. If the encoder has performed a three-channel downmix, the decoder performs a three-channel upmix.

Optionally, audio post-processing includes but is not limited to general decoding processes, such as time-frequency inverse transform, time-domain noise shaping inverse transform, frequency-domain noise shaping inverse transform, frequency band extension inverse transform and other modules, and also includes decoding processing for certain types of signal characteristics, such as multi-channel decoding processing, HOA channel decoding processing, object metadata decoding processing, etc.

In some embodiments, the decoder obtains each channel signal by performing the above steps. Referring to FIG2E , after obtaining the audio stream, the audio stream is demultiplexed, and then bit allocation inverse quantization entropy coding, inter-channel upmixing, and post-processing are performed to obtain decoded channel signals.

Among them, post-processing includes frequency band extension decoding, inverse time domain noise shaping, inverse frequency domain noise shaping, inverse time-frequency transformation, etc. The inter-channel group upmixing includes upmixing the M1 channel, S11 channel, and S12 channel into L channel, R channel, and C channel in three-channel groups to obtain channels; and upmixing the M2 channel and S2 channel into LS channel and RS channel in two-channel groups to obtain channels, and the LFE channel is not processed.

In some embodiments, the names of information, etc. are not limited to the names recorded in the embodiments, and terms such as "information", "message", "signal", "signaling", "report", "configuration", "indication", "instruction", "command", "channel", "parameter", "domain", "field", "symbol", "symbol", "code element", "codebook", "codeword", "codepoint", "bit", "data", "program", and "chip" can be used interchangeably.

In some embodiments, terms such as "uplink", "uplink", "physical uplink" can be interchangeable, and terms such as "downlink", "downlink", "physical downlink" can be interchangeable, and terms such as "side", "sidelink", "side communication", "sidelink communication", "direct connection", "direct link", "direct communication", "direct link communication" can be interchangeable.

In some embodiments, "obtain", "obtain", "get", "receive", "transmit", "bidirectional transmission", "send and/or receive" can be interchangeable, and can be interpreted as receiving from other entities, obtaining from protocols, obtaining from high levels, obtaining by self-processing, autonomous implementation, etc.

In some embodiments, terms such as "send", "transmit", "report", "send", "transmit", "bidirectional transmission", "send and/or receive" can be used interchangeably.

In some embodiments, terms such as "moment", "time point", "time", and "time position" can be interchangeable, and terms such as "duration", "period", "time window", "window", and "time" can be interchangeable.

In some embodiments, terms such as "certain", "preset", "preset", "set", "indicated", "some", "any", and "first" can be interchangeable, and "specific A", "preset A", "preset A", "set A", "indicated A", "some A", "any A", and "first A" can be interpreted as A pre-defined in a protocol, etc., or as A obtained through setting, configuration, or indication, etc., and can also be interpreted as specific A, some A, any A, or first A, etc., but is not limited to this.

The grouping method involved in the embodiment of the present disclosure may include at least one of steps S2101 to S2106. For example, step S2101 may be implemented as an independent embodiment, step S2102 may be implemented as an independent embodiment, step S2103 may be implemented as an independent embodiment, step S2104 may be implemented as an independent embodiment, step S2105 may be implemented as an independent embodiment, step S2106 may be implemented as an independent embodiment, step S2101 and step S2102 may be implemented as independent embodiments ...6 may be implemented as an independent embodiment, step S2103 may be implemented as an independent embodiment, step S2104 may be implemented as an independent embodiment, step S2105 may be implemented as an independent embodiment, step S2106 may be implemented as an independent embodiment, step S2106 may be implemented as an independent embodiment, step S210 Step S2103 and step S2104 can be implemented as independent embodiments, step S2105 and step S2106 can be implemented as independent embodiments, step S2101, step S2102, step S2103, and step S2104 can be implemented as independent embodiments, step S2101, step S2102, step S2105, and step S2106 can be implemented as independent embodiments, step S2103, step S2104, step S2105, and step S2106 can be implemented as independent embodiments, but are not limited to this.

In some embodiments, step S2101 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2102 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2103 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2104 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2105 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2106 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2101 and step S2102 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S2103 and step S2104 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, step S215 and step S2106 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

In some embodiments, reference may be made to other optional implementations recorded before or after the description corresponding to FIG. 2 .

FIG3A is a flow chart of a grouping method according to an embodiment of the present disclosure, which is applied to an encoder. As shown in FIG3A , an embodiment of the present disclosure relates to a grouping method, which includes:

Step S3101: The encoder groups multiple channels to obtain at least one channel group.

The optional implementation of step S3101 can refer to the optional implementation of step S2101 in FIG. 2 and other related parts in the embodiment involved in FIG. 2 , which will not be described in detail here.

Step S3102: the encoder downmixes the audio signal in each channel group to obtain a downmixed audio signal.

The optional implementation of step S3102 can refer to the optional implementation of step S2102 in FIG. 2 and other related parts in the embodiment involved in FIG. 2 , which will not be described in detail here.

Step S3103: the encoder encodes the downmixed audio signal to obtain an audio stream.

The optional implementation of step S3103 can refer to the optional implementation of step S2103 in FIG. 2 and other related parts in the embodiment involved in FIG. 2 , which will not be described in detail here.

Step S3104: the encoder sends the first information and the audio stream.

The optional implementation of step S3103 can refer to the optional implementation of step S2104 in FIG. 2 and other related parts in the embodiment involved in FIG. 2 , which will not be described in detail here.

The grouping method involved in the embodiment of the present disclosure may include at least one of steps S3101 to S3104. For example, step S3101 may be implemented as an independent embodiment, step S3102 may be implemented as an independent embodiment, step S3103 may be implemented as an independent embodiment, step S3104 may be implemented as an independent embodiment, or at least two steps may be combined, but are not limited thereto.

In some embodiments, step S3101 is optional, step S3102 is optional, step S3103 is optional, and step S3104 is optional. In different embodiments, one or more of these steps may be omitted or replaced, but the present invention is not limited thereto.

FIG3B is a flow chart of a grouping method according to an embodiment of the present disclosure, which is applied to an encoder. As shown in FIG3B , an embodiment of the present disclosure relates to a grouping method, which includes:

Step S3201: The encoder groups multiple channels to obtain at least one channel group.

The optional implementation of step S3201 can refer to step S2101 of FIG. 2 , step S3101 of FIG. 3A , and other related parts of the embodiments involved in FIG. 2 and FIG. 3A , which will not be described in detail here.

FIG4A is a flow chart of a grouping method according to an embodiment of the present disclosure, which is applied to a decoder. As shown in FIG4A , an embodiment of the present disclosure relates to a grouping method, and the method includes:

Step S4101: The decoder decodes the first information to obtain channel information of at least one channel group.

The optional implementation of step S4101 can refer to step S2105 of FIG. 2 and other related parts of the embodiment involved in FIG. 2 , which will not be described in detail here.

Step S4102: When the decoder determines that the channel grouping does not include a channel grouping with three channels, it performs two-channel upmixing on the audio stream; when the decoder determines that the channel grouping includes a channel grouping with three channels, it performs three-channel upmixing on the audio stream.

The optional implementation of step S4102 can refer to step S2106 of FIG. 2 and other related parts of the embodiment involved in FIG. 2 , which will not be described in detail here.

The grouping method involved in the embodiment of the present disclosure may include at least one of step S4101 to step S4102. For example, step S4101 may be implemented as an independent embodiment, step S4102 may be implemented as an independent embodiment, or at least two steps may be combined, but not limited thereto.

In some embodiments, step S4101 is optional, step S4102 is optional, and one or more of these steps may be omitted or replaced in different embodiments, but the present invention is not limited thereto.

FIG4B is a flow chart of a grouping method according to an embodiment of the present disclosure, which is applied to a decoder. As shown in FIG4B , an embodiment of the present disclosure relates to a grouping method, and the method includes:

Step S4201: The decoder decodes the first information to obtain channel information of at least one channel group.

The optional implementation of step S4201 can refer to step S2105 of FIG. 2 and other related parts of the embodiment involved in FIG. 2 , which will not be described in detail here.

In some embodiments, there are N first channel groups in the at least one channel group, the first channel group includes three channels, and there are M second channel groups in the at least one channel group, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer.

In some embodiments, the channel information includes at least one of the following:

The number of channel groups including two channels;

channel identifiers of channels included in a channel group including two channels;

An energy parameter of a channel grouping comprising two channels;

The number of channel groups including three channels;

channel identifiers of channels included in a channel group including three channels;

An energy parameter of a channel grouping comprising three channels;

The energy parameter is used to adjust the energy of the channels in the channel group.

In some embodiments, the method further comprises:

When it is determined that the channel groups do not include a channel grouping for three channels, two-channel upmixing is performed on the audio stream.

In some embodiments, the method further comprises:

When it is determined that the channel groups include a channel grouping with three channels, three-channel upmixing is performed on the audio stream.

FIG5 is a flow chart of a grouping method according to an embodiment of the present disclosure. As shown in FIG5 , the embodiment of the present disclosure relates to a grouping method, and the method includes:

Step S5101: the encoder groups multiple channels to obtain at least one channel group.

In some embodiments, there are N first channel groups in the at least one channel group, the first channel group includes three channels, and there are M second channel groups in the at least one channel group, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer.

Step S5102: The decoder decodes the first information to obtain channel information of at least one channel group.

The optional implementation of step S5101 can refer to step S2101 in FIG. 2 , step S3101 in FIG. 3A , and other related parts in the embodiments involved in FIG. 2 and FIG. 4A , which will not be described in detail here.

The optional implementation of step S5102 can refer to step S2105 of FIG. 2 , step S4101 of FIG. 4A , and other related parts of the embodiments involved in FIG. 2 and FIG. 3A , which will not be described in detail here.

In some embodiments, the above method may include the method of the above-mentioned embodiments of the coding and decoding system side, encoder side, decoder side, etc., which will not be repeated here.

FIG6 is a flow chart of a grouping method according to an embodiment of the present disclosure. As shown in FIG6 , the embodiment of the present disclosure relates to a grouping method, and the method includes:

Step S6101: the encoding end encodes multiple channels by combining two-channel sum and difference encoding and three-channel sum and difference encoding.

The encoder completes audio preprocessing and enters the channel pair downmixing module. Audio preprocessing includes but is not limited to general encoding processes, such as transient analysis, time-frequency transformation, time-domain noise shaping, frequency-domain noise shaping, and frequency band extension. It also includes audio preprocessing for certain types of signals. Features are processed, such as multi-channel encoding, HOA channel encoding, object metadata encoding, etc.

The inter-channel group downmix module introduces a three-channel sum-difference coding method and combines it with a two-channel sum-difference coding framework. The module includes a decision module and a downmix module.

The decision module is used to determine which sum and difference coding method or combination to use. The decision criterion is the correlation between channels, which is compared with the group pair threshold. The decision result is that the L, R, and C channels are 3-channel sum and difference coded, LS and LS are 2-channel sum and difference coded, and the LFE channel is not processed.

The downmix module performs 3-channel sum and difference encoding on the L, R, and C channels, and 2-channel sum and difference encoding on LS and LS.

After the inter-channel group downmixing module, the three-channel sum difference coding downmixed channels (M1 channel, S11 channel, S12 channel), the two-channel sum difference coding downmixed channels (M2 channel, S2 channel) and the un-downmixed channels (LFE channel) are all passed through the bit allocation quantization entropy coding module, and the bit stream is multiplexed to form the coded bit stream E.

In the embodiments of the present disclosure, part or all of the steps and their optional implementations may be arbitrarily combined with part or all of the steps in other embodiments, or may be arbitrarily combined with optional implementations of other embodiments.

The embodiments of the present disclosure also propose a device for implementing any of the above methods, for example, a device is proposed, the above device includes a unit or module for implementing each step performed by the encoder in any of the above methods. For another example, another device is also proposed, including a unit or module for implementing each step performed by the decoder in any of the above methods.

It should be understood that the division of the units or modules in the above device is only a division of logical functions, which can be fully or partially integrated into one physical entity or physically separated in actual implementation. In addition, the units or modules in the device can be implemented in the form of a processor calling software: for example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or implement the functions of the units or modules of the above device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory inside the device or a memory outside the device. Alternatively, the units or modules in the device may be implemented in the form of hardware circuits, and the functions of some or all of the units or modules may be implemented by designing the hardware circuits. The hardware circuits may be understood as one or more processors; for example, in one implementation, the hardware circuits are application-specific integrated circuits (ASICs), and the functions of some or all of the above units or modules may be implemented by designing the logical relationship of the components in the circuits; for another example, in another implementation, the hardware circuits may be implemented by programmable logic devices (PLDs), and Field Programmable Gate Arrays (FPGAs) may be used as an example, which may include a large number of logic gate circuits, and the connection relationship between the logic gate circuits may be configured by configuring the configuration files, thereby implementing the functions of some or all of the above units or modules. All units or modules of the above devices may be implemented in the form of software called by the processor, or in the form of hardware circuits, or in the form of software called by the processor, and the remaining part may be implemented in the form of hardware circuits.

In the embodiments of the present disclosure, the processor is a circuit with signal processing capability. In one implementation, the processor may be a circuit with instruction reading and execution capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which may be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor may implement certain functions through the logical relationship of hardware circuits, and the logical relationship of the above hardware circuits may be fixed or reconfigurable, such as a processor being an application-specific integrated circuit (ASIC). A hardware circuit implemented by an ASIC (Integrated Circuit) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

FIG7A is a schematic diagram of the structure of the encoding and decoding device proposed in an embodiment of the present disclosure. As shown in FIG7A, the encoding and decoding device 7100 may include: at least one of a transceiver module 7101, a processing module 7102, etc. In some embodiments, the processing module 7102 is used to group multiple channels to obtain at least one channel group, wherein there are N first channel groups in the at least one channel group, and the first channel group includes three channels, and there are M second channel groups in the at least one channel group, and the second channel group includes two channels, wherein N is 1 and M is a non-negative integer. Optionally, the above-mentioned transceiver module 7101 is used to perform at least one of the communication steps such as sending and/or receiving performed by the encoder in any of the above methods (for example, step S2101 but not limited thereto), which will not be repeated here. Optionally, the above-mentioned processing module is used to perform at least one of the other steps performed by the encoder in any of the above methods, which will not be repeated here.

Optionally, the processing module 7102 is used to execute at least one of the communication steps such as processing performed by the encoder in any of the above methods, which will not be repeated here.

FIG7B is a schematic diagram of the structure of the coding and decoding device proposed in the embodiment of the present disclosure. As shown in FIG7B , the coding and decoding device 7200 may include: at least one of a transceiver module 7201 and a processing module 7202. In some embodiments, the processing module 7202 is used to decode the first information to obtain channel information of at least one channel grouping, wherein there are N first channel groups in the at least one channel grouping, and the first channel grouping includes three channels, and there are M second channel groups in the at least one channel grouping, and the second channel grouping includes two channels, wherein N is 1 and M is a non-negative integer. Optionally, the above-mentioned transceiver module is used to execute at least one of the communication steps such as sending and/or receiving (such as step S2102 but not limited thereto) executed by the decoder in any of the above methods, which will not be repeated here.

Optionally, the processing module 7202 is used to execute at least one of the communication steps such as processing performed by the decoder in any of the above methods, which will not be repeated here.

In some embodiments, the transceiver module may include a sending module and/or a receiving module, and the sending module and the receiving module may be separate or integrated. Optionally, the transceiver module may be interchangeable with the transceiver.

In some embodiments, the processing module can be a module or include multiple submodules. Optionally, the multiple submodules respectively execute all or part of the steps required to be executed by the processing module. Optionally, the processing module can be replaced with the processor.

FIG8A is a schematic diagram of the structure of a communication device 8100 proposed in an embodiment of the present disclosure. The communication device 8100 may be a decoder (e.g., an access network device, a core network device, etc.), or an encoder (e.g., a user device, etc.), or a chip, a chip system, or a processor that supports a decoder to implement any of the above methods, or a chip, a chip system, or a processor that supports an encoder to implement any of the above methods. The communication device 8100 may be used to implement the method described in the above method embodiment, and the details may refer to the description in the above method embodiment.

As shown in FIG8A , the communication device 8100 includes one or more processors 8101. The processor 8101 may be a general purpose processor or a processor. The communication device 8100 may be a dedicated processor, such as a baseband processor or a central processing unit. The baseband processor may be used to process the communication protocol and the communication data, and the central processing unit may be used to control the coding and decoding device (such as a base station, a baseband chip, an encoder device, an encoder device chip, a DU or a CU, etc.), execute a program, and process the data of the program. The communication device 8100 is used to execute any of the above methods.

In some embodiments, the communication device 8100 further includes one or more memories 8102 for storing instructions. Optionally, all or part of the memory 8102 may also be outside the communication device 8100.

In some embodiments, the communication device 8100 further includes one or more transceivers 8103. When the communication device 8100 includes one or more transceivers 8103, the transceiver 8103 performs at least one of the communication steps such as sending and/or receiving in the above method (for example, step S2101, step S2102, step S2103, step S2104, but not limited thereto).

In some embodiments, the transceiver may include a receiver and/or a transmitter, and the receiver and the transmitter may be separate or integrated. Optionally, the terms such as transceiver, transceiver unit, transceiver, transceiver circuit, etc. may be replaced with each other, the terms such as transmitter, transmission unit, transmitter, transmission circuit, etc. may be replaced with each other, and the terms such as receiver, receiving unit, receiver, receiving circuit, etc. may be replaced with each other.

In some embodiments, the communication device 8100 may include one or more interface circuits 8104. Optionally, the interface circuit 8104 is connected to the memory 8102, and the interface circuit 8104 may be used to receive signals from the memory 8102 or other devices, and may be used to send signals to the memory 8102 or other devices. For example, the interface circuit 8104 may read instructions stored in the memory 8102 and send the instructions to the processor 8101.

The communication device 8100 described in the above embodiments may be a decoder or an encoder, but the scope of the communication device 8100 described in the present disclosure is not limited thereto, and the structure of the communication device 8100 may not be limited by FIG. 8A. The communication device may be an independent device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data and programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, an encoder device, an intelligent encoder device, a cellular phone, a wireless device, a handheld device, a mobile unit, a vehicle-mounted device, a decoder, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

8B is a schematic diagram of the structure of a chip 8200 provided in an embodiment of the present disclosure. In the case where the communication device 8100 may be a chip or a chip system, reference may be made to the schematic diagram of the structure of the chip 8200 shown in FIG8B , but the present disclosure is not limited thereto.

The chip 8200 includes one or more processors 8201, and the chip 8200 is used to execute any of the above methods.

In some embodiments, the chip 8200 further includes one or more interface circuits 8202. Optionally, the interface circuit 8202 is connected to the memory 8203. The interface circuit 8202 can be used to receive signals from the memory 8203 or other devices, and the interface circuit 8202 can be used to send signals to the memory 8203 or other devices. For example, the interface circuit 8202 can read instructions stored in the memory 8203 and send the instructions to the processor 8201.

In some embodiments, the interface circuit 8202 performs at least one of the communication steps such as sending and/or receiving in the above method, and the processor 8201 performs at least one of the other steps.

In some embodiments, terms such as interface circuit, interface, transceiver pin, and transceiver may be used interchangeably.

In some embodiments, the chip 8200 further includes one or more memories 8203 for storing instructions. Optionally, all or part of the memory 8203 may be outside the chip 8200.

The present disclosure also proposes a storage medium, on which instructions are stored, and when the instructions are executed on the communication device 8100, the communication device 8100 executes any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited to this, and it can also be a storage medium readable by other devices. Optionally, the storage medium can be a non-transitory storage medium, but is not limited to this, and it can also be a temporary storage medium.

The present disclosure also proposes a program product, which, when executed by the communication device 8100, enables the communication device 8100 to execute any of the above methods. Optionally, the program product is a computer program product.

The present disclosure also proposes a computer program, which, when executed on a computer, causes the computer to execute any one of the above methods.

Claims (21)

A grouping method, characterized in that the method is performed by an encoder, and the method comprises: A plurality of channels are grouped to obtain at least one channel group, wherein the at least one channel group includes N first channel groups, the first channel group includes three channels, and the at least one channel group includes M second channel groups, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer. The method according to claim 1, characterized in that the step of grouping the plurality of channels to obtain at least one channel group comprises: Obtaining a similarity between any two channels of the multiple channels; The plurality of channels are grouped based on the similarity between any two channels to obtain the at least one channel group. The method according to claim 2, characterized in that the grouping of the multiple channels based on the similarity between any two channels to obtain the at least one channel group comprises: For any two channels among the multiple channels, determine the two channels with the greatest similarity as a candidate channel group; Excluding the other channels except the two channels with the greatest similarity, the two channels with the greatest similarity are determined as a candidate channel group until one independent channel remains or no channel remains; The at least one channel grouping is determined based on the obtained candidate channel groupings and/or the independent channels. The method according to claim 3, characterized in that the determining the at least one channel grouping based on the obtained candidate channel groupings and/or the independent channels comprises: Obtaining a similarity between the independent channel and each channel in each candidate channel group; When the similarity between the independent channel and each channel in the first candidate channel group is greater than a similarity threshold, determining the independent channel and each channel in the first candidate channel group as a first channel group; Each of the remaining candidate channel groups is determined as a second channel group. The method according to claim 1, characterized in that the step of grouping the plurality of channels to obtain at least one channel group comprises: The multiple channels are grouped to directly obtain the N first channel groups and the M second channel groups. The method according to claim 5, characterized in that the grouping of the multiple channels to directly obtain the N first channel groups and the M second channel groups comprises: By performing a global search on the multiple channels, the multiple channels are grouped according to the similarity between any two channels to obtain N first channel groups and the M second channel groups. The method according to claim 1, characterized in that the step of grouping the plurality of channels to obtain at least one channel group comprises: Grouping the multiple channels to obtain multiple candidate channel groups including two channels and at least one independent channel; An independent channel among the at least one independent channel is divided into one candidate channel group among the plurality of candidate channel groups to obtain the N first channel groups and the M second channel groups. The method according to any one of claims 1 to 7, characterized in that the method further comprises: Downmixing the audio signal in each of the channel groups to obtain a downmixed audio signal; The downmixed audio signal is encoded to obtain an audio stream. The method according to any one of claims 1 to 8, characterized in that the method further comprises: First information is sent, where the first information is used to indicate channel information of the plurality of channel groups. The method according to claim 9, wherein the channel information comprises at least one of the following: The number of channel groups including two channels; channel identifiers of channels included in a channel group including two channels; An energy parameter of a channel grouping comprising two channels; The number of channel groups including three channels; channel identifiers of channels included in a channel group including three channels; An energy parameter of a channel grouping comprising three channels; The energy parameter is used to adjust the energy of the channels in the channel group. A grouping method, characterized in that the method is performed by a decoder, and the method comprises: The first information is decoded to obtain channel information of at least one channel grouping, where there are N first channel groups in the at least one channel grouping, the first channel grouping includes three channels, and there are M second channel groups in the at least one channel grouping, the second channel grouping includes two channels, where N is 1 and M is a non-negative integer. The method according to claim 11, characterized in that the channel information includes at least one of the following: The number of channel groups including two channels; channel identifiers of channels included in a channel group including two channels; An energy parameter of a channel grouping comprising two channels; The number of channel groups including three channels; channel identifiers of channels included in a channel group including three channels; An energy parameter of a channel grouping comprising three channels; The energy parameter is used to adjust the energy of the channels in the channel group. The method according to claim 11 or 12, characterized in that the method further comprises: When it is determined that the channel groups do not include a channel grouping for three channels, two-channel upmixing is performed on the audio stream. The method according to claim 11 or 12, characterized in that the method further comprises: When it is determined that the channel groups include a channel grouping with three channels, three-channel upmixing is performed on the audio stream. A grouping method, characterized in that the method comprises: The encoder groups the multiple channels to obtain at least one channel group, wherein the at least one channel group includes N first channel groups, the first channel group includes three channels, and the at least one channel group includes M second channel groups, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer; The decoder decodes the first information to obtain channel information of the at least one channel grouping, wherein at least one channel grouping among the multiple channel groups includes three channels. A coding and decoding device, characterized in that the coding and decoding device comprises: A processing module is used to group multiple channels to obtain at least one channel group, wherein there are N first channel groups in the at least one channel group, the first channel group includes three channels, and there are M second channel groups in the at least one channel group, the second channel group includes two channels, wherein N is 1 and M is a non-negative integer. A coding and decoding device, characterized in that the coding and decoding device comprises: A processing module is used to decode the first information to obtain channel information of at least one channel grouping, where there are N first channel groups in the at least one channel grouping, the first channel grouping includes three channels, and there are M second channel groups in the at least one channel grouping, the second channel grouping includes two channels, wherein N is 1 and M is a non-negative integer. A coding and decoding device, characterized in that the coding and decoding device comprises: one or more processors; The processor is used to execute the grouping method according to any one of claims 1 to 10. A coding and decoding device, characterized in that the coding and decoding device comprises: one or more processors; The processor is used to execute the grouping method described in any one of claims 11 to 14. A coding and decoding system, characterized in that it comprises an encoder and a decoder, wherein the encoder is configured to implement the grouping method described in any one of claims 1 to 10, and the decoder is configured to implement the grouping method described in any one of claims 11 to 14. A storage medium storing instructions, characterized in that when the instructions are executed on a communication device, the communication device executes the grouping method as described in any one of claims 1 to 10, or executes the grouping method as described in any one of claims 11 to 14.