KR100932790B1 - Multitrack Downmixing Device Using Correlation Between Sound Sources and Its Method - Google Patents

Abstract

The present invention relates to a multitrack downmixing apparatus using the cross-correlation between sound sources and a method thereof, and to downmixing individual sound source signals into a multitrack signal having a high cross-correlation value by using cross-correlation between the respective sound sources. An apparatus and method for multitrack downmixing using cross-correlation between sound sources, which can more faithfully recover an individual sound source from a mixed multitrack signal, are provided.

To this end, the present invention provides a multitrack downmixing apparatus comprising: signal conversion means for converting an individual sound source signal into a frequency band; Spatial information calculating means for calculating spatial information between respective sound sources from the converted individual sound source signals; Mixing combination determination means for determining mixing combination information according to a cross-correlation value using the calculated spatial information between the respective sound sources; And multitrack downmixing means for downmixing the converted individual sound source signals into multitrack signals according to the determined mixing combination information.

Cross-correlation, Multitrack, Individual Source Signal, Spatial Information, Mixing Combination, Closed Loop, Mixing Matrix Information, Multitrack Downmixing

Description

Multitrack downmixing apparatus and its method using cross-correlation between sound sources {APPARATUS AND METHOD OF MULTI-TRACK DOWN-MIXING USING CROSS CORRELATION BETWEEN VOICE SOURCE}

The present invention relates to a multitrack downmixing apparatus using the cross-correlation between sound sources and a method thereof, and more particularly, to downlink an individual sound source signal into a multitrack signal having a high cross-correlation value using cross-correlation between each sound source. The present invention relates to a multitrack downmixing apparatus using the cross-correlation between sound sources and a method thereof, which can faithfully recover individual sound sources from downmixed multitrack signals by mixing.

The present invention is derived from the research conducted as part of the IT growth engine technology development project of the Ministry of Information and Communication and the Ministry of Information and Telecommunications Research and Development. [Task Management Number: 2007-S-004-01] Development].

Once complete, the current acoustic content cannot be interacted with except for very limited adjustments such as volume control and timbre conversion. This adjustment refers to the adjustment of the overall signal, and adjustments to the individual sound sources contained in the sound content are currently impossible.

Recently, an interactive music service has been introduced that allows the user to make various adjustments (e.g. selection and level adjustments) of each individual sound source by processing the individual instrument and singer's voice as separate individual sound sources. . However, the interactive music service has a problem in that a large amount of transmission and storage device is required because the separate sound sources must be processed separately.

In order to solve this problem, the conventional multi-channel audio coding technique called "MPEG Surround" and the spatial audio object coding (SAOC) technique, which is an object-based audio encoding technique, are compressed using a mono or multichannel downmix. Do this.

Looking specifically at conventional audio downmixing / upmixing techniques, such conventional audio downmixing techniques can be used to determine the factors of quantity (e.g., level differences between sheep, delay between sheep, cross correlation between sheep, etc.). Analyze multichannel or multiple object sources. In addition, the conventional downmixing technique downmixes a multichannel or a plurality of object sources into one signal based on the analysis result, and transmits them together with a positive factor.

In a conventional audio upmixing technique, the user terminal restores the original multichannel or object sound source from the downmixed signal using the transmitted amount factor.

In short, the conventional audio downmixing / upmixing technique mixes a plurality of object sources into one downmix signal, and then reconstructs them using a positive factor. Here, the amount shift factor is calculated for each band divided by the nonlinear filter bank reflecting the auditory characteristics. De-correlation by cross-correlation from mixed signals in bands mitigates the converging of sound images.

However, such a conventional audio downmixing / upmixing technique has to perform a process of restoring the original sound from the downmix signal, and thus there is a problem in that independent service is not possible because the sound quality of the restored individual channel or sound source signal is degraded.

Therefore, the prior art as described above has a problem that the error occurs in the process of restoring the original sound from the downmix signal, the quality of the restored individual channel or sound source signal is degraded, so that independent service is not possible. It is a subject of the present invention.

Therefore, the present invention downmixes the individual sound source signals into multitrack signals having a high cross-correlation combination by using the cross-correlation between the respective sound sources, thereby more faithfully restoring the individual sound sources from the downmixed multitrack signals. An object of the present invention is to provide a multitrack downmixing apparatus using the cross-correlation between sound sources and a method thereof.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention which are not mentioned above can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

In order to solve the problem, the present invention is characterized by downmixing an individual sound source signal into a multitrack signal having a combination having a high cross-correlation value by using cross-correlation between each sound source.

More specifically, the present invention provides a multitrack downmixing apparatus comprising: signal conversion means for converting an individual sound source signal into a frequency band; Spatial information calculating means for calculating spatial information between respective sound sources from the converted individual sound source signals; Mixing combination determination means for determining mixing combination information according to a cross-correlation value using the calculated spatial information between the respective sound sources; And multitrack downmixing means for downmixing the converted individual sound source signals into multitrack signals according to the determined mixing combination information.

Meanwhile, the present invention provides a multitrack downmixing method, comprising: a signal conversion step of converting an individual sound source signal into a frequency band; Calculating spatial information between the respective sound sources from the converted individual sound source signals; A mixing combination determination step of determining mixing combination information according to a cross-correlation value using the calculated spatial information between the respective sound sources; And a multitrack downmixing step of downmixing the converted individual sound source signals into a multitrack signal according to the determined mixing combination information.

The present invention as described above, by downmixing the individual sound source signal to a multitrack signal having a combination of high correlation value using the cross-correlation between each sound source, it is possible to faithfully recover the individual sound source from the downmixed multitrack signal It can be effective. That is, according to the present invention, the mixed sound sources can be more faithfully separated by downmixing a plurality of sound sources having a high cross-correlation value to several tracks than downmixing a plurality of sound sources into one signal.

Therefore, the present invention has the effect of reducing the mixing of other sound sources as much as possible when removing or reproducing any sound source by user control as much as possible, and reducing the channel capacity than when transmitting individual sound sources separately.

The above objects, features, and advantages will become more apparent from the detailed description given hereinafter with reference to the accompanying drawings, and accordingly, those skilled in the art to which the present invention pertains may share the technical idea of the present invention. It will be easy to implement. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an embodiment of a multitrack downmixing apparatus using cross-correlation between sound sources according to the present invention.

As shown in FIG. 1, the multitrack downmixing apparatus 100 according to the present invention includes a signal converter 110, a spatial information calculator 120, a mixing combination determiner 130, and a multitrack downmixer ( 140). Here, the signal converter 110 includes a plurality of filter banks 111.

The multitrack downmixing apparatus 100 according to the present invention receives a plurality of individual sound source signals and performs multitrack downmixing using cross-correlation between each sound source signal for each frequency band. This is to perform multitrack audio compression. Here, the individual sound source is defined as a basic unit for generating sound, and may be used as an object sound source in an interactive music service. The individual sound source signal may include one micro-recorded sound signal as needed. In addition, the individual sound source signal may include a sound signal of a mono or stereo channel generated by mixing some of the individual sound source signals.

In detail, the signal converter 110 receives a plurality of individual sound source signals (for example, a sound source signal 1, a sound source signal 2, and a sound source signal n). A plurality of individual sound source signals are signal converted in the filter bank 111 corresponding to each of the individual sound sources. That is, the plurality of filter banks 111 converts the respective individual sound source signals to the plurality of frequency bands. At this time, the individual sound source signal is converted into a frequency band for the corresponding bandwidth among the entire frequency bandwidth (for example, 20 kHz) divided into a plurality of subbands.

Each filter bank 111 corresponds to a plurality of individual sound source signals and performs a function of converting individual sound source signals. For example, the filter bank 111 receives the sound source 1 and converts it into a plurality of frequency band signals corresponding to each frequency bandwidth. That is, the filter bank 111 converts the plurality of subband signals corresponding to the sound source 1 in the entire frequency band (for example, 20 kHz).

The spatial information calculator 120 analyzes a plurality of frequency band signals converted by the signal converter 110 and calculates spatial information of each sound source included in the corresponding bandwidth. The spatial information includes level difference values and cross-correlation values between sound sources. That is, the spatial information calculator 120 calculates the level difference value and the cross-correlation value between sound sources as the spatial information of each sound source unit. In this case, the spatial information calculator 120 obtains all of the cross-correlation values for the two channels, and determines the final cross-correlation value as the average of the cross-correlation values when the sound source of the stereo channel.

Here, the cross-correlation value has a value between "0" and "1". If the cross-correlation value is close to "0", it can be seen that the cross-correlation between the two sound sources becomes small. In addition, when the cross-correlation value is close to "1", it can be seen that the cross-correlation between the two sound sources increases. Meanwhile, the level difference value between sound sources refers to a level difference value between sound sources capable of comparing a plurality of sound source signals included in a corresponding frequency band.

The mixing combination determination unit 130 determines the mixing combination information by selecting sound sources having high correlation with each other by using the spatial information calculated by the spatial information calculating unit 120, that is, the level difference value and the cross-correlation value between the sound sources. . The mixing combination information relates to sound sources to be mixed together among a plurality of sound sources. That is, the mixing combination determination unit 130 selects sound sources having high correlation. Here, the selection of the highly correlated sound sources as a mixing combination is for upmixing without error between the sound sources when upmixing the multitrack downmixed signal. In addition, this is to prevent the sound quality degradation of the sound source signal to upmix.

In this case, the mixing combination determination unit 130 groups the plurality of groups according to the magnitude of the cross-correlation value between the sound sources. Here, a plurality of groups means a multitrack. The mixing combination determination unit 130 determines the sound source configuration of the track to be downmixed, that is, the mixing combination, by grouping the sources having high correlation with each other in order.

Meanwhile, the mixing combination determiner 130 separately processes mono and stereo sound sources according to the channel of the input sound source. That is, the combination of the stereo sound sources is downmixed into the stereo channel, and the mono sound source is mixed into the mono or stereo channel by the mixing matrix. When mono sources are downmixed to stereo, panning information between the two channels is included in the mixing matrix.

The multitrack downmixer 140 receives the respective sound source signals converted by the signal converter 110 and according to the mixing combination information determined by the mixing combination determination unit 130 and additionally the mixing matrix information from the outside. Each sound source signal from the signal converter 110 is downmixed into a multitrack signal. Here, the mixing matrix information is additional information in the present invention and is predetermined for the mixing matrix in response to input by a user or in accordance with various standards.

The multitrack downmixer 140 stores or transmits the downmixed multitrack signal together with additional information (eg, level difference, cross-correlation value, mixing combination information, and mixing matrix information). The multitrack downmixing unit 140 may use a conventional compression codec as needed.

2 is a block diagram of an embodiment of a multitrack upmixing apparatus according to the present invention.

As shown in FIG. 2, the multitrack upmixing apparatus 200 according to the present invention includes a multitrack upmixing unit 210 and a signal converter 220.

The multitrack signal downmixed by the multitrack downmixing apparatus 100 is transmitted to the multitrack upmixing apparatus 200 through a medium. Here, the multitrack upmixing apparatus 200 may be included in the user terminal.

In detail, the multitrack upmix unit 210 receives the downmixed multitrack signal and additional information. The multitrack upmixing unit 210 upmixes the downmixed multitrack signal using the received additional information. That is, the multitrack upmixing unit 210 performs a multitrack upmixing process using the level difference between the sound sources, the mixing combination information, and the mixing matrix information included in the additional information.

The signal converter 220 converts the signal upmixed by the multitrack upmixer 210 and restores the original individual sound source signal (for example, sound source 1, sound source 2, to sound source n). At this time, the sound source signal of each band can be restored by applying a gain from the downmixing multitrack signal transmitted by the level difference between the sound sources.

Meanwhile, one track signal transmitted from the multitrack downmixing apparatus 100 is downmixed by being combined with a sound source signal having a large cross-correlation value in the downmixing apparatus 100. Therefore, the multitrack upmixing apparatus 200 can restore the sound source signal having a large cross-correlation value, that is, only gain of each sound source having high similarity between the sound sources.

In addition, the multitrack upmixing apparatus 200 may prevent the direction of the restored sound source from being changed by adjusting the cross-correlation according to the original cross-correlation value through the non-correlation process between the respective sound sources.

3 is a diagram for explaining a method of grouping according to a magnitude of cross-correlation value.

As described above, the spatial information calculator 120 calculates a cross-correlation value between each sound source. An example of the cross-correlation between each of the sound sources is shown in FIG.

As shown in Fig. 3, each sound source (sound source 1 to sound source 6) 301 to 306 is disposed on the circumference. The order of magnitude (eg, 1, 2, ..., 15) of the cross-correlation values between each sound source (sound sources 1 to 6) is displayed. Here, the order of the size of the cross-correlation value is not a corresponding value, but represents the order of the size of the cross-correlation value.

In the method of grouping the sound sources of the tracks to be downmixed according to the cross-correlation value, when the cross-correlation is connected in high order, the sound sources in which the closed loop is formed are downmixed to one track.

The grouping method ends the grouping when the cross-correlation value falls below a predetermined threshold (eg, 0.5). The remaining sound sources are combined in pairs with large cross-correlation values and downmixed into one track. In addition, the last remaining sound source is transmitted to one track without downmixing.

Through this grouping process, each sound source (sources 1 to 6) 301 to 306 is a first track composed of three sound sources, a second track composed of two sound sources, and a third track composed of one sound source. Are grouped together. In other words, it can be seen that three downmixing tracks have been created.

4 is a flowchart illustrating a multitrack downmixing method using cross-correlation between sound sources according to the present invention.

4 illustrates the processing procedure of the mixing combination determination unit 130 in detail.

First, the signal converter 110 receives a plurality of individual sound source signals (for example, sound source signal 1, sound source signal 2, to sound source signal n) and converts them into a plurality of frequency bands corresponding to the individual sound source signals (402). ). At this time, the individual sound source signal is converted into a frequency band for the corresponding bandwidth among the entire frequency bandwidth (for example, 20 kHz) divided into a plurality of subbands.

Thereafter, the spatial information calculator 120 calculates spatial information of each sound source included in the corresponding bandwidth of the plurality of frequency band signals converted by the signal converter 110 (404). The spatial information includes level difference values and cross-correlation values between sound sources. That is, the spatial information calculating unit 120 calculates the level difference and cross-correlation between sound sources as spatial information in units of sound sources.

Then, the mixing combination determination unit 130 receives the cross-correlation value of each sound source band of the spatial information calculated by the spatial information calculation unit 120, and selects a sound source pair having the largest cross-correlation value among the sound sources. Search (406).

The mixing combination determination unit 130 compares the cross-correlation value of the sound source pair having the largest cross-correlation value with a threshold value (for example, 0.5) to determine whether the largest cross-correlation value is less than the threshold value. (408).

As a result of the check 408, if the cross-correlation value is greater than or equal to the threshold value, the mixing combination determination unit 130 checks whether the connection between the sound sources having the largest cross-correlation value is connected to the closed loop (410). This is to check whether a sound source signal connected to one track has a closed loop shape when the sound source signals are connected in a high order to each other.

As a result of the check 410, if the connection between the sound sources having the largest cross-correlation value is already connected to the registered closed loop, the mixing combination determination unit 130 is already a full track and thus disconnects and transitions to the process of "406". Done. Thereafter, the mixing combination determination unit 130 performs a process of searching for the largest cross-correlation value among the remaining sound sources, and performing a "406" process. On the other hand, if the connection between the sound sources having the largest cross-correlation value is not connected to the registered closed loop, the mixing combination determination unit 130 is connected to the already registered track instead of being connected to the closed loop. Check if there is (412).

As a result of the check 412, if the connection between the sound source is connected to the registered track, the mixing combination determination unit 130 adds the current sound source to the configuration of the track (414). The mixing combination determination unit 130 checks whether the configuration of the added track forms a closed loop (418). On the other hand, if it is not connected to the registered track, the mixing combination determination unit 130 registers a new track (416), and performs again from the process "406".

As a result of the check 418, when the configuration of the added track forms a closed loop, the mixing combination determination unit 130 registers the closed loop (420) and performs the process again from the "406" process. On the other hand, if the configuration of the added track does not form a closed loop, the mixing combination determination unit 130 is to perform again from the "406" process. That is, if the mixing combination determination unit 130 is not connected to the closed loop and is not connected to the already registered track, the mixing combination determination unit 130 registers a new track, adds the current sound source to the track configuration, and then performs the process again from the "406" process. do.

On the other hand, if the check result 408, the largest cross-correlation value is less than the threshold value, the mixing combination determination unit 130 determines the mixing combination, and generates the mixing combination information according to the determined mixing combination (422).

Subsequently, the multitrack downmixer 140 receives the respective sound source signals converted by the signal converter 110 and according to the mixing combination information determined by the mixing combination determination unit 130 and additionally the mixing matrix information. Each of the converted sound source signals is downmixed into a multitrack signal (424).

On the other hand, the method of the present invention as described above can be written in a computer program. And the code and code segments constituting the program can be easily inferred by a computer programmer in the art. In addition, the written program is stored in a computer-readable recording medium (information storage medium), and read and executed by a computer to implement the method of the present invention. The recording medium may include any type of recording medium that can be read by a computer.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

1 is a configuration diagram of an embodiment of a multitrack downmixing apparatus using cross-correlation between sound sources according to the present invention;

2 is a block diagram of an embodiment of a multitrack upmixing apparatus according to the present invention;

3 is an explanatory diagram of a method of grouping according to a size of a cross-correlation value;

4 is a flowchart illustrating a multitrack downmixing method using cross-correlation between sound sources according to the present invention.

* Explanation of symbols for the main parts of the drawings

100: multitrack downmixing device 110: signal conversion unit

120: spatial information calculation unit 130: mixing combination determination unit

140: multitrack downmixing unit 200: multitrack upmixing device

210: multitrack upmixer 220: signal converter

Claims (10)

In a multitrack downmixing device, Signal conversion means for converting an individual sound source signal into a frequency band; Spatial information calculating means for calculating spatial information between respective sound sources from the converted individual sound source signals; Mixing combination determination means for determining mixing combination information according to a cross-correlation value using the calculated spatial information between the respective sound sources; And Multitrack downmixing means for downmixing the converted individual sound source signals into multitrack signals according to the determined mixing combination information Multitrack downmixing device using the cross-correlation between the sound source. The method of claim 1, The mixing combination determination means, Multitrack downmixing apparatus using the cross-correlation between the sound source, characterized in that for grouping the sound source having a high cross-correlation value by using the calculated spatial information between each sound source. The method of claim 2, The mixing combination determination means, And a plurality of sound sources that form a closed loop as sound sources grouped in one track by connecting the cross-correlation values in the order of high sound sources. The method of claim 3, wherein The mixing combination determination means, If the cross-correlation value is less than a predetermined threshold value, the multi-track downmixing apparatus using the cross-correlation between the sound source, characterized in that for combining the remaining ungrouped sound sources. The method according to any one of claims 1 to 4, The multitrack downmixing means, And downmixing the converted individual sound source signals into multitrack signals according to the determined mixing combination information and external mixing matrix information. The method according to any one of claims 1 to 4, The spatial information calculating means, When the converted individual sound source signal has a stereo channel, the multi-track downmixing apparatus using the sound cross-correlation, characterized in that for calculating the average cross-correlation value for the left / right channel. In the multitrack downmix method, A signal conversion step of converting the individual sound source signals into frequency bands; Calculating spatial information between the respective sound sources from the converted individual sound source signals; A mixing combination determination step of determining mixing combination information according to a cross-correlation value using the calculated spatial information between the respective sound sources; And A multitrack downmixing step of downmixing the converted individual sound source signals into a multitrack signal according to the determined mixing combination information Multitrack downmixing method using cross-correlation between sound sources comprising a. The method of claim 7, wherein The mixing combination determination step, A multitrack downmixing method using cross-correlation between sound sources, wherein sound sources having a high cross-correlation value are grouped using the calculated spatial information between each sound source. The method of claim 8, The mixing combination determination step, A method of multitrack downmixing using cross-correlation between sound sources, wherein the cross-correlation values are connected in order of sound sources to determine closed sound sources as sound sources grouped in one track. The method of claim 9, The mixing combination determination step, If the cross-correlation value is less than a predetermined threshold value, the rest of the ungrouped sound sources are combined.

Images