CN114073106B - Binaural beamforming microphone array - Google Patents

Abstract

A binaural beamformer comprising two beamforming filters may be communicatively coupled to the microphone array to generate two beamformed outputs, one for the left ear and the other for the right ear. The beamforming filters may be configured such that they are orthogonal to each other such that the white noise components in the binaural output are substantially uncorrelated and the desired signal components in the binaural output are highly correlated. Thus, the human auditory system may better separate the desired signal from white noise and may increase the intelligibility of the desired signal.

Description

Binaural beamforming microphone array

Technical Field

The present disclosure relates to microphone arrays, and in particular, to binaural beamforming microphone arrays.

Background Art

Microphone arrays have been used in a wide range of applications, including, for example, hearing aids, smart headsets, smart speakers, voice communications, automatic speech recognition (ASR), human-machine interfaces, etc. The performance of a microphone array depends largely on its ability to extract the signal of interest in a noisy and/or reverberant environment. As a result, many techniques have been developed to maximize the gain of the signal of interest and suppress the effects of noise, interference, and/or reflections. One such technique is called beamforming, which filters the received signal based on the spatial configuration of the signal source and the microphones to focus on sounds originating from a specific location. However, in practical situations, conventional beamformers with high gain lack the ability to handle noise amplification (e.g., amplification of white noise in a specific frequency range).

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example and not limitation in the figures of the accompanying drawings.

FIG. 1 is a simplified diagram illustrating an environment in which an example microphone array system may be configured to operate according to an embodiment of the present disclosure.

FIG. 2 is a simplified block diagram showing an example microphone array system according to an embodiment of the present disclosure.

3 is a graph illustrating different phase relationships between a signal of interest and a noise signal and the effect of such phase relationships on the ambiguity of the signal of interest.

4 is a simplified diagram illustrating an environment in which an example binaural beamformer may be configured to operate in accordance with an embodiment of the present disclosure.

5 is a flow chart illustrating a method that may be performed by an example binaural beamformer including two orthogonal beamforming filters.

6 is a line graph showing simulated output interaural coherence of an example binaural beamformer described herein and a conventional beamformer combining a desired signal and a white noise signal.

FIG. 7 is a block diagram illustrating an example computer system according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

FIG. 1 is a simplified block diagram showing an environment 100 in which a microphone array 102 can be configured for operation. The microphone array 102 can be associated with one or more applications, including, for example, hearing aids, smart headphones, smart speakers, voice communications, automatic speech recognition (ASR), human-machine interfaces, etc. The environment 100 can include multiple audio signal sources. These audio signals can include a signal of interest 104 (e.g., a speech signal), a noise signal 106 (e.g., diffuse noise), an interference signal 108, a white noise signal 110 (e.g., noise generated from the microphone array 102 itself), and/or similar signals. The microphone array 102 may include a plurality of (e.g., M) microphones (e.g., acoustic sensors) configured to operate in series. The microphones may be placed on a platform (e.g., a straight or curvilinear platform) to receive the signals 104, 106, 108, and/or 110 from their respective sources/positions. For example, the microphones may be arranged according to a specific geometric relationship to each other (e.g., along a line, on the same plane surface, spaced apart from each other at a specific distance in three-dimensional space, etc.). Each microphone in the microphone array 102 can capture a version of an audio signal originating from a source at a specific angle of incidence at a specific time relative to a reference point (e.g., a reference microphone position in the microphone array 102). The time of sound capture can be recorded to determine the time delay of each microphone relative to the reference point. The captured audio signal can be converted into one or more electronic signals for further processing.

The microphone array 102 may include or be communicatively coupled to a processing device, such as a digital signal processor (DSP) or a central processing unit (CPU). The processing device may be configured to process (e.g., filter) signals received from the microphone array 102 and generate an audio output 112 having certain properties (e.g., noise reduction, speech enhancement, sound source separation, dereverberation, etc.). For example, the processing device may be configured to filter the signals received via the microphone array 102 so that the signal of interest 104 may be extracted and/or enhanced, while other signals (e.g., signals 106, 108, and/or 110) may be suppressed to minimize their possible adverse effects on the signal of interest.

FIG. 2 is a simplified block diagram showing an example microphone array system 200 as described herein. As shown in FIG. 2 , the system 200 may include a microphone array 202, an analog-to-digital converter (ADC) 204, and a processing device 206. The microphone array 202 may include a plurality of microphones arranged to receive audio signals from different sources and/or at different angles. In an example, the position of the microphone may be specified relative to a coordinate system (x, y). The coordinate system may include an origin (O) to which the microphone position may be specified, wherein the origin may coincide with the position of one of the microphones. The angular position of the microphone may also be defined with reference to the coordinate system. The source signal may be a plane wave from a far field and propagate at the speed of sound (e.g., c=340 m/s) and impact the microphone array 202.

Each microphone in the microphone array 202 may receive a version of the source signal with a certain time delay and/or phase shift. The electronic components of the microphone may convert the received sound signal into an electronic signal that may be sent to the ADC 204. In an example embodiment, the ADC 204 may further convert the electronic signal into one or more digital signals.

The processing device 206 may include an input interface (not shown) to receive the digital signal generated by the ADC 204. The processing device 206 may further include a pre-processor 208 configured to prepare the digital signal for further processing. For example, the pre-processor 208 may include a hardware circuit and/or a software program to convert the digital signal into a frequency domain representation using, for example, a short-time Fourier transform or other suitable type of frequency domain transform technique.

The output of the preprocessor 208 may be further processed by the processing device 206, for example, via a beamformer 210. The beamformer 210 may be operable to apply one or more filters (e.g., spatial filters) to the received signals to achieve spatial selectivity of the signals. In one embodiment, the beamformer 210 may be configured to process the phase and/or amplitude of the captured signals so that signals at specific angles may experience constructive interference, while other signals may experience destructive interference. The processing of the beamformer 210 may result in the formation of a desired beam pattern (e.g., a directivity pattern) that may enhance audio signals from one or more specific directions. The ability of such a beam pattern to maximize the ratio of its sensitivity in the observation direction (e.g., the angle of incidence of the audio signal associated with the maximum sensitivity) to its average sensitivity in all directions may be quantified by one or more parameters, including, for example, a directivity factor (DF).

The processing device 206 may also include a post-processor 212 configured to transform the signal generated by the beamformer 210 into a suitable form for output. For example, the post-processor 212 may be operative to convert the estimate provided by the beamformer 210 for each frequency subband back into the time domain so that the output of the microphone array system 200 may be understandable to a hearing receiver.

The signal and/or filtering described herein can be understood from the following description. For a source signal of interest that is a plane wave from an azimuth angle θ, propagating at the speed of sound (e.g., c=340 m/s) in an anechoic acoustic environment and impinging on a microphone array (e.g., microphone array 202) comprising 2M omnidirectional microphones, the corresponding steering vector of length 2M can be represented as follows:

Wherein, J may represent an imaginary unit, i.e., J ² = -1, ω = 2πf may represent an angular frequency, f>0 is a temporal frequency, τ ₀ = δ/c may represent a delay between two adjacent sensors at an angle θ = 0, δ is an array element spacing, and the superscript ^T may represent a transposition operator. The wavelength of an acoustic wave may be represented by λ = c/f.

Based on the steering vector defined above, the frequency domain observation signal vector with a length of 2M can be expressed as

y(ω)＝[Y ₁ (ω) Y ₂ (ω) … Y _2M (ω)] ^T

=x(ω)+v(ω)

=d(ω, θ _s )X(ω)+v(ω),

Where _Ym (ω) may represent the mth microphone signal, x(ω)=d(ω, _θs )X(ω), X(ω) may represent the zero-mean source signal of interest (e.g., the desired signal), d(ω, _θs ) may represent the signal propagation vector (e.g., which may take the same form as the steering vector), and v(ω) may represent a zero-mean additive noise signal vector defined similarly to y(ω).

Based on the above, the 2M×2M covariance matrix of y(ω) can be derived as

where E[·] can represent mathematical expectation, and the superscript ^H can represent the conjugate-transposition operator. It can be expressed as the variance of X(ω), The variance matrix of v(ω) can be expressed as The variance of the noise V ₁ (ω) at the first sensor or microphone may be represented, and Γ _v (ω)=Φ _v (ω)/φ _V1 (ω) (e.g., by normalizing Φ _v (ω) with φ _V1 (ω)) may represent a pseudo-coherence matrix of the noise. The variance of the noise may be assumed to be the same between multiple sensors or microphones (e.g., between all sensors or microphones).

The sensor spacing δ described herein can be assumed to be smaller than the acoustic wavelength λ (e.g., δ << λ), where λ = c/f. This may mean that ωτ ₀ is smaller than 2π (e.g., ωτ ₀ << 2π) and the true acoustic pressure difference can be approximated by the finite difference of the microphone outputs. Furthermore, it can be assumed that the desired source signal will propagate from an angle θ = 0 (e.g., in an end-fire direction). Thus, y(ω) can be expressed as

y(ω)＝d(ω，0)X(ω)+v(ω)

And at endfire, the value of the beam pattern of the beamformer may be equal to 1 or have a maximum value.

In an example implementation of a beamformer filter, complex weights may be applied at the output of one or more microphones (eg, at each microphone) of the microphone array 102.

The weighted outputs are summed together to obtain an estimate of the source signal as follows:

Z(ω)＝h ^H (ω)y(ω)

=X(ω)h ^H (ω)d(ω,0)+h ^H (ω)v(ω)

Where Z(ω) may represent an estimate of the desired signal X(ω) and h(ω) may represent a spatial linear filter of length 2M comprising complex weights applied to the outputs of the microphones. The distortion-free constraint in the direction of the signal source may be calculated as:

h ^H (ω)d(ω,0)＝1,

And the directivity factor (DF) of the beamformer can be defined as:

in For i,j=1,2,...,2M,[Γ _d (ω)] _i,j may represent the pseudo-coherence matrix of spherical isotropic (eg, diffuse) noise and may be

It is derived as:

Based on the definitions and/or calculations shown above, by maximizing DF and considering the distortion-free constraints shown above, the beamformer (referred to as a super-directional beamformer) can be expressed as follows:

(For example, considering the array geometry described herein), the DF corresponding to such a beamformer may have a maximum value, which may be expressed as:

The example beamformers described herein are capable of generating beam patterns that are frequency invariant (e.g., due to an increase or maximization of DF). However, an increase in DF may result in greater noise amplification, such as amplification of white noise generated by hardware elements of microphones in the microphone array 102 (e.g., in the low frequency range). In order to reduce the adverse effects of noise amplification on the signal of interest, one may consider deploying a smaller number of microphones in the microphone array 102, normalizing the matrix Γ _d (ω), and/or designing the microphone array 102 with an extremely low self-noise level. However, these approaches may be costly and difficult to implement, or may negatively affect other aspects of the beamformer performance (e.g., resulting in a decrease in DF, a change in the shape of the beam pattern, and/or a greater frequency dependence of the beam pattern).

Embodiments of the present disclosure explore the influence of the perceived position and/or direction of an audio signal on the intelligibility of a signal in a human auditory system (e.g., at a frequency lower than, for example, 1kHz) to solve the noise amplification problem described herein. The perception of a speech signal in a human binaural auditory system can be classified as in-phase and out-of-phase, while the perception of a noise signal (e.g., a white noise signal) can be classified as in-phase, random phase, or out-of-phase. As cited herein, "in-phase" can mean that two signal streams arriving at a binaural receiver (e.g., a receiver having two receiving channels such as a pair of headphones, a person having two ears, etc.) have substantially the same phase (e.g., approximately the same phase). "Out-of-phase" can mean that the phases of the two signal streams arriving at the binaural receiver differ by approximately 180°. "Random phase" can mean that the phase relationship between the two signal streams arriving at the binaural receiver is random (e.g., the respective phases of the signal streams differ by a random amount).

FIG. 3 is a diagram showing different phase scenarios associated with a signal of interest (e.g., a speech signal) and a noise signal (e.g., white noise), and the effect of the interaural phase relationship on the localization of these signals. The left column shows that the phase relationship between binaural noise signal streams can be classified as in-phase, random phase, and out-of-phase. The top row shows that the phase relationship between binaural speech signal streams can be classified as in-phase and out-of-phase. The rest of FIG. 3 shows a combination of the phase relationship of the speech signal and the noise signal perceived by the binaural receiver when the signals coexist in the environment. For example, cell 302 depicts a scenario in which the speech stream and the white noise stream are both in-phase at the binaural receiver (e.g., as a result of monophonic beamforming), and cell 304 depicts a scenario in which the speech stream arriving at the binaural receiver is in-phase, while the noise stream arriving at the receiver has a random phase relationship.

The intelligibility of speech signals can vary based on the combination of the phase relationship of the speech signal and white noise. Table 1 below shows the intelligibility ranking based on the phase relationship between speech and noise, where the anti-phase and out-of-phase conditions correspond to higher levels of intelligibility, while the in-phase conditions correspond to lower levels of intelligibility.

Table 1 - Intelligibility ranking based on speech/noise phase relationship

Intelligibility voice noise Classification 1 Out of phase In phase Inverted 2 In phase Out of phase Inverted 3 In phase Random Phase Out of phase 4 Out of phase Random Phase Out of phase 5 In phase In phase In phase 6 Out of phase Out of phase In phase

When the speech signal and the noise are perceived as coming from the same direction (e.g., as in the case of being in phase), the human auditory system will have difficulty separating the speech from the noise, and the intelligibility of the speech signal will be affected. Therefore, binaural filtering such as binaural linear filtering can be performed in conjunction with beamforming (e.g., fixed beamforming) to generate binaural outputs (e.g., two output streams) having a phase relationship corresponding to the anti-phase or out-of-phase situation shown above. Each of the binaural outputs may include a signal component corresponding to a signal of interest (e.g., a speech signal) and a noise component corresponding to a noise signal (e.g., white noise). Filtering may be applied in such a way that the noise component of the output stream becomes uncorrelated (e.g., having a random phase relationship), while the signal component of the output stream remains correlated (e.g., being in phase with each other) and/or enhanced. Therefore, the desired signal and white noise may be perceived as coming from different directions and are better separated to improve intelligibility.

Fig. 4 is a simplified block diagram of a microphone array 402 configured to apply binaural filtering to improve the intelligibility of a desired signal in an environment 400. Environment 400 may be similar to the environment 100 depicted in Fig. 1, wherein respective sources of a signal 404 of interest and a white noise signal 410 coexist. Similar to the microphone array 102 of Fig. 1, microphone array 402 may include multiple (e.g., M) microphones (e.g., acoustic sensors) configured to operate in series . These microphones may be placed to capture different versions of a signal 404 of interest (e.g., source audio signal) from its position, for example, at different angles and/or at different times. The microphone may also capture one or more other audio signals (e.g., noise 406 and/or interference 408), the audio signal including white noise 410 generated by the electronic components of microphone array 402 itself.

The microphone array 402 may include or may be communicatively coupled to a processing device such as a digital signal processor (DSP) or a central processing unit (CPU). The processing device may be configured to apply binaural filtering to the signal of interest 404 and/or the white noise signal 410 and generate a plurality of outputs for the binaural receiver. For example, the processing device may apply a first beamformer filter _h1 to the signal of interest 404 and the white noise signal 410 to generate a first audio output stream. The processing device may also apply a second beamformer filter _h2 to the signal of interest 404 and the white noise signal 410 to generate a second audio output stream. Each of the first and second audio output streams may include a white noise component 412a and a desired signal component 412b. The white noise component 412a may correspond to the white noise signal 410 (e.g., a filtered version of the white noise signal), and the desired signal component 412b may correspond to the signal of interest 404 (e.g., a filtered version of the signal of interest). Filters _hi and _h2 may be designed to be orthogonal to each other so that the white noise component 412a becomes uncorrelated (e.g., has a random phase relationship or an interaural coherence (IC) of approximately zero) in the first and second audio output streams. Filters _hi and _h2 may also be configured in such a manner that the desired signal component 412b is in phase with each other (e.g., has an IC of approximately one) in the first and second audio output streams. Thus, a binaural receiver of the first and second audio outputs may perceive the signal of interest 404 and the white noise signal 410 as coming from different locations and/or directions, and thus may improve the intelligibility of the signal of interest.

In one embodiment, binaural linear filtering may be performed in conjunction with fixed beamforming. Two complex-valued linear filters (e.g., h ₁ (ω) and h ₂ (ω)) may be applied to the observed signal vector, such as y(ω) as described herein. The respective lengths of the filters may depend on the number of microphones included in the associated microphone array. For example, if the associated microphone array includes 2M microphones, the length of the filter may be 2M.

Two estimates (e.g., Z ₁ (ω) and Z ₂ (ω)) of a source signal (e.g., X(ω)) can be obtained in response to binaural filtering of the signal. The estimates can be expressed as

And the variance of _Zi (ω) can be expressed as

wherein Γ _v (ω), Φ _y (ω), Φ _v (ω), φ _X (ω), φ _V1 (ω) and d(ω,0) have the same meanings as described herein.

Based on the above, two distortion-free constraints can be determined as

And the input signal-to-noise ratio (SNR) and output SNR can be calculated as

and

In at least some scenarios (e.g., when h ₁ (ω) = i _i and h ₂ (ω) = i _j , where i _i and i _j are the i-th and j-th columns of the 2M×2M identity matrix I _2M , respectively), the binaural output SNR may be equal to the input SNR (e.g., oSNR[i _i (ω), i _j (ω)] = iSNR(ω)). Based on the input SNR and the output SNR, the binaural SNR gain may be determined, for example, as

Other metrics associated with binaural beamforming may also be determined, which may include, for example, binaural white noise gain (WNG) denoted as W[h ₁ (ω), h ₂ (ω)], binaural directivity factor (DF) denoted as D[h ₁ (ω), h ₂ (ω)], and binaural beam pattern denoted as |B[h ₁ (ω), h ₂ (ω), θ]| ^2. These metrics may be calculated as follows:

The meaning of Γ _d (ω) has been explained above.

The localization of binaural signals in the human auditory system can depend on another metric, which is referred to as the interaural coherence (IC) of the signal in this article. The value of IC (or the modulus of IC) can increase or decrease according to the correlation of the binaural signal. For example, when the two audio streams of the source signal are highly correlated (for example, when the two audio streams are in phase with each other, or when the human auditory system perceives the two audio streams as coming from a single signal source), the value of IC can reach a maximum value (for example, 1). When the two audio streams of the source signal are substantially unrelated (for example, when the two audio streams have a random phase relationship, or when the human auditory system perceives the two streams as coming from two independent sources), the value of IC can reach a minimum value (for example, 0). The value of IC can indicate other binaural clues (for example, interaural time difference (ITD), interaural level difference (ILD), the width of the sound field, etc.) that the brain is used to locate the sound, or can be related to other binaural clues that the brain is used to locate the sound. As the IC of the sound decreases, the ability of the brain to locate the sound may be reduced accordingly.

The effect of interaural coherence can be determined and/or understood as follows. Let A(ω) and B(ω) be two zero-mean complex-valued random variables. The coherence function (CF) between A(ω) and B(ω) can be defined as

Wherein the superscript * denotes a complex conjugate operator. The value of γ _AB (ω) may satisfy the following relationship: 0≤|γ _AB (ω)| ² ≤1. For one or more pairs (eg, for any pair) of microphones or sensors (i, j), the input IC of the noise may correspond to the CF between V _i (ω) and V _j (ω), as shown below.

The input IC of white noise, i.e., γ _w (ω), and the input IC of diffuse noise, i.e., γ _d (ω), can be given as follows

γ _w (ω)＝0

The output IC of the noise can be defined as the CF between the filtered noise in Z ₁ (ω) and Z ₂ (ω) as shown below.

In at least some scenarios (e.g., when _h1 (ω)= _ii and _h2 (ω)= _ij ), the input and output ICs may be equal, i.e., γ[ _i (ω), _ij (ω)]=γ[ _h1 (ω), _h2 (ω)]. The output ICs of white noise, i.e., _γw [ _h1 (ω), _h2 (ω)], and diffuse noise, i.e., _γd [ _h1 (ω), _h2 (ω)], may be determined as

and

When filters h ₁ (ω) and h ₂ (ω) are collinear, the following may hold:

in, It can be a complex-valued number, and |γ[h ₁ (ω), h ₂ (ω)]|, |γ _w [h ₁ (ω), h ₂ (ω)]|, and |γ _d [h ₁ (ω), h ₂ (ω)]| can all have values close to 1 (e.g., |γ[h ₁ (ω), h ₂ (ω)]|＝|γ _w [h ₁ (ω), h ₂ (ω)]|＝|γ _w [h ₁ (ω), h ₂ (ω)]|＝1). Therefore, not only the desired source signal will be perceived as coherent (e.g., completely coherent), other signals (e.g., noise) will also be perceived as coherent, and the combined signal (e.g., the desired source signal plus noise) may be perceived as coming from the same direction. As a result, it will be difficult for the human auditory system to separate the signals, and the intelligibility of the desired signal may be affected.

When filters _h1 (ω) and _h2 (ω) are orthogonal to each other (e.g., _h1H (ω) _h2 (ω)=0), the separation between the desired source signal and noise (e.g., white ^noise ) can be improved. The following explains how such orthogonal filters can be derived, and their effect on the separation between the desired signal and noise, as well as on the enhanced intelligibility of the desired signal.

The matrix Γ _d (ω) described herein may be symmetric and may be diagonalized as

U ^T (ω)Γ _d (ω)U(ω)=Λ(ω)

in

U(ω)＝[u ₁ (ω) u ₂ (ω) … u _2M (ω)]

It can be an orthogonal matrix that satisfies the following conditions

U ^T (ω) U (ω) = U (ω) U ^T (ω) = I _2M

and

Λ(ω)=diag[λ ₁ (ω), λ ₂ (ω),..., λ _2M (ω)]

Can be a diagonal matrix.

The orthogonal vectors u ₁ (ω), u ₂ (ω), ..., u _2M (ω) may be eigenvectors corresponding to eigenvalues λ ₁ (ω), λ ₂ (ω), ..., λ _2M (ω), respectively, of the matrix Γ _d (ω), where λ ₁ (ω) ≥ λ ₂ (ω) ≥ ... ≥ _{λ 2M} (ω) > 0. Thus, the orthogonal filter of the output IC that can maximize the diffuse noise described herein may be determined as

The first maximum mode of CF can be as follows:

With corresponding vectors q _{+, 1} (ω) and q _{-, 1} (ω), where

All M maximum modes of CF (from m=1, 2, ..., M) can satisfy the following conditions

with corresponding vectors q _{+, m} (ω)) and q _{-, m} (ω), where

and

Based on the above, the following may be true:

From two vector sets q _{+, m} (ω) and q _{-, m} (ω), with m = 1, 2, ..., M, two semi-orthogonal matrices of size 2M × M can be formed as:

Q ₊ (ω) = [q _{+, 1} (ω) q _{+, 2} (ω) … q _{+, M} (ω)],

Q _- (ω) = [q _{-, 1} (ω) q _{-, 2} (ω) ... q _{-, M} (ω)],

in

I _M is the M×M identity matrix.

The following may also be true:

in

Λ _- (ω)=diag[λ _{-, 1} (ω), λ _{-, 2} (ω),..., λ _{-, M} (ω)],

Λ ₊ (ω)=diag[λ _{+, 1} (ω), λ _{+, 2} (ω),..., λ _{+, M} (ω)],

are two diagonal matrices of size M×M, with diagonal elements λ _−,m (ω)=λ _m (ω)−λ _2M−m+1 (ω) and λ _+,m (ω)=λ _m (ω)+λ _2M−m+1 (ω).

Let N be a positive integer with size 2≤N≤M, and two semi-orthogonal matrices of size 2M×M can be defined as follows:

Q _+,:N (ω)

=[q _{+, 1} (ω) q _{+, 2} (ω) … q _{+, N} (ω)], Q _{-,: N} (ω)

=[q _{-, 1} (ω) q _{-, 2} (ω) … q _{-, N} (ω)]

In an example implementation, the orthogonal filters described herein may take the following form:

in

A common complex-valued filter of length N can be represented. For such an orthogonal filter, the output IC of the diffuse noise can be calculated as

in

Λ _{-, N} (ω) = diag [λ _{-, 1} (ω), λ _{-, 2} (ω), ..., λ _{-, N} (ω)]

Λ _{+, N} (ω) = diag [λ _{+, 1} (ω), λ _{+, 2} (ω), ..., λ _{+, N} (ω)]

and

Based on the above, the binaural WNG, DF, and power beam patterns can be determined as follows:

and

in

can be a matrix of size N × 2, and the distortion-free constraint can be

Where N≥2.

From the above, we can conclude that the variance of _Zi (ω) is:

where for φ _Z1 (ω), Q _±,:N (ω) = Q _+,:N (ω), and for φ _Z2 (ω), Q _±,:N (ω) = Q _−,:N (ω). In the case of diffuse plus white noise (e.g., Γ _d (ω) = Γ _d (ω) + I _2M ), the variance of _Zi (ω) can be simplified to

This indicates that φ _Z1 (ω) may be equal to φ _Z2 (ω) (eg, φ _Z1 (ω)=φ _Z2 (ω)).

Furthermore, the cross-correlation of the two estimates Z ₁ (ω) and Z ₂ (ω) can be determined as follows:

In the case of diffuse plus white noise (eg, Γ _d (ω) = Γ _d (ω) + I _2M ), the cross-correlation may become

This may not rely on white noise. For Γ _v (ω) = Γ _d (ω) + I _2M , the output IC of the estimated signal may be determined as

As can be seen from the above, in some scenarios (e.g., for large input SNRs), the localization clues of the estimated signal may (e.g., mainly) depend on the localization clues of the desired signal, while in other scenarios (e.g., for low SNRs), the localization clues of the estimated signal may (e.g., mainly) depend on the localization clues of the diffuse plus white noise. Therefore, a first binaural beamformer (e.g., a binaural super-directional beamformer) can be obtained by minimizing the sum of the filtered diffuse noise signals subject to the distortion-free constraint described herein. The summation can be performed, for example, in the following manner:

From this we can conclude that:

And the corresponding DF can be determined as:

Therefore, the first binaural beamformer can be represented by:

The second binaural beamformer (e.g., the second binaural super-directional beamformer) can be obtained by maximizing the DF described herein.

The DF shown above can be rewritten as:

in

C′(ω,0) ^C′H (ω,0) may represent an NxN Hermitian matrix and the rank of the matrix may be equal to 2. Since there are two constraints (e.g., distortion-free constraints) to satisfy, two eigenvectors may be considered, denoted as _t′1 (ω) and _t′2 (ω). These eigenvectors may correspond to two non-empty eigenvalues of the matrix C′(ω,0) ^C′H (ω,0), denoted as λt′1( _ω ) and _λt′2 (ω). Thus, a filter that maximizes the DF with two degrees of freedom as rewritten above (since two constraints are to be satisfied) may be as follows:

in

α′(ω)＝[α′ ₁ (ω) α′ ₂ (ω)] ^T ≠0

can be any complex-valued vector of length 2, and T′ _1:2 (ω) can be determined as:

T′ _1:2 (ω)=[t′ ₁ (ω) t′ ₂ (ω)]

Therefore, the filter that maximizes the above DF can be expressed as:

And the corresponding DF can be determined as:

Based on the above, we can conclude the following:

And the second binaural beamformer can be determined as:

By including two sub-beamforming filters in a binaural beamformer (e.g., one for each binaural channel) and making the filters orthogonal to each other, the IC of the white noise component in the binaural output of the beamformer can be reduced (e.g., minimized). In some embodiments, the IC of the diffuse noise component in the binaural output of the beamformer can also be increased (e.g., maximized). The signal components (e.g., the signal of interest) in the binaural output of the beamformer can be in phase while the white noise components in the output can have a random phase relationship. In this way, when receiving the binaural output from the beamformer, the human auditory system can better separate the signal of interest from the white noise and reduce the effect of white noise amplification.

5 is a flow chart illustrating a method 500 that may be performed by an example beamformer (e.g., beamformer 210 of FIG. 2 ) including two orthogonal filters. Method 500 may be performed by processing logic including hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions running on a processing device to perform hardware emulation), or a combination thereof.

In order to simplify the description, the method is depicted and described as a series of actions. However, the actions according to the present disclosure may occur in various orders and/or simultaneously, and with other actions not presented and described herein. In addition, all the actions shown may not be required to implement the method according to the disclosed subject matter. In addition, these methods can alternatively be represented as a series of interrelated states via state diagrams or events. In addition, it should be understood that the methods disclosed in this specification can be stored on an article of manufacture, so that such methods can be transmitted and transferred to a computing device. The term article of manufacture used herein is intended to cover a computer program accessible from any computer-readable device or storage medium.

5 , at 502 , method 500 may be performed by a processing device (e.g., processing device 206) associated with a microphone array (e.g., microphone array 102 in FIG. 1 , 202 in FIG. 2 , or 402 in FIG. 4 ). At 504 , the processing device may receive an audio input signal including a source audio signal (e.g., a signal of interest) and a noise signal (e.g., white noise). At 506 , the processing device may apply a first beamformer filter to the audio input signal including the signal of interest and the noise signal to generate a first audio output designated for a first channel receiver. The first audio output may include a first source signal component (e.g., representing the signal of interest) and a first noise component (e.g., representing white noise) characterized by respective first phases. At 508 , the processing device may apply a second beamformer filter to the audio input signal including the signal of interest and the noise signal to generate a second audio output designated for a second channel receiver. The second audio output may include a second source signal component (e.g., representing the signal of interest) and a second noise component (e.g., representing white noise) characterized by respective second phases. The first and second beamformer filters may be constructed in a manner such that the noise components of the two outputs are uncorrelated (e.g., have a random phase relationship) and the source signal components of the two outputs are correlated (e.g., in phase with each other). At 510, the first and second audio outputs may be provided to respective channel receivers or respective audio channels. For example, the first audio output may be provided to a first channel receiver (e.g., for a left ear), while the second audio output may be designated for a second channel receiver (e.g., for a right ear). The interaural coherence (IC) of the white noise component in the output may be minimized (e.g., having a value of approximately zero), while the interaural coherence (IC) of the signal component in the output may be maximized (e.g., having a value of approximately one).

FIG6 is a line graph comparing the simulated output IC of the example binaural beamformer described herein with the simulated output IC of the conventional beamformer in combination with the desired signal and white noise. The upper half of the graph shows that the output IC of the desired signal of both the binaural and conventional beamformers is equal to one, while the lower half of the graph shows that the output IC of the white noise of the binaural beamformer is equal to zero and the output IC of the white noise of the conventional beamformer is equal to one. This indicates that in the two output signals of the binaural beamformer, the signal component (e.g., the desired signal) is substantially correlated, while the white noise component is substantially uncorrelated. Thus, the output signals correspond to the out-of-phase situation discussed herein, in which the desired signal and the white noise are perceived as coming from two separate directions/positions in space.

The binaural beamformers described herein may also have one or more other desirable characteristics. For example, although the beam pattern generated by the binaural beamformer may vary depending on the number of microphones included in the microphone array associated with the beamformer, the beam pattern may be substantially invariant with respect to frequency (e.g., substantially frequency invariant). Furthermore, when compared to conventional beamformers of the same order (e.g., first order, second order, third order, and fourth order), the binaural beamformer may not only provide better separation between the desired signal and the white noise signal, but also produce a higher white noise gain (WNG).

FIG. 7 is a block diagram showing a machine in the example form of a computer system 700 according to an example embodiment, in which an instruction set or sequence of instructions can be executed to make the machine perform any of the methods discussed herein. In an alternative embodiment, the machine operates as a standalone device, or can be connected (e.g., networked) to other machines. In a networked deployment, the machine can operate as a server or client machine in a server-client network environment, or can act as a peer machine in a peer (or distributed) network environment. The machine can be an in-vehicle system, a wearable device, a personal computer (PC), a tablet PC, a hybrid tablet, a personal digital assistant (PDA), a mobile phone, or any machine capable of (sequentially or otherwise) executing instructions specifying the actions to be taken by the machine. In addition, although only a single machine is shown, the term "machine" should also be understood to include any collection of machines that individually or collectively execute a set (or multiple sets) of instructions to perform any one or more methods discussed herein. Similarly, the term "processor-based system" should be considered to include any set of one or more machines controlled or operated by a processor (e.g., a computer) to individually or collectively execute instructions to perform any one or more methods discussed herein.

The example computer system 700 includes at least one processor 702 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both, a processor core, a computing node, etc.), a main memory 704, and a static memory 706, which communicate with each other via a link 708 (e.g., a bus). The computer system 700 may further include a video display unit 710, an alphanumeric input device 712 (e.g., a keyboard), and a user interface (UI) navigation device 714 (e.g., a mouse). In one embodiment, the video display unit 710, the input device 712, and the UI navigation device 714 are incorporated into a touch screen display. The computer system 700 may additionally include a storage device 716 (e.g., a drive unit), a signal generating device 718 (e.g., a speaker), a network interface device 720, and one or more sensors (not shown), such as a global positioning system (GPS) sensor, a compass, an accelerometer, a gyroscope, a magnetometer, or other sensors.

The storage device 716 includes a machine-readable medium 722 having stored thereon one or more sets of data structures and instructions 724 (e.g., software) that embody or are utilized by one or more of the methodologies or functions described herein. The instructions 724 may also reside, in whole or in part, within the main memory 704, the static memory 706, and/or the processor 702 during execution by the computer system 700, the main memory 704, the static memory 706, and the processor 702 also constituting machine-readable media.

Although the machine-readable medium 722 is shown as a single medium in the example embodiment, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) storing one or more instructions 724. The term "machine-readable medium" should also be considered to include any tangible medium that is capable of storing, encoding, or carrying instructions executed by a machine and causing the machine to perform any one or more methods of the present disclosure, or capable of storing, encoding, or carrying data structures utilized by or associated with such instructions. Thus, the term "machine-readable medium" should be considered to include, but is not limited to, solid-state memory and optical and magnetic media. Specific examples of machine-readable media include volatile or non-volatile memory, including, but not limited to, for example, semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

The instructions 724 may also be sent or received over a communication network 726 using a transmission medium via the network interface device 720 using any of a number of well-known transmission protocols (e.g., HTTP). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, a mobile phone network, a plain old telephone (POTS) network, and wireless data networks (e.g., Wi-Fi, 3G and 4G LTE/LTE-A or WiMAX networks). The term "transmission medium" shall be deemed to include any intangible medium that is capable of storing, encoding, or carrying instructions for execution by the machine, and includes digital or analog communications signals or other intangible media to facilitate communication of such software.

In the foregoing description, many details are set forth. However, it will be apparent to those of ordinary skill in the art who benefit from the present disclosure that the present disclosure may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form rather than in detail to avoid making the present disclosure unclear.

Certain portions of the detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the art of data processing to most effectively convey the substance of their work to others skilled in the art. Here, an algorithm is generally considered to be a self-consistent sequence of steps that produces a desired result. These steps are steps that require physical manipulation of physical quantities. Typically, although not necessarily, these quantities take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated. Mainly for general reasons, it has proven convenient at times to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, etc.

It should be kept in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise clearly indicated from the following discussion, it should be understood that throughout the description, discussions utilizing terms such as "segment," "analyze," "determine," "enable," "identify," "modify," etc., refer to actions and processes of a computer system or similar electronic computing device that manipulate and transform data represented as physical (e.g., electronic) quantities in the computer system's registers and memories into other data represented as physical quantities in the computer system's memories or other such information storage, transmission, or display devices.

The words "example" or "exemplary" are used herein to mean serving as an example, instance or illustration. Any aspect or design described herein as "example" or "exemplary" is not necessarily to be interpreted as being preferred or advantageous over other aspects or designs. On the contrary, the use of the words "example" or "exemplary" is intended to present concepts in a specific way. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clearly seen from the context, "X includes A or B" is intended to mean any natural inclusive arrangement. That is, if X includes A; X includes B; or X includes A and B, then "X includes A or B" is satisfied in any of the above cases. In addition, the articles "one" and "an" used in this application and the appended claims should generally be interpreted as meaning "one or more", unless otherwise specified or clearly pointed to the singular form from the context. In addition, unless so described, the use of the terms "embodiment" or "one embodiment" or "implementation" or "one implementation" throughout the text is not intended to represent the same embodiment or implementation.

References throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment. Thus, the phrases "in one embodiment" or "in an embodiment" appearing in various places throughout this specification are not necessarily all referring to the same embodiment. Additionally, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or".

It should be understood that the above description is intended to be illustrative rather than restrictive. After reading and understanding the above description, many other embodiments will be apparent to those skilled in the art. Therefore, the scope of the present disclosure should be determined with reference to the appended claims and the full scope of equivalents given by these claims.

Claims (22)

1. A method implemented by a processing device communicatively coupled to a microphone array comprising M microphones, where M is greater than 1, the method comprising:

receiving an audio input signal comprising a source audio signal and a noise signal from the microphone array;

filtering, by a processing device executing a first beamformer filter associated with the microphone array, the audio input signal to generate a first audio output signal designated for a first channel receiver, the first audio output signal including a first audio signal component corresponding to the source audio signal and a first noise component corresponding to the noise signal;

Filtering, by a processing device executing a second beamformer filter associated with the microphone array, the audio input signal to generate a second audio output signal designated for a second audio receiver, the second audio output signal including a second audio signal component corresponding to the source audio and a second noise component corresponding to the noise signal, wherein the filtering executed by the second beamformer filter is substantially orthogonal to the filtering executed by the first beamformer filter such that the first noise component is substantially uncorrelated with the second noise component; and

the first audio output signal is provided to the first channel receiver and the second audio output signal is provided to the second channel receiver.

2. The method of claim 1, wherein the first and second audio signal components are substantially in phase with each other, and wherein the first and second noise components have a random phase relationship with each other.

3. The method of claim 1, wherein an inter-ear coherence value between the first noise component and the second noise component has a value substantially equal to zero.

4. The method of claim 1, wherein an inter-aural coherence value between the first audio signal component and the second audio signal component is substantially equal to one.

5. The method of claim 1, wherein the first audio signal component is substantially correlated with the second audio signal component.

6. The method of claim 1, wherein an inner product of a first vector corresponding to the first beamformer filter and a second vector corresponding to the second beamformer filter is substantially equal to zero.

7. The method of claim 1, wherein providing the first audio output signal to the first channel receiver and providing the second audio output signal to the second channel receiver comprises: simultaneously providing the first audio output signal to the first channel receiver and the second audio output signal to the second channel receiver.

8. The method of claim 1, wherein the first channel receiver is configured to provide the first audio output to a left ear of a user and the second channel receiver is configured to provide the second audio output to a right ear of the user.

9. The method of claim 1, further comprising applying beamforming to the source audio signal to create a substantially frequency-invariant beam pattern.

10. The method of claim 1, wherein filtering performed by at least one of the first beamformer filter or the second beamformer filter maximizes a directivity factor associated with the microphone array without distortion constraints.

11. A microphone array system, comprising:

storing data; and

processing means communicatively coupled to the data store and M microphones of the microphone array, wherein M is greater than 1 to:

receiving an audio input signal comprising a source audio signal and a noise signal from the microphone array;

filtering the audio input signals by executing a first beamformer filter associated with the microphone array to generate a first audio output signal designated for a first channel receiver, the first audio output comprising a first audio signal component corresponding to the source audio signal and a first noise component corresponding to the noise signal;

filtering the audio input signals by performing a second beamformer filter associated with the microphone array to generate a second audio output designated for a second audio receiver, the second audio output signal comprising a second audio signal component corresponding to the source audio and a second noise component corresponding to the noise signal, wherein the filtering performed by the second beamformer filter is substantially orthogonal to the filtering performed by the first beamformer filter such that the first noise component is substantially uncorrelated with the second noise component; and

The first audio output signal is provided to the first channel receiver and the second audio output signal is provided to the second channel receiver.

12. The microphone array system of claim 11, wherein the first audio signal component and the second audio signal component are substantially in phase with each other, and wherein the first noise component and the second noise component have a random phase relationship with each other.

13. The microphone array system of claim 11, wherein an inter-ear coherence value between the first noise component and the second noise component has a value substantially equal to zero.

14. The microphone array system of claim 11 wherein an inter-ear coherence value between the first audio signal component and the second audio signal component is substantially equal to one.

15. The microphone array system of claim 11, wherein the first audio signal component is substantially correlated with the second audio signal component.

16. The microphone array system of claim 11, wherein an inner product of a first vector corresponding to the first beamformer filter and a second vector corresponding to the second beamformer filter is substantially equal to zero.

17. The microphone array system of claim 11, wherein to provide the first audio output signal to the first channel receiver and the second audio output signal to the second channel receiver, the processing means simultaneously provides the first audio output signal to the first channel receiver and the second audio output signal to the second channel receiver.

18. The microphone array system of claim 11, wherein the first channel receiver is configured to provide the first audio output to a left ear of a user and the second channel receiver is configured to provide the second audio output to a right ear of the user.

19. The microphone array system of claim 11 wherein the processing means is further configured to apply beamforming to the source audio signal to create a substantially frequency-invariant beam pattern.

20. The microphone array system of claim 11, wherein at least one of the first beamformer filter or the second beamformer filter executed by the processing device maximizes a directivity factor associated with the microphone array without distortion constraints.

21. A non-transitory machine-readable storage medium storing instructions that, when executed, cause a processing device to:

receiving an audio input signal comprising a source audio signal and a noise signal from a microphone array of M microphones, wherein M is greater than 1;

filtering the audio input signals by executing a first beamformer filter associated with the microphone array to generate a first audio output signal designated for a first channel receiver, the first audio output comprising a first audio signal component corresponding to the source audio signal and a first noise component corresponding to the noise signal;

filtering the audio input signals by performing a second beamformer filter associated with the microphone array to generate a second audio output signal designated for a second audio receiver, the second audio output signal comprising a second audio signal component corresponding to the source audio and a second noise component corresponding to the noise signal, wherein the filtering performed by the second beamformer filter is substantially orthogonal to the filtering performed by the first beamformer filter such that the first noise component is substantially uncorrelated with the second noise component; and

The first audio output is provided to the first channel receiver and the second audio output is provided to the second channel receiver.

22. The non-transitory machine-readable storage medium of claim 21, wherein the first audio signal component and the second audio signal component are substantially in phase with each other, and wherein the first noise component and the second noise component have a random phase relationship with each other.