A SIMPLE AND COMUTATIONALLY EFFICIENT ALGORITHM

FOR REAL-TIME BLIND SOURCE SEPARATION OF SPEECH

MIXTURES

Tarig Ballal, Nedelko Grbic and Abbas Mohammed

Department of Signal Processing, Blekinge Institute of Technology, 372 25 Ronneby, Sweden

Keywords: BSS, blind source separation, speech enhancement, speech analysis, speech synthesis.

Abstract: In this paper we exploit the amplitude diversity provided by two sensors to achieve blind separation of two

speech sources. We propose a simple and highly computationally efficient method for separating sources

that are W-disjoint orthogonal (W-DO), that are sources whose time-frequency representations are disjoint

sets. The Degenerate Unmixing and Estimation Technique (DUET), a powerful and efficient method that

exploits the W-disjoint orthogonality property, requires extensive computations for maximum likehood pa-

rameter learning. Our proposed method avoids all the computations required for parameters estimation by

assuming that the sources are "cross high-low diverse (CH-LD)", an assumption that is explained later and

that can be satisfied exploiting the sensors settings/directions. With this assumption and the W-disjoint or-

thogonality property, two binary time-frequency masks that can extract the original sources from one of the

two mixtures, can be constructed directly from the amplitude ratios of the time-frequency points of the two

mixtures. The method works very well when tested with both artificial and real mixtures. Its performance is

comparable to DUET, and it requires only 2% of the computations required by the DUET method. More-

over, it is free of convergence problems that lead to poor SIR ratios in the first parts of the signals. As with

all binary masking approaches, the method suffers from artifacts that appear in the output signals.

1 GENERAL INFORMATION

Blind source separation (BSS) consists of recovering

unobserved signals or “sources” from several ob-

served mixtures (Cardoso, 1998). Several algorithms

based on different source assumptions have been pro-

posed. Some common assumptions are that the

sources are statistically independent (Bell et al.,

1995), are statistically orthogonal (Weinstein et al.,

1993), are non-stationary (Parra et al., 2000), or can

be generated by finite dimensional model spaces

(Broman et al., 1999).

The Degenerate Unmixing and Estimation Tech-

nique (DUET) algorithm (Jourjine et al., 2000)

(Rickard et al., 2001) (Yilmaz, et al., 2004) and other

proposed methods (Bofill et al., 2000) exploit the

approximate W-disjoint orthogonality property of

speech signals to perform source separation. Two

signals are said to be W-disjoint orthogonal (W-DO)

when their time-frequency representations, are dis-

joint sets (Jourjine et al., 2000) (Rickard et al., 2001).

DUET uses an online algorithm to perform gradient

search for the mixing parameters, and simultaneously

construct binary time-frequency masks that are used

to partition one of the mixtures to recover the original

source signals. DUET was proved to be powerful in

speech source separation. Additionally, it is proved to

be more computationally efficient as compared to

other existing methods (Rickard et al., 2001). How-

ever, the idea that separation of W-DO sources re-

quires only classifying the time-frequency points of

mixtures has motivated us to look for a simpler ap-

proach. In other words, for W-DO sources the source

separation problem is as simple as classifying the

time-frequency points of a mixtures as belonging to

one source or another.

In this paper we propose a simple and highly com-

putationally efficient approach to achieve the above-

mentioned classification. For this purpose we have

introduced an additional assumption, that is the

sources are "cross high-low diverse (CH-LD)". In a

system with two sensors, two sources are said to be

CH-LD, if the two sources are not both close to the

same sensor. A source is close to a sensor, if its energy

105

Ballal T., Grbic N. and Mohammed A. (2006).

A SIMPLE AND COMUTATIONALLY EFFICIENT ALGORITHM FOR REAL-TIME BLIND SOURCE SEPARATION OF SPEECH MIXTURES.

In Proceedings of the International Conference on Signal Processing and Multimedia Applications, pages 105-109

DOI: 10.5220/0001571901050109

Copyright

c

SciTePress

at that sensor is higher than its energy at the other

sensor.

Obviously, such diversity can be obtained from the

spatial domain. To provide such diversity, sensors

settings/directions can be exploited. A real case that

supports our assumption is the case of two micro-

phones with two speakers each associated with one of

the microphones. If the distance between the micro-

phones is relatively large as compared to that between

the speakers and the microphones, we will get two

mixtures of two sources (near speaker and interfer-

ence from far speaker) that exactly satisfy the re-

quired assumption.

With this new assumption and the W-disjoint or-

thogonality property, two binary time-frequency

masks that can extract the original sources from any

of the two mixtures, can be constructed directly from

the amplitude ratios of the time-frequency points of

the two mixtures. The organization of this paper is as

follows. In section 2 we define the source assump-

tions. In section 3 we derive a simple signal model

based on the source assumptions. In section 4 the pro-

posed algorithm is presented. In section 5, we discuss

the results obtained from practical tests. Finally, a

summary of this paper is given in section 6.

2 SOURCE ASSUMPTIONS

There are two basic assumptions required by our pro-

posed approach:

• First: the sources should be W-

disjoint orthogonal (W-DO).

• Second: the sources should be cross

high-low diverse (CH-LD).

The first assumption requires that the time-

frequency representations of the source signals con-

tained in a mixture should be disjoint (or non-

overlapping). This condition generated a concept,

which is referred to as the W-disjoint orthogonality

(Jourjine et al., 2000) (Rickard et al., 2001). For W-

disjoint orthogonal (W-DO) sources, only one source

should be active in each time-frequency point of the

time-frequency representation of the sources.

Given a windowing function

)(tW , two signals

)(ts

i

and )(ts

j

are said to be W-disjoint orthogonal

(W-DO) if the supports of the short-time Fourier

transforms (STFTs) of

)(ts

i

and )(ts

j

are disjoint

(Jourjine et al., 2000) (Rickard et al., 2001).

The STFT of

)(ts

j

is defined as (Allen et al., 1977)

dtetwtsS

ti

jj

ω

ττω

−

∞

∞−

−=

∫

)()(),(

(1)

The support of

),(

τ

ω

j

S is denoted as the set of the

),(

τ

ω

pairs for which

0),( ≠

τ

ω

j

S

.

Since the W-disjoint orthogonality assumption is

not exactly satisfied for many categories of signals,

the concept of approximate W-disjoint orthogonality

introduced in (Rickard et al., 2001) provides a practi-

cal version for the basic assumption. Approximate W-

disjoint orthogonality assumes that at each point of

the time-frequency representation of a mixture, the

power of, at most, one source signal will be dominant.

In other words, the assumption that a non-active

source contributes zero energy is replaced by assum-

ing that it contributes relatively low energy as com-

pared to the dominant source. Additionally, if the

source signals have sparse representations in the time-

frequency domain, the W-disjoint orthogonality can

be sufficiently satisfied as for speech signals (Araki et

al., 2004).

The second assumption requires that at least one of

the two sources has two different (one high and one

relatively low) amplitudes in the two mixtures, and

the two sources are not both high (or both low) in the

same mixture. To illustrate this assumption, let us

assume a simple instantaneous mixing model with

two mixtures of two sources:

2211111

sasax

+

=

(2)

2221122

sasax

+

=

(3)

Taking the STFT for both (2) and (3) yields

),(),(),(

2211111

τ

ω

τ

ω

τ

ω

SaSaX +

=

(4)

),(),(),(

2221122

τ

ω

τ

ω

τ

ω

SaSaX +

=

(5)

The CH-LD is fully satisfied when one of the two

following statements is fulfilled:

1

),(

),(

1

),(

),(

222

221

112

111

≤>

τω

τω

τω

τω

Sa

Sa

and

Sa

Sa

(6)

or

1

),(

),(

1

),(

),(

112

111

222

221

≤>

τω

τω

τω

τω

Sa

Sa

and

Sa

Sa

(7)

Simplifying (6) and (7), the CH-LD can be fully

satisfied by satisfying either

SIGMAP 2006 - INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING AND MULTIMEDIA

APPLICATIONS

106

11

22

21

12

11

≤>

a

a

and

a

a

(8)

or

11

12

11

22

21

≤>

a

a

and

a

a

(9)

From (8) and (9), we deduce that the CH-LD de-

pends mainly on the sensor setting and not on the

source signals. For a typical interference cancellation

problem (8) and (9) are normally satisfied. For gen-

eral source separation problems a variety of sensor

settings that satisfies (8) and (9) do exist.

3 SIGNAL MODEL

Let's start with the mixing model described by (2),

(3), (4) and (5), respectively. First we try to introduce

the W-disjoint orthogonality assumption into the

model. Assuming that source

}2,1{, ∈ks

k

, is the

active source at time-frequency point

),(

τ

ω

, (4) and

(5) become

),(),(

11

τ

ω

τ

ω

kk

SaX = (10)

),(),(

22

τ

ω

τ

ω

kk

SaX = (11)

From (10), (11) and in order to separate the

sources, we need to determine which source is active

at each time-frequency point. In other words, we need

to determine the values of

k

that satisfy (10) and (11)

at each time-frequency point.

4 PROPOSED ALGORITHM

Our proposed algorithm constructs two binary time-

frequency masks,

}2,1{),,( =Φ i

i

τ

ω

, by testing

the ratio

),(

),(

2

1

τω

τω

X

X

for all time-frequency points.

The masks are constructed simply using

otherwise

i

X

X

if

i

0

}2,1{,1

),(

),(

1),(

2

1

∈>=Φ

τω

τω

τω

(12)

ijj

ij

≠

∈Φ−=Φ },2,1{,),(1),(

τ

ω

τ

ω

(13)

(12) and (13) stem directly from (6), (7), (10) and

(11).

The time-frequency representation of the original

sources can be obtained using

}2,1{),,(),(),(

1

∈

Φ

=

jXS

jj

τ

ω

τ

ω

τ

ω

(14)

),(

2

τ

ω

X

can be used instead of

),(

1

τ

ω

X

in

(14) and will yield a scaled and phase shifted version

of

),(

τ

ω

j

S providing a spatial diversity the can

further be exploited to improve the outputs. Finally,

the inverse transform is used to obtain the original

sources.

It is noticed that for instantaneous mixing, a simi-

lar algorithm can be derived without the CH-LD as-

sumption be satisfied. For an instantaneous mixing

model, the attenuations parameters are fixed leading

to ratios of absolute values in (6), (7), (8) and (9) that

are equal to constants (e.g.

j

c , }2,1{=j ). In this

case a mask can be constructed according to

otherwise

ic

X

X

if

ii

0

}2,1{,

),(

),(

1),(

2

1

∈==Φ

τω

τω

τω

(15)

For real mixing models with reverberations, the

ratios of absolute values in (6), (7), (8) and (9) will

take random values that cluster around some two val-

ues constituting two clusters corresponding to the two

sources. To be able to demix the sources, these clus-

ters should be separate and no intersection should

occur between them. If no inter-cluster intersection

takes place, two clusters corresponding to the two

CH-LD sources will be separated by a surface (imag-

ine a 3-D plot of the ratios over the

),(

τ

ω

plane) for

which the ratio equals unity. If the sources are not

CH-LD demixing the sources is still possible if we

find the separating surface, which is beyond the scope

of this paper. Finally, if reverberations cause inter-

cluster intersection, separating the sources using the

proposed method will not be possible in this case. But

generally, practical tests with real echoic mixtures

have proved the separation of sources even when re-

verberation is present.

5 RESULTS

The algorithm was implemented and tested using both

artificial mixtures and real mixtures. Up to 22 dB SIR

(signal to interference ratio) gain has been achieved

with instantaneous artificial mixtures, up to 5 dB with

echoic (i.e., containing reverberations in addition to

the main signals) real mixtures. The interference here

is the energy contribution from the other (undesired)

source that should ideally be completely masked. For

A SIMPLE AND COMUTATIONALLY EFFICIENT ALGORITHM FOR REAL-TIME BLIND SOURCE

SEPARATION OF SPEECH MIXTURES

107

the same mixtures the DUET showed approximately

the same performance. Block size and block overlap

were respectively 512 and 384 for the STFT. Fig. 1

shows two speech sources Separated from two echoic

real mixtures using our proposed method, and using

DUET. The mixtures are from an office room re-

cording done by Te-Won Lee

(http://inc2.ucsd.edu/~tewon/). Two Speakers have

been recorded speaking simultaneously. Speaker 1

says the digits from one to ten in English and speaker

2 counts at the same time the digits in Spanish (uno

dos, etc.). The recording was done in a normal office

room. The distance between the speakers and the mi-

crophones was about 60cm in a square ordering. The

figure illustrates the efficiency of our proposed

method.

Since the proposed algorithm does not require any

parameter learning, convergence problems are

avoided. This explains the improved SIR in the first

few milliseconds as compared to that of our imple-

mentation of the DUET method and as reflected in

Fig. 1.

We noticed that, when white noise is present, the

DUET normally fails. Our proposed algorithm was

seen to withstand white noise. The algorithm has been

verified to work even with low input signal to noise

ratios, but still the noise remains in the outputs. Ad-

dressing this problem and generalizing the method for

cases with more than two sources are two important

future research goals.

As with all binary masking approaches, an impor-

tant drawback that should also be addressed by future

research is the presence of artifacts in the form of

distortions. Using continuous masks instead of binary

masks is supposed to solve the problem. Araki et al.

(Araki et al., 2004) has addressed the artifacts prob-

lem associated with the DUET method and were able

to reduce the artifacts by combing DUET with ICA

(Independent Component Analysis). Combining our

method with ICA in a similar way can be proposed as

a solution to the associated artifacts problem.

6 CONCLUSIONS

In this paper we proposed a new method for blind

source separation of W-disjoint orthogonal sources

using time-frequency masks. Our focus was on sepa-

rating two sources from two mixtures. We also intro-

duced the cross high-low diversity assumption, an

assumption that can be satisfied exploiting the sensors

setting/directions. The method uses the amplitude

ratios of the time-frequency representations of two

mixtures to directly construct binary time-frequency

masks to separate the sources. The method has shown

performance that is comparable to that of the DUET

method despite using only 2% of the computations

required by DUET. Moreover, it is free of conver-

gence problems. As with all binary masking ap-

proaches, the method suffers from artifacts that appear

in the output signals.

(a)

(b)

Figure 1: Separation of two speech sources from two echoic

real mixtures. The figure shows the spectrograms of the

separated sources: a) using our proposed method and b)

using DUET. Sources have different permutation in each

case. While DUET does not have a specific assumption

about source permutation, our proposed method assumes

that source 1 is the one that has its higher energy at sensor 1,

and source 2 is the one that has its higher energy at sensor 2.

As appears, there are no significant differences between the

separated sources in fig. (a) and those in fig. (b). This illus-

trates the major advantage of our proposed method; that is

despite using only 2% of the computations required by

DUET, it can achieve results that are comparable to those

achieved by DUET.

SIGMAP 2006 - INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING AND MULTIMEDIA

APPLICATIONS

108

REFERENCES

Cardoso, J.-F., 1998. Blind Signal Separation: Statistical

Principles. In Proceedings of the IEEE, vol. 86, no. 10,

pp. 2009–2025.

Bell, A.J. and Sejnowski, T.J., 1995. An Information

Maximization Approach to Blind Separation and Blind

Deconvolution. Neural Computation, pp. 1129-1 159.

Weinstein, E., Feder, M. and Oppenheim, A., 1993 Mul-

tichannel Signal Separation by Decorrelation. IEEE

Transaction on Speech and Audio Processing, vol. 1,

no. 4, pp. 405-413.

Parra, L. and Spence, C., 2000. Convolutive Blind Source

Separation Based On Multiple Decorrelations. IEEE

Transactions on Speech and Audio Processing, March

2000.

Broman, H., Lindgren, U., Sahlin, H and Stoica, P, 1999.

Source Separation: A TITO System Identification Ap-

proach. Signal Processing, vol. 73, pp. 169-183.

Jourjine, A., Rickard, S. and Yilmaz, O., 2000. Blind Sepa-

ration of Disjoint Orthogonal Signals: Demixing N

Sources from 2 Mixtures. Proceedings of the 2000

IEEE International Conference on Acoustics, Speech,

and Signal Processing, Istanbul, Turkey.

Rickard, S., Balan, R. and Rosca, J., 2001. Real- Time

Time-Frequency Based Blind Source Separation. Pro-

ceedings of the International Workshop of Independent

Component Analysis and Blind Source Separation, San

Diego, CA.

Yilmaz, O Rickard, S, July 2004. Blind Separation of

Speech Mixtures via Time-Frequency Masking. IEEE

Transactions on Signal Processing, Vol. 52.

Bofill, P. and Zibulevsky, M., 2000. Blind Separation of

More Sources than Mixtures Using Sparsity of Their

Short-time Fourier Transform. International Workshop

on Independent Component Analysis and Blind Signal

Separation, Helsinki, Finland.

Allen, Jont B., June1977. Short Term Spectral Analysis,

Synthesis and Modification by Discrete Fourier Trans-

form. IEEE Transactions on Acoustics, Speech, and

Signal Processing, vol. 25.

Araki, S., Makino, S., Sawada H. and Mukai, R., Sept.

2004. Underdetermined Blind Separation of Convolut-

ive Mixtures of Speech with Directivity Pattern Based

Mask and ICA. ICA2004 (Fifth International Confer-

ence on Independent Component Analysis and Blind

Signal Separation), pp. 898-905.

http://inc2.ucsd.edu/~tewon/

A SIMPLE AND COMUTATIONALLY EFFICIENT ALGORITHM FOR REAL-TIME BLIND SOURCE

SEPARATION OF SPEECH MIXTURES

109