DYNAMIC COMPONENTS OF LINEAR STABLE MIXTURES FROM FRACTIONAL LOW ORDER MOMENTS Thomas Fabricius, Preben Kidmose and Lars Kai Hansen Technical University of Denmark Department for Mathematical Modelling Richard Petersens Plads bldg. 321 2800 Lyngby Denmark ABSTRACT The second moment based independent component analy- sis scheme of Molgedey and Schuster is generalized to frac- tional low order moments, relevant for linear mixtures of heavy tail stable processes. The Molgedey and Schuster al- gorithm stands out by allowing explicitly construction of the independent components. Surpricingly, this turns out to be possible also for decorrelation based on fractional low order moments. Keywords: Blind Source Separation (BSS), Dynamic Com- ponents, Independent Component Analysis (ICA), Stable process, Fractional lower order moments, Cocktail party prob- lem 1. INTRODUCTION Reconstruction of independent components from linear mix- tures is an important signal processing research area with numerous practical applications [1, 2]. Typically, indepen- dent component analysis (ICA) proceeds from non-linear transformations [3, 4] or from temporal correlations [5, 6]. The second moment based Molgedey-Schuster approach as- sumes that the independent sources have different autocor- relation functions [5, 7, 8]. The main virtue of the approach is that an explicit solution (reconstruction of sources and mixing matrix) is possible. In [8] we applied the Molgedey- Schuster algorithm to image mixtures and we proposed a minor modification of the algorithm that relieves a prob- lem of the original approach, namely that it occasionally produces complex mixing coefficients and source signals. Attias and Schreiner proposed a very rich ICA framework based on higher order statistics and decorrelation [9, 10], allowing for completely general and learnable source distri- This work is funded by the Danish Research Councils through the THOR Center for Neuroinformatics. butions, however at the price of significant additional com- putation. In this contribution we analyze dynamic decorrelation for heavy tail stable random signal, and we provide a gener- alization of the Molgedey-Schuster algorithm based on frac- tional low order moments rather than second moments. 2. STABLE DISTRIBUTIONS AND SOME PROPERTIES Random vectors following a stable distribution are stable with respect to addition, i.e., if and , are stable then so is the sum . A random vector follows a strictly symmetric stable distribution iff its characteristic function has the form [11] (1) where and is a finite Borel measure called the spectral measure, is the unit sphere and is the char- acteristic exponent. A random variable that is distributed by a symmetric stable distribution with characteristic expo- nent is denoted . In the class of stable dis- tributions we find the normal distribution and the so-called Cauchy distribution . In general the low or- der stable distributions have heavy tails, and are supposed to model, e.g., speech signals. Because of the heavy tail- s, higher order moments do not exists, hence, ICA methods based on second moments are not well-defined for these sig- nals. For multivariate stable variables the spectral measure carries information about the dependencies among the in- dividual components and this may be used for independent component analysis [12]. Here we show that independen- t dynamic components can be reconstructed explicitly a la Molgedey Schuster, even if the signals follow stable distri- butions.
4
Embed
DYNAMIC COMPONENTS OF LINEAR STABLE MIXTURES FROM ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DYNAMIC COMPONENTS OF LINEAR STABLE MIXTURES FROM FRACTIONAL LOWORDER MOMENTS
The secondmomentbasedindependentcomponentanaly-sisschemeof Molgedey andSchusteris generalizedto frac-tional low order moments,relevant for linear mixturesofheavy tail stableprocesses.TheMolgedey andSchusteral-gorithmstandsoutbyallowingexplicitly constructionof theindependentcomponents.Surpricingly, this turnsout to bepossiblealsofor decorrelationbasedonfractionallow ordermoments.
Reconstructionof independentcomponentsfrom linearmix-turesis an importantsignalprocessingresearchareawithnumerouspracticalapplications[1, 2]. Typically, indepen-dent componentanalysis(ICA) proceedsfrom non-lineartransformations[3, 4] or from temporalcorrelations[5, 6].ThesecondmomentbasedMolgedey-Schusterapproachas-sumesthat the independentsourceshave differentautocor-relationfunctions[5, 7, 8]. Themainvirtueof theapproachis that an explicit solution (reconstructionof sourcesandmixing matrix) is possible.In [8] weappliedtheMolgedey-Schusteralgorithm to imagemixturesand we proposedaminor modificationof the algorithm that relieves a prob-lem of the original approach,namely that it occasionallyproducescomplex mixing coefficientsand sourcesignals.Attias andSchreinerproposeda very rich ICA frameworkbasedon higher order statisticsand decorrelation[9, 10],allowing for completelygeneralandlearnablesourcedistri-
This work is fundedby the DanishResearchCouncils throughtheTHORCenterfor Neuroinformatics.
butions,however at thepriceof significantadditionalcom-putation.
In this contribution we analyzedynamicdecorrelationfor heavy tail stablerandomsignal,andweprovideagener-alizationof theMolgedey-Schusteralgorithmbasedonfrac-tional low ordermomentsratherthansecondmoments.
2. STABLE DISTRIBUTIONS AND SOMEPROPERTIES
Randomvectorsfollowing a stabledistribution are stablewith respectto addition,i.e., if
�and � , arestablethenso
is thesum ��� ��� � . A randomvectorfollowsa strictlysymmetricstabledistribution iff its characteristicfunctionhastheform [11]��� � � ��������� ������ � ����������� "!$## ��&% ##('*) �,+.-/�0� 1�2��32 � (1)
where547698
and ) �,+.-/� is a finite Borel measurecalledthespectralmeasure,
-is theunit sphereand
�is thechar-
acteristicexponent.A randomvariable: thatis distributedby a symmetricstabledistributionwith characteristicexpo-nent
�is denoted:<; -=�>-
. In the classof stabledis-tributions we find the normal distribution
� �?� and theso-calledCauchydistribution
� �A@ . In generalthelow or-der stabledistributionshave heavy tails, andaresupposedto model,e.g.,speechsignals. Becauseof the heavy tail-s,higherordermomentsdo not exists,hence,ICA methodsbasedonsecondmomentsarenotwell-definedfor thesesig-nals. For multivariatestablevariablesthespectralmeasurecarriesinformationaboutthe dependenciesamongthe in-dividual componentsandthis maybeusedfor independentcomponentanalysis[12]. Herewe show that independen-t dynamiccomponentscanbe reconstructedexplicitly a laMolgedey Schuster, even if thesignalsfollow stabledistri-butions.
Our derivation is basedon the generalizedcovariationbetweenrandomvariables: and B jointly
-=�>-asdefined
in [11] C :EDFB9G ' �IH !KJMLON 'QPSRUT ) �,+.-/� (2)
with thedefinition V N '�PWRXT �ZY VOY 'QPSR sign� V � . Notethat
C :5DUB9G ' � 1 is necessarybut not sufficient for : and B tobe independent.The covariationhasa convenientpseudo-linearity property. Let : R , :$[ , B R and B\[ be jointly
-/�-and B R , B\[ independentthenC ] R : R � ] [�:^[_DF` R B R � `a[�Bb[0G ' �] R C : R DUB R G ' ` 'QPSRR � ] [ C :^[_DUB R G ' ` 'QPSRR �] R C : R DUB\[aG ' ` 'QPSR[ � ] [ C :^[_DUB\[aG ' ` 'QPSR[ (3)
From the covariationwe canderive the covariation coeffi-cient cbdfe g � C :EDFBhG 'C BiDFBhG ' (4)
which hastheproperty[11]cbdfe g �kjml :nB N(o PSRUTXpj l BqB N(o PSRUT p (5)
for @hrts 2u� . By equation(5) wecanestimatethecovari-ationcoefficient from data.
3. DECORRELATION BASED ON FRACTIONALLOW ORDER MOMENTS
We arenow equippedto generalizetheMolgedey-Schusterapproach,andstartout by definingthe matrix of observedsignals
�with elements:$v e w ��x7y>z{U| RS} v e { - { e w 1 r�~ 2�� (6)
where } v e { arethemixing coefficientsforming themixingmatrix
�,- { e w is an instanceof the � w,� -/�- sourcesignal
written asanelementin thematrix � and ��� is thenumberof sources.Wenow assumethatthedifferentsourcesignalsareindependent,C - { e w D -S� e w��&� G ' � 1 �������� 4�6�� (7)
The covariationcoefficient between: v e w and : 8 e w��&� canbefoundusingequations(4) and(6)cMd>�>� �Ue d>��� ���_� � � x�y z{U| RW} v
e { - { e w D x7y z� | RO} 8 e � - � e w��W��� '� x7yz� | R } 8e ��-S� e w��W� D x7y>z� | R } 8 e ��-S� e w��&� � '
(8)
Finally, we usethat the sourcesareindependent,hence,e-quation(3) is valid, providingc d �=� � e d ��� ���_� � x7y>z{U| R x7y>z� | RW} v e { C - { e w D - � e w��W� G ' } N '�PWRXT8 e �x�y>z{U| R x7y>z� | RO} 8 e { C - { e w��W� D -S� e w��W� G ' } N '�PWRXT8 e �
(9)
Wedefinematrices�^� ��� with elements � � �v e 8 � c d �>� � e d ��� �¡�¢� ,£ � ��� with thenominatorof equation(8) aselements¤ � ���v e 8 � y z¥{U| R y z¥ � | R } ve { C - { e w D -S� e w��W� G ' } N 'QPSRUT8 e � (10)
anda diagonalmatrix ¦ with thedenominatorfrom equa-tion (8) aselements§ 8 e 8 � y z¥{U| R y z¥ � | R } 8
e { C - { e w��W� D -S� e w��W� G ' } N 'QPSRUT8 e � (11)
Assumingstationarity, ¦ is independentof � , sothat � � � � �£ � ��� ¦ PSR . Fromequation(10) it is seenthat£ � � � decom-
by solving theeigenvalueproblem, ¬ � ���°¯ � ¯²± (14)
if we identify
¯ � � and
± � ¨ � � � ¨ �® � PWR .In short,we have shown our main result, namelythat
it is possibleto recover the mixing matrix from fractionallow order moments. The sourcesignalsare subsequentlyestimatedby �E� � PSR � .
3.1. Sample estimates of Fractional Low Order Moments
Equation(5) providestheestimateof theelementsof �$� ���³ c d �>� � e d ��� ���_� � R� P � x � PSRaP�w | :$v e w : N(o PSRUT8 e w��W�R��x � PSRw | : 8 e w : N´o PWRXT8 e w (15)
¨ � � � ¨ �® � PSR comparedto theiraccuracy. In this paperwe will not addressthis further, butsuggestusing �ª�»@ .
4. EXPERIMENTAL RESULTS
We illustrate the performanceof the generalizationof theMolgedey-Schusteralgorithmby analyzingmixing of syn-theticdatafollowing astabledistribution,andby mixing oftwo speechsignals.
4.1. Synthetic data
We performexperimentson mixing of two signalsin twochannels.In thefirst wemix two independentunit-varianceGaussiansignalsby themixing matrix� �½¼ 1Q� ¾_¿�À¢ÀÁ1M� À¢À. �1Q� Ã� @ ÀÁ1M� ¿ � ¾¢ÄÆÅcontainingtwo non-orthogonalnormalizedrows.Thesourcesignalshave differenttemporaldependencies,oneis whitenoise,while the otheris low-passfiltered with a FIR-filterwith two coefficients Ç �È@ and Ç R � 1M� ¾
. We show inFigure1 scatterplotsof the“true signals”andtherecoveredsignalsfrom the estimatedmixing matrix estimatedfromthe conventionalMolgedey-Schusteralgorithm. Next wemix two similarstablesignalsdrawn from distributionswith� �É@ �(Ä , hencewith heavy tails, by the samemixing ma-trix. In Figure2 we show the scatterplotsof the true andestimatedsignals.
The two different scenarioslead to similar estimatedmixing matricesÊmËÍÌ�Î�Ï Ð Î Ñ0ÒÓÎ_Ï Ò"Ô�Õ"ÖÎ�Ï Ô�Õ_×aÔØÎ_Ï Ð Ù�Ð"ÖÆÚfÛ ÊmËÍÌ�Î�Ï Ð Ñ Ü ÖØÎ_Ï Ñ Ù�Î ÑÎ�Ï Ò�Ô Ö�ÔØÎ_Ï Ð�Î�Ò¢×5Údemonstratingthatit is possiblereliablyto reconstructinde-pendentstablesourcesignalsby thegeneralizedMolgedey-Schusterscheme,in caseswherethesecondmomentcorre-lationswould not convergefor largesamples.
4.2. Speech data
Finally, weconsidertheblind separationof amixtureof twospeechsignals.The signalsaretwo sourcesignalsusedin[13]. We usethesamemixing matrix asin theaboveexam-ple. In Figure3 we show thescatterplotof thetrueandes-timatedsignalsusingthe conventionalMolgedey-Schusteralgorithmandin Figure4 usingthenew schemeusing sE�@ � ¾ . The recoveredsignalsarevery similar, indicatingthattheconventionalalgorithmis robustto theunderlyingdistri-bution of thesourcesignals.We conjecturethatthis robust-nessis relatedto the fact that the conventionalMolgedey-Schusteralgorithmproceedsfrom the ratio of two secondordermoments,andthattheratiomight havebetterstatisti-cal propertiesthanthe two individual moments,this is thetopic for on-goingresearch.
5. CONCLUSION
Wehavederivedanew algorithmgeneralizingthatof Molge-dey andSchuster[5] basedonfractionallow ordermoments.We haveprovideda schemefor estimatingtherelevantmo-mentsfrom finite samplesanddemonstratedtheviability ofthenew algorithmin asyntheticdataset.However, wealsofound that the original algorithmis fairly robust to possi-bly diverging secondordermoments,andconjecturedthattheexplanationcouldberelatedto thefact thattheconven-tionalalgorithmis basedontheratiobetweentwo divergingterms,possiblyimproving thestatisticalproperties.
−10 −5 0 5 10−10
−5
0
5
10
Est
imat
ed S
ourc
e S
1
True Source S1
−10 −5 0 5 10−10
−5
0
5
10
Est
imat
ed S
ourc
e S
2
True Source S1
−10 −5 0 5 10−10
−5
0
5
10
Est
imat
ed S
ourc
e S
1
True Source S2
−10 −5 0 5 10−10
−5
0
5
10
Est
imat
ed S
ourc
e S
2
True Source S2
Fig. 1. Original Gaussiansource,i.e.� ��� signalsagainst
recoveredsourcesignalswhenusingtheoriginalMolgedeyandSchusteralgorithm(s3��� ). In this setup
� �Ý@ 1¢1_1¢1andoneof the sourceswaswhite, the otherfiltered by theFIR-filter in thetext. Thealgorithmwasrunat �ª�»@
−100 −50 0 50 100−100
−50
0
50
100
Est
imat
ed S
ourc
e S
1
True Source S1
−100 −50 0 50 100−100
−50
0
50
100
Est
imat
ed S
ourc
e S
2
True Source S1
−100 −50 0 50 100−100
−50
0
50
100
Est
imat
ed S
ourc
e S
1
True Source S2
−100 −50 0 50 100−100
−50
0
50
100
Est
imat
ed S
ourc
e S
2
True Source S2
Fig. 2. Samesetupasin Fig.1but now thesourcesignalsaregeneratedwith
[9] H. Attias andC.E.Schreiner, “Blind sourceseparationanddeconvolutionby dynamiccomponentanalysis,” Neural Net-works for Signal ProcessingVII: Proceedingsof the 1997IEEE Workshop,456-465(1997, pp.456–465,1997.