BIOL 582 Lecture Set 22 One-Way MANOVA, Part II Post-hoc exercises Discriminant Analysis.

BIOL 582

Lecture Set 22One-Way MANOVA, Part II

Post-hoc exercisesDiscriminant Analysis

Example MANOVA: Bumpus Data

1. State Null/Alternative hypotheses2. Define model (evaluate assumptions)3. Calculate Sf for the error of the full model4. Define the reduced “null” model (always contains just an intercept)5. Calculate Sr for the error of the reduced model6. Calculate a multivariate test statistic Sf using and/or Sr . (Note: one can use

only Sf to get a test statistic if performing a randomization test.)7. Evaluate the probability of the test statistic if the null hypothesis were true

a. By converting the test statistic to an F stat, which approximately follows and F distribution

b. By performing a randomization test to create an empirical probability distribution

8. Generate plots/tables9. Maybe do a discriminant analysis

BIOL 582 Multivariate ANOVA

Example MANOVA: Bumpus Data

8. Generate plots/tables9. Maybe do a discriminant analysis

• These two are linked because we are in a position where we might want to do multiple comparisons. Therefore, what we plot or how we present a table might be contingent upon what a multiple comparisons test reveals

• Also, multiple comparisons can be done several ways

• Let’s start by looking at a PC plot of the data• (Because we used the original variables in the MANOVA, we must use a

covariance matrix)


Example MANOVA: Bumpus Data8. Generate plots/tables9. Maybe do a discriminant analysis


> Y.cov.pca<-princomp(Y,cor=F)> par(mfrow=c(1,3))> # PC 2 vs PC 1> plot(Y.cov.pca$scores[,1],Y.cov.pca$scores[,2],type='n',asp=1)> points(Y.cov.pca$scores[group=="m.TRUE",1],Y.cov.pca$scores[group=="m.TRUE",2],pch=22,bg='blue')> points(Y.cov.pca$scores[group=="m.FALSE",1],Y.cov.pca$scores[group=="m.FALSE",2],pch=22,col='blue')> points(Y.cov.pca$scores[group=="f.TRUE",1],Y.cov.pca$scores[group=="f.TRUE",2],pch=21,bg='red')> points(Y.cov.pca$scores[group=="f.FALSE",1],Y.cov.pca$scores[group=="f.FALSE",2],pch=21,col='red')> > # PC 3 vs PC 1> plot(Y.cov.pca$scores[,1],Y.cov.pca$scores[,3],type='n',asp=1)> points(Y.cov.pca$scores[group=="m.TRUE",1],Y.cov.pca$scores[group=="m.TRUE",3],pch=22,bg='blue')> points(Y.cov.pca$scores[group=="m.FALSE",1],Y.cov.pca$scores[group=="m.FALSE",3],pch=22,col='blue')> points(Y.cov.pca$scores[group=="f.TRUE",1],Y.cov.pca$scores[group=="f.TRUE",3],pch=21,bg='red')> points(Y.cov.pca$scores[group=="f.FALSE",1],Y.cov.pca$scores[group=="f.FALSE",3],pch=21,col='red')> > # PC 3 vs PC 2> plot(Y.cov.pca$scores[,2],Y.cov.pca$scores[,3],type='n',asp=1)> points(Y.cov.pca$scores[group=="m.TRUE",2],Y.cov.pca$scores[group=="m.TRUE",3],pch=22,bg='blue')> points(Y.cov.pca$scores[group=="m.FALSE",2],Y.cov.pca$scores[group=="m.FALSE",3],pch=22,col='blue')> points(Y.cov.pca$scores[group=="f.TRUE",2],Y.cov.pca$scores[group=="f.TRUE",3],pch=21,bg='red')> points(Y.cov.pca$scores[group=="f.FALSE",2],Y.cov.pca$scores[group=="f.FALSE",3],pch=21,col='red')



> # open circles = dead; closed = survived; blue = male; red = female



> # 3D scatterplot> library(scatterplot3d)> a<-scatterplot3d(Y.cov.pca$scores[,1],Y.cov.pca$scores[,2],Y.cov.pca$scores[,3],type='p',asp=1)> a$points(Y.cov.pca$scores[group=="m.TRUE",1],Y.cov.pca$scores[group=="m.TRUE",2],Y.cov.pca$scores[group=="m.TRUE",3],pch=22,bg='blue')> a$points(Y.cov.pca$scores[group=="m.FALSE",1],Y.cov.pca$scores[group=="m.FALSE",2],Y.cov.pca$scores[group=="m.FALSE",3],pch=22,col='blue')> a$points(Y.cov.pca$scores[group=="f.TRUE",1],Y.cov.pca$scores[group=="f.TRUE",2],Y.cov.pca$scores[group=="f.TRUE",3],pch=21,bg='red')> a$points(Y.cov.pca$scores[group=="f.FALSE",1],Y.cov.pca$scores[group=="f.FALSE",2],Y.cov.pca$scores[group=="f.FALSE",3],pch=21,col='red')


Have to go to R for this one….


> # Interactive 3D scatterplot> library(rgl)> col.ind<-as.numeric(group) # f.F = black, f.T = red, m.F = green, m.T =blue> plot3d(Y.cov.pca$scores[,1],Y.cov.pca$scores[,2],Y.cov.pca$scores[,3],col=col.ind)> # note that this distorts aspect ratio


• So it appears that males and females tend to differ in morphology, but survived versus dead birds, it’s a little hard to distinguish.

• What options are there for multiple comparisons1. Hotelling’s T 2 (inferential)2. Randomization Test3. Discriminant analysis (exploratory but multi-faceted)


Hotelling’s T 2 is basically a multivariate t-test between any two multivariate means

Where W is the error (within-group) covariance matrix, found as

Hotelling’s T 2 can be converted to an F value by





Hotelling’s T 2 is basically a multivariate t-test between any two multivariate means

Note, Hotelling’s T 2 is like a t stat that measures differences in means, which in multivariate practice is expressed as a squared distance

The square root, d, is the simplest way to express the difference in means

Example MANOVA: Hotelling T 2 tests


> # Hotelling's T2 comparisons> > m.means<-predict(lm.group, data.frame(group=levels(group))) # multivariate means> rownames(m.means)<-levels(group) # label means> m.means AE BHL FL TTL SW SKLfemale.FALSE 241.6500 31.38000 17.91716 28.55595 15.19936 20.71243female.TRUE 242.3793 31.53793 18.14699 28.92972 15.30394 21.05397male.FALSE 246.2273 31.59091 18.07095 28.71355 15.32948 21.40527male.TRUE 247.6744 31.67209 18.21771 28.89102 15.32329 21.74004

> mean.diffs<-dist(m.means) # all elements are differences in means> means.n<-by(AE,group,length) # the sample sizes for each mean> > g<-nrow(m.means) # number of group means> > # Create empty matrices for results> HT2.matrix<-F.matrix<-P.matrix<-matrix(0,g,g,dimnames=list(c(levels(group)),c(levels(group))))> W<-1/(lm.group$df.residual)*Sf # error covariance matrix> for(i in 1:g){+ y1<-m.means[i,];n1<-means.n[i] # by rows+ for(j in 1:g){+ y2<-m.means[j,];n2<-means.n[j] # by rows also+ T2<-n1*n2/(n1+n2)*t(y1-y2)%*%solve(W)%*%(y1-y2)+ F<-(n1+n2-ncol(Y)-1)/((n1+n2-lm.group$rank)*ncol(Y))*T2+ P<-1-pf(F,ncol(Y),(n1+n2-ncol(Y)-1))+ # fill in result matrices+ HT2.matrix[i,j]<-T2;F.matrix[i,j]<-F;P.matrix[i,j]<-P + }+ }> >

Example MANOVA: Hotelling T 2 tests


> > mean.diffs female.FALSE female.TRUE male.FALSEfemale.TRUE 0.9364565 male.FALSE 4.6412666 3.8712015 male.TRUE 6.1361939 5.3416972 1.5053122

> HT2.matrix female.FALSE female.TRUE male.FALSE male.TRUEfemale.FALSE 0.000000 2.965359 16.530255 28.514610female.TRUE 2.965359 0.000000 22.752539 34.003285male.FALSE 16.530255 22.752539 0.000000 4.481377male.TRUE 28.514610 34.003285 4.481377 0.000000

> F.matrix female.FALSE female.TRUE male.FALSE male.TRUEfemale.FALSE 0.0000000 0.4612781 2.6172904 4.5107858female.TRUE 0.4612781 0.0000000 3.6272163 5.4171901male.FALSE 2.6172904 3.6272163 0.0000000 0.7198999male.TRUE 4.5107858 5.4171901 0.7198999 0.0000000

> P.matrix female.FALSE female.TRUE male.FALSE male.TRUEfemale.FALSE 1.0000000000 0.8328550545 0.026042668 0.0008545144female.TRUE 0.8328550545 1.0000000000 0.003596766 0.0001367368male.FALSE 0.0260426678 0.0035967655 1.000000000 0.6347116030male.TRUE 0.0008545144 0.0001367368 0.634711603 1.0000000000

Note that females and males differ, but there are no differences in morphology between surviving and non-surviving birds

Example MANOVA: Randomization test


> # Randomization Test approach> > P<-matrix(1,g,g)> permute<-999> md.obs<-as.matrix(mean.diffs) # make sure R know this is a matrix> for(i in 1:permute){+ Y.r<-Y[sample(nrow(Y)),] # shuffle rows, not all values+ lm.group.r<-lm(Y.r~group)+ m.means.r<-predict(lm.group.r, data.frame(group=levels(group)))+ mean.diffs.r<-as.matrix(dist(m.means.r)) # random mean differences+ P<-ifelse(mean.diffs.r>=md.obs,P+1,P+0) # logical comparisons+ }> dimnames(P)<-dimnames(md.obs)> P.values<-P/(permute+1) # P-values>> mean.diffs female.FALSE female.TRUE male.FALSEfemale.TRUE 0.9364565 male.FALSE 4.6412666 3.8712015 male.TRUE 6.1361939 5.3416972 1.5053122

> P.values female.FALSE female.TRUE male.FALSE male.TRUEfemale.FALSE 1.000 0.620 0.003 0.001female.TRUE 0.620 1.000 0.003 0.001male.FALSE 0.003 0.003 1.000 0.216male.TRUE 0.001 0.001 0.216 1.000>

Note that females and males differ, but there are no differences in morphology between surviving and non-surviving birds




• Here is one way to present results (could do it the same way with Hotelling’s T 2


Table 1. Morphological distances (below diagonal) and corresponding P-values (above diagonal), based on 1,000 random permutations of shuffling subjects among the sex-survival groups. Bolded values represent sexual dimorphisms within survival type; italicized values are survival-type comparisons, within sex.

Female Malenon-survived survived non-survived survived

Femalenon-survived 0.620 0.003 0.001survived 0.936 0.003 0.001

Malenon-survived 4.641 3.871 0.216survived 6.136 5.342 1.505




• Discriminant analyses are useful, but often misused• Rather than ask if groups are different, one asks (1) if subjects can be

assigned to the correct group, and (2) which variables might be best for distinguishing group differences

• Also, a plot can be made to show group distinction (this is where misuse if often made… but not the only place)


Example MANOVA: Bumpus Data / Discriminant Analysis (DA)

Part 1: ClassificationWithout getting into too much detail, the classification part of DA measures the probability that any subject belongs to a certain group. This can be done several ways (and the way it is generally done is not the best). The typical way is to use the same data used to estimate group means. Then the generalized Mahalanobis distance is measured for every ith subject to the jth group mean. Based on these distances, the posterior classification probability of belonging to each group is determined (smaller distance = higher probability).

The prior classification probability is determined by group size (prior-probabilities = nj / Σnj)

Comparison of prior and posterior probabilities can help elucidate whether groups are clustered in the multivariate data space



Part 2: Variable loadingRecall that the multivariate equivalent to the F value was the F matrix:

A linear discriminant analysis (LDA) performs an eigenanalysis on this matrix. The eigenvectors are called canonical vectors. The variables that load highest on these vectors (or just the first vector) best characterize differences among groups. Projecting data onto these vectors produces a scatterplot (Canonical variate plot) that shows the best statistical discrimination of groups, if it exists. The plot does not show actual relationships of objects; rather it shows statistical relationships. If groups are distinct, the scatter will indicate it.

Sometimes LDA is called a canonical variates analysis (CVA), which sounds much like principal components analysis.



These are really superficial descriptions of DA. A better treatment of the subject requires a multivariate stats course. However, be aware that some who work with multivariate data use such an analysis as a post-hoc method for dealing with MANOVA results.


> #LDA Example> library(MASS)> > lda.group<-lda(group~Y,cv=True) # reverse of linear model formula> lda.fit<-predict(lda.group,group) # calculates post-probabilities and CVs> fit.table<-table(group,lda.fit$class) # a summary table of classifications> fit.table # rows are actual; columns are predicted group female.FALSE female.TRUE male.FALSE male.TRUE female.FALSE 3 7 8 2 female.TRUE 1 15 10 3 male.FALSE 2 7 17 18 male.TRUE 0 2 15 26> > # classification success> CS<-sum(diag(fit.table))/nrow(Y)*100> CS # very low classification....[1] 44.85294> > # However, looking at the table reveals that males tend to be classified as males> # which suggests males might be more distinct> # Even if there was a high classification success rate> # one has to use caution, because the same values were used to calculate> # group means (i.e., inherently better rate to expect than equal prior probability)


How does the classification work?


> lda.group$prior # what is expected by chancefemale.FALSE female.TRUE male.FALSE male.TRUE 0.1470588 0.2132353 0.3235294 0.3161765

> lda.fit$posterior[1:10,] # the first 10 posterior probabilities female.FALSE female.TRUE male.FALSE male.TRUE1 0.09162555 0.05118162 0.4930559 0.36413692 0.16343518 0.28885553 0.3121966 0.23551273 0.11406832 0.14269727 0.4342863 0.30894814 0.04710183 0.06797484 0.4262030 0.45872035 0.14647994 0.30878952 0.2968586 0.24787196 0.08462697 0.07996460 0.4483835 0.38702507 0.13939473 0.14526545 0.3868157 0.32852418 0.08688642 0.13789299 0.4070071 0.36821359 0.10429253 0.08706685 0.3712054 0.437435310 0.06296977 0.18031920 0.3780656 0.3786454>




> (lda.fit$posterior-lda.group$prior)[1:10,]

# look for highest positive difference in each row

female.FALSE female.TRUE male.FALSE male.TRUE1 -0.0554332753 -0.09587720 0.34599707 0.2170781172 -0.0498001173 0.07562023 0.09896126 0.0222774453 -0.2094610894 -0.18083214 0.11075691 -0.0145813324 -0.2690746455 -0.24820163 0.11002655 0.1425438495 -0.0005788862 0.16173070 0.14979977 0.1008131166 -0.1286083231 -0.13327069 0.23514816 0.1737896757 -0.1841346788 -0.17826396 0.06328632 0.0049946748 -0.2292900492 -0.17828348 0.09083063 0.0520370109 -0.0427662902 -0.05999197 0.22414653 0.29037643510 -0.1502655276 -0.03291610 0.16483031 0.165410143




> (lda.fit$posterior-lda.group$prior)[1:10,]

# look for highest positive difference in each row

female.FALSE female.TRUE male.FALSE male.TRUE1 -0.0554332753 -0.09587720 0.34599707 0.2170781172 -0.0498001173 0.07562023 0.09896126 0.0222774453 -0.2094610894 -0.18083214 0.11075691 -0.0145813324 -0.2690746455 -0.24820163 0.11002655 0.1425438495 -0.0005788862 0.16173070 0.14979977 0.1008131166 -0.1286083231 -0.13327069 0.23514816 0.1737896757 -0.1841346788 -0.17826396 0.06328632 0.0049946748 -0.2292900492 -0.17828348 0.09083063 0.0520370109 -0.0427662902 -0.05999197 0.22414653 0.29037643510 -0.1502655276 -0.03291610 0.16483031 0.165410143

> lda.fit$class[1:10] [1] male.FALSE male.FALSE male.FALSE male.TRUE female.TRUE male.FALSE male.FALSE [8] male.FALSE male.TRUE male.TRUE Levels: female.FALSE female.TRUE male.FALSE male.TRUE

> Compare to actual> group[1:10] [1] male.TRUE male.FALSE male.FALSE male.TRUE male.TRUE male.FALSE male.TRUE male.FALSE [9] male.TRUE male.FALSELevels: female.FALSE female.TRUE male.FALSE male.TRUE



> # Variable loadings> lda.groupCall:lda(group ~ Y, cv = True)

Prior probabilities of groups:female.FALSE female.TRUE male.FALSE male.TRUE 0.1470588 0.2132353 0.3235294 0.3161765

Group means: YAE YBHL YFL YTTL YSW YSKLfemale.FALSE 241.6500 31.38000 17.91716 28.55595 15.19936 20.71243female.TRUE 242.3793 31.53793 18.14699 28.92972 15.30394 21.05397male.FALSE 246.2273 31.59091 18.07095 28.71355 15.32948 21.40527male.TRUE 247.6744 31.67209 18.21771 28.89102 15.32329 21.74004

Coefficients of linear discriminants: LD1 LD2 LD3YAE -0.2141290 -0.1153824 -0.015984182YBHL 0.1042882 -0.3799127 -0.036351076YFL 0.2203001 1.1769264 -0.989261124YTTL 0.4893930 0.3703951 0.272006957YSW 0.3982302 -0.4163508 2.997840427YSKL -0.4043300 0.7320153 -0.005272685

Proportion of trace: LD1 LD2 LD3 0.9034 0.0820 0.0146 > > # note: the number of CVs = g-1, where g is group levels> # proportion of trace is the amount of information explained> # by each CV. This indicates how many dimensions are neeed> # to explain group differences, most likely

TTL, SW, and SKL are the variables that best describe group differences. SW and SKL differ in sign, meaning groups might have either long-slender skulls or short-wide skulls. Wide skulls positive covary with long legs, in distinguishing groups



> # CV plot> CV.scores<-lda.fit$x[,1:2]> > plot(CV.scores,asp=1,type='n')> points(CV.scores[group=='female.FALSE',],col='red')> points(CV.scores[group=='female.TRUE',],pch=21,bg='red')> points(CV.scores[group=='male.FALSE',],pch=22,col='blue')> points(CV.scores[group=='male.TRUE',],pch=22,bg='blue')

The reason for low classification is apparent now. This is a good example of how one can have sufficient statistical power but not have very meaningful conclusions about the biology. Remember this plot is only an abstract statistical visual aid. Some people make the mistake of showing these plots as “data plots” But the axes do not correspond to linear combinations of variables that express dispersion in the data; they correspond to linear combinations that best separate groups

Just for reference…..


CV1

CV2

CV1

CV2

CV1

CV2

Classification success (discrimination ability)

BIOL 582 Lecture Set 22 One-Way MANOVA, Part II Post-hoc exercises Discriminant Analysis.

Documents