Multidimensionality Multi-dimensionality & Factor analysis.

Multidimensionality

Multi-dimensionality &Factor analysis

Multidimensionality

Multidimensionality

• Adding dimensions: Multidimensionality

• Compressing dimensions: Factor analysis

• Intelligence testing and g

Multidimensionality

Adding new dimensions

• Most tests measure single dimensional constructs– We have already seen one case of dual-

dimensionality in IQ– It is easy to imagine extending into a third

dimension• e.g. Gould’s ‘football’, correlating growth of three

body parts

Multidimensionality


• Mathematically, there is no need to stop at three dimensions

– Correlations, Euclidean distance, orthogonality [to be explained] etc. are all well-defined in higher dimensional spaces

• Informally, it is easy to imagine high dimensional spaces

– e.g. Think of the qualities that make up a good car, or a good apartment, or a good mate

Multidimensionality


• Visually, it is not so difficult either…at least for a few more dimensions

– Imagine using color for the 4th dimension, and size of point for the fifth dimension [etc. etc.]

D1

D2 D3 = SizeD4 = Colour

Multidimensionality

What is a dimension?

• Dimensions are information = they are differences that make a difference

Multidimensionality

What is a factor?

• A factor is a dimension

Multidimensionality

Correlation and information

• What is the relation between a correlation and information/probability?– A significant correlation between x and y tells

us that x contains information about y– Another way of saying this is that x and y are

not independent, in the sense we used the term in discussing probability

Multidimensionality

Correlation and information

• However, note that the dependence may be roundabout or even spurious if it depends on relations between features that are very common or just accidental

• Examples:– In American towns and cities, the correlation between

the number of churches and the number of violent crimes is about 0.85: Why?

– Income and homelessness are positively correlated: Why?

– Outside temperature and rape rates are positively correlated: Why?

Multidimensionality

Relations between dimensions

• Any relation between any two dimensions (which means: any correlation) can be expressed in terms of two orthogonal (=right-angle) components– By definition, these two components that have nothing

to do with each other, each containing no ‘amount’ of the other dimension

– If you know something about one dimension, you know nothing about an orthogonal dimension

– If knowing something about one dimension does give you information about another: they are not orthogonal = they are correlated

Multidimensionality

Relating two dimensions

Dimension 1

Dim

ensi

on 2

Pure vertical component

Pure horizontal component

Multidimensionality

A concrete example

Picture arrangement score

Dig

it s

pan

scor

e Pure vertical component

Pure horizontal component

Multidimensionality

Pure dimensions

• When two dimensions are orthogonal, they are ‘pure’ dimensions, wholly separable from one another – No information about one is contained in the other

• By the same token, when two dimensions are not orthogonal, they are ‘contaminated’ by at least one common component– Information about one is contained in the other– i.e. Gould: 14 independent bone measurements reduce

to the single dimension of 'size'

Multidimensionality

Relating two dimensions

Dimension 1

Dim

ensi

on 2

Dimensions 1 and 2 here both contain a little bit of ‘horizontalness’ and a little bit of ‘verticality’

Multidimensionality

The need for orthogonality• Dimensions which are orthogonal are independent =

whatever happens on one dimension has no effect on what happens on the other dimension (or, equivalently, knowing the value on one dimension provides no information about value along the other dimension or they correlate with r = 0).– Example: Beauty and intelligence are orthogonal (if not,

the world sure is unfair!); height and weight are not orthogonal

• For this reason, the theoretical ‘true’ dimensionality of a thing (the number of things we need to know to know ~everything that thing) is the value of ~all orthogonal dimensions relevant to that thing

Multidimensionality

Information and orthogonality

Dimension 1

Dim

ensi

on 2

Here we know something about Dimension2 (the value of B) when we know the value along Dimension 1(the value of A)- precisely because the dimensions are not orthogonal.

A

B

Multidimensionality

Information and orthogonality

Dimension 1

Dim

ensi

on 2

A

B

This is mathematically equivalent to saying that the dimensions are correlated. For the curious, the cosine of the angle = r

Multidimensionality

Factor A

Fac

tor

B

Example: Four correlations of test results with factors

Multidimensionality

We rotate the dimensionshere to get 2 orthogonal dimensions.

Factor A'

Fact

or B

'

MultidimensionalityFactor A

Fac

tor

B

F1: General intelligence (g)

F2: Verbal load

Non-verbal subtests Verbal subtests

Multidimensionality

Test results as factors• Note (to repeat) that this eliminates any hard distinction

between the identified dimensions (say, the labels on a graph of one subtest against another) and some alleged ‘real’ factors– Beware of falling in love with the axes labels on a graph– these are just factors that you have chosen (or have other reason) to

graph– they are no more (or less) true than a rotated graph in which

‘abstract’ factors now defined the axes, and text scores are vectors (regression lines) on the graph

• One hard idea to grasp is that the differences here are merely notational = no information is gained or lost through rotation, and the labels are just a convenience

Multidimensionality

Correlation matrix

• We can represent the relationship between many dimensions with a correlation matrix

Dimensions 1 2 3 4 51 12 0.3 13 0.8 0.6 14 0.2 0.7 0.1 15 0.8 0.3 0.9 0.5 1

Multidimensionality

Factor analysis• Factor analysis is a method for reducing a correlation

matrix with a large number of dimensions to (usually) their orthogonal dimensions, called principal components– In other words, it is a method of dimensional reduction – It looks for ‘contamination’ of one dimension by others

(= inter-correlations), and tries to re-present the matrix without that ‘contamination’

– Preference expression is an everyday example of such dimensional reduction, as when we ask ‘If you had pick just three qualities to look for in your next car, what would you pick?’

Multidimensionality

What is a factor?• Factors are not Platonic 'true' objects gifted to man from

the benevolent inhabitants of the Heavens– A factor is a mathematical construct, containing

abstracted information about the inter-relations between other information/dimensions

– it is no better than those measures: GIGO

"The 'factors', in short, are to be regarded as convenient mathematical abstractions, not as concrete mental 'faculties', lodged in separate 'organs' of the brain.

Cyril Burt, 1937

Multidimensionality

What is a factor?– mathematically, a factor is a (usually but not

necessarily) linear combination of correlations between the items in the analysis

– What does it mean to be a ‘linear combination’?• The factors are produced produced by adding in

portions of the variance accounted for by some dimensions (for example, subtests) and subtracting out portion of the variance accounted for by others

• e.g. F1 = aX + bY + cZ, if X,Y,Z are correlations in the input table, and a,b,c are (positive or negative) real number coefficients

Multidimensionality

The factor table• The principal components are taken out in an ordered fashion,

from those accounting for the most variance (strong loading or weighting) to those accounting for the least (weak loading or weighting)

• They are presented in a factor table

FACTORS A B C D

Item1 0.81 0.21 -0.11 -0.12

Item2 -0.13 0.76 0.72 -0.04

Item3 0.08 0.12 0.34 0.88

...

Multidimensionality

Are factors ‘real’?• The danger of reification: Just because a factor can

be extracted does not necessarily mean it has any relevant reality– Some principal component can always be extracted so

long as most correlations are in the same direction• In modern factor analysis, the axes are also rotated to see if

one rotation is better able to account for the data than another.– Because different rotations lead to different

descriptions, Gould warns against reifying any particular description

Multidimensionality

" 'When I use a word', Humpty Dumpty said, in rather a scornful tone, 'it means just what I choose it to mean - neither more nor less.' "

Lewis CarrollThrough the Looking-Glass

Multidimensionality

Spearman's g

• Spearman (1904) invented factor analysis as a way of studying correlations between mental test scores

• He called the first principal component (accounting for about 50-60% of variance) g = general intelligence

• People have argued ever since whether g is real


Fac

tor

B

F1: General intelligence (g)

F2: Verbal load



Fac

tor

B

F1: Performance IQ

F2: Verbal IQ


Multidimensionality

Cyril Burt's Error

• Intelligence is an abused concept

• Burt cut off 80% of people tested from access to higher education, entirely on the basis of a single test

Multidimensionality

Questions• Is a factor more or less 'real' than a 'traditional'

construct? – How can we evaluate the ‘reality’ of any given

factor/dimension?

• What advantage is there is discussing psychometric results in terms of non-orthogonal factors?

• Should people's lives be planned around abstract mathematical constructs?

• Should there be legal limits on what constructs can be defined?

Multidimensionality Multi-dimensionality & Factor analysis.

Documents