Nonparametric Bayes and human cognition Tom Griffiths Department of Psychology Program in Cognitive Science University of California, Berkeley.

Nonparametric Bayes and human cognition

Tom GriffithsDepartment of Psychology

Program in Cognitive Science

University of California, Berkeley

data

hypothesisStatistics about the mind

Analyzing psychological data• Dirichlet process mixture models for capturing

individual differences(Navarro, Griffiths, Steyvers, & Lee, 2006)

• Infinite latent feature models…– …for features influencing similarity

(Navarro & Griffiths, 2007; 2008)– …for features influencing decisions

()

Statistics in the mind

data

hypothesis

data

hypothesisStatistics about the mind

Flexible mental representations• Dirichlet

Categorization

How do people represent categories?

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

catdog ?



cat



cat

Prototypes

(Posner & Keele, 1968; Reed, 1972)



Prototype







cat

cat

cat



cat



cat

Exemplars

(Medin & Schaffer, 1978; Nosofsky, 1986)

Store everyinstance

(exemplar)in memory







cat

cat

cat



cat



cat

Something in between

(Love et al., 2004; Vanpaemel et al., 2005)







cat

cat

cat

A computational problem• Categorization is a classic inductive problem

– data: stimulus x– hypotheses: category c

• We can apply Bayes’ rule:

and choose c such that P(c|x) is maximized

€

P(c | x) =p(x | c)P(c)

p(x | c)P(c)c

∑

Density estimation• We need to estimate some probability

distributions– what is P(c)?– what is p(x|c)?

• Two approaches:– parametric– nonparametric

• These approaches correspond to prototype and exemplar models respectively

(Ashby & Alfonso-Reese, 1995)

Parametric density estimation

Assume that p(x|c) has a simple form, characterized by parameters (indicating the

prototype)

QuickTime™ and aTIFF (LZW) decompressor


Pro

babi

lity

den

sity

x

Nonparametric density estimation

Approximate a probability distribution as a sum of many “kernels” (one per data point)

estimated functionindividual kernelstrue function

n = 10

Pro

babi

lity

den

sity

x

Something in between

mixture distributionmixture components

Pro

babi

lity

x

Use a “mixture” distribution, with more than one component per data point

(Rosseel, 2002)



Anderson’s rational model(Anderson, 1990, 1991)

• Treat category labels like any other feature• Define a joint distribution p(x,c) on features using

a mixture model, breaking objects into clusters• Allow the number of clusters to vary…

€

P(cluster j)∝n j j is old

α j is new

⎧ ⎨ ⎩

a Dirichlet process mixture model(Neal, 1998; Sanborn et al., 2006)

A unifying rational model• Density estimation is a unifying framework

– a way of viewing models of categorization

• We can go beyond this to define a unifying model– one model, of which all others are special cases

• Learners can adopt different representations by adaptively selecting between these cases

• Basic tool: two interacting levels of clusters– results from the hierarchical Dirichlet process

(Teh, Jordan, Beal, & Blei, 2004)

The hierarchical Dirichlet process



A unifying rational modelclusterexemplarcategory

€

γ∈ (0,∞)









€

γ→ ∞

€

α → ∞€

α ∈ (0,∞)€

α → 0

exemplar



prototype



Anderson

HDP+, and Smith & Minda (1998)

• HDP+, will automatically infer a representation using exemplars, prototypes, or something in between (with α being learned from the data)

• Test on Smith & Minda (1998, Experiment 2)111111011111101111110111111011111110000100

000000100000010000001000000010000001111101

Category A: Category B:

exceptions

HDP+, and Smith & Minda (1998)





Pro

babi

lity

of

A prototype

exemplar HDP



Log

-lik

elih

ood

exemplar

prototype

HDP


are needed to see this picture.The promise of HDP+,+

• In HDP+,+, clusters are shared between categories

– a property of hierarchical Bayesian models

• Learning one category has a direct effect on the prior on probability densities for the next category















QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.









Learning the features of objects

• Most models of human cognition assume objects are represented in terms of abstract features

• What are the features of this object?

• What determines what features we identify?



(Austerweil & Griffiths, submitted)





Binary matrix factorization

Binary matrix factorization

How should we infer the number of features?

The nonparametric approach

Use the Indian buffet process as a prior on Z

Assume that the total number of features is unbounded, but only a finite number will be

expressed in any finite dataset

(Griffiths & Ghahramani, 2006)










An experiment…


Training Testing

Cor

rela

ted

Fac

tori

al

See

nU

nsee

nS

huff

led


Results

Conclusions

• Approaching cognitive problems as computational problems allows cognitive science and machine learning to be mutually informative

• Machine

Credits

CategorizationAdam SanbornKevin CaniniDan Navarro

Learning featuresJoe Austerweil

MCMC with peopleAdam Sanborn

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.



Computational Cognitive Science Labhttp://cocosci.berkeley.edu/



Nonparametric Bayes and human cognition Tom Griffiths Department of Psychology Program in Cognitive Science University of California, Berkeley.

Documents

mind slide

berkeley slide

decisions slide

prototype cat slide

memory cat slide

unifying model

exemplar models

data test