Top Banner
Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in Computer Science Simon D. Levy Computer Science Department Washington & Lee University Fourth Regional Wittgenstein Workshop 11 March 2012 Socrates Plato Aristotle Aquinas Descartes Locke Hume Kant Frege Wittgenstein Angry Birds Phone Sunday, March 11, 12
55

Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Sep 27, 2018

Download

Documents

truongnga
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Wittgenstein's ComputerWittgenstein (Early and Late) and Modeling in Computer Science

Simon D. LevyComputer Science DepartmentWashington & Lee University

Fourth Regional Wittgenstein Workshop11 March 2012

Socrates Plato Aristotle

Aquinas Descartes Locke

Hume Kant Frege

Wittgenstein Angry Birds Phone

Sunday, March 11, 12

Page 2: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

In Memoriam

Ruth Barcan Marcus 1921 - 2012

Sunday, March 11, 12

Page 3: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Origins: 1939

WITTGENSTEIN: I won’t say anything which anyone can dispute. Or if anyone does dispute it, I will let that point drop and pass on to say something else.

TURING: I understand but I don’t agree that it is simply a question of giving new meanings to words.

WITTGENSTEIN: Turing doesn’t object to anything I say. He agrees with every word.

TURING: I see your point.

WITTGENSTEIN: I don’t have a point.

[Diamond 1976]Sunday, March 11, 12

Page 4: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Computing: 1946

Sunday, March 11, 12

Page 5: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Computing: "2004"

Sunday, March 11, 12

Page 7: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Computing: 2010

Sunday, March 11, 12

Page 8: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Hierarchy

Sunday, March 11, 12

Page 9: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Hierarchy

Sunday, March 11, 12

Page 10: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Localist Representations + Explicit Rules =

Brittle Computations

cats

chase

dogs

AGENT PATIENT

Sunday, March 11, 12

Page 11: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Localist Representations + Explicit Rules =

Brittle Computations

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 12: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sequential pointer-following:Who chases cats?

Sunday, March 11, 12

Page 13: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 14: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 15: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 16: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 17: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 18: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 19: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 20: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 21: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 22: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 23: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

One small error = total failure

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &12

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Page 24: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Usage

The confusions which occupy us arise when language is like an engine idling, not when it is doing work.

Sunday, March 11, 12

Page 25: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Usage

Sunday, March 11, 12

Page 26: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Rule-Following

All this, however, can only appear in the right light when one has attained greater clarity about the concepts of understanding, meaning, and thinking. For it will then also become clear what can lead us (and did lead me) to think that if anyone utters a sentence and means or understands it he is operating a calculus according to definite rules. 

Sunday, March 11, 12

Page 27: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Family Resemblances

I can think of no better expression to characterize these similarities than "family resemblances"; for the various resemblances between members of a family: build, features, colour of eyes, gait, temperament, etc. etc. overlap and cries-cross in the same way.-And I shall say: 'games' form a family.

Sunday, March 11, 12

Page 28: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Critique of Pure Reductionism

Suppose everyone had a box with something in it: we call it a "beetle". No one can look into anyone else's box, and everyone says he knows what a beetle is only by looking at his beetle. — Here it would be quite possible for everyone to have something different in his box.

Sunday, March 11, 12

Page 29: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Blue and Brown Books

Sunday, March 11, 12

Page 30: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

WALK

WALKED

Sunday, March 11, 12

Page 31: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

ROAR

ROARED

Sunday, March 11, 12

Page 32: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

SPEAK

SPOKE

Sunday, March 11, 12

Page 33: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

GO

WENT

Sunday, March 11, 12

Page 34: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

ignores(mary, john)

Mary won’t give John the time of day.

Sunday, March 11, 12

Page 35: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Challenges (Jackendoff 2002)

Sunday, March 11, 12

Page 36: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

I. The Binding Problem

+

? ? ? ?Sunday, March 11, 12

Page 37: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

II. The Problem of Two

+

? ? ?Sunday, March 11, 12

Page 38: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

III. The Problem of Variables

ignores(X, Y)

X won’t give Y the time of day.

Sunday, March 11, 12

Page 39: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Vector Symbolic Architectures

(Plate 1991; Kanerva 1994; Gayler 1998)

Sunday, March 11, 12

Page 40: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Tensor Product Binding(Smolensky 1990)

Sunday, March 11, 12

Page 41: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Binding

Sunday, March 11, 12

Page 42: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Bundling

+ =

Sunday, March 11, 12

Page 43: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Unbinding (query)

Sunday, March 11, 12

Page 44: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Lossy

Sunday, March 11, 12

Page 45: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Lossy

Sunday, March 11, 12

Page 46: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Cleanup

Hebbian / Hopfield /

Attractor Net

Sunday, March 11, 12

Page 47: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Reduction(Holographic; Plate 2003)

Sunday, March 11, 12

Page 48: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Reduction(Binary;

Kanerva 1994,Gayler 1998)

Sunday, March 11, 12

Page 49: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Composition / Recursion

Sunday, March 11, 12

Page 50: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Variables

X

john

Sunday, March 11, 12

Page 51: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Family Resemblances

• Both "primitive" concepts (cat, dog, chase) and "complex" ones (dogs chase cats unless the cats are bigger than them, etc.) are represented in the same way (N-dimensional vector)

• So we can use the same metric (e.g. Euclidean distance) to describe resemblances between objects of arbitrary complexity.

Sunday, March 11, 12

Page 52: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Rule-Following without Rules

• VSA can generalize ("learn a rule") from a single exemplar, because anything can be a "variable" (c.f. Moser 2012)

• E.g. What is the dollar of Mexico?

Sunday, March 11, 12

Page 53: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

“What is the Dollar of Mexico?” (Kanerva 2009)• Let X = <country>, Y = <currency>,

A = <USA>, B = <Mexico>

• Then A = X*U + Y*D, B = X*M + Y*PD*A*B =

D*(X*U + Y*D) * (X*M + Y*P) =

(D*X*U + D*Y*D) * (X*M + Y*P) =

(D*X*U + Y) * (X*M + Y*P) =

D*X*U*X*M + D*X*U*Y*P + Y*X*M + Y*Y*P =

P + noise

Sunday, March 11, 12

Page 54: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

The Random Beetle in the Box

• VSA representations are essentially random.

• So any two individual agents' internal representations of the same concepts will be unrelated to each other, though perfectly functional.

• VSA reps can also be expressed as neural spike trains (Stewart / Bekolay Eliasmith 2011), making VSA a plausible candidate for perceptual grounding of symbols (à la Barsalou 1999 ).

Sunday, March 11, 12

Page 55: Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

SourcesBarsalou, L.W. (1999) Perceptual symbols systems. Behavioral and Brain Sciences 22.

Diamond, C., ed. (1976) Wittgenstein's Lectures on the Foundations of Mathematics, Cambridge, 1939, Hassocks: Harvester Press.

Dror, I. E. and M. Dascal (1997) Can Wittgenstein help free the mind from rules? The philosophical foundations of connectionism. In D. Johnson and C. Erneling (eds.), The Future of the Cognitive Revolution. Oxford University Press.

Kanerva, P. (2009) Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors. Cognitive Computation 1(2).

Klagge, J.C. (1989) Wittgenstein and neuroscience. Synthese 78.

Moser, C. (2012) Schmagency and the formal character of constitutive principles. Presented at the Fourth Regional Wittgenstein Workshop (10 March 2012), Lexington, Virginia.

Stern, D.G. (1991) Models of memory: Wittgenstein and cognitive science. Philosophical Psychology 4 (2).T

Stewart, T., T. Bekolay, and C. Eliasmith (2011) Neural representations of compositional structures: representing and manipulating vector spaces with spiking neurons. Connection Science 23 (2).

Wittgenstein, L. (1921/1975) Tractatus Logico-Philosophicus (trans. D.F. Pears and B.F. McGuiness). Routledge.

Wittgenstein, L. (1933/1965) The Blue and Brown Books. Harper Torchbooks.

Wittgenstein, L. (1953/1973) Philosophical Investigations (3rd Edition, trans. G.E.M. Anscombe). Prentice Hall.

Sunday, March 11, 12