What is Cognitive Science? What's in the brain, that ink may character? Shakespeare Sonnet 108; Warren McCulloch, Proceedings of the 1964 International.

What is Cognitive Science?

What's in the brain, that ink may character?Shakespeare Sonnet 108; Warren McCulloch, Proceedings of the 1964 International Congress for Logic, Methodology and Philosophy of Science

What’s in the mind that we may know it?

http://ruccs.rutgers.edu/faculty/pylyshyn.html

Zenon Pylyshyn, Rutgers Center for Cognitive Science

Cognitive science is a delicate mixture of the obvious and the incredible

Granny was almost right:

Behavior really is governed by what we believe and what we want (together with the mechanisms for representing and for drawing inferences from these)

It’s emic, not etic properties that matterKenneth Pike

What determines our behavior is not how the world is, but how we represent it as being If we describe behavior in relation to the objective

properties of the world, we would have to conclude that behavior is essentially stimulus-independent e.g., Chomsky’s review of B.F.Skinner’s book on language

Every behavioral regularity (other than physical ones like falling) is cognitively penetrable

or how we perceive it to be

It’s emic states that

matter!

The central role of representation presents some serious problems for a natural science

It’s what representations are about that matters (mostly)

But how can the fact that a belief is about some particular thing have an observable consequence?

• e.g. How can the presence of “holy grail” in a belief determine behavior when the holy grail does not exist?

Another way to view this is to note that we often characterize and explain behavior by referring to an agent’s reasons for doing something. Reasons seem to bridge the gap between beliefs and actions.

Can we naturalize representation? In a natural science if “X causes Y” then X must

exist and be causally connected to Y!• But in Cognitive Science X need not exist in order to play a

role in explaining behavior Y. Even when X does exist, it is not X’s physical properties that are relevant!

The North Star & navigationIt’s not what is actually present before you that determines

your behavior – it’s what you see it as. If you see a wind- swept bush, this in itself does not evoke any particular behavior. But if you see it as a wild animal you are

likely to cringe or run.We see in categories called concepts.

Dretske: Reasons in a world of causes

This dilemma is sometimes referred to as Brentano’s problem or the problem of intentionality

What determines what we do is what our mental states are about, but aboutness is not a category of natural science. There are no laws involving aboutness!

That is why Franz Brentano concluded that psychology was beyond the reach of natural science.

There are other properties that are special to cognitively determined behavior

1. The Semantic determinants of most cognitive behavior. To capture regularities in cognitively-caused behavior our theories must use symbols that refer to things in the world, or to meanings. Stimuli that mean the same thing can be interchanged

in many generalizations of cognitive science.

2. The Cognitive Penetrability of most cognitive processes. Almost any regularity can be systematically altered in a quasi-rational way by imparting new information.

Properties that are special to cognition… 3. The productivity and systematicity of systems of mental

representation. Systems of mental representation are structured so that if they are capable of representing certain situations then they are also capable of representing an unbounded number of other related situations. This leads to the requirements that representations be compositional, and that they have constituent structure.

4. The critical role of "Cognitive Capacity". Because of an organism's ecological or social niche, only a small fraction of its behavioral repertoire is ever actually observed. Nonetheless an adequate cognitive theory must account for the behavioral repertoire that is compatible with the organism's structure (it’s functional architecture). We call this behavioral repertoire its cognitive capacity.

Is it hopeless to think we can have a natural science of cognition?

Along comes The computational theory of mind

“the only straw afloat”

The major historical milestones

• Brentano’s recognition of the problem of intentionality

• The formalist movement in the foundations of mathematics: Hilbert, Goedel, Russell & Whitehead, Turing, Church, …

• Representational/Computational theory of mind: Newell & Simon, Chomsky, Fodor…

Intelligent systems behave the way they do because* of what the represent

● But “what they represent” cannot cause behavior because only physical properties have causes and functional states can represent almost anything (people, unicorns, electrons, anger, love – anything we can conceptualize).

So in order to cause behavior, the representations must be instantiated (‘encoded’) in physical properties

* Note: The nontechnical word “because” is misleading. In science if X causes Y it means either that there is a fundamental law of physics relating X and Y, or that one can derive the connection between X and Y in terms of a complex series of physical laws.

Intelligent systems behave the way they do because of what the represent

In the past ~70 years two sets of important ideas have provided a possible solution to our dilemma:

1. We have learned that to encode knowledge in physical form we must first encode it in symbolic form → symbolic logic tells us how

2. When knowledge (beliefs, goals etc) has been encoded in symbolic form then the symbolic codes can be instantiated in physical form→ computer science tells us how.

How to make a purely mechanical system reason about things it does not understand or know about? The discovery of symbolic logic.

(1) Married(John, Mary) or Married(John, Susan) and the equation or “statement”,(2) not[Married(John, Susan)]. from these two statements you can conclude,(3) Married(John, Mary)

But notice that (3) follows from (1) and (2) regardless of what is in the parts of the equation not occupied by the terms or or not so that you could write down the equations without mentioning marriage or John or Mary or, for that matter, anything having to do with the world. If you replaced these expressions with letters P and Q, the inference would still hold:

(1') P or Q (2') not Q therefore, (3') P

Cognitive Science and the Tri-Level Hypothesis

Intelligent systems are organized at three (or more) distinct levels:

1. The physical or biological level2. The symbolic or syntactic level3. The knowledge or semantic level

This means that different regularities may require appeal to different levels

Calculator example

• Why is the calculator’s printing faint and irregular? Why are parts of numbers missing in the LED display?

• Why does it take longer to multiply large numbers than small ones, whereas it takes the same length of time to add large numbers as small numbers?

• Why does it take longer to calculate trigonometrical functions than sums?

• Why is it especially fast at calculating the logarithm of 1?

• Why is it that when one of the keys (labeled ) is pressed after a number is entered, the calculator prints what appears to be the square root of that number? Will it always do so?

• When the answer to an arithmetic problem is too long to fit in the display window, why are some of the digits left off?

Does intentionality (and the trilevel hypothesis) only apply to high-level processes such as reasoning?

No They apply to all cognitive processes, including visual perception. Consider examples from vision:

• ‘Red light and yellow light mix to produce orange light’ • This remains true for any way of producing red light and

any way of producing yellow light: You can produce yellow by using a filter that allows light of

580 nm (nanometer) wavelength to pass, or combinations of filters that allow a certain mixture of light of 530 nm and 650 nm wavelengths to pass, or other mixtures...

But so long as one light looks yellow (or is represented as yellow) and the other looks red (or is represented as red) the “law” will hold.

• Represented as (or seeing as) is the crucial property

Vision patterns can present many different percepts Seeing as: It’s what you see the figure as that determines behavior (e.g.,

reaching for some part of a figure) – not its physical properties. What you see one part as determines what you see another part as. There must be a representation that contains the interpreted pictorial pattern

More Examples from Vision.

Does intentionality (and the trilevel hypothesis) only apply to high-level processes such as reasoning?

Examples from language

•John gave the book to Fred because he finished reading it•John gave the book to Fred because he wanted to read it

• The city council refused to give the workers a permit for a demonstration because they feared violence

• The city council refused to give the workers a permit for a demonstration because they were communists

If cognition is at a different level of organization from the physical level, how can

we ever tell what it’s like (how it works)?

We are limited only by the imagination of the scientist, e.g., in the past century we have used Relative complexity evidence (RT, error rates…) Intermediate state evidence

Eye tracking Stage analysis (additive factors method) Event Related Potentials (EEG) fMRI clinical observations of brain damage Psychophysical methods (SDT) Etc…

The central role of representation creates some serious problems for a natural science A serious problem arises because we are not

directly aware of our representations We are not aware of our thoughts …

What we are usually aware of is what our thoughts are about, not properties of the representation itself

Need to distinguish properties of our thoughts and properties of what they are about (e.g. mental images)

We are not aware of deciding, choosing or willing an action; or we are aware but are confabulating (Wegner)

Introspective evidence is just one type of evidence and it has turned out to be unreliable

Weak vs Strong Equivalence Is cognitive science concerned only with developing

models that generate the same Input-Output behavior as people exhibit in carrying out certain tasks?

A theory that correctly predicts I-O behavior is said to be weakly equivalent to the psychological process it is supposed to explain. It is what some people mean by “simulating behavior”.

Everyone in Cognitive Science is interested in strong equivalence – we want to explain not only the observed behavior, but also how it is generated. The how will usually takes the form of an algorithm.

Simulating the Input-Output function

Black BoxInput Output

Can we do any better than I-O simulation without looking inside the black box?

If all you have is observed behavior, how can you go beyond I-O simulation?

Simulating the Input-Output function

Think about this for a few minutes:

Is there any way to find out HOW a person does a simple problem such as adding two 4 digit numbers? Like 67 + 44 = ??

What are possible sources of evidence that may be relevant to this question?

Modeling the Actual Process (the algorithm used)

Black BoxInput Output

If all you have is observed behavior, how can you go beyond I-O simulation (mimicry)?

Answer: Not all observations are Inputs or Outputs: some are meta-behavior or indexes of processes.

Index of process

Example:Sternberg memory search experiment

• The initial input consists of the instructions and the presentation of the memory set (n letters).

• On each trial the particular input to the black box consists of the presentation of a target letter.

• The output consists of a binary response (present or absent). The time taken to respond is also recorded. That’s called the “Reaction Time”.

• Is the reaction time part of the output, to be generated by the theory, as another response?

Example of the input-output of a computational model of the Sternberg

task

Inputs: Memory set is (e.g.) C, D, H, NInputs: Probe (e.g., C or F)Output: Pairs of Responses and Reaction Times

(e.g. output is something like “Yes, 460 msecs”)Does it matter how the Output is derived?

It doesn’t if all you care about is I-O behavior It does if you care about Strong Equivalence (i.e.,

HOW it works)

Example of the input-output of a computational model of the Sternberg

task• Same inputs as before (Memory set is C,D,H,N)• Input-Output can be shown as a list:

Input to model Model prints out

C Yes 460 ms

N Yes 530 ms

H Yes 520 ms

R No 600 ms

M No 620 ms

If a model printed out these results, would it be weakly- or strongly-equivalent to a person?

Example of a weakly equivalent model of the Sternberg task

1. Store memory set as a list L. Call the list size = n2. Read target item, call it (If there is no , then quit)3. Search through L for until found or end of list encountered

4. If found in the list, assign Ans =“yes” otherwise Ans =“no”(That provides the answer, but what about the time RT ?)

5. If Ans =“yes”, set RT = 500 + n * K Random(20 x 50)

6. If Ans =“no” , set RT = 800 + n * K Random(20 x 50)

7. Print Ans , Print RT 8. End of trial, repeat from #2.

Is this the way people do it? How do you know?

Results of the Sternberg memory search taskWhat do they tell us about how people do it? Is this Input-Output equivalent or is it strongly equivalent to human performance?

Self-terminating searchExhaustive search

What reasons do you have for doubting that people do it this way?

Because in this case time should not be one of the computed outputs, but a measure of how many steps it took.

The same is true of intermediate states (e.g., evidence includes what subjects say, error rates, eye tracking, judgments about the output, and so on.)

Reaction time is one of the main sources of evidence in cog sci. Question: Is time always a valid index of processing

complexity?

Of course we can’t always be sure we have the right method or instrument

To achieve strong equivalence you first have to establish the architecture which supports cognition

• The architecture consists of the fixed properties of the mind/brain which encode and processes information.

• Although we are far from being able to describe the architecture of mind in any detail, we can provide some of its necessary properties which, in turn, will constrain what the architecture must be like and which proposals are plausible and which are implausible.

• So far we are able to postulate the nature of the form of representation, the nature of some of the mechanisms involved in mental computation

Next lecture: A different way to view architecture – in terms of cognitive capacity