Modeling Computation

Modeling Computation

Rosen, ch. 12Rosen, ch. 12

Modeling Computation

• We learned earlier the concept of an We learned earlier the concept of an algorithm.algorithm.– A description of a computational procedure.A description of a computational procedure.

• Now, how can we model the computer Now, how can we model the computer itself, and what it is doing when it carries itself, and what it is doing when it carries out an algorithm?out an algorithm?– For this, we want to model the abstract process For this, we want to model the abstract process

of of computationcomputation itself. itself.

Early Models of Computation

• Recursive Function TheoryRecursive Function Theory– Kleene, Church, Turing, Post, 1930’s Kleene, Church, Turing, Post, 1930’s (before computers!!)(before computers!!)

• Turing Machines Turing Machines – Turing, 1940’s – Turing, 1940’s (defined: computable) (defined: computable) • RAM Machines RAM Machines – von Neumann, 1940’s – von Neumann, 1940’s (“real computer”)(“real computer”)• Cellular Automata Cellular Automata – von Neumann, 1950’s – von Neumann, 1950’s (Wolfram 2005; physics of our world?)(Wolfram 2005; physics of our world?)• Finite-state machines, pushdown automataFinite-state machines, pushdown automata

– various people, 1950’svarious people, 1950’s• VLSI models VLSI models – 1970s– 1970s• Parallel RAMs, etc. Parallel RAMs, etc. – 1980’s– 1980’s

§12.1 – Languages & Grammars

• Phrase-Structure GrammarsPhrase-Structure Grammars• Types of Phrase-Structure GrammarsTypes of Phrase-Structure Grammars• Derivation TreesDerivation Trees• Backus-Naur FormBackus-Naur Form

Computers as Transition Functions• A computer (or really any physical system) can be A computer (or really any physical system) can be

modeled as having, at any given time, a specific state modeled as having, at any given time, a specific state ssSS from some (finite or infinite) from some (finite or infinite) state space state space SS..

• Also, at any time, the computer receives an Also, at any time, the computer receives an input symbolinput symbol iiII and produces an and produces an output symboloutput symbol ooOO. . – Where Where II and and OO are sets of symbols. are sets of symbols.

• Each “symbol” can encode an arbitrary amount of data.Each “symbol” can encode an arbitrary amount of data.• A computer can then be modeled as simply being a A computer can then be modeled as simply being a

transition function transition function TT::SS××II → → SS××OO..– Given the old state, and the input, this tells us what the Given the old state, and the input, this tells us what the

computer’s new state and its output will be a moment later.computer’s new state and its output will be a moment later.• Every model of computing we’ll discuss can be viewed Every model of computing we’ll discuss can be viewed

as just being some special case of this general picture.as just being some special case of this general picture.

Language Recognition Problem

• Let a Let a language language LL be any set of some arbitrary objects be any set of some arbitrary objects ss which will be dubbed “sentences.”which will be dubbed “sentences.”– ““legal” or “grammatically correct” sentences of the language.legal” or “grammatically correct” sentences of the language.

• Let the Let the language recognition problemlanguage recognition problem for for LL be: be:– Given a sentence Given a sentence ss, is it a legal sentence of the language , is it a legal sentence of the language LL? ?

• That is, is That is, is ssLL??

• Surprisingly, this simple problem is as general as our Surprisingly, this simple problem is as general as our very notion of computation itself! Hmm…very notion of computation itself! Hmm…

• Ex: addition ‘language’ Ex: addition ‘language’ “num1-num2-(num1+num2)”“num1-num2-(num1+num2)”

Vocabularies and Sentences

• Remember the concept of strings Remember the concept of strings ww of of symbols symbols ss chosen from an alphabet chosen from an alphabet ΣΣ– An alternative terminology for this concept: An alternative terminology for this concept:

• SentencesSentences σσ of of wordswords υυ chosen from a chosen from a vocabulary vocabulary VV..– No essential difference in concept or notation!No essential difference in concept or notation!

• Empty sentence (or string): Empty sentence (or string): λλ (length 0) (length 0)• Set of all sentences over Set of all sentences over VV: : Denoted Denoted VV**..

Grammars

• A formal A formal grammargrammar GG is any compact, precise is any compact, precise mathematical definition of a language mathematical definition of a language LL..– As opposed to just a raw listing of all of the language’s As opposed to just a raw listing of all of the language’s

legal sentences, or just examples of them.legal sentences, or just examples of them.• A grammar implies an algorithm that would A grammar implies an algorithm that would

generate all legal sentences of the language.generate all legal sentences of the language.– Often, it takes the form of a set of recursive definitions.Often, it takes the form of a set of recursive definitions.

• A popular way to specify a grammar recursively is A popular way to specify a grammar recursively is to specify it as a to specify it as a phrase-structure grammarphrase-structure grammar..

Phrase-Structure Grammars• A A phrase-structure grammarphrase-structure grammar (abbr. PSG) (abbr. PSG)

GG = ( = (VV,,TT,,SS,,PP)) is a 4-tuple, in which: is a 4-tuple, in which:– VV is a vocabulary (set of words) is a vocabulary (set of words)

• The “template vocabulary” of the language.The “template vocabulary” of the language.– TT VV is a set of words called is a set of words called terminalsterminals

• Actual words of the language.Actual words of the language.• Also, Also, NN : :≡ ≡ VV − − TT is a set of special “words” called is a set of special “words” called

nonterminalsnonterminals. (Representing concepts like “noun”). (Representing concepts like “noun”)– SSNN is a special nonterminal, the is a special nonterminal, the start symbol.start symbol.– PP is a set of is a set of productionsproductions (to be defined). (to be defined).

• Rules for substituting one sentence fragment for another.Rules for substituting one sentence fragment for another.

A phrase-structure grammar is a special case of the more general concept of a string-rewriting system, due to Post.

Productions• A A production pproduction pPP is a pair is a pair pp=(=(bb,,aa) of sentence fragments ) of sentence fragments

aa, , bb (not necessarily in (not necessarily in LL), which may generally contain a ), which may generally contain a mix of both terminals and nonterminals.mix of both terminals and nonterminals.– We often denote the production as We often denote the production as bb → → aa..

• Read “replace b by a”Read “replace b by a”– Call Call bb the “before” string, the “before” string, a a goes goes the “after” string.the “after” string.– It is a kind of recursive definition meaning that It is a kind of recursive definition meaning that

If If lbr lbr LLTT, then , then larlar LLTT.. ((LLTT = sentence = sentence “templates”)“templates”)

• That is, if That is, if lbrlbr is a legal sentence template, then so is is a legal sentence template, then so is larlar..• That is, we can substitute That is, we can substitute aa in place of in place of bb in any sentence template. in any sentence template.

Languages from PSGs• The recursive definition of the language The recursive definition of the language LL defined defined

by the PSG: by the PSG: GG = ( = (VV, , TT, , SS, , PP))::– Rule 1: Rule 1: SS LLTT ( (LLTT is is LL’s ’s template languagetemplate language))

• The start symbol is a sentence template (member of The start symbol is a sentence template (member of LLTT).).– Rule 2: Rule 2:

((bb→→aa))PP: : ll,,rrVV*: *: lbrlbr LLTT → → larlar LLTT

• Any production, after substituting in any fragment of any Any production, after substituting in any fragment of any sentence template, yields another sentence template.sentence template, yields another sentence template.

– Rule 3: Rule 3: ((σσ LLTT: ¬: ¬nnNN: : nnσσ) → ) → σσLL• All sentence templates that contain no nonterminal symbols All sentence templates that contain no nonterminal symbols

are sentences in are sentences in LL..

Abbreviatethis usinglbr lar.(read, “lar is directly derivable from lbr”).

PSG Example – English Fragment

We have We have GG = ( = (VV, , TT, , SS, , PP)), where:, where:• VV = { = {(sentence)(sentence), , (noun phrase)(noun phrase), ,

(verb phrase)(verb phrase), , (article)(article), , (adjective)(adjective),, (noun)(noun), , (verb)(verb), , (adverb)(adverb), , aa, , thethe, , largelarge,,

hungryhungry, , rabbitrabbit, , mathematicianmathematician, , eatseats, , hopshops,, quicklyquickly, , wildlywildly}}

• TT = { = {aa, , thethe, , largelarge, , hungryhungry, , rabbitrabbit, , mathematicianmathematician,, eatseats, , hopshops, , quicklyquickly, , wildlywildly}}

• SS = = (sentence)(sentence)• PP = = (see next slide)(see next slide)

Productions for our Language

PP = { = { (sentence)(sentence) → → (noun phrase)(noun phrase) (verb phrase)(verb phrase),,(noun phrase)(noun phrase) → → (article) (adjective) (noun)(article) (adjective) (noun),,(noun phrase)(noun phrase) → → (article) (noun)(article) (noun),,(verb phrase)(verb phrase) → → (verb) (adverb)(verb) (adverb),,(verb phrase)(verb phrase) → → (verb)(verb), , (article)(article) → → aa, , (article)(article) → → thethe,,(adjective)(adjective) → → largelarge, , (adjective)(adjective) → → hungryhungry,,(noun)(noun) → → rabbitrabbit, , (noun)(noun) → → mathematicianmathematician,,(verb)(verb) → → eatseats, , (verb)(verb) → → hopshops,,(adverb)(adverb) → → quicklyquickly, , (adverb)(adverb) → → wildlywildly } }

Backus-Naur Form

sentencesentence noun phrasenoun phrase verb phraseverb phrasenoun phrasenoun phrase articlearticle [[adjectiveadjective]] nounnounverb phraseverb phrase verbverb [[adverbadverb]]articlearticle a a | | thetheadjectiveadjective large large | hungry| hungrynounnoun rabbit rabbit | mathematician| mathematicianverbverb eats eats | | hopshopsadverbadverb quickly quickly | | wildlywildly

Square brackets []mean “optional”

Vertical barsmean “alternatives”

A Sample Sentence Derivation(sentence)(sentence)

(noun phrase) (verb phrase) (noun phrase) (verb phrase) (article) (adj.) (noun) (verb phrase)(article) (adj.) (noun) (verb phrase) (art.) (adj.) (noun) (verb) (adverb)(art.) (adj.) (noun) (verb) (adverb) thethe (adj.) (noun) (verb) (adverb)(adj.) (noun) (verb) (adverb)

the the largelarge (noun) (verb) (adverb)(noun) (verb) (adverb) the large rabbit the large rabbit (verb) (adverb)(verb) (adverb)

the large rabbitthe large rabbit hopshops (adverb)(adverb) the large rabbit hops quicklythe large rabbit hops quickly

On each step,we apply a production to a fragment of the previous sentence template to get a new sentence template. Finally, we end up with a sequence of terminals (real words), that is, a sentence of our language L.

Another Example

• Let Let GG = ({ = ({aa, , bb, , AA, , BB, , SS}, }, {{aa, , bb}, }, SS,, { {SS → → ABaABa, , AA → → BBBB, , BB → → abab, , ABAB → →

bb})})..• One possible derivation in this grammar is:One possible derivation in this grammar is:

SS ABaABa AabaAaba BBabaBBaba BababaBababa abababa.abababa.

V T

P

Derivability• Recall that the notation Recall that the notation ww00 ww11 means that means that

((bb→→aa))PP: : ll,,rrV*V*: : ww00 = = lbrlbr ww1 1 = = larlar..– The template The template ww11 is directly derivable from is directly derivable from ww00..

• If If ww22,…,…wwnn-1-1: : ww00 ww11 ww22 … … wwnn, then we , then we write write ww00 * * wwnn, and say that , and say that wwnn is derivable from is derivable from ww00. . – The sequence of steps wThe sequence of steps wii w wi+1i+1 is called a derivation is called a derivation

of wof wnn from w from w00..• Note that the relation Note that the relation * is just the transitive * is just the transitive

closure of the relation closure of the relation ..

A Simple Definition of L(G)

• The language The language LL((GG)) (or just (or just LL) that is ) that is generated by a given phrase-structure generated by a given phrase-structure grammar grammar GG=(=(VV,,TT,,SS,,PP)) can be defined by: can be defined by:

LL((GG) = {) = {ww TT* | * | SS * * ww}}• That is, L is simply the set of strings of That is, L is simply the set of strings of

terminals that are derivable from the start terminals that are derivable from the start symbol.symbol.

Generating Infinite Languages

• A simple PSG can easily generate an A simple PSG can easily generate an infinite language.infinite language.

• Example: Example: SS → 11→ 11SS, , SS → 0 ( → 0 (TT = {0,1}). = {0,1}).• The derivations are:The derivations are:

– SS 0 0– SS 11 11SS 110 110– SS 11 11SS 1111S 1111S 11110 11110– and so on…and so on…

L = {(11)*0} – theset of all strings consisting of somenumber of concaten-ations of 11 with itself,followed by 0.

Another example

• Construct a PSG that generates the language Construct a PSG that generates the language LL = = {{00nn11nn | | nnNN}}..– 00 and and 11 here represent symbols being concatenated here represent symbols being concatenated nn

times, not integers being raised to the times, not integers being raised to the nnth power.th power.• Solution strategy:Solution strategy: Each step of the derivation Each step of the derivation

should preserve the invariant that the number of should preserve the invariant that the number of 00’s = the number of ’s = the number of 11’s in the template so far, and ’s in the template so far, and all all 00’s come before all ’s come before all 11’s.’s.

• Solution:Solution: SS → → 00SS11, , SS → → λλ..

Types of Grammars - Chomsky hierarchy of languages

• Venn Diagram of Grammar Types:Venn Diagram of Grammar Types:Type 0 – Phrase-structure Grammars

Type 1 – Context-Sensitive

Type 2 –Context-Free

Type 3 –Regular

Defining the PSG Types• Type 1: Context-Sensitive PSG:Type 1: Context-Sensitive PSG:

– All after fragments are either longer than the corresponding All after fragments are either longer than the corresponding before fragments, or empty:before fragments, or empty:

if b if b →→ a, then a, then ||bb| < || < |aa| | aa = = λλ ..• Type 2: Context-Free PSG:Type 2: Context-Free PSG:

– All before fragments have length 1: All before fragments have length 1: if b if b →→ a, then a, then ||bb| = 1| = 1 ( (bb NN).).

• Type 3: Regular PSGs:Type 3: Regular PSGs:– All after fragments are either single terminals, or a pair of a All after fragments are either single terminals, or a pair of a

terminal followed by a nonterminal.terminal followed by a nonterminal. if b if b →→ a, then a, then aa TT aa TNTN..

Classifying grammarsGiven a grammar, we need to be able to find the Given a grammar, we need to be able to find the

smallest class in which it belongs. This can be smallest class in which it belongs. This can be determined by answering three questions: determined by answering three questions:

Are the left hand sides of all of the productions single Are the left hand sides of all of the productions single non-terminals? non-terminals?

• If yes, does each of the productions create at most If yes, does each of the productions create at most one non-terminal and is it on the right?one non-terminal and is it on the right? Yes – regularYes – regular No – context-free No – context-free

• If not, can any of the rules reduce the length of a If not, can any of the rules reduce the length of a string of terminals and non-terminals?string of terminals and non-terminals?

Yes – unrestricted Yes – unrestricted No – context-sensitive No – context-sensitive

Grammar

Productions of the form:xA String of variables and terminals

),,,( PSTVG

Variables Terminalsymbols

Startvariable

Variable

Definition: Context-Free Grammars

ExampleExample

  The language { The language { aannbbnnccnn | n | n 1} is context-sensitive 1} is context-sensitive but not context free.but not context free.

A grammar for this language is given by:A grammar for this language is given by:S S aSBC | aBC aSBC | aBCCB CB BC BCaB aB ab abbB bB bb bbbC bC bc bccC cC cc cc

A derivation from this grammar is:-A derivation from this grammar is:- S S aSBC aSBC

aaBCBCaaBCBC (using (using S S aBC) aBC) aabCBCaabCBC (using (using aB aB ab ab)) aabBCCaabBCC (using (using CB CB BC BC)) aabbCCaabbCC (using (using bB bB bb) bb) aabbcCaabbcC (using (using bC bC bc bc)) aabbccaabbcc (using (using cC cC cc cc))

  which derives which derives aa22bb22cc22..