Top Banner
CS 373: Theory of Computation Gul Agha Mahesh Viswanathan University of Illinois, Urbana-Champaign Fall 2010 Agha-Viswanathan CS373
80

CS 373: Theory of Computation

Feb 10, 2022

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: CS 373: Theory of Computation

CS 373: Theory of Computation

Gul Agha Mahesh Viswanathan

University of Illinois, Urbana-Champaign

Fall 2010

Agha-Viswanathan CS373

Page 2: CS 373: Theory of Computation

Closure Properties

Part I

Closure Properties of Turing Machines

Agha-Viswanathan CS373

Page 3: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Boolean Operators

Proposition

Decidable languages are closed under union, intersection, andcomplementation.

Proof.

Given TMs M1, M2 that decide languages L1, and L2

A TM that decides L1 ∪ L2: on input x , run M1 and M2 on x ,and accept iff either accepts. (Similarly for intersection.)

A TM that decides L1: On input x , run M1 on x , and acceptif M1 rejects, and reject if M1 accepts. �

Agha-Viswanathan CS373

Page 4: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Boolean Operators

Proposition

Decidable languages are closed under union, intersection, andcomplementation.

Proof.

Given TMs M1, M2 that decide languages L1, and L2

A TM that decides L1 ∪ L2: on input x , run M1 and M2 on x ,and accept iff either accepts. (Similarly for intersection.)

A TM that decides L1: On input x , run M1 on x , and acceptif M1 rejects, and reject if M1 accepts. �

Agha-Viswanathan CS373

Page 5: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Boolean Operators

Proposition

Decidable languages are closed under union, intersection, andcomplementation.

Proof.

Given TMs M1, M2 that decide languages L1, and L2

A TM that decides L1 ∪ L2: on input x , run M1 and M2 on x ,and accept iff either accepts.

(Similarly for intersection.)

A TM that decides L1: On input x , run M1 on x , and acceptif M1 rejects, and reject if M1 accepts. �

Agha-Viswanathan CS373

Page 6: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Boolean Operators

Proposition

Decidable languages are closed under union, intersection, andcomplementation.

Proof.

Given TMs M1, M2 that decide languages L1, and L2

A TM that decides L1 ∪ L2: on input x , run M1 and M2 on x ,and accept iff either accepts. (Similarly for intersection.)

A TM that decides L1: On input x , run M1 on x , and acceptif M1 rejects, and reject if M1 accepts. �

Agha-Viswanathan CS373

Page 7: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Boolean Operators

Proposition

Decidable languages are closed under union, intersection, andcomplementation.

Proof.

Given TMs M1, M2 that decide languages L1, and L2

A TM that decides L1 ∪ L2: on input x , run M1 and M2 on x ,and accept iff either accepts. (Similarly for intersection.)

A TM that decides L1: On input x , run M1 on x , and acceptif M1 rejects, and reject if M1 accepts. �

Agha-Viswanathan CS373

Page 8: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Regular Operators

Proposition

Decidable languages are closed under concatenation and KleeneClosure.

Proof.

Given TMs M1 and M2 that decide languages L1 and L2.

A TM to decide L1L2:

On input x , for each of the |x |+ 1ways to divide x as yz : run M1 on y and M2 on z , and acceptif both accept. Else reject.

A TM to decide L∗1: On input x , if x = ε accept. Else, foreach of the 2|x |−1 ways to divide x as w1 . . .wk (wi 6= ε): runM1 on each wi and accept if M1 accepts all. Else reject. �

Agha-Viswanathan CS373

Page 9: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Regular Operators

Proposition

Decidable languages are closed under concatenation and KleeneClosure.

Proof.

Given TMs M1 and M2 that decide languages L1 and L2.

A TM to decide L1L2: On input x , for each of the |x |+ 1ways to divide x as yz : run M1 on y and M2 on z , and acceptif both accept. Else reject.

A TM to decide L∗1: On input x , if x = ε accept. Else, foreach of the 2|x |−1 ways to divide x as w1 . . .wk (wi 6= ε): runM1 on each wi and accept if M1 accepts all. Else reject. �

Agha-Viswanathan CS373

Page 10: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Regular Operators

Proposition

Decidable languages are closed under concatenation and KleeneClosure.

Proof.

Given TMs M1 and M2 that decide languages L1 and L2.

A TM to decide L1L2: On input x , for each of the |x |+ 1ways to divide x as yz : run M1 on y and M2 on z , and acceptif both accept. Else reject.

A TM to decide L∗1:

On input x , if x = ε accept. Else, foreach of the 2|x |−1 ways to divide x as w1 . . .wk (wi 6= ε): runM1 on each wi and accept if M1 accepts all. Else reject. �

Agha-Viswanathan CS373

Page 11: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Regular Operators

Proposition

Decidable languages are closed under concatenation and KleeneClosure.

Proof.

Given TMs M1 and M2 that decide languages L1 and L2.

A TM to decide L1L2: On input x , for each of the |x |+ 1ways to divide x as yz : run M1 on y and M2 on z , and acceptif both accept. Else reject.

A TM to decide L∗1: On input x , if x = ε accept. Else, foreach of the 2|x |−1 ways to divide x as w1 . . .wk (wi 6= ε): runM1 on each wi and accept if M1 accepts all. Else reject. �

Agha-Viswanathan CS373

Page 12: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Inverse Homomorphisms

Proposition

Decidable languages are closed under inverse homomorphisms.

Proof.

Given TM M1 that decides L1, a TM to decide h−1(L1) is: Oninput x , compute h(x) and run M1 on h(x); accept iff M1

accepts. �

Agha-Viswanathan CS373

Page 13: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Inverse Homomorphisms

Proposition

Decidable languages are closed under inverse homomorphisms.

Proof.

Given TM M1 that decides L1, a TM to decide h−1(L1) is:

Oninput x , compute h(x) and run M1 on h(x); accept iff M1

accepts. �

Agha-Viswanathan CS373

Page 14: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Inverse Homomorphisms

Proposition

Decidable languages are closed under inverse homomorphisms.

Proof.

Given TM M1 that decides L1, a TM to decide h−1(L1) is: Oninput x , compute h(x) and run M1 on h(x); accept iff M1

accepts. �

Agha-Viswanathan CS373

Page 15: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

Decidable languages are not closed under homomorphism

Proof.

We will show a decidable language L and a homomorphism h suchthat h(L) is undecidable

Let L = {xy | x ∈ {0, 1}∗, y ∈ {a, b}∗, x = 〈M,w〉, and yencodes an integer n such that the TM M on input w willhalt in n steps }L is decidable: can simply simulate M on input w for n steps

Consider homomorphism h: h(0) = 0, h(1) = 1,h(a) = h(b) = ε.

h(L) = HALT which is undecidable. �

Agha-Viswanathan CS373

Page 16: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

Decidable languages are not closed under homomorphism

Proof.

We will show a decidable language L and a homomorphism h suchthat h(L) is undecidable

Let L = {xy | x ∈ {0, 1}∗, y ∈ {a, b}∗, x = 〈M,w〉, and yencodes an integer n such that the TM M on input w willhalt in n steps }L is decidable: can simply simulate M on input w for n steps

Consider homomorphism h: h(0) = 0, h(1) = 1,h(a) = h(b) = ε.

h(L) = HALT which is undecidable. �

Agha-Viswanathan CS373

Page 17: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

Decidable languages are not closed under homomorphism

Proof.

We will show a decidable language L and a homomorphism h suchthat h(L) is undecidable

Let L = {xy | x ∈ {0, 1}∗, y ∈ {a, b}∗, x = 〈M,w〉, and yencodes an integer n such that the TM M on input w willhalt in n steps }

L is decidable: can simply simulate M on input w for n steps

Consider homomorphism h: h(0) = 0, h(1) = 1,h(a) = h(b) = ε.

h(L) = HALT which is undecidable. �

Agha-Viswanathan CS373

Page 18: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

Decidable languages are not closed under homomorphism

Proof.

We will show a decidable language L and a homomorphism h suchthat h(L) is undecidable

Let L = {xy | x ∈ {0, 1}∗, y ∈ {a, b}∗, x = 〈M,w〉, and yencodes an integer n such that the TM M on input w willhalt in n steps }L is decidable: can simply simulate M on input w for n steps

Consider homomorphism h: h(0) = 0, h(1) = 1,h(a) = h(b) = ε.

h(L) = HALT which is undecidable. �

Agha-Viswanathan CS373

Page 19: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

Decidable languages are not closed under homomorphism

Proof.

We will show a decidable language L and a homomorphism h suchthat h(L) is undecidable

Let L = {xy | x ∈ {0, 1}∗, y ∈ {a, b}∗, x = 〈M,w〉, and yencodes an integer n such that the TM M on input w willhalt in n steps }L is decidable: can simply simulate M on input w for n steps

Consider homomorphism h: h(0) = 0, h(1) = 1,h(a) = h(b) = ε.

h(L) = HALT which is undecidable. �

Agha-Viswanathan CS373

Page 20: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

Decidable languages are not closed under homomorphism

Proof.

We will show a decidable language L and a homomorphism h suchthat h(L) is undecidable

Let L = {xy | x ∈ {0, 1}∗, y ∈ {a, b}∗, x = 〈M,w〉, and yencodes an integer n such that the TM M on input w willhalt in n steps }L is decidable: can simply simulate M on input w for n steps

Consider homomorphism h: h(0) = 0, h(1) = 1,h(a) = h(b) = ε.

h(L) =

HALT which is undecidable. �

Agha-Viswanathan CS373

Page 21: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

Decidable languages are not closed under homomorphism

Proof.

We will show a decidable language L and a homomorphism h suchthat h(L) is undecidable

Let L = {xy | x ∈ {0, 1}∗, y ∈ {a, b}∗, x = 〈M,w〉, and yencodes an integer n such that the TM M on input w willhalt in n steps }L is decidable: can simply simulate M on input w for n steps

Consider homomorphism h: h(0) = 0, h(1) = 1,h(a) = h(b) = ε.

h(L) = HALT which is undecidable. �

Agha-Viswanathan CS373

Page 22: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Boolean Operators

Proposition

R.E. languages are closed under union, and intersection.

Proof.

Given TMs M1, M2 that recognize languages L1, L2

A TM that recognizes L1 ∪ L2: on input x , run M1 and M2 onx in parallel, and accept iff either accepts. (Similarly forintersection; but no need for parallel simulation) �

Agha-Viswanathan CS373

Page 23: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Boolean Operators

Proposition

R.E. languages are closed under union, and intersection.

Proof.

Given TMs M1, M2 that recognize languages L1, L2

A TM that recognizes L1 ∪ L2: on input x , run M1 and M2 onx in parallel, and accept iff either accepts. (Similarly forintersection; but no need for parallel simulation) �

Agha-Viswanathan CS373

Page 24: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Boolean Operators

Proposition

R.E. languages are closed under union, and intersection.

Proof.

Given TMs M1, M2 that recognize languages L1, L2

A TM that recognizes L1 ∪ L2: on input x , run M1 and M2 onx in parallel, and accept iff either accepts.

(Similarly forintersection; but no need for parallel simulation) �

Agha-Viswanathan CS373

Page 25: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Boolean Operators

Proposition

R.E. languages are closed under union, and intersection.

Proof.

Given TMs M1, M2 that recognize languages L1, L2

A TM that recognizes L1 ∪ L2: on input x , run M1 and M2 onx in parallel, and accept iff either accepts. (Similarly forintersection; but no need for parallel simulation) �

Agha-Viswanathan CS373

Page 26: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Complementation

Proposition

R.E. languages are not closed under complementation.

Proof.

Atm is r.e. but Atm is not. �

Agha-Viswanathan CS373

Page 27: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Regular Operations

Proposition

R.E languages are closed under concatenation and Kleene closure.

Proof.

Given TMs M1 and M2 recognizing L1 and L2

A TM to recognize L1L2:

On input x , do in parallel, for eachof the |x |+ 1 ways to divide x as yz : run M1 on y and M2 onz , and accept if both accept. Else reject.

A TM to recognize L∗1: On input x , if x = ε accept. Else, doin parallel, for each of the 2|x |−1 ways to divide x as w1 . . .wk

(wi 6= ε): run M1 on each wi and accept if M1 accepts all.Else reject. �

Agha-Viswanathan CS373

Page 28: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Regular Operations

Proposition

R.E languages are closed under concatenation and Kleene closure.

Proof.

Given TMs M1 and M2 recognizing L1 and L2

A TM to recognize L1L2: On input x , do in parallel, for eachof the |x |+ 1 ways to divide x as yz : run M1 on y and M2 onz , and accept if both accept. Else reject.

A TM to recognize L∗1: On input x , if x = ε accept. Else, doin parallel, for each of the 2|x |−1 ways to divide x as w1 . . .wk

(wi 6= ε): run M1 on each wi and accept if M1 accepts all.Else reject. �

Agha-Viswanathan CS373

Page 29: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Regular Operations

Proposition

R.E languages are closed under concatenation and Kleene closure.

Proof.

Given TMs M1 and M2 recognizing L1 and L2

A TM to recognize L1L2: On input x , do in parallel, for eachof the |x |+ 1 ways to divide x as yz : run M1 on y and M2 onz , and accept if both accept. Else reject.

A TM to recognize L∗1:

On input x , if x = ε accept. Else, doin parallel, for each of the 2|x |−1 ways to divide x as w1 . . .wk

(wi 6= ε): run M1 on each wi and accept if M1 accepts all.Else reject. �

Agha-Viswanathan CS373

Page 30: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Regular Operations

Proposition

R.E languages are closed under concatenation and Kleene closure.

Proof.

Given TMs M1 and M2 recognizing L1 and L2

A TM to recognize L1L2: On input x , do in parallel, for eachof the |x |+ 1 ways to divide x as yz : run M1 on y and M2 onz , and accept if both accept. Else reject.

A TM to recognize L∗1: On input x , if x = ε accept. Else, doin parallel, for each of the 2|x |−1 ways to divide x as w1 . . .wk

(wi 6= ε): run M1 on each wi and accept if M1 accepts all.Else reject. �

Agha-Viswanathan CS373

Page 31: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

R.E. languages are closed under both inverse homomorphisms andhomomorphisms.

Proof.

Let TM M1 recognize L1.

A TM to recognize h−1(L1):

On input x , compute h(x) andrun M1 on h(x); accept iff M1 accepts.

A TM to recognize h(L1): On input x , start going through allstrings w , and if h(w) = x , start executing M1 on w , usingdovetailing to interleave with other executions of M1. Acceptif any of the executions accepts. �

Agha-Viswanathan CS373

Page 32: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

R.E. languages are closed under both inverse homomorphisms andhomomorphisms.

Proof.

Let TM M1 recognize L1.

A TM to recognize h−1(L1):On input x , compute h(x) andrun M1 on h(x); accept iff M1 accepts.

A TM to recognize h(L1): On input x , start going through allstrings w , and if h(w) = x , start executing M1 on w , usingdovetailing to interleave with other executions of M1. Acceptif any of the executions accepts. �

Agha-Viswanathan CS373

Page 33: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

R.E. languages are closed under both inverse homomorphisms andhomomorphisms.

Proof.

Let TM M1 recognize L1.

A TM to recognize h−1(L1):On input x , compute h(x) andrun M1 on h(x); accept iff M1 accepts.

A TM to recognize h(L1):

On input x , start going through allstrings w , and if h(w) = x , start executing M1 on w , usingdovetailing to interleave with other executions of M1. Acceptif any of the executions accepts. �

Agha-Viswanathan CS373

Page 34: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

R.E. languages are closed under both inverse homomorphisms andhomomorphisms.

Proof.

Let TM M1 recognize L1.

A TM to recognize h−1(L1):On input x , compute h(x) andrun M1 on h(x); accept iff M1 accepts.

A TM to recognize h(L1): On input x , start going through allstrings w , and if h(w) = x , start executing M1 on w

, usingdovetailing to interleave with other executions of M1. Acceptif any of the executions accepts. �

Agha-Viswanathan CS373

Page 35: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

R.E. languages are closed under both inverse homomorphisms andhomomorphisms.

Proof.

Let TM M1 recognize L1.

A TM to recognize h−1(L1):On input x , compute h(x) andrun M1 on h(x); accept iff M1 accepts.

A TM to recognize h(L1): On input x , start going through allstrings w , and if h(w) = x , start executing M1 on w , usingdovetailing to interleave with other executions of M1.

Acceptif any of the executions accepts. �

Agha-Viswanathan CS373

Page 36: CS 373: Theory of Computation

Closure PropertiesDecidable LanguagesRecursively Enumerable Languages

Homomorphisms

Proposition

R.E. languages are closed under both inverse homomorphisms andhomomorphisms.

Proof.

Let TM M1 recognize L1.

A TM to recognize h−1(L1):On input x , compute h(x) andrun M1 on h(x); accept iff M1 accepts.

A TM to recognize h(L1): On input x , start going through allstrings w , and if h(w) = x , start executing M1 on w , usingdovetailing to interleave with other executions of M1. Acceptif any of the executions accepts. �

Agha-Viswanathan CS373

Page 37: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Part II

Grammars

Agha-Viswanathan CS373

Page 38: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Parenthesis Matching

Problem: Describe the the set of arithmetic expressions withcorrectly matched parenthesis.

Solution: Ignoring numbers and variables, and focussing only onparenthesis, correctly matched expressions can be defined as

The ε is a valid expression

A valid string (6= ε) must either be

The concatenation of two correctly matched expressions, orIt must begin with ( and end with ) and moreover, once thefirst and last symbols are removed, the resulting string mustcorrespond to a valid expression.

Agha-Viswanathan CS373

Page 39: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Parenthesis Matching

Problem: Describe the the set of arithmetic expressions withcorrectly matched parenthesis.Solution: Ignoring numbers and variables, and focussing only onparenthesis, correctly matched expressions can be defined as

The ε is a valid expression

A valid string (6= ε) must either be

The concatenation of two correctly matched expressions, orIt must begin with ( and end with ) and moreover, once thefirst and last symbols are removed, the resulting string mustcorrespond to a valid expression.

Agha-Viswanathan CS373

Page 40: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Parenthesis Matching

Problem: Describe the the set of arithmetic expressions withcorrectly matched parenthesis.Solution: Ignoring numbers and variables, and focussing only onparenthesis, correctly matched expressions can be defined as

The ε is a valid expression

A valid string (6= ε) must either be

The concatenation of two correctly matched expressions, orIt must begin with ( and end with ) and moreover, once thefirst and last symbols are removed, the resulting string mustcorrespond to a valid expression.

Agha-Viswanathan CS373

Page 41: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Parenthesis Matching

Problem: Describe the the set of arithmetic expressions withcorrectly matched parenthesis.Solution: Ignoring numbers and variables, and focussing only onparenthesis, correctly matched expressions can be defined as

The ε is a valid expression

A valid string (6= ε) must either be

The concatenation of two correctly matched expressions, orIt must begin with ( and end with ) and moreover, once thefirst and last symbols are removed, the resulting string mustcorrespond to a valid expression.

Agha-Viswanathan CS373

Page 42: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Parenthesis Matching

Problem: Describe the the set of arithmetic expressions withcorrectly matched parenthesis.Solution: Ignoring numbers and variables, and focussing only onparenthesis, correctly matched expressions can be defined as

The ε is a valid expression

A valid string (6= ε) must either be

The concatenation of two correctly matched expressions, or

It must begin with ( and end with ) and moreover, once thefirst and last symbols are removed, the resulting string mustcorrespond to a valid expression.

Agha-Viswanathan CS373

Page 43: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Parenthesis Matching

Problem: Describe the the set of arithmetic expressions withcorrectly matched parenthesis.Solution: Ignoring numbers and variables, and focussing only onparenthesis, correctly matched expressions can be defined as

The ε is a valid expression

A valid string (6= ε) must either be

The concatenation of two correctly matched expressions, orIt must begin with ( and end with ) and moreover, once thefirst and last symbols are removed, the resulting string mustcorrespond to a valid expression.

Agha-Viswanathan CS373

Page 44: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Parenthesis MatchingGrammar

Taking E to be the set of correct expressions, the inductivedefinition can be succinctly written as

E → εE → EEE → (E )

Agha-Viswanathan CS373

Page 45: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

English Sentences

English sentences can be described as

〈S〉 → 〈NP〉〈VP〉〈NP〉 → 〈CN〉 | 〈CN〉〈PP〉〈VP〉 → 〈CV 〉 | 〈CV 〉〈PP〉〈PP〉 → 〈P〉〈CN〉〈CN〉 → 〈A〉〈N〉〈CV 〉 → 〈V 〉 | 〈V 〉〈NP〉〈A〉 → a | the〈N〉 → boy | girl | flower〈V 〉 → touches | likes | sees〈P〉 → with

Agha-Viswanathan CS373

Page 46: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

English SentencesExamples

noun-phrs︷ ︸︸ ︷a︸︷︷︸

article

boy︸︷︷︸noun

verb-phrs︷︸︸︷sees︸︷︷︸verb

noun-phrs︷ ︸︸ ︷the︸︷︷︸

article

boy︸︷︷︸noun

verb-phrs︷ ︸︸ ︷sees︸︷︷︸verb

a flower︸ ︷︷ ︸noun-phrs

Agha-Viswanathan CS373

Page 47: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

English SentencesExamples

noun-phrs︷ ︸︸ ︷a︸︷︷︸

article

boy︸︷︷︸noun

verb-phrs︷︸︸︷sees︸︷︷︸verb

noun-phrs︷ ︸︸ ︷the︸︷︷︸

article

boy︸︷︷︸noun

verb-phrs︷ ︸︸ ︷sees︸︷︷︸verb

a flower︸ ︷︷ ︸noun-phrs

Agha-Viswanathan CS373

Page 48: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Applications

Such rules (or grammars) play a key role in

Parsing programming languages and natural languages

Markup Languages like HTML and XML.

Modelling software

Agha-Viswanathan CS373

Page 49: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Grammars

Definition

A grammar is G = (V ,Σ,R,S) where

V is a finite set of variables also called nonterminals orsyntactic categories. Each variable represents a language.

Σ is a finite set of symbols, disjoint from V , called terminals,that form the strings of the language.

R is a finite set of rules or productions. Each production is ofthe form α→ β where α, β ∈ (V ∪ Σ)∗

S ∈ V is the start symbol; it is the variable that representsthe language being defined. Other variables represent auxiliarylanguages that are used to define the language of the startsymbol.

Agha-Viswanathan CS373

Page 50: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Grammars

Definition

A grammar is G = (V ,Σ,R,S) where

V is a finite set of variables also called nonterminals orsyntactic categories. Each variable represents a language.

Σ is a finite set of symbols, disjoint from V , called terminals,that form the strings of the language.

R is a finite set of rules or productions. Each production is ofthe form α→ β where α, β ∈ (V ∪ Σ)∗

S ∈ V is the start symbol; it is the variable that representsthe language being defined. Other variables represent auxiliarylanguages that are used to define the language of the startsymbol.

Agha-Viswanathan CS373

Page 51: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Grammars

Definition

A grammar is G = (V ,Σ,R,S) where

V is a finite set of variables also called nonterminals orsyntactic categories. Each variable represents a language.

Σ is a finite set of symbols, disjoint from V , called terminals,that form the strings of the language.

R is a finite set of rules or productions. Each production is ofthe form α→ β where α, β ∈ (V ∪ Σ)∗

S ∈ V is the start symbol; it is the variable that representsthe language being defined. Other variables represent auxiliarylanguages that are used to define the language of the startsymbol.

Agha-Viswanathan CS373

Page 52: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Grammars

Definition

A grammar is G = (V ,Σ,R,S) where

V is a finite set of variables also called nonterminals orsyntactic categories. Each variable represents a language.

Σ is a finite set of symbols, disjoint from V , called terminals,that form the strings of the language.

R is a finite set of rules or productions. Each production is ofthe form α→ β where α, β ∈ (V ∪ Σ)∗

S ∈ V is the start symbol; it is the variable that representsthe language being defined. Other variables represent auxiliarylanguages that are used to define the language of the startsymbol.

Agha-Viswanathan CS373

Page 53: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Grammars

Definition

A grammar is G = (V ,Σ,R,S) where

V is a finite set of variables also called nonterminals orsyntactic categories. Each variable represents a language.

Σ is a finite set of symbols, disjoint from V , called terminals,that form the strings of the language.

R is a finite set of rules or productions. Each production is ofthe form α→ β where α, β ∈ (V ∪ Σ)∗

S ∈ V is the start symbol; it is the variable that representsthe language being defined. Other variables represent auxiliarylanguages that are used to define the language of the startsymbol.

Agha-Viswanathan CS373

Page 54: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Example of a CFG

Example

Let Gpar = (V ,Σ,R,S) be

V = {E}Σ = {(, )}R = {E → ε,E → EE ,E → (E )}S = E

Agha-Viswanathan CS373

Page 55: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Palindromes

Example

A string w is a palindrome if w = wR .

For example,madaminedenimadam(“Madam, in Eden, I’m Adam”)Gpal = ({S}, {0, 1},R,S) defines palindromes over {0, 1}, where Ris

S → εS → 0S → 1S → 0S0S → 1S1

Or more briefly, R = {S → ε | 0 | 1 | 0S0 | 1S1}

Agha-Viswanathan CS373

Page 56: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Palindromes

Example

A string w is a palindrome if w = wR . For example,madaminedenimadam(“Madam, in Eden, I’m Adam”)

Gpal = ({S}, {0, 1},R,S) defines palindromes over {0, 1}, where Ris

S → εS → 0S → 1S → 0S0S → 1S1

Or more briefly, R = {S → ε | 0 | 1 | 0S0 | 1S1}

Agha-Viswanathan CS373

Page 57: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Palindromes

Example

A string w is a palindrome if w = wR . For example,madaminedenimadam(“Madam, in Eden, I’m Adam”)Gpal = ({S}, {0, 1},R,S) defines palindromes over {0, 1}, where Ris

S → εS → 0S → 1S → 0S0S → 1S1

Or more briefly, R = {S → ε | 0 | 1 | 0S0 | 1S1}

Agha-Viswanathan CS373

Page 58: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Palindromes

Example

A string w is a palindrome if w = wR . For example,madaminedenimadam(“Madam, in Eden, I’m Adam”)Gpal = ({S}, {0, 1},R,S) defines palindromes over {0, 1}, where Ris

S → εS → 0S → 1S → 0S0S → 1S1

Or more briefly, R = {S → ε | 0 | 1 | 0S0 | 1S1}

Agha-Viswanathan CS373

Page 59: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Arithmetic Expressions

Consider the language of all arithmetic expressions (E ) built out ofintegers (N) and identifiers (I ), using only + and ∗

Gexp = ({E , I ,N}, {a, b, 0, 1, (, ),+, ∗,−},R,E ) where R is

E → I | N | E + E | E ∗ E | (E )I → a | b | Ia | IbN → 0 | 1 | N0 | N1 | − N | + N

Agha-Viswanathan CS373

Page 60: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Arithmetic Expressions

Consider the language of all arithmetic expressions (E ) built out ofintegers (N) and identifiers (I ), using only + and ∗Gexp = ({E , I ,N}, {a, b, 0, 1, (, ),+, ∗,−},R,E ) where R is

E → I | N | E + E | E ∗ E | (E )I → a | b | Ia | IbN → 0 | 1 | N0 | N1 | − N | + N

Agha-Viswanathan CS373

Page 61: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

More Examples

Example

Consider the grammar G with Σ = {a, b, c}, V = {S ,B,C ,H} and

S → aSBC | aBC CB → HB HB → HCHC → BC aB → ab bB → bbbC → bc cC → cc

Consider the grammar G with Σ = {a} with

S → $Ca# | a | ε Ca→ aaC $D → $CC#→ D# | E aD → Da aE → Ea$E → ε

Agha-Viswanathan CS373

Page 62: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

More Examples

Example

Consider the grammar G with Σ = {a, b, c}, V = {S ,B,C ,H} and

S → aSBC | aBC CB → HB HB → HCHC → BC aB → ab bB → bbbC → bc cC → cc

Consider the grammar G with Σ = {a} with

S → $Ca# | a | ε Ca→ aaC $D → $CC#→ D# | E aD → Da aE → Ea$E → ε

Agha-Viswanathan CS373

Page 63: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Derivation

Expand the start symbol using one of its rules. Then expand theresulting string by replacing one of its substrings that matches theLHS of a rule by the RHS. Repeat until you get a string ofterminals.

For the rules

E → I | N | E + E | E ∗ E | (E )I → a | b | Ia | IbN → 0 | 1 | N0 | N1 | − N | + N

we have

E ⇒ E ∗ E ⇒ E ∗ N ⇒ E ∗ −N ⇒ E ∗ −N ⇒ E ∗ −1⇒ (E ) ∗ −1⇒ (E + E ) ∗ −1⇒ (E + I ) ∗ −1⇒ (E + a) ∗ −1⇒ (I + a) ∗ −1⇒ (a + a) ∗ −1

Agha-Viswanathan CS373

Page 64: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Derivation

Expand the start symbol using one of its rules. Then expand theresulting string by replacing one of its substrings that matches theLHS of a rule by the RHS. Repeat until you get a string ofterminals.For the rules

E → I | N | E + E | E ∗ E | (E )I → a | b | Ia | IbN → 0 | 1 | N0 | N1 | − N | + N

we have

E ⇒ E ∗ E ⇒ E ∗ N ⇒ E ∗ −N ⇒ E ∗ −N ⇒ E ∗ −1⇒ (E ) ∗ −1⇒ (E + E ) ∗ −1⇒ (E + I ) ∗ −1⇒ (E + a) ∗ −1⇒ (I + a) ∗ −1⇒ (a + a) ∗ −1

Agha-Viswanathan CS373

Page 65: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Derivation

Expand the start symbol using one of its rules. Then expand theresulting string by replacing one of its substrings that matches theLHS of a rule by the RHS. Repeat until you get a string ofterminals.For the rules

E → I | N | E + E | E ∗ E | (E )I → a | b | Ia | IbN → 0 | 1 | N0 | N1 | − N | + N

we have

E ⇒ E ∗ E ⇒ E ∗ N ⇒ E ∗ −N ⇒ E ∗ −N ⇒ E ∗ −1⇒ (E ) ∗ −1⇒ (E + E ) ∗ −1⇒ (E + I ) ∗ −1⇒ (E + a) ∗ −1⇒ (I + a) ∗ −1⇒ (a + a) ∗ −1

Agha-Viswanathan CS373

Page 66: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Formal Definition

Definition

Let G = (V ,Σ,R,S) be a grammar. We say γ1αγ2 ⇒G γ1βγ1,where γ1, γ2α, β ∈ (V ∪ Σ)∗ if α→ β is a rule of G .

We say α∗⇒G β if either α = β or there are α0, α1, . . . αn such that

α = α0 ⇒G α1 ⇒G α2 ⇒G · · · ⇒G αn = β

Notation

When G is clear from the context, we will write ⇒ and∗⇒ instead

of ⇒G and∗⇒G .

Agha-Viswanathan CS373

Page 67: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Formal Definition

Definition

Let G = (V ,Σ,R,S) be a grammar. We say γ1αγ2 ⇒G γ1βγ1,where γ1, γ2α, β ∈ (V ∪ Σ)∗ if α→ β is a rule of G .

We say α∗⇒G β if either α = β or there are α0, α1, . . . αn such that

α = α0 ⇒G α1 ⇒G α2 ⇒G · · · ⇒G αn = β

Notation

When G is clear from the context, we will write ⇒ and∗⇒ instead

of ⇒G and∗⇒G .

Agha-Viswanathan CS373

Page 68: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Formal Definition

Definition

Let G = (V ,Σ,R,S) be a grammar. We say γ1αγ2 ⇒G γ1βγ1,where γ1, γ2α, β ∈ (V ∪ Σ)∗ if α→ β is a rule of G .

We say α∗⇒G β if either α = β or there are α0, α1, . . . αn such that

α = α0 ⇒G α1 ⇒G α2 ⇒G · · · ⇒G αn = β

Notation

When G is clear from the context, we will write ⇒ and∗⇒ instead

of ⇒G and∗⇒G .

Agha-Viswanathan CS373

Page 69: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Language of a Grammar

Definition

The language of a grammar G = (V ,Σ,R,S), denoted L(G ) is thecollection of strings over the terminals derivable from S using therules in R.

In other words,

L(G ) = {w ∈ Σ∗ | S ∗⇒ w}

Agha-Viswanathan CS373

Page 70: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

GrammarExamplesLanguage of a Grammar

Language of a Grammar

Definition

The language of a grammar G = (V ,Σ,R,S), denoted L(G ) is thecollection of strings over the terminals derivable from S using therules in R. In other words,

L(G ) = {w ∈ Σ∗ | S ∗⇒ w}

Agha-Viswanathan CS373

Page 71: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example I

Example

Consider the grammar G with Σ = {a, b, c}, V = {S ,B,C ,H} and

S → aSBC | aBC CB → HB HB → HCHC → BC aB → ab bB → bbbC → bc cC → cc

Some derivations of G are

S ⇒ aBC ⇒ abC ⇒ abc

S ⇒ aSBC ⇒ aaSBCBC ⇒ aaaBCBCBC ⇒ aaaBHBCBC ⇒ aaaBHCCBC⇒ aaaBBCCBC ⇒ aaaBBCHBC ⇒ aaaBBCHCC ⇒ aaaBBCBCC⇒ aaaBBHBCC ⇒ aaaBBHCCC ⇒ aaaBBBCCC ⇒ aaabBBCCC⇒ aaabbBCCC ⇒ aaabbbC ⇒ aaabbbcCC ⇒ aaabbbccC ⇒ aaabbbccc

L(G ) = {anbncn | n ≥ 0}

Agha-Viswanathan CS373

Page 72: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example I

Example

Consider the grammar G with Σ = {a, b, c}, V = {S ,B,C ,H} and

S → aSBC | aBC CB → HB HB → HCHC → BC aB → ab bB → bbbC → bc cC → cc

Some derivations of G are

S ⇒ aBC ⇒ abC ⇒ abc

S ⇒ aSBC ⇒ aaSBCBC ⇒ aaaBCBCBC ⇒ aaaBHBCBC ⇒ aaaBHCCBC⇒ aaaBBCCBC ⇒ aaaBBCHBC ⇒ aaaBBCHCC ⇒ aaaBBCBCC⇒ aaaBBHBCC ⇒ aaaBBHCCC ⇒ aaaBBBCCC ⇒ aaabBBCCC⇒ aaabbBCCC ⇒ aaabbbC ⇒ aaabbbcCC ⇒ aaabbbccC ⇒ aaabbbccc

L(G ) = {anbncn | n ≥ 0}

Agha-Viswanathan CS373

Page 73: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example I

Example

Consider the grammar G with Σ = {a, b, c}, V = {S ,B,C ,H} and

S → aSBC | aBC CB → HB HB → HCHC → BC aB → ab bB → bbbC → bc cC → cc

Some derivations of G are

S ⇒ aBC ⇒ abC ⇒ abc

S ⇒ aSBC ⇒ aaSBCBC ⇒ aaaBCBCBC ⇒ aaaBHBCBC ⇒ aaaBHCCBC⇒ aaaBBCCBC ⇒ aaaBBCHBC ⇒ aaaBBCHCC ⇒ aaaBBCBCC⇒ aaaBBHBCC ⇒ aaaBBHCCC ⇒ aaaBBBCCC ⇒ aaabBBCCC⇒ aaabbBCCC ⇒ aaabbbC ⇒ aaabbbcCC ⇒ aaabbbccC ⇒ aaabbbccc

L(G ) = {anbncn | n ≥ 0}

Agha-Viswanathan CS373

Page 74: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example I

Example

Consider the grammar G with Σ = {a, b, c}, V = {S ,B,C ,H} and

S → aSBC | aBC CB → HB HB → HCHC → BC aB → ab bB → bbbC → bc cC → cc

Some derivations of G are

S ⇒ aBC ⇒ abC ⇒ abc

S ⇒ aSBC ⇒ aaSBCBC ⇒ aaaBCBCBC ⇒ aaaBHBCBC ⇒ aaaBHCCBC⇒ aaaBBCCBC ⇒ aaaBBCHBC ⇒ aaaBBCHCC ⇒ aaaBBCBCC⇒ aaaBBHBCC ⇒ aaaBBHCCC ⇒ aaaBBBCCC ⇒ aaabBBCCC⇒ aaabbBCCC ⇒ aaabbbC ⇒ aaabbbcCC ⇒ aaabbbccC ⇒ aaabbbccc

L(G ) = {anbncn | n ≥ 0}

Agha-Viswanathan CS373

Page 75: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example II

Example

Consider the grammar G with Σ = {a} with

S → $Ca# | a | ε Ca→ aaC $D → $CC#→ D# | E aD → Da aE → Ea$E → ε

The following are derivations in this grammar

S ⇒ $Ca#⇒ $aaC#⇒ $aaE ⇒ $aEa⇒ $Eaa⇒ aa

S ⇒ $Ca#⇒ $aaC#⇒ $aaD#⇒ $aDa#⇒ $Daa#⇒ $Caa#⇒ $aaCa#⇒ $aaaaC#⇒ $aaaaE ⇒ $aaaEa⇒ $aaEaa⇒ $aEaaa⇒ $Eaaaa⇒ aaaa

L(G ) = {ai | i is a power of 2}

Agha-Viswanathan CS373

Page 76: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example II

Example

Consider the grammar G with Σ = {a} with

S → $Ca# | a | ε Ca→ aaC $D → $CC#→ D# | E aD → Da aE → Ea$E → ε

The following are derivations in this grammar

S ⇒ $Ca#⇒ $aaC#⇒ $aaE ⇒ $aEa⇒ $Eaa⇒ aa

S ⇒ $Ca#⇒ $aaC#⇒ $aaD#⇒ $aDa#⇒ $Daa#⇒ $Caa#⇒ $aaCa#⇒ $aaaaC#⇒ $aaaaE ⇒ $aaaEa⇒ $aaEaa⇒ $aEaaa⇒ $Eaaaa⇒ aaaa

L(G ) = {ai | i is a power of 2}

Agha-Viswanathan CS373

Page 77: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example II

Example

Consider the grammar G with Σ = {a} with

S → $Ca# | a | ε Ca→ aaC $D → $CC#→ D# | E aD → Da aE → Ea$E → ε

The following are derivations in this grammar

S ⇒ $Ca#⇒ $aaC#⇒ $aaE ⇒ $aEa⇒ $Eaa⇒ aa

S ⇒ $Ca#⇒ $aaC#⇒ $aaD#⇒ $aDa#⇒ $Daa#⇒ $Caa#

⇒ $aaCa#⇒ $aaaaC#⇒ $aaaaE ⇒ $aaaEa⇒ $aaEaa⇒ $aEaaa⇒ $Eaaaa⇒ aaaa

L(G ) = {ai | i is a power of 2}

Agha-Viswanathan CS373

Page 78: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example II

Example

Consider the grammar G with Σ = {a} with

S → $Ca# | a | ε Ca→ aaC $D → $CC#→ D# | E aD → Da aE → Ea$E → ε

The following are derivations in this grammar

S ⇒ $Ca#⇒ $aaC#⇒ $aaE ⇒ $aEa⇒ $Eaa⇒ aa

S ⇒ $Ca#⇒ $aaC#⇒ $aaD#⇒ $aDa#⇒ $Daa#⇒ $Caa#⇒ $aaCa#⇒ $aaaaC#⇒ $aaaaE ⇒ $aaaEa⇒ $aaEaa

⇒ $aEaaa⇒ $Eaaaa⇒ aaaa

L(G ) = {ai | i is a power of 2}

Agha-Viswanathan CS373

Page 79: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example II

Example

Consider the grammar G with Σ = {a} with

S → $Ca# | a | ε Ca→ aaC $D → $CC#→ D# | E aD → Da aE → Ea$E → ε

The following are derivations in this grammar

S ⇒ $Ca#⇒ $aaC#⇒ $aaE ⇒ $aEa⇒ $Eaa⇒ aa

S ⇒ $Ca#⇒ $aaC#⇒ $aaD#⇒ $aDa#⇒ $Daa#⇒ $Caa#⇒ $aaCa#⇒ $aaaaC#⇒ $aaaaE ⇒ $aaaEa⇒ $aaEaa⇒ $aEaaa⇒ $Eaaaa⇒ aaaa

L(G ) = {ai | i is a power of 2}

Agha-Viswanathan CS373

Page 80: CS 373: Theory of Computation

IntroductionFormal Definition

Examples

Example II

Example

Consider the grammar G with Σ = {a} with

S → $Ca# | a | ε Ca→ aaC $D → $CC#→ D# | E aD → Da aE → Ea$E → ε

The following are derivations in this grammar

S ⇒ $Ca#⇒ $aaC#⇒ $aaE ⇒ $aEa⇒ $Eaa⇒ aa

S ⇒ $Ca#⇒ $aaC#⇒ $aaD#⇒ $aDa#⇒ $Daa#⇒ $Caa#⇒ $aaCa#⇒ $aaaaC#⇒ $aaaaE ⇒ $aaaEa⇒ $aaEaa⇒ $aEaaa⇒ $Eaaaa⇒ aaaa

L(G ) = {ai | i is a power of 2}Agha-Viswanathan CS373