Computational Models - Lecture 61tau-cm2015b.wdfiles.com/local--files/course-schedule/BLecture6h.pdf · Part I Push-Down Automata Iftach Haitner and Ronitt Rubinfeld (TAU) Computational

Computational Models - Lecture 61

Handout Mode

Iftach Haitner and Ronitt Rubinfeld.

Tel Aviv University.

April 27/29, 2015

1Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University. Also with

modifications of Yishay Mansour.

Iftach Haitner and Ronitt Rubinfeld (TAU) Computational Models Lecture 6 April 27/29, 2015 1 / 51

Outline

I Push Down Automata (PDA)I Closure properties for CFL and testing properties.I Equivalence of CFGs and PDAs

I Sipser’s book, 2.1, 2.2 & 2.3


Part I

Push-Down Automata


Diagram Notation

When drawing the automata diagram, we use the following notationI Transition from state q to state q′ labelled by a,b → c means

(q′, c) ∈ δ(q,a,b),and informally means the automata

I read a from inputI pop b from stackI push c onto stack

I Meaning of ε transitions ((informally):I a = ε : don’t read inputI b = ε: don’t pop any symbolI c = ε: don’t push any symbol


How to define δ̂(q,w , s) contains (q′, s′)?

Given (start) state q, substring w of the input, and s, s′ descriptions ofstrings on a stack:There is a legal way to get from state q with stack contents s to state q′

with stack contents s′ by reading from w at each step.


Model of Computation

The following is with respect to M = (Q,Σ, Γ, δ,q0,F ).

Definition 1 (δ∗)

For w ∈ Σ∗ let δ̂(q,w , s) be all pairs (q′, s′) ∈ Q × Γ∗ for which existw ′1, . . . ,w

′m ∈ Σε, states r1, . . . , rm ∈ Q and strings s0, s1, . . . sm ∈ Γ∗

s.t.:1. w = w ′1, . . . ,w

′m, r0 = q, rm = q′, s0 = s and sm = s′

2. For every i ∈ {0, . . . ,m − 1} exist a,b ∈ Γε and t ∈ Γ∗ s.t.:2.1 (ri+1,b) ∈ δ(ri ,w ′i+1,a)2.2 si = at and si+1 = bt

Namely, (q′, s′) ∈ δ̂(q0,w , ε) if after reading w (possibly within-between ε moves), M can find itself in state q′ and stack values′.

I M accepts w ∈ Σ∗ if ∃q′ ∈ F such that (q′, t) ∈ δ̂(q0,w , ε) forsome t .


Knowing when stack is empty

It is convenient to be able to know when the stack is empty, but there isno built-in mechanism to do that.

Solution1. Start by pushing $ onto stack.2. When you see it again, stack is empty.


A PDA for L1 = {0n1n : n ≥ 0}

Claim 20011 ∈ L(P).

Proof: For input read bits w ′, stack contents si , current state ri , takew ′1 = ε w ′2 = 0 w ′3 = 0 w ′4 = 1 w ′5 = 1 w ′6 = ε

s0 = ε s1 = $ s2 = 0$ s3 = 00$ s4 = 0$ s5 = $ s6 = εr0 = q1 r1 = q2 r2 = q2 r3 = q2 r4 = q3 r5 = q3 r6 = q4


A PDA for L1 = {0n1n : n ≥ 0}

We want to show that L(P) = L1 = {0n1n : n ≥ 0}What do we need to prove?

Claim 3

I δ̂(q1, ε, ε) = {(q1, ε), (q2, $)}.I δ̂(q1,0k , ε) = {(q2,0k $)}, for k ≥ 1.I δ̂(q1,0k1i , ε) = {(q3,0k−i$)}, for k > i ≥ 1.I δ̂(q1,0k1k , ε) = {(q3, $), (q4, ε)}, for k ≥ 1.I δ̂(q1,w , ε) = ∅, for w 6∈ {0k1i |k ≥ i ≥ 0}.


A PDA for L2 = {aibjck : i = j ∨ i = k}


A PDA for L2 = {aibjck : i = j ∨ i = k}, cont.

I Non-determinism is essential here!I Unlike finite automata, non-determinism does add power.I But we saw deterministic algorithm to decide membership in any

CFL (and as we see later, CFLs are exactly the languagesdecided by PDAs)!

I How to prove that non-determinism adds power?...

I Does not seem trivial or immediate.I Another example: L = {xnyn : n ≥ 0} ∪ {xny2n : n ≥ 0} is

accepted by a non-deterministic PDA, but not by a deterministicone. (Proof? Book!)


PDA Languages

The Push-Down Automata Languages, LPDA, is the set of alllanguages that can be described by some PDA:

I LPDA = {L(M) : M is a PDA}

It is immediate that LPDA ) LDFA: every DFA is just a PDA that ignoresthe stack.

I LCFG ⊆ LPDA ?I LPDA ⊆ LCFG ?I LPDA = LCFG !!!

Proof in last hour of class.


Part II

Closure Properties


Simple Closure Properties of Context-Free Languages

I CFL’s are closed underI Union:S → S1 | S2I Concatenation:S → S1S2I Star:Snew → ε | Sold | SoldSnew

I What about complement and intersection?


Intersection

S1 → A1B1 S2 → A2B2A1 → 0A11|ε A2 → 0A2|εB1 → 2B1|ε B2 → 1B22|ε

L1 = 0n1n2∗ L2 = 0∗1n2n

I L1 ∩ L2 = 0n1n2n

I L1 and L2 are CFLs (why?),I But L1 ∩ L2 is not a CFL.I But can’t we run two PDA’s in parallel, and accept iff both

accept??I What about intersection of a CFL with a regular language?


When CFL Intersects Regular Language

I Are the context free languages closed under intersection with aregular language?

I That is, if L1 is context free languages, and L2 is regular, mustL1 ∩ L2 be context free languages?

I YES!I Run PDA L1 and DFA L2 “in parallel” (just like the intersection of

two regular languages).I Formal details omitted (but you should be able to figure them out).


Applications

I Is L = {w ∈ {0,1,2}∗ : #0(w) = #1(w) = #2(w)} context free?I L ∩ 0∗1∗2∗ = {0n1n2n : n ≥ 0} is not context free.I 0∗1∗2∗ is regular.I Context free languages intersected with a regular languages are

context free.I So L is not a context free language

I This could also be established using pumping lemma, but proofabove is more elegant.


Complementation

The fact that CFLs are not closed under intersection, but are closedunder union, implies they are not closed under complementation, asL1 ∩ L2 = L1 ∪ L2.

We give a simple example where L is not CFL but L is.

I Take L = {ww : w ∈ {0,1}∗}.I L is not a CFL (why?)I We prove that L is a CFL


Complementation cont.

I For any y ∈ L, eitherI y ’s length is odd, orI y ’s length is even, 2`, and ∃i ≥ 1 such that yi 6= y`+i .

I It suffice to construct a PDA/CFG for Leven – the even lengthmembers of L (why?)

I Let Lσeven = {{0,1}kσ{0,1}j{0,1}kσ{0,1}j : k , j ≥ 0}I Note that Leven = L0

even ∪ L1even

I and that Lσeven = {{0,1}kσ{0,1}k{0,1}jσ{0,1}j : k , j ≥ 0}I CFG for L0

evenS → ABA→ CAC | 0B → CBC | 1C → 0 | 1


A PDA for L0even

Idea: Guess k , j ≥ 0, and accept w if it is of the form:{0,1}k0{0,1}k{0,1}j1{0,1}j


Homomorphism and Inverse Homomorphism

I Homomorphism: replaces each letter with a wordI Example: h(1) = aba, h(0) = aa

h(010) = aa aba aaL1 = {0n1n|n ≥ 0}, h(L1) = {a2n(aba)n|n ≥ 0}.

I Claim: Assuming that L is a CFL, then so is h(L)

I Proof? use the grammarI Inverse homomorphism: h−1(w) = {x : h(x) = w},

h−1(L) = {x : h(x) ∈ L}I Example: L2 = {anbnai |n, i ≥ 0}, h−1(L2) = {10i ,0i |i ≥ 0}.I Claim: Assuming that L is a CFL, then so is h−1(L)

I Proof? use the automata (see next)


CFL’s are Closed Under Inverse Homomorphism

I Idea: let P be a PDA for L. On input word w , emulate P(h(w))

I But we cannot afford to store h(w)!I Solution, compute h(w) “on demand":

Algorithm 4Input w :

1. Initialized a “buffer" Buff to h(a), where a is the first letter of w

2. Emulate a running of P with Buff as its input string.Each time Buff is fully read by P, set Buff = h(a), where a is the nextletter in w (if exists)

3. Accept iff P does

I How do we implement Buff?


CFL’s are Closed Under Inverse Homomorphism

Given PDA (Q,Σ, Γ, δ,q0,F ) and homomorphism h : Σ1 → Σ∗, wedefine a new PDA (Q′,Σ1, Γ, δ

′,q′0,F′).

I Let k = maxσ1∈Σ1 |h(σ1)| and Σ̄ = ∪0≤i≤k Σi .I Q′ = Q × Σ̄.I q′0 = [q0, ε].I F ′ = F × {ε}.I δ′ is define as follows:

I if (p, γ) ∈ δ(q, ε,Y ) then ([p, x ], γ) ∈ δ′([q, x ], ε,Y ), for any x ∈ Σ̄.I if (p, γ) ∈ δ(q,a,Y ) then ([p, x ], γ) ∈ δ′([q,ax ], ε,Y ), for any x ∈ Σ̄.I ([q,h(a)],Y ) ∈ δ′([q, ε],a,Y ), for a ∈ Σ1 and Y ∈ Γ.


Part III

Algorithmic Questions


Emptiness of CFGs

Question 5Given a CFG, G, is L(G) = ∅?

In other words, is there a string generated by G?

Theorem 6There is an algorithm that solves this problem (and always halts).

Possible approaches for a proof:I Bad Idea: We know how to test whether w ∈ L(G) for any string

w , so just try it for each w ...I Better Idea: Can the start variable generate a string of terminals?I A more holistic approach: Can a particular variable generate a

string of terminals?


Checking Emptiness

Idea: Mark variables that can produce a string of terminals

1. Mark all terminal symbols in G.2. Repeat until no new variable become marked:

Mark any A where A→ U1U2 . . .Uk and all Ui have already beenmarked.

3. Remove all unmarked variables, and any rule they appear in.4. If S is removed, then L(G) = ∅.

Correctness?


CFGs Fullness

Question 7Given a CFG G, is L(G) = Σ∗?

I We just saw an algorithm to determine, given a CFG G, whetherL(G) = ∅

I L(G) = Σ∗ iff L(G) = ∅. Why not modify the algorithm so itdetermines emptiness of the complement?

I Unfortunately, CFGs are not closed under complement.

Fact 8There is no algorithm to solve CFG fullness.

I We are not prepared to prove this remarkable fact (yet).


Finiteness of CFGs

Question 9Given a CFG G, is |L(G)| finite?

First, a useful subroutine.


Removing redundant variables and terminals

1. Mark all terminal symbols in G.2. Repeat until no new variable become marked:

Mark any A where A→ U1U2 . . .Uk and all Ui have already beenmarked.

3. Remove all unmarked variables, and any rule they appear in.4. If S is removed, then L(G) = ∅.5. Remove any variable A not reachable from S.6. Remove any terminal which does not appear in some rule.


Back to finiteness of CFGs

Question 10Given a CFG G, is |L(G)| finite?

1. Remove redundant variables and terminals.2. Turn into a CNF form3. Create a graph C whose nodes are variables and its directed

edges are derivations.4. Return TRUE iff C has a no cycles.

Correctness?


CFGs Inherent Ambiguity

Question 11Given a CFG G, is L(G) inherently ambiguous?

This means that for any CFG that generates L(G), there is a word inthe language with two different parse trees.

Fact 12There is no algorithm to solve CFG inherent ambiguity.

I We will not prove this fact, yet you want to know it to put things incontext.


When Are Two CFGs equivalent?

Question 13Given two CFG G1 and G2, test if L(G1) = L(G2).Is there an algorithm to solve this problem?


Part IV

Equivalence Theorem


The CFG–PDA Equivalence Theorem

Theorem 14LPDA = LCFG : A language is context free if and only if somepushdown automata accepts it.

This time (unlike the regular expression vs. regular languagestheorem), both the proof “if” part and of the “only if” part are non trivial.

Proof sketch follows.


CFL =⇒ PDA

Lemma 15

LCFG ⊆ LPDA : If a language is context free, then some PDA accepts it.

I Let L be a context-free language, and let G = (V ,Σ,R,S) becontext-free grammar for L

I We build a PDA P = (Q,Σ, Γ, δ,q0,F ), such that on input w it“figures out" if there is a derivation of w using G.

Question 16How does P figure out which substitution to make?

Answer: It guesses.


Simplifying Assumptions

1. In a single move, a PDA can push a whole word (from some fixedset) into the stack (first letter at the top)Can we justify it?

2. When deriving a word from a CFL, we always substitute the leftmost variableDoes it change the derived language?


Informal Description of P

Algorithm 17 (P)1. Push S$ on stack2. While top of the stack t is not $:3. If t is variable A,

(non-deterministically) select rule A→ α and substitute.4. If t is a terminal a,

read next input and compare; Reject if different.5. Accept if end of input and stack is empty


State Diagram for P


Example

consider the CFG:S → 0S1|ε.The related PDA:


Claim: L(P) = L(G)

Claim 18

S ∗→ α iff α = α1α2 such that (qloop, α2$) ∈ δ̂(qloop, α1, $).

Does the above yields that L(P) = L(G)?


S ∗→ α =⇒ α = α1α2 such that (qloop, α2$) ∈ δ̂(qloop, α1,S$)

Proof by induction on the number of derivations steps used to yield αfrom S.

I 1 derivation steps: hence there is a rule S → α. Thus(qloop, α$) ∈ δ̂(qloop, ε,S$), and the proof follows for α1 = ε andα2 = α.

I Assume S ∗→ α in k > 1 derivation steps, and let α′ be the stringderived by the first (k − 1) steps.

I By i.h α′ = α′1α′2 such that (qloop, α

′2$) ∈ δ̂(qloop, α

′1,S$)

I Write α′2 = w1Aw2 where A is the left most variable in α′2.I The k ’th derivation step replaces this occurrence of A with a string

t (why?)I It is easy to see that (qloop, tw2$) ∈ δ̂(qloop, α

′1w1,S$).

I To complete the proof take α1 = α′1w1 and α2 = tw2.


α = α1α2 such that (qloop, α2$) ∈ δ̂(qloop, α1, $) =⇒ S ∗→ α

Proof by induction on the number of steps used by P to process α1.I A single step: α1 = ε and α2 = S$, and the proof follows since

S ∗→ S.I Assume α1 was processed in k > 1 steps, and let α′1 and α′2 be

the input string read and the stack value before the last stepI Note that (qloop, α

′2$) ∈ δ̂(qloop, α

′1, $).

I By i.h S ∗→ α′ = α′1α′2.

I In case the k ’th move of P is reading an input character, thenα1α2 = α′1α

′2, and therefore S ∗→ α1α2

I Otherwise, α′1 = α1, α′2 = Aw and α2 = tw for some ruleA→ t ∈ R

I Hence S ∗→ α1α2


PDA =⇒ CFL

Lemma 19LPDA ⊆ LCFG: If a PDA accepts a language then it is context free.

We prove the lemma by constructing a CFG G for a language Laccepted by a PDA PLet P = (Q,Σ, Γ, δ,q0,F ). We assume wlg. that:

I A single accepting state qa ∈ F .I P empties the stack before acceptingI Each transition either pops or pushes

Can we justify the above?


Proof Idea

I Suppose string x takes P from state p with empty stack to state qwith empty stack:

I First move that touches the stack must be a push, last must be apop.

I In between:I Either stack is empty only at start and finish:I Simulate by Apq → aArsb, where a,b are first and last symbols in

x , r is state that p can reach in a step and s is state that can reachq in a step.

I or stack was empty at some point in between:I Simulate by Apq → Apr Arq where r is intermediate state and P has

empty stack.


Defining G = (V ,Σ,R,S)

I V = {Apq : p,q ∈ Q}Idea: Apq will generate all strings that take P from p with an emptystack, to q with an empty stack

I S = Aq0,qa

I Initially R = ∅ and1. Add {Apq → Ap,r Ar ,q : p,q, r ∈ Q} to R2. Add {Aqq → ε : q ∈ Q} to R3. For all p, r , s,q ∈ Q, a,b ∈ Σε and t ∈ Γ such that

3.1 (r , t) ∈ δ(p, a, ε) and3.2 (q, ε) ∈ δ(s, b, t)

add Apq → aAr ,sb to R


Example PDA to CFG

Aq1,q4 → Aq2,q3

Aq2,q3 → 0Aq2,q31Aq2,q3 → 0Aq2,q21.Aq2,q2 → ε.


PDA Computation corresponding to Apq → Ap,r Ar ,q


PDA Computation corresponding to Apq → aAr ,sb


Claim: L(G) = L(P)

Claim 20

Apq∗→ w ∈ Σ∗ iff (q, ε) ∈ δ̂(p,w , ε)

Proof by induction on the number of derivation rules/ transitions


A Short Summary

I Regular Languages ≡ Finite Automata.I Context Free Languages ≡ Push Down Automata.I Closure properties of regular languages and of CFLs.I Most algorithmic problems for finite automata are solvable.I Some algorithmic problems for finite automata are not solvable.I Pumping lemmata for both classes of languages.I There are additional languages out there.


The View Over The Horizon

context free

regular

decidable

enumerable


Computational Models - Lecture 61tau-cm2015b.wdfiles.com/local--files/course-schedule/BLecture6h.pdf · Part I Push-Down Automata Iftach Haitner and Ronitt Rubinfeld (TAU) Computational

Documents