Computational Models - Lecture 5 1 Handout Mode Iftach Haitner and Yishay Mansour. Tel Aviv University. March 17/19, 2014 1 Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 1 / 55
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Computational Models - Lecture 5 1
Handout Mode
Iftach Haitner and Yishay Mansour.
Tel Aviv University.
March 17/19, 2014
1Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 1 / 55
Outline
Push Down Automata (PDA)
Closure properties for CFL and testing properties.
Equivalence of CFGs and PDAs
Sipser’s book, 2.1, 2.2 & 2.3
Announcement: Two new recitations, Thursday 12-13 and 14-15.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 2 / 55
Part I
Push-Down Automata
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 3 / 55
String Generators and String Acceptors
Regular expressions are string generators – they tell us how togenerate all strings in a language L
Finite Automata (DFA, NFA) are string acceptors – they tell us if aspecific string w is in L
CFGs are string generators
Are there string acceptors for CFLs?
YES! Push-down automata
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 4 / 55
A Finite Automaton
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 5 / 55
A PushDown Automaton
(ignore the ’$’ sign)Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 6 / 55
Example 1 — PDA for L1 = {0n1n : n ≥ 0}
Informally:
1 Read input symbols1 Push each read 0 on the stack2 Pop a 0 for each read 1
2 Accept if stack is empty after last symbol read, and no 0 appearsafter 1
Recall that L1 is not regular
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 7 / 55
Example 2 — PDA for L2 = {aibjck : i = j ∨ i = k}
Informally:
Read and push a’sEither pop and match with b’s or pop and match with c’s
A non-deterministic choice
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 8 / 55
Pushdown Automaton (PDA) — Formal Definition
A PDA is a 6-tuple (Q,Σ, Γ, δ,q0,F ), where
Q is a finite set called the states,
Σ is a finite set called the input alphabet,
Γ is a finite set called the stack alphabet,
δ : Q × Σε × Γε → P(Q × Γε) is the transition function,2
q0 ∈ Q is the starting state, and
F ⊆ Q is the set of accepting states.
2Xε := X ∪ {ε}.Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 9 / 55
The language accepted by a PDA
A pushdown automaton (PDA) M accepts a string w , if there is a“computation" of M on w (see next slide) that leads to anaccepting state.
The language accepted by M, denoted L(M), is the set of allstrings w ∈ Σ∗ accepted by M.
A (non-deterministic) PDA may have many computations on asingle string
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 10 / 55
Model of ComputationThe following is with respect to M = (Q,Σ, Γ, δ,q0,F ).
Definition 1 ( δ∗)
For w ∈ Σ∗ let δ̂(q,w , s) be all pairs (q′, s′) ∈ Q × Γ∗ for which existw ′
1, . . . ,w′m ∈ Σε, states r1, . . . , rm ∈ Q and strings s0, s1, . . . sm ∈ Γ∗
s.t.:1 w = w ′
1, . . . ,w′m, r0 = q, rm = q′, s0 = s and sm = s′
2 For every i ∈ {0, . . . ,m − 1} exist a,b ∈ Γε and t ∈ Γ∗ s.t.:1 (ri+1, b) ∈ δ(ri ,w ′
i+1, a)2 si = at and si+1 = bt
Namely, (q′, s′) ∈ δ̂(q0,w , ε) if after reading w (possibly within-between ε moves), M can find itself in state q′ and stack values′.
M accepts w ∈ Σ∗ if ∃q′ ∈ F such that (q′, t) ∈ δ̂(q0,w , ε) forsome t .
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 11 / 55
Diagram Notation
When drawing the automata diagram, we use the following notation
Transition a,b → c from state q to q′ means (q′, c) ∈ δ(q,a,b),and informally means the automata
◮ read a from input◮ pop b from stack◮ push c onto stack
Meaning of ε transitions ((informally):◮ a = ε : don’t read input◮ b = ε: don’t pop any symbol◮ c = ε: don’t push any symbol
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 12 / 55
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 13 / 55
A PDA for L1 = {0n1n : n ≥ 0}
We want to show that L(P) = L1 = {0n1n : n ≥ 0}What do we need to prove?
Claim 3
δ̂(q1, ε, ε) = {(q1, ε), (q2, $)}.
δ̂(q1,0k , ε) = {(q2,0k$)}, for k ≥ 1.
δ̂(q1,0k1i , ε) = {(q3,0k−i$)}, for k > i ≥ 1.
δ̂(q1,0k1k , ε) = {(q3, $), (q4, ε)}, for k ≥ 1.
δ̂(q1,w , ε) = ∅, for w 6∈ {0k1i |k ≥ i ≥ 0}.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 14 / 55
Knowing when stack is empty
It is convenient to be able to know when the stack is empty, but there isno built-in mechanism to do that.
Solution1 Start by pushing $ onto stack.2 When you see it again, stack is empty.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 15 / 55
A PDA for L2 = {aibjck : i = j ∨ i = k}
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 16 / 55
A PDA for L2 = {aibjck : i = j ∨ i = k}, cont.
Non-determinism is essential here!
Unlike finite automata, non-determinism does add power.
But we saw deterministic algorithm to deicide any CFL (and as wesee later, CFLs are exactly the languages decided by PDAs)!
How to prove that non-determinism adds power?...Does not seem trivial or immediate.
Another example: L = {xnyn : n ≥ 0} ∪ {xny2n : n ≥ 0} isaccepted by a non-deterministic PDA, but not by a deterministicone. (Proof? Book!)
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 17 / 55
PDA Languages
The Push-Down Automata Languages, LPDA, is the set of alllanguages that can be described by some PDA:
LPDA = {L(M) : M is a PDA}
It is immediate that LPDA ) LDFA: every DFA is just a PDA that ignoresthe stack.
LCFG ⊆ LPDA ?
LPDA ⊆ LCFG ?
LPDA = LCFG !!!
Proof in last hour of class.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 18 / 55
Part II
Closure Properties
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 19 / 55
Simple Closure Properties of Context-Free Languages
CFL’s are closed under◮ Union:S → S1 | S2◮ Concatenation:S → S1S2◮ Star:Snew → ε | Sold | Sold Snew
What about complement and intersection?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 20 / 55
Intersection
S1 → A1B1 S2 → A2B2
A1 → 0A11|ε A2 → 0A2|εB1 → 2B1|ε B2 → 1B22|ε
L1 = 0n1n2∗ L2 = 0∗1n2n
L1 ∩ L2 = 0n1n2n
L1 and L2 are CFLs (why?),
But L1 ∩ L2 is not a CFL.
But can’t we run two PDA’s in parallel, and accept iff bothaccept??
What about intersection of a CFL with a regular language?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 21 / 55
When CFL Intersects Regular Language
Are the context free languages closed under intersection with aregular language?
That is, if L1 is context free languages, and L2 is regular, mustL1 ∩ L2 be context free languages?
Run PDA L1 and DFA L2 “in parallel” (just like the intersection oftwo regular languages).
Formal details omitted (but you should be able to figure them out).
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 22 / 55
Applications
Is L = {w ∈ {0,1,2}∗ : #0(w) = #1(w) = #2(w)} context free?◮ L ∩ 0∗1∗2∗ = {0n1n2n : n ≥ 0} is not context free.◮ 0∗1∗2∗ is regular.◮ Context free languages intersected with a regular languages are
context free.◮ So L is not a context free language
This could also be established using pumping lemma, but proofabove is more elegant.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 23 / 55
Complementation
The fact that CFLs are not closed under intersection, but are closedunder union, implies they are not closed under complementation, asL1 ∩ L2 = L1 ∪ L2.
We give a simple example where L is not CFL but L is.
Take L = {ww : w ∈ {0,1}∗}.
L is not a CFL (why?)
We prove that L is a CFL
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 24 / 55
Complementation cont.
For any y ∈ L, either◮ y ’s length is odd, or◮ y ’s length is even, 2ℓ, and ∃i ≥ 1 such that yi 6= yℓ+i .
It suffice to construct a PDA/CFG for Leven – the even lengthmembers of L (why?)
Let Lσ
even = {{0,1}kσ{0,1}j{0,1}kσ{0,1}j : k , j ≥ 0}
Note that Leven = L0even ∪ L
1even
and that Lσ
even = {{0,1}kσ{0,1}k{0,1}jσ{0,1}j : k , j ≥ 0}
CFG for L0even
S → ABA → CAC | 0B → CBC | 1C → 0 | 1
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 25 / 55
A PDA for L0even
Idea: Guess k , j ≥ 0, and accept w if it is of the form:{0,1}k 0{0,1}k{0,1}j1{0,1}j
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 26 / 55
Homomorphism and Inverse Homomorphism
Homomorphism: replaces each letter with a word
Example: h(1) = aba, h(0) = aah(010) = aa aba aaL1 = {0n1n|n ≥ 0}, h(L1) = {a2n(aba)n|n ≥ 0}.
Claim: Assuming that L is a CFL, then so is h−1(L)
Proof? use the automata (see next)
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 27 / 55
CFL’s are Closed Under Inverse Homomorphism
Idea: let P be a PDA for L. On input word w , emulate P(h(w))
But we cannot afford to store h(w)!
Solution, compute h(w) “on demand":
Algorithm 4
Input w :
1 Initialized a “buffer" Buff to h(a), where a is the first letter of w
2 Emulate a running of P with Buff as its input string.
Each time Buff is fully read by P, set Buff = h(a), where a is the nextletter in w (if exists)
3 Accept iff P does
How do we implement Buff?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 28 / 55
CFL’s are Closed Under Inverse Homomorphism
Given PDA (Q,Σ, Γ, δ,q0,F ) and homomorphism h : Σ1 → Σ∗, wedefine a new PDA (Q′,Σ1, Γ, δ
′,q′0,F
′).
Let k = maxσ1∈Σ1 |h(σ1)| and Σ̄ = ∪0≤i≤kΣi .
Q′ = Q × Σ̄.
q′0 = [q0, ε].
F ′ = F × {ε}.δ′ is define as follows:
◮ if (p, γ) ∈ δ(q, ε,Y ) then ([p, x ], γ) ∈ δ′([q, x ], ε,Y ), for any x ∈ Σ̄.◮ if (p, γ) ∈ δ(q, a,Y ) then ([p, x ], γ) ∈ δ′([q, ax ], ε,Y ), for any x ∈ Σ̄.◮ ([q, h(a)],Y ) ∈ δ′([q, ε], a,Y ), for a ∈ Σ1 and Y ∈ Γ.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 29 / 55
Part III
Algorithmic Questions
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 30 / 55
Emptiness of CFGs
Question 5Given a CFG, G, is L(G) = ∅?
In other words, is there a string generated by G?
Theorem 6There is an algorithm that solves this problem (and always halts).
Possible approaches for a proof:
Bad Idea: We know how to test whether w ∈ L(G) for any stringw , so just try it for each w ...
Better Idea: Can the start variable generate a string of terminals?
A more holistic approach: Can a particular variable generate astring of terminals?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 31 / 55
Removing redundant variables and terminals of a CFG
1 Mark all terminal symbols in G.2 Repeat until no new variable become marked:
Mark any A where A → U1U2 . . .Uk and all Ui have already beenmarked.
3 Remove all unmarked variables, and any rule they appear in.4 If S is removed, then L(G) = ∅.5 Remove any variable A not reachable from S.6 Remove any terminal which does not appear in some rule.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 32 / 55
Checking Emptiness
Algorithm 7
Input a CFG G1 Remove redundant variables and terminals to get G′.2 Return TRUE (i.e., L(G′) = ∅) iff S is removed
Correctness?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 33 / 55
CFGs Fullness
Question 8Given a CFG G, is L(G) = Σ∗?
We just saw an algorithm to determine, given a CFG G, whetherL(G) = ∅
L(G) = Σ∗ iff L(G) = ∅. Why not modify the algorithm so itdetermines emptiness of the complement?
Unfortunately, CFGs are not closed under complement.
Fact 9There is no algorithm to solve CFG fullness.
We are not prepared to prove this remarkable fact (yet).
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 34 / 55
Finiteness of CFGs
Question 10Given a CFG G, is |L(G)| finite?
1 Remove redundant variables and terminals.2 Turn into a CNF form3 Create a graph C whose nodes are variables and its directed
edges are derivations.4 Return TRUE iff C has a no cycles.
Correctness?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 35 / 55
CFGs Inherent Ambiguity
Question 11Given a CFG G, is L(G) inherently ambiguous?
This means that for any CFG that generates L(G), there is a word inthe language with two different parse trees.
Fact 12There is no algorithm to solve CFG inherent ambiguity.
We will not prove this fact, yet you want to know it to put things incontext.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 36 / 55
When Are Two CFGs equivalent?
Question 13Given two CFG G1 and G2, test if L(G1) = L(G2).Is there an algorithm to solve this problem?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 37 / 55
Part IV
Equivalence Theorem
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 38 / 55
The CFG–PDA Equivalence Theorem
Theorem 14LPDA = LCFG : A language is context free if and only if somepushdown automata accepts it.
This time (unlike the regular expression vs. regular languagestheorem), both the proof “if” part and of the “only if” part are non trivial.
Proof sketch follows.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 39 / 55
CFL =⇒ PDA
Lemma 15
LCFG ⊆ LPDA : If a language is context free, then some PDA accepts it.
Let L be a context-free language, and let G = (V ,Σ,R,S) becontext-free grammar for L
We build a PDA P = (Q,Σ, Γ, δ,q0,F ), such that on input w it“figures out" if there is a derivation of w using G.
Question 16How does P figure out which substitution to make?
Answer: It guesses.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 40 / 55
Simplifying Assumptions
1 In a single move, a PDA can push a whole word (from some fixedset) into the stack (first letter at the top)
Can we justify it?2 When deriving a word form a CFL, we always substitute the left
most variable
Does it change the derived language?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 41 / 55
Informal Description of P
Algorithm 17 ( P)
1 Push S$ on stack2 While top of the stack t is not $:3 If t is variable A,
(non-deterministically) select rule A → α and substitute.4 If t is a terminal a,
read next input and compare; Reject if different.5 Accept if end of input and stack is empty
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 42 / 55
State Diagram for P
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 43 / 55
Example
consider the CFG:S → 0S1|ε.
The related PDA:
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 44 / 55
Claim: L(P) = L(G)
Claim 18
S ∗→ α iff α = α1α2 such that (qloop, α2$) ∈ δ̂(qloop, α1, ε).
Does the above yields that L(P) = L(G)?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 45 / 55
S ∗→ α =⇒ α = α1α2 such that (qloop, α2$) ∈ δ̂(qloop, α1, $)
Proof by induction on the number of derivations steps used to yield α
from S.
0 derivation steps: hence α = S. Since (qloop,S$) ∈ δ̂(qloop, ε, $),the proof follows for α1 = ε and α2 = S.
Assume S ∗→ α in k > 0 derivation steps, and let α′ be the string
derived by the first (k − 1) steps.
By i.h α′ = α′1α
′2 such that (qloop, α
′2$) ∈ δ̂(qloop, α
′1, $)
Write α′2 = w1Aw2 where A is the left most variable in α′
2.
The k ’th derivation step replaces this occurrence of A with a stringt (why?)
It is easy to see that (qloop, tw2$) ∈ δ̂(qloop, α′1w1, $).
To complete the proof take α1 = α′1w1 and α2 = tw2.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 46 / 55
α = α1α2 such that (qloop, α2$) ∈ δ̂(qloop, α1, $) =⇒ S ∗→ α
Proof by induction on the number of steps used by P to process α1.
A single step: α1 = ε and α2 = S$, and the proof follows sinceS ∗→ S.
Assume α1 was processed in k > 1 steps, and let α′1 and α′
2 bethe input string read and the stack value before the last step
Note that (qloop, α′2$) ∈ δ̂(qloop, α
′1, $).
By i.h S ∗→ α′ = α′
1α′2.
In case the k ’th move of P is reading an input character, thenα1α2 = α′
1α′2, and therefore S ∗
→ α1α2
Otherwise, α′1 = α1, α′
2 = Aw and α2 = tw for some ruleA → t ∈ R
Hence S ∗→ α1α2
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 47 / 55
PDA =⇒ CFL
Lemma 19LPDA ⊆ LCFG: If a PDA accepts a language then it is context free.
We prove the lemma by constructing a CFG G for a language Laccepted by a PDA PLet P = (Q,Σ, Γ, δ,q0,F ). We assume wlg. that:
◮ A single accepting state qa ∈ F .◮ P empties the stack before accepting◮ Each transition either pops or pushes
Can we justify the above?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 48 / 55
Defining G = (V ,Σ,R,S)
V = {Apq : p,q ∈ Q}
Idea: Apq will generate all strings that take P from p with an emptystack, to q with an empty stack
S = Aq0,qa
Initially R = ∅ and1 Add {Apq → Ap,r Ar ,q : p, q, r ∈ Q} to R2 Add {Aqq → ε : q ∈ Q} to R3 For all p, r , s, q ∈ Q, a, b ∈ Σε and t ∈ Γ such that
1 (r , t) ∈ δ(p, a, ε) and2 (q, ε) ∈ δ(s, b, t)
add Apq → aAr ,sb to R
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 49 / 55
Example PDA to CFG
Aq1,q4 → Aq2,q3
Aq2,q3 → 0Aq2,q31Aq2,q3 → 0Aq2,q21.Aq2,q2 → ε.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 50 / 55
PDA Computation corresponding to Apq → Ap,rAr ,q
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 51 / 55
PDA Computation corresponding to Apq → aAr ,sb
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 52 / 55
Claim: L(G) = L(P)
Claim 20
Apq∗→ w ∈ Σ∗ iff (q, ε) ∈ δ̂(p,w , ε)
Proof by induction on the number of derivation rules/ transitions
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 53 / 55
A Short Summary
Regular Languages ≡ Finite Automata.
Context Free Languages ≡ Push Down Automata.
Closure properties of regular languages and of CFLs.
Most algorithmic problems for finite automata are solvable.
Some algorithmic problems for finite automata are not solvable.
Pumping lemmata for both classes of languages.
There are additional languages out there.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 54 / 55
The View Over The Horizon
context free
regular
decidable
enumerable
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 March 17/19, 2014 55 / 55