Computational Models - Lecture 7 1 Iftach Haitner and Yishay Mansour. Tel Aviv University. May 7/9, 2012 1 Based on slides by Benny Chor, Tel Aviv University, modifying slides by Maurice Herlihy, Brown University. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 1 / 56
57
Embed
Iftach Haitner and Yishay Mansour.tau-cm2012.wdfiles.com/local--files/course-schedule/Blecture7.pdf · Computational Models - Lecture 71 Iftach Haitner and Yishay Mansour. Tel Aviv
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Computational Models - Lecture 7 1
Iftach Haitner and Yishay Mansour.
Tel Aviv University.
May 7/9, 2012
1Based on slides by Benny Chor, Tel Aviv University, modifying slides by MauriceHerlihy, Brown University.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 1 / 56
Talk Outline
Church-Turing Thesis
Enumerators
Decidability vs. Enumerability
Encoding of Turing Machines and Universal Turing Machines
The Halting/Acceptance problem
Beyond Enumerable and co-Enumerable
Sipser’s book, 3.2, 3.3, 4.1, 4.2
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 2 / 56
Part I
Church-Turing Thesis
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 3 / 56
What is an Algorithm?
Informally:1 A recipe
2 A procedure3 A computer program4 Who cares? I know it when I see it :-(5 The notion of algorithm has long history in Mathematics (starting
with Euclid’s gcd algorithm), but not precisely defined until 20’thcentury
Informal notions rarely questioned, still they were insufficient
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 4 / 56
Computation Model
Many models have been proposed for general-purposecomputation. Remarkably, all “reasonable” models were shown tobe equivalent to Turing machines.
The notion of an algorithm is model-independent!
We don’t really care about Turing machines per se.
We do care about understanding computation, and because oftheir simplicity, Turing machines are a good model to use.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 5 / 56
Models Equivalent to TM
All “reasonable” programming languages (e.g., Java, Pascal, C,Python, Scheme, Mathematica, Maple, Cobol,. . . ).
λ-calculus of Alonzo Church
Turing machines of Alan Turing
Recursive functions of Godel and Kleene
Counter machines of Minsky
Normal algorithms of Markov
Unrestricted grammars
Two stack automata
Random access machines (RAMs)...
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 6 / 56
Church-Turing Thesis
“The intuitive notion of reasonable models of computationequals Turing machine algorithms”.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 7 / 56
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 8 / 56
Part II
Enumerators
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 9 / 56
Enumerators
A language is enumerable, if it is accepted by some Turing Machine.
Question 3But why enumerable?
1 010
aaabba
Definition 4 (enumerator)
A TM M is an enumerator for a language L, if M “outputs" all thestrings in L and nothing else.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 10 / 56
Having Enumerator ⇔ Being in RE
Intuition 5Enumerator is a program with no input, writes to a write-only file, andeach line is interpreted as a word that it outputs.
Theorem 6A language is in RE iff it has an enumerator.
Will show
If E enumerates language L, then some TM M accepts L.
If M accepts L, then some enumerator E enumerates it.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 11 / 56
Having Enumerator =⇒ Being in RE
Claim 7If a TM E enumerates a language L, then some TM M accepts L.
Proof:
Algorithm 8 (TM M)
On input w , run E .
Every time E outputs a string v :
If v = w , accept.
♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 12 / 56
Being in RE =⇒ Having Enumerator
Claim 9If a TM M accepts L, then some enumerator TM E enumerates L.
Proof: Let s1, s2, s3, . . . be a list of all strings in Σ∗ (e.g., strings inlexicographic order).
Algorithm 10 (TM E)
Repeat the following for i = 1, 2, 3, . . .Run M for i steps on each input s1, s2, . . . , si .
For any accepting computation, output the corresponding s.
♣
Note that with this procedure, each output is duplicated infinitely often.
Question 11Can this duplication be avoided?
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 13 / 56
Having “In order" Enumerator ⇔ Being in R
Theorem 12A language L is decidable if and only if there is some enumerator Ethat enumerates L in lexicographic order.
Proof: ? Left as an exercise. ♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 14 / 56
Part III
Decidability vs. Enumerability
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 15 / 56
Decidability vs. Enumerability
RE – the class of enumerable languages
coRE – the class of languages whose complement is enumerable
R – the class of decidable languages.
L ∈ R =⇒ L ∈ REL ∈ R =⇒ L ∈ RThm (next slide): R = RE ∩ coRE
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 16 / 56
Theorem R = RE ∩ coRETheorem 13R = RE ∩ coRE .
Proof:R ⊆ RE ∩ coRE
I L ∈ R =⇒ L ∈ REI L ∈ R =⇒ L ∈ R =⇒ L ∈ RE =⇒ L ∈ coRE
R ⊇ RE ∩ coREFor L ∈ RE ∩ coRE , let M1 be a TM that accepts L, and let M2 be aTM that accepts L.
Algorithm 14 ( M - a decider for L)
Input: w .
I Run both M1 and M2 in “in parallel".
I Accept if M1 accepts
I Reject if M2 accepts
We next show that M decides L.♣Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 17 / 56
M decides L
Claim 15M decides L
Proof:
Every string is in L or in L (of course not in both).
Thus either M1 or M2 accepts the input w .
Since M halts whenever M1 or M2 accepts, M always halts (andhence is a decider).
Moreover, M accepts strings in L and rejects strings in L.
♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 18 / 56
Emulating TM’s In Parallel
Question 16What does it mean to emulate M1,M2 in parallel?
Answer: M has two tapes, one for each machine.
Algorithm 17 (TM M)
Do (forever)
1 Emulate the next step of M1
2 Emulate the next step of M2
3 If this is accepting configuration for some Mi , halt and return i .
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 19 / 56
Revised View of the World of Languages
context free
regular
decidable
enumerable
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 20 / 56
Part IV
Encodings and Universal TM
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 21 / 56
Encodings
Input to a Turing machine is a string of symbols.
We want algorithms that work on graphs, matrices, polynomials,Turing machines, etc.
Need to choose an encoding for objects (can often be done inmany reasonable ways).
Sometimes it is helpful to distinguish between X , the object, and〈X 〉, its encoding.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 22 / 56
Encoding of Turing Machines
Just like any piece of code in any programming language, Turingmachines can be encoded as strings.Such encoding will enable us
To check (by an algorithm) that a given string is a legal encodingof a TM.
To build a universal machine that can read such encoding andemulates the encoded TM on any input string.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 23 / 56
Standard Encoding of Turing Machines
Definition 18 (Encoding of 〈M〉 of a TM M)
Let M = (Q,Σ, Γ, δ, q0, qa, qr ) be a TM. Assume wlg. that
Q = {q1, . . . qm}, where q0, qa and qr are indicated by states q1,q2 and q3 respectively.
Γ = {γ1, . . . , γs}, where 0, 1 and are indicated by γ1, γ2 and γ3.
The directions L and R be indicated by D1 and D2.
To encode M, we only encode the transition function δ. For each ruleδ(qi , γj) = (qk , γ`,Db), we add the string 0i10j10k10`10b.
Different rules are separated by 11.
Fact 19There exists a TM (called universal TM) that on input 〈M,w〉 (encodedby (1100∗100∗100∗100∗100∗)∗111(0 ∪ 1)∗, can check that 〈M〉encoded a TM, and can emulates M(w).
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 24 / 56
The Universal Turing Machine
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 25 / 56
Universal Turing Machines
Algorithm 20 (Universal TM U)
On input (〈M,w〉, where 〈M,w〉 are binary strings separated by 111.
Checks that 〈M,w〉 is a proper encoding of a TM.Emulate M(w)(how?)
I Accept, if M enters its accept stateI Reject, if M enters its reject state
Notice that as a consequence, if M on input w enters an infinite loop,so does U on input 〈M,w〉.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 26 / 56
Universal Turing Machines (2)
The universal machine U obviously has a fixed number of states(100 should do).
Despite this, it can simulate machines M with many more states.
Universal machines inspired the development of stored-programcomputers in the 40s and 50s.
Most of you have seen a universal machine, and have even usedone!
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 27 / 56
Universal Turing Machines (3)
For example, Dr. Scheme (interpreter) is a universal Schememachine.
It accepts a two part input: “Above the line” – the program(corresponding to 〈M〉), and “below the line” the input to run it on(corresponding to w).
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 28 / 56
Part V
The Acceptance & Halting Problems
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 29 / 56
The Acceptance & Halting Problems
Of the most philosophically important theorems of the theory ofcomputation.
Computers (and computation) are not omnipotent – they arelimited in a very fundamental way.Many common problems are unsolvable, e.g., does a program sortan array of integers?
Note that these problems are well defined: both program andspecification are precise mathematical objects.
Hey, proving program ∼= specification should be just like provingthat triangle 1 ∼= triangle 2 . . .
Well, this is not the case!
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 30 / 56
CFG, NFA, DFA RemindersLet ECFG = {〈G〉 | G is a CFG and L(G) = ∅}
We saw that ECFG is a decidable language.
ACFG = {〈M,w〉 | M is a PDA accepting the string w}.Saw that the language ACFG is decidable.
ANFA = {〈M,w〉 | M is an NFA accepting the string w}.
ADFA = {〈M,w〉 | M is a DFA accepting the string w}.
Saw both ANFA and ADFA are also decidable.
What would happen with Turing Machines?
ATM = {〈M,w〉|M is a TM that accepts w}
Theorem 21 (The Acceptance Problem is undecidable)
ATM is undecidable.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 31 / 56
The Acceptance Problem
ATM = {〈M,w〉|M is a TM that accepts w}
Before approaching the proof of undecidability, we first prove
Theorem 22ATM is recursively enumerable (namely in RE).
Proof: The universal machine accepts ATM. ♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 32 / 56
Proving Thm 20
We prove Thm 20 by contradiction.Suppose a TM, H, is a decider forATM. Namely,
H(〈M,w)〉) ={
accepts if M accepts wreject if M does not accept w
We construct a new TM, D, with H as a subroutine.
Algorithm 23 ( D)
On input 〈M〉Run H on input 〈M,M〉.Output the opposite of what H outputs:
I Reject if H accepts, andI Accept if H rejects.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 33 / 56
Proving Thm 20 cont.
What happens if we run D on its own description?
D(〈D〉) ={
reject if D accepts 〈D〉accept if D rejects 〈D〉
Oh, oh...Or, more accurately, a contradiction (to what?) ♣
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 34 / 56
Explaining the contradictionConsider the algorithm D on input < D >.
Algorithm 24 ( D)
On input 〈D〉Run H on input 〈D,D〉.Output the opposite of what H outputs:
I Reject if H accepts (i.e., D accepts < D >), andI Accept if H rejects (i.e., D does not accepts < D >).
Question 25What should be the output of D(< D >)?
Answer:If D accepts < D > then H(< D,D >) accepts, so D(< D >)rejects.If D does not accepts < D > then H(< D,D >) rejects, soD(< D >) accepts.contradiction
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 35 / 56
Self Reference
Don’t be confused by the notion of running a machine on its owndescription!
Actually, you should get used to it.
Notion of self-reference comes up again and again in diverseareas.
This notion of self-reference is the basic idea behind Gödel’srevolutionary result.
Compilers do this all the time . . . .
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 36 / 56
A Non-Enumerable Language
We already saw a non-decidable language: ATM.
Can we do better (i.e., worse)?
We now display a language that isn’t even recursively enumerable. . . .
Corollary 26
If L is not decidable, then either L or L is not enumerable.
Proof: Assume otherwise, then by Thm 12 L is decidable. ♣
Corollary 27
ATM is not Enumerable
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 37 / 56
The Halting Problem
HTM = {〈M,w〉|M is a TM and M halts on input w}
Theorem 28HTM is undecidable.
Proof idea:
Similar to ATM.
Alternatively, by a reduction to ATM.
Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 7 May 7/9, 2012 38 / 56
The World as we (currently) Know It
enumerableco-enumerable decidable
A TMA DFAA TM
???
Question 29Are there any languages in the area marked ??? ?