Computer Language Theory

1

Computer Computer Language TheoryLanguage Theory

Chapter 4: DecidabilityChapter 4: Decidability

2

Limitations of Algorithmic Limitations of Algorithmic SolvabilitySolvability

In this Chapter we investigate the power of In this Chapter we investigate the power of algorithms to solve problemsalgorithms to solve problems Some can be solved algorithmically and some Some can be solved algorithmically and some

cannotcannot Why we study unsolvabilityWhy we study unsolvability

Useful because then can realize that searching for Useful because then can realize that searching for an algorithmic solution is a waste of timean algorithmic solution is a waste of time

Perhaps the problem can be simplifiedPerhaps the problem can be simplified Gain an perspective on computability and its limitsGain an perspective on computability and its limits In my view also related to complexity (Chapter 7)In my view also related to complexity (Chapter 7)

First we study whether there is an algorithmic solution and First we study whether there is an algorithmic solution and then we study whether there is an “efficient” (polynomial-then we study whether there is an “efficient” (polynomial-time) onetime) one

3

Chapter 4.1Chapter 4.1Decidable LanguagesDecidable Languages

4

Decidable LanguagesDecidable Languages We start with problems that are We start with problems that are

decidabledecidable We first look at problems concerning We first look at problems concerning

regular languages and then those for regular languages and then those for context-free languagescontext-free languages

5

Decidable Problems for Regular Decidable Problems for Regular LanguagesLanguages

We give algorithms for testing whether a finite We give algorithms for testing whether a finite automaton accepts a string, whether the automaton accepts a string, whether the language of a finite automaton is empty, and language of a finite automaton is empty, and whether two finite automata are equivalentwhether two finite automata are equivalent

We represent the problems by languages not We represent the problems by languages not finite automatafinite automata Let ALet ADFADFA={(B, w)|B is a DFA that accepts string w}={(B, w)|B is a DFA that accepts string w} The problem of testing whether a DFA B accepts an The problem of testing whether a DFA B accepts an

input w is the same as testing whether (B,w) is a input w is the same as testing whether (B,w) is a member of the language Amember of the language ADFADFA..

Showing that the language is decidable is the same Showing that the language is decidable is the same thing as showing that the computational problem is thing as showing that the computational problem is decidabledecidable

6

AADFADFA is a Decidable is a Decidable LanguageLanguage

Theorem: ATheorem: ADFADFA is a decidable language is a decidable language Proof Idea: Present a TM M that decides AProof Idea: Present a TM M that decides ADFADFA

M = On input (B,w), where B is a DFA and w is a string:M = On input (B,w), where B is a DFA and w is a string:1.1. Simulate B on input wSimulate B on input w2.2. If the simulation ends in an accept state, then accept; else If the simulation ends in an accept state, then accept; else

reject.reject. Background for actual proof:Background for actual proof:

Books description too simple– leads to wrong Books description too simple– leads to wrong understandingunderstanding

The TM M cannot “be” the DFA BThe TM M cannot “be” the DFA B If it could, then it would be simple. Both would have essentially If it could, then it would be simple. Both would have essentially

the same transition functions (TM just needs to move right over the same transition functions (TM just needs to move right over w as each symbol read).w as each symbol read).

7

Outline of ProofOutline of Proof Must take B as input, described as a string, and then Must take B as input, described as a string, and then

simulate itsimulate it This means the algorithm for simulating any DFA must be embodied This means the algorithm for simulating any DFA must be embodied

in the TM’s state transitionsin the TM’s state transitions Think about this. Given a current state and input symbol, scan the Think about this. Given a current state and input symbol, scan the

tape for the encoded transition function and then use that info to tape for the encoded transition function and then use that info to determine new statedetermine new state

The actual proof would describe how a TM simulates a DFAThe actual proof would describe how a TM simulates a DFA Can assume B is represented by its 5 components and then we have Can assume B is represented by its 5 components and then we have

ww Note that the TM must be able to handle Note that the TM must be able to handle anyany DFA, not just this one DFA, not just this one

Keep track of current state and position in w by writing on the tapeKeep track of current state and position in w by writing on the tape Initially current state is q0 and current position is leftmost symbol of wInitially current state is q0 and current position is leftmost symbol of w

The states and position are updated using the transition function The states and position are updated using the transition function δδ TM M’s TM M’s δδ not the same as DFA B’s not the same as DFA B’s δδ

When M finishes processing, accept if in an accept state; else reject. When M finishes processing, accept if in an accept state; else reject. The implementation will make it clear that will complete in finite The implementation will make it clear that will complete in finite time.time.

8

AANFANFA is a Decidable is a Decidable LanguageLanguage

Proof Idea:Proof Idea: Because we have proven decidability for DFAs, Because we have proven decidability for DFAs,

all we need to do is convert the NFA to a DFA.all we need to do is convert the NFA to a DFA. N = On input (B,w) where B is an NFA and w is a stringN = On input (B,w) where B is an NFA and w is a string

1.1. Convert NFA B to an equivalent DFA C, using the Convert NFA B to an equivalent DFA C, using the procedure for conversion given in Theorem 1.39procedure for conversion given in Theorem 1.39

2.2. Run TM M on input (C,w) using the theorem we just provedRun TM M on input (C,w) using the theorem we just proved3.3. If M accepts, then accept; else rejectIf M accepts, then accept; else reject

Running TM M in step 2 means incorporating M Running TM M in step 2 means incorporating M into the design of N as a subprocedureinto the design of N as a subprocedure

Note that these proofs allow the TM to be Note that these proofs allow the TM to be described at the highest of the 3 levels we described at the highest of the 3 levels we discussed in Chapter 3 (and even then, without discussed in Chapter 3 (and even then, without most of the details!). most of the details!).

9

Computing whether a DFA accepts Computing whether a DFA accepts any Stringany String

EEDFADFA is a decidable language is a decidable language Proof:Proof:

A DFA accepts some string iff it is possible to reach the accept state A DFA accepts some string iff it is possible to reach the accept state from the start state. How can we check this?from the start state. How can we check this?

We can use a marking algorithm similar to the one used in Chapter We can use a marking algorithm similar to the one used in Chapter 3.3.

T = On input (A) where A is a DFA:T = On input (A) where A is a DFA:1.1. Mark the start state of AMark the start state of A2.2. Repeat until no new states get marked:Repeat until no new states get marked:

3.3. Mark any state that has a transition coming into it from any state already Mark any state that has a transition coming into it from any state already markedmarked

4.4. If no accept state is marked, accept; otherwise rejectIf no accept state is marked, accept; otherwise reject In my opinion this proof is clearer than most of the previous ones In my opinion this proof is clearer than most of the previous ones

because the pseudo-code above specifies enough details to make it because the pseudo-code above specifies enough details to make it clear to the reader how to implement itclear to the reader how to implement it

One assumption is that all transitions include valid alphabet One assumption is that all transitions include valid alphabet symbols, but since this is part of the definition of a DFA, no need to symbols, but since this is part of the definition of a DFA, no need to worry about this. worry about this.

10

EQEQDFADFA is a Decidable is a Decidable LanguageLanguage

EQEQDFADFA={(A,B)|A and B are DFAs and L(A)=L(B)}={(A,B)|A and B are DFAs and L(A)=L(B)} Proof ideaProof idea

Construct a new DFA C from A and B, where C accepts Construct a new DFA C from A and B, where C accepts only those strings accepted by either A or B but not both.only those strings accepted by either A or B but not both.

If A and B accept the same language then C will accept If A and B accept the same language then C will accept nothing and we can use the previous proof to check for this.nothing and we can use the previous proof to check for this.

So, the proof is:So, the proof is: F = On input (A,B) where A and B are DFAs:F = On input (A,B) where A and B are DFAs:

1.1. Construct DFA C (more on this in a minute …)Construct DFA C (more on this in a minute …)2.2. Run TM T from the proof from last slide on input (C)Run TM T from the proof from last slide on input (C)3.3. If T accepts, then accept. If T rejects, then rejectIf T accepts, then accept. If T rejects, then reject

11

Constructing CConstructing C

L(C) = (L(A) ∩ L(B)’) L(C) = (L(A) ∩ L(B)’) (L(A)’ ∩ L(B)) (L(A)’ ∩ L(B)) We used proofs by construction that regular We used proofs by construction that regular

languages are closed under languages are closed under , ∩ , and , ∩ , and complementcomplement

We can use those constructions to construct a We can use those constructions to construct a FA that accepts L(C)FA that accepts L(C)

L(A)

L(B)

L(C)

Complement symbol

12

AACFGCFG is a Decidable is a Decidable LanguageLanguage

Proof Idea:Proof Idea: For CFG G and string w want to determine For CFG G and string w want to determine

whether G generates w. One idea is to use G to go whether G generates w. One idea is to use G to go through all derivations. This will not work, why?through all derivations. This will not work, why?

Because this method will yield a TM that is a recognizer, Because this method will yield a TM that is a recognizer, not a decider. That is, it could infinite loop and never not a decider. That is, it could infinite loop and never stop (e.g., imagine applying a rule like A →xA repeatedlystop (e.g., imagine applying a rule like A →xA repeatedly

But since we know the length of w, we can exploit this. But since we know the length of w, we can exploit this. How?How?

A string w of length n will have a derivation that uses A string w of length n will have a derivation that uses 2n-1 steps if the CFG is in Chomsky-Normal Form.2n-1 steps if the CFG is in Chomsky-Normal Form.

So first convert to Chomsky-Normal FormSo first convert to Chomsky-Normal Form Then list all derivations of length 2n-1 steps. If any Then list all derivations of length 2n-1 steps. If any

generates w, then accept, else reject.generates w, then accept, else reject. This is a variant of breadth first search, but instead of This is a variant of breadth first search, but instead of

extended the depth 1 at a time we allow it to go 2n-1 at a extended the depth 1 at a time we allow it to go 2n-1 at a time. As long as finite depth extension, we are okaytime. As long as finite depth extension, we are okay

13

EECFGCFG is a Decidable is a Decidable LanguageLanguage

How can you do this? What is the brute How can you do this? What is the brute force approach?force approach? Try all possible strings w. Will this work?Try all possible strings w. Will this work? The number is not bounded, so this would not be The number is not bounded, so this would not be

decidable. It would be TM-recognizable as long decidable. It would be TM-recognizable as long as you work breadth firstas you work breadth first

Instead, think of this as a graph problem where Instead, think of this as a graph problem where you want to know if you can reach a string of you want to know if you can reach a string of terminals from the start stateterminals from the start state

Do you think it is easier to work forward or backwards?Do you think it is easier to work forward or backwards? Answer: backwardsAnswer: backwards

14

EECFGCFG is a Decidable is a Decidable Language (cont)Language (cont)

Proof Idea: Proof Idea: Can the start variable generate a string of Can the start variable generate a string of

terminals?terminals? Determine for each variable if it can generate Determine for each variable if it can generate

any string of terminals and if so, mark itany string of terminals and if so, mark it Keep working backwards so that if the right-Keep working backwards so that if the right-

side of any rule has only marked items, then side of any rule has only marked items, then mark the LHSmark the LHS For example, if XFor example, if X YZ and Y and Z are marked, YZ and Y and Z are marked,

then mark Xthen mark X If you mark S, then done; if nothing else to mark If you mark S, then done; if nothing else to mark

and S not marked, then rejectand S not marked, then reject

15

EQEQCFGCFG is not a Decidable is not a Decidable LanguageLanguage

We cannot reuse the reasoning to We cannot reuse the reasoning to show that EQshow that EQDFADFA is a decidable is a decidable language since CFGs are not closed language since CFGs are not closed under complement and intersectionunder complement and intersection

As it turns out, EQAs it turns out, EQCFGCFG is not is not decidabledecidable

We will learn in Chapter 5 how to We will learn in Chapter 5 how to prove things undecidableprove things undecidable

16

Every Context-Free Language is Every Context-Free Language is DecidableDecidable

Note that a few slides back we showed that a Note that a few slides back we showed that a Context-Free Grammar (CFG) is decidable, not Context-Free Grammar (CFG) is decidable, not a CFLa CFL But since we already know that CFG’s define CFLs But since we already know that CFG’s define CFLs

this is not an issue (i.e., we can ignore PDAs if it is this is not an issue (i.e., we can ignore PDAs if it is easier to prove with a CFG than a PDA).easier to prove with a CFG than a PDA).

The proof in the book is so trivial it may The proof in the book is so trivial it may confuse youconfuse you Essentially it is the same proof we saw beforeEssentially it is the same proof we saw before That is a CFL is decidable because we can build a That is a CFL is decidable because we can build a

TM to decide membership of any stringTM to decide membership of any string This leads us to the following picture of the This leads us to the following picture of the

hierarchy of languageshierarchy of languages

17

Hierarchy of Classes of Hierarchy of Classes of LanguagesLanguages

Regular

Context-Free

Decidable

Turing-recognizable

We proved Regular Context-free since we can convert a FA into a CFG

We just proved that every Context-free language is decidable

From the definitions in Chapter 3 it is clear that every Decidable language is trivially Turing-recognizable. We hinted that not every Turing-recognizable language is Decidable. Next we prove that!

18

Chapter 4.2Chapter 4.2The Halting ProblemThe Halting Problem

19

The Halting ProblemThe Halting Problem One of the most philosophically important One of the most philosophically important

theorems in the theory of computationtheorems in the theory of computation There is a specific problem that is algorithmically There is a specific problem that is algorithmically

unsolvable.unsolvable. In fact, ordinary/practical problems may be unsolvableIn fact, ordinary/practical problems may be unsolvable

Software verificationSoftware verification Given a computer program and a precise specification of what Given a computer program and a precise specification of what

the program is supposed to do (e.g., sort a list of numbers)the program is supposed to do (e.g., sort a list of numbers) Come up with an algorithm to prove the program works as Come up with an algorithm to prove the program works as

requiredrequired This cannot be done!This cannot be done! But wait, can’t we prove a sorting algorithm works?But wait, can’t we prove a sorting algorithm works? Note: the input has two parts: specification and task. The Note: the input has two parts: specification and task. The

proof is not only to prove it works for a specific task, like proof is not only to prove it works for a specific task, like sorting numbers.sorting numbers.

Our first undecidable problem:Our first undecidable problem: Does a TM accept a given input string?Does a TM accept a given input string?

Note: we have shown that a CFL is decidable and a CFG can Note: we have shown that a CFL is decidable and a CFG can be simulated by a TM. This does not yield a contradiction. TMs be simulated by a TM. This does not yield a contradiction. TMs are more expressive than CFGs.are more expressive than CFGs.

20

Halting Problem IIHalting Problem II AATMTM = {(M,w)|M is a TM and M accepts w} = {(M,w)|M is a TM and M accepts w} AATMTM is undecidable is undecidable

It can only be undecidable due to a loop of M on w.It can only be undecidable due to a loop of M on w. If we could determine if it will loop forever, then If we could determine if it will loop forever, then

could reject. Hence Acould reject. Hence ATMTM is often called the halting is often called the halting problem.problem.

As we will show, it is impossible to determine if a TM will As we will show, it is impossible to determine if a TM will always halt (i.e., on every possible input).always halt (i.e., on every possible input).

Note that this is Turing recognizable:Note that this is Turing recognizable: Simulate M on input w and if it accept, then accept; if it ever Simulate M on input w and if it accept, then accept; if it ever

rejects, then rejectrejects, then reject We start with the diagonalization methodWe start with the diagonalization method

21

Diagonalization MethodDiagonalization Method In 1873 mathematician Cantor was concerned In 1873 mathematician Cantor was concerned

with the problem of measuring the sizes of with the problem of measuring the sizes of infinite sets.infinite sets. How can we tell if one infinite set is bigger than How can we tell if one infinite set is bigger than

another or if they are the same size?another or if they are the same size? We cannot use the counting method that we would use for We cannot use the counting method that we would use for

finite sets. Example: how many even integers are there?finite sets. Example: how many even integers are there? What is larger: the set of even integers or the set of all What is larger: the set of even integers or the set of all

strings over {0,1} (which is the set of all integers)strings over {0,1} (which is the set of all integers) Cantor observed that two finite sets have the same Cantor observed that two finite sets have the same

size if each element in one set can be paired with the size if each element in one set can be paired with the element in the otherelement in the other

This can work for infinite setsThis can work for infinite sets

22

Function Property Function Property DefinitionsDefinitions

From basic discrete math (e.g., CS 1100)From basic discrete math (e.g., CS 1100) Given a set A and B and a function Given a set A and B and a function ff from A from A

to Bto B ff is one-to-one if it never maps two elements in A is one-to-one if it never maps two elements in A

to the same element in Bto the same element in B The function The function add-twoadd-two is one-to-one whereas is one-to-one whereas absolute-absolute-valuevalue is not is not

ff is onto if every item in B is reached from some is onto if every item in B is reached from some value in a (i.e., f(a) = b for every b value in a (i.e., f(a) = b for every b B). B).

For example, if A and B are the set of integers, then For example, if A and B are the set of integers, then add-add-twotwo is onto but if A and B are the positive integers, then is onto but if A and B are the positive integers, then it is not onto since b = 1 is never hit.it is not onto since b = 1 is never hit.

A function that is one-to-one and onto has a (one-A function that is one-to-one and onto has a (one-to-one) correspondenceto-one) correspondence

This allows all items in each set to be pairedThis allows all items in each set to be paired

23

An Example of Pairing An Example of Pairing Set ItemsSet Items

Let N be the set of natural numbers {1, 2, 3, Let N be the set of natural numbers {1, 2, 3, …} and let E be the set of even natural …} and let E be the set of even natural numbers {2, 4, 6, …}.numbers {2, 4, 6, …}.

Using Cantor’s definition of size we can see Using Cantor’s definition of size we can see that N and E have the same size.that N and E have the same size. The correspondence f from N to E is f(n) = 2n.The correspondence f from N to E is f(n) = 2n.

This may seem bizarre since E is a proper This may seem bizarre since E is a proper subset of N, but it is possible to pair all items, subset of N, but it is possible to pair all items, since f(n) is a 1:1 correspondence, so we say since f(n) is a 1:1 correspondence, so we say they are the same size.they are the same size.

Definition:Definition: A set is A set is countablecountable if either it is finite or it has the if either it is finite or it has the

same size as N, the set of natural numberssame size as N, the set of natural numbers

24

Example: Rational Example: Rational NumbersNumbers

Let Q = {m/n: m,n Let Q = {m/n: m,n N}, the set of positive N}, the set of positive Rational NumbersRational Numbers

Q seems much larger than N, but according Q seems much larger than N, but according to our definition, they are the same size.to our definition, they are the same size. Here is the 1:1 correspondence between Q and NHere is the 1:1 correspondence between Q and N We need to list all of the elements of Q and then We need to list all of the elements of Q and then

label the first with 1, the second with 2, etc.label the first with 1, the second with 2, etc. We need to make sure each element in Q is listed only We need to make sure each element in Q is listed only

onceonce

25

Correspondence between N Correspondence between N and Qand Q To get our list, we make an infinite matrix To get our list, we make an infinite matrix

containing all the positive rational numbers.containing all the positive rational numbers. Bad way is to make the list by going row-to-row. Since Bad way is to make the list by going row-to-row. Since

11stst row is infinite, would never get to the second row row is infinite, would never get to the second row Instead use the diagonals, not adding the values that Instead use the diagonals, not adding the values that

are equivalentare equivalent So the order is 1/1, 2/1, ½, 3/1, 1/3, …So the order is 1/1, 2/1, ½, 3/1, 1/3, …

This yields a correspondence between Q and NThis yields a correspondence between Q and N That is, N=1 corresponds to 1/1, N=2 corresponds to That is, N=1 corresponds to 1/1, N=2 corresponds to

2/1, N=3 corresponds to ½ etc.2/1, N=3 corresponds to ½ etc.

1/11/1 1/21/2 1/31/3 1/41/4 1/51/52/12/1 2/22/2 2/32/3 2/42/4 2/52/53/13/1 3/23/2 3/33/3 3/43/4 3/53/54/14/1 4/24/2 4/34/3 4/44/4 4/54/55/15/1 5/25/2 5/35/3 5/45/4 5/55/5

26

Theorem: R is Theorem: R is UncountableUncountable

A real number is one that has a decimal A real number is one that has a decimal representation and R is set of Real representation and R is set of Real NumbersNumbers Includes those that cannot be represented Includes those that cannot be represented

with a finite number of digits, like Pi and with a finite number of digits, like Pi and square root of 2square root of 2

Will show that there can be no pairing of Will show that there can be no pairing of elements between R and Nelements between R and N Will find some x that is always not in the Will find some x that is always not in the

pairings and thus a proof by contradictionpairings and thus a proof by contradiction

27

Finding a New Value xFinding a New Value x To the right is an example mappingTo the right is an example mapping

Assume that it is completeAssume that it is complete I now describe a method that will be I now describe a method that will be

guaranteed to generate a value x not already guaranteed to generate a value x not already in the infinite listin the infinite list

Generate x to be a real number between 0 and Generate x to be a real number between 0 and 1 as follows1 as follows To ensure that x ≠ f(1), pick a digit not equal to To ensure that x ≠ f(1), pick a digit not equal to

the first digit after the decimal point. Any value the first digit after the decimal point. Any value not equal to 1 will work. Pick 4 so we have .4not equal to 1 will work. Pick 4 so we have .4

To x ≠ f(2), pick a digit not equal to the second To x ≠ f(2), pick a digit not equal to the second digit. Any value not equal to 5 will work. Pick 6. digit. Any value not equal to 5 will work. Pick 6. We have .46We have .46

Continue, choosing values along the “diagonal” Continue, choosing values along the “diagonal” of digits (i.e., if we took the f(n) column and put of digits (i.e., if we took the f(n) column and put one digit in each column of a new table). one digit in each column of a new table).

When done, we are guaranteed to have a When done, we are guaranteed to have a value x not already in the list since it differs in value x not already in the list since it differs in at least one position with every other number at least one position with every other number in the list.in the list.

nn f(n)f(n)11 3.3.11415415

9…9…22 55.555.55555

5…5…33 0.120.123344

5…5…44 0.5000.50000

0000.. ..

28

ImplicationsImplications The theorem we just proved about R The theorem we just proved about R

being uncountable has an important being uncountable has an important application in the theory of computationapplication in the theory of computation It shows that some languages are not It shows that some languages are not

decidable or even Turing-recognizable, decidable or even Turing-recognizable, because there are uncountably many because there are uncountably many languages yet only countably many Turing languages yet only countably many Turing Machines.Machines.

Because each Turing machine can recognize a Because each Turing machine can recognize a single language and there are more languages single language and there are more languages than Turing machines, some languages are not than Turing machines, some languages are not recognized by any Turing machine.recognized by any Turing machine.

Corollary: some languages are not Turing-recognizableCorollary: some languages are not Turing-recognizable

29

Some Languages are Not Turing-Some Languages are Not Turing-recognizablerecognizable

Proof:Proof: All strings All strings ∑* is countable∑* is countable

With only a finite number of strings of each length, we may form a list of ∑* With only a finite number of strings of each length, we may form a list of ∑* by writing down all strings of length 0, length 1, length 2, etc.by writing down all strings of length 0, length 1, length 2, etc.

The set of all Turing Machines M is countable since each TM M has an The set of all Turing Machines M is countable since each TM M has an encoding into a string <M>encoding into a string <M>

If we simply omit all strings that do not represent valid TM’s, we obtain a list If we simply omit all strings that do not represent valid TM’s, we obtain a list of all Turing Machinesof all Turing Machines

The set of all languages L over ∑ is uncountableThe set of all languages L over ∑ is uncountable the set of all possible binary sequences B is uncountablethe set of all possible binary sequences B is uncountable

The same diagonalization proof we used to prove R is uncountableThe same diagonalization proof we used to prove R is uncountable L is uncountable because it has a correspondence with BL is uncountable because it has a correspondence with B

See book for details, but assuming ∑ = {sSee book for details, but assuming ∑ = {s11, s, s22, …} we can encode any s, …} we can encode any sii as a as a binary sequence. Thus, there is a 1:1 mapping.binary sequence. Thus, there is a 1:1 mapping.

Since B is uncountable and L and B are of equal size, L is uncountableSince B is uncountable and L and B are of equal size, L is uncountable Since the set of TMs is countable and the set of languages is not, we Since the set of TMs is countable and the set of languages is not, we

cannot put the set of languages into a correspondence with the set of cannot put the set of languages into a correspondence with the set of Turing Machines. Thus there exists some languages without a Turing Machines. Thus there exists some languages without a corresponding Turing machinecorresponding Turing machine

30

Halting Problem is Halting Problem is UndecidableUndecidable

Prove that halting problem is Prove that halting problem is undecidableundecidable We started this a while ago …We started this a while ago … Let ALet ATMTM = {<M,w>| M is a TM and accepts = {<M,w>| M is a TM and accepts

w}w} Proof Technique:Proof Technique:

Assume AAssume ATMTM is decidable and obtain a is decidable and obtain a contradictioncontradiction

A diagonalization proofA diagonalization proof

31

Proof: Halting Problem is Proof: Halting Problem is UndecidableUndecidable

Assume AAssume ATMTM is decidable is decidable Let H be a decider for ALet H be a decider for ATMTM

On input <M,w>, where M is a TM and w is a string, H halts and On input <M,w>, where M is a TM and w is a string, H halts and accepts if M accepts w; otherwise it rejectsaccepts if M accepts w; otherwise it rejects

Construct a TM D using H as a subroutineConstruct a TM D using H as a subroutine D calls H to determine what M does when the input string is its D calls H to determine what M does when the input string is its

own description <M>. own description <M>. D then outputs the opposite of H’s answerD then outputs the opposite of H’s answer In summary:In summary:

D(<M>) accepts if M does not accept <M> and rejects if M accepts D(<M>) accepts if M does not accept <M> and rejects if M accepts <M><M>

Now run D on its own descriptionNow run D on its own description D(<D>) = accept if D does not accept <D> and reject if D accepts D(<D>) = accept if D does not accept <D> and reject if D accepts

DD A contradiction so H cannot be a decider for AA contradiction so H cannot be a decider for ATMTM

32

The Diagonalization The Diagonalization ProofProof

<M1<M1>>

<M2<M2>>

<M3<M3>>

<M4<M4>>

…… <D><D>

M1M1 AcceAcceptpt

RejeRejectct

AcceAcceptpt

RejeRejectct

…… AcceAcceptpt

M2M2 AcceAcceptpt

AcceAcceptpt

AcceAcceptpt

AcceAcceptpt

…… AcceAcceptpt

M3M3 RejeRejectct

RejeRejectct

RejeRejectct

RejeRejectct

…… RejeRejectct

M4M4 AcceAcceptpt

AcceAcceptpt

RejeRejectct

RejeRejectct

…… AcceAcceptpt

..DD RejeReje

ctctRejeRejectct

AcceAcceptpt

AcceAcceptpt

…… ??

..

33

Slightly more concrete Slightly more concrete versionversion

You write a program, halts(P, X) in C that takes as You write a program, halts(P, X) in C that takes as input any C program, P, and the input to that input any C program, P, and the input to that program, Xprogram, X Your program halts(P, X) analyzes P and returns “yes” if Your program halts(P, X) analyzes P and returns “yes” if

P will halt on X and “no” if P will not halt on XP will halt on X and “no” if P will not halt on X You now write a short procedure foo(X):You now write a short procedure foo(X):

foo(X) {a: if halts(X,X) then goto a else halt}foo(X) {a: if halts(X,X) then goto a else halt}This program halts if P does not halt on X; otherwise it This program halts if P does not halt on X; otherwise it

does haltdoes halt Does foo(foo) halt?Does foo(foo) halt?

It halts if and only if halts(foo,foo) returns noIt halts if and only if halts(foo,foo) returns no It halts if and only if it does not halt. Contradiction.It halts if and only if it does not halt. Contradiction.

Thus we have proven that you cannot write a Thus we have proven that you cannot write a program to determine if an arbitrary program will program to determine if an arbitrary program will halt or loophalt or loop

34

What does this mean?What does this mean? Recall what was said earlierRecall what was said earlier

The halting problem is not some contrived problemThe halting problem is not some contrived problem The halting problem asks whether we can tell if The halting problem asks whether we can tell if

some TM M will accept an input stringsome TM M will accept an input string We are asking if the language below is decidableWe are asking if the language below is decidable

AATMTM = {(M,w)|M is a TM and M accepts w} = {(M,w)|M is a TM and M accepts w} It is It is notnot decidable decidable

But as I keep emphasizing, M is a input variable too!But as I keep emphasizing, M is a input variable too! Of course some algorithms are decidable, like sorting Of course some algorithms are decidable, like sorting

algorithmsalgorithms It is Turing-recognizable (we covered this earlier)It is Turing-recognizable (we covered this earlier)

Simulate the TM on w and if it accepts/rejects, then Simulate the TM on w and if it accepts/rejects, then accept/reject.accept/reject.

Actually the halting problem is special because it Actually the halting problem is special because it gets at the heart of the mattergets at the heart of the matter

35

Co-Turing RecognizableCo-Turing Recognizable A language is co-Turing recognizable if it is A language is co-Turing recognizable if it is

the complement of a Turing-recognizable the complement of a Turing-recognizable languagelanguage

Theorem: A language is decidable if and only Theorem: A language is decidable if and only if it is Turing-recognizable and co-Turing-if it is Turing-recognizable and co-Turing-recognizablerecognizable Why? To be Turing-recognizable, we must accept Why? To be Turing-recognizable, we must accept

in finite time. If we don’t accept, we may reject or in finite time. If we don’t accept, we may reject or loop (it which case it is not decidable).loop (it which case it is not decidable).

Since we can invert any “question” by taking the Since we can invert any “question” by taking the complement, taking the complement flips the accept and complement, taking the complement flips the accept and reject answers. Thus, if we invert the question and it is reject answers. Thus, if we invert the question and it is Turing-recognizable, then that means that we would get Turing-recognizable, then that means that we would get the answer to the original reject question in finite time.the answer to the original reject question in finite time.

36

More Formal ProofMore Formal Proof Theorem: A language is decidable iff it is Turing-Theorem: A language is decidable iff it is Turing-

recognizable and co-Turing-recognizablerecognizable and co-Turing-recognizable Proof (2 directions)Proof (2 directions)

Forward direction easy. If it is decidable, then both it Forward direction easy. If it is decidable, then both it and its complement are Turing-recognizableand its complement are Turing-recognizable

Other direction: Other direction: Assume A and A’ are Turing-recognizable and let M1 Assume A and A’ are Turing-recognizable and let M1

recognize A and M2 recognize A’recognize A and M2 recognize A’ The following TM will decide AThe following TM will decide A M = On input wM = On input w

1.1. Run both M1 and M2 on input w in parallelRun both M1 and M2 on input w in parallel2.2. If M1 accepts, accept; if M2 accepts, then rejectIf M1 accepts, accept; if M2 accepts, then reject

Every string is in either A or A’ so every string w must be Every string is in either A or A’ so every string w must be accepted by either M1 or M2. Because M halts whenever accepted by either M1 or M2. Because M halts whenever M1 or M2 accepts, M always halts and so is a decider.M1 or M2 accepts, M always halts and so is a decider.

Furthermore, it accepts all strings in A and rejects all not Furthermore, it accepts all strings in A and rejects all not in A, so M is also a decider for A and thus A is decidablein A, so M is also a decider for A and thus A is decidable

37

ImplicationImplication Thus for any undecidable language, Thus for any undecidable language,

either the language or its either the language or its complement is not Turing-complement is not Turing-recognizablerecognizable

38

Complement of AComplement of ATMTM is not is not Turing-recognizableTuring-recognizable

AATMTM’ is not Turing-recognizable’ is not Turing-recognizable Proof:Proof:

We know that AWe know that ATMTM is Turing-recognizable is Turing-recognizable but not decidablebut not decidable

If AIf ATMTM’ were also Turing-recognizable, ’ were also Turing-recognizable, then Athen ATMTM would be decidable, which it is would be decidable, which it is notnot

Thus AThus ATMTM’ is not Turing-recognizable’ is not Turing-recognizable

39

Computer Computer Language TheoryLanguage Theory

Chapter 5: ReducibilityChapter 5: Reducibility

Due to time constraints we are only going to cover the first 3 pages of this chapter. However, we cover the notion of reducibility in depth when we cover Chapter 7.

40

What is Reducibility?What is Reducibility? A reduction is a way of converting one A reduction is a way of converting one

problem to another such that the solution to problem to another such that the solution to the second can be used to solve the firstthe second can be used to solve the first We say that problem A is reducible to problem BWe say that problem A is reducible to problem B Example: finding your way around NY City is Example: finding your way around NY City is

reducible to the problem of finding and reading a reducible to the problem of finding and reading a mapmap

If A reduces to B, what can we say about the If A reduces to B, what can we say about the relative difficulty of problem A and B?relative difficulty of problem A and B?

A can be no harder than B since the solution to B solves A can be no harder than B since the solution to B solves AA

A could be easier (the reduction is “inefficient” in a A could be easier (the reduction is “inefficient” in a sense)sense)

In example above, A is easier than B since B can solve In example above, A is easier than B since B can solve any routing problemany routing problem

41

Practice on ReducibilityPractice on Reducibility In our previous class work, did we In our previous class work, did we

reduce NFAs to DFAs or DFAs to NFAs?reduce NFAs to DFAs or DFAs to NFAs? We reduced NFAs to DFAsWe reduced NFAs to DFAs

We showed that an NFA can be reduced (i.e., We showed that an NFA can be reduced (i.e., converted) to a DFA via a set of simple stepsconverted) to a DFA via a set of simple steps

That means that NFA can not be any more That means that NFA can not be any more powerful than a DFApowerful than a DFA

Based only on the reduction, the NFA could be Based only on the reduction, the NFA could be less powerfulless powerful

But since we know this is not possible, since an But since we know this is not possible, since an DFA is a degenerate form of an NFA, we DFA is a degenerate form of an NFA, we showed they have the same expressive powershowed they have the same expressive power

42

How Reducibility is used to How Reducibility is used to Prove Languages Prove Languages

Undecidable Undecidable If A is reducible to B and B is decidable then If A is reducible to B and B is decidable then

what can we say?what can we say? A is decidable (since A can only be “easier”)A is decidable (since A can only be “easier”)

If A is reducible to B and A is decidable then If A is reducible to B and A is decidable then what can we say?what can we say? Nothing (so this is not useful for us)Nothing (so this is not useful for us)

If A is undecidable and reducible to B, then If A is undecidable and reducible to B, then what can we say about B?what can we say about B? B must be undecidable (B can only be harder B must be undecidable (B can only be harder

than A)than A) This is the most useful part for Chapter 5, since This is the most useful part for Chapter 5, since

this is how we can prove a language undecidablethis is how we can prove a language undecidable We can leverage past proofs and not start from scratchWe can leverage past proofs and not start from scratch

43

Example: Prove HALTExample: Prove HALTTMTM is is UndecidableUndecidable

Need to reduce ANeed to reduce ATM TM to HALTto HALTTMTM, where A, where ATM TM already already proven to be undecidableproven to be undecidable Can use HALTCan use HALTTM TM to solve Ato solve ATMTM

Proof by contradictionProof by contradiction Assume HALTAssume HALTTMTM is decidable and show this implies A is decidable and show this implies ATMTM is is

decidabledecidable Assume TM R that decides HALTAssume TM R that decides HALTTMTM

Use R to construct S a TM that decides AUse R to construct S a TM that decides ATMTM

Pretend you are S and need to decide APretend you are S and need to decide ATMTM so if given input <M, w> so if given input <M, w> must output accept if M accepts w and reject if M loops on w or must output accept if M accepts w and reject if M loops on w or rejects w.rejects w.

First try: simulate M on w and if it accepts then accept and if rejects First try: simulate M on w and if it accepts then accept and if rejects then reject. But in trouble if it loops.then reject. But in trouble if it loops.

This is bad because we need to be a decider and This is bad because we need to be a decider and

44

Example: Prove HALTExample: Prove HALTTMTM is is UndecidableUndecidable

Instead, use assumption that have TM R Instead, use assumption that have TM R that decides HALTthat decides HALTTMTM

Now can test if M halts on wNow can test if M halts on w If R indicates that M does halt on w, you can If R indicates that M does halt on w, you can

use the simulation and output the same use the simulation and output the same answeranswer

If R indicates that M does not halt, then If R indicates that M does not halt, then reject since infinite looping on w means it reject since infinite looping on w means it will never be accepted.will never be accepted.

The formal solution on next slideThe formal solution on next slide

45

Solution: Solution: HALTHALTTMTM is is UndecidableUndecidable

Assume TM R decides HALTAssume TM R decides HALTTMTM

Construct TM S to decide AConstruct TM S to decide ATM as follows as followsS = “On input <M, w>, an encoding of a S = “On input <M, w>, an encoding of a

TM M and a string w:TM M and a string w:1.1. Run TM R on input <M, w>Run TM R on input <M, w>2.2. If R rejects, If R rejects, rejectreject3.3. If R accepts, simulate M on w until it haltsIf R accepts, simulate M on w until it halts4.4. If M has accepted, If M has accepted, acceptaccept; If M has rejected, ; If M has rejected,

rejectreject””