Theory of Computation (X)

Theory of Computation (X)

Yijia ChenFudan University

Review

Decidability

Decidable Languages

Decidable problems concerning regular languages

ADFA ={〈B,w〉

∣∣ B is a DFA that accepts input string w}.

TheoremADFA is a decidable language.

ANFA ={〈B,w〉

∣∣ B is an NFA that accepts input string w}.

TheoremANFA is a decidable language.

AREX ={〈R,w〉

∣∣ R is a regular expression that generates w}.

TheoremAREX is a decidable language.

Testing the emptiness

EDFA ={〈A〉

∣∣ A is a DFA and L(A) = ∅}.

TheoremEDFA is a decidable language.

Testing equality

EQDFA ={〈A,B〉

∣∣ A are B are DFAs and L(A) = L(B)}.

TheoremEQDFA is a decidable language.

Decidable problems concerning context-free languages

ACFG ={〈G ,w〉

∣∣ G is a CFG that generates w}.

TheoremACFG is a decidable language.

Testing the emptiness

ECFG ={〈G〉

∣∣ G is a CFG and L(G) = ∅}.

TheoremECFG is a decidable language.

Testing equality

EQCFG ={〈G ,H〉

∣∣ G are H are CFGs and L(G) = L(H)}.

TheoremEQCFG is not decidable.

TheoremEvery context-free language is decidable.

Relationship among classes of languages

4.2 UNDECIDABILITY 201

FIGURE 4.10

The relationship among classes of languages

4.2UNDECIDABILITY

In this section, we prove one of the most philosophically important theorems ofthe theory of computation: There is a specific problem that is algorithmicallyunsolvable. Computers appear to be so powerful that you may believe that allproblems will eventually yield to them. The theorem presented here demon-strates that computers are limited in a fundamental way.

What sorts of problems are unsolvable by computer? Are they esoteric,dwelling only in the minds of theoreticians? No! Even some ordinary prob-lems that people want to solve turn out to be computationally unsolvable.

In one type of unsolvable problem, you are given a computer program anda precise specification of what that program is supposed to do (e.g., sort a listof numbers). You need to verify that the program performs as specified (i.e.,that it is correct). Because both the program and the specification are mathe-matically precise objects, you hope to automate the process of verification byfeeding these objects into a suitably programmed computer. However, you willbe disappointed. The general problem of software verification is not solvable bycomputer.

In this section and in Chapter 5, you will encounter several computationallyunsolvable problems. We aim to help you develop a feeling for the types ofproblems that are unsolvable and to learn techniques for proving unsolvability.

Now we turn to our first theorem that establishes the undecidability of a spe-cific language: the problem of determining whether a Turing machine accepts a

Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional

content at any time if subsequent rights restrictions require it.

Undecidability

Testing membership

ATM ={〈M,w〉

∣∣ M is a TM and M accepts w}.

TheoremATM is not decidable.

TheoremATM is Turing-recognizable.

Proof.U on 〈M,w〉:

1. Simulate M on w .

2. If M enters its accept state, then accept; if it enters its reject state, reject.

U is a universal Turing machine first proposed by Alan Turing in 1936. Thismachine is called universal because it is capable of simulating any other Turingmachine from the description of that machine.

The diagonalization method

Functions

DefinitionLet f : A→ B be a function.

1. f is one-to-one if f (a) 6= f (a′) whenever a 6= a′.

2. f is onto if for every b ∈ B there is an a ∈ A with f (a) = b.

A and B are the same size if there is a one-to-one, onto function d : A→ B.

A function that is both one-to-one and onto is a correspondence.

injective one-to-onesurjective ontobijective one-to-one and onto

Cantor’s Theorem

DefinitionA is countable if it is either finite or has the same size as N.

TheoremR is not countable.

Corollary

Some languages are not Turing-recognizable.

Proof

We fix an alphabet Σ.

1. Σ∗ is countable.

2. The set of all TMs is countable, as every M can be identified with a string〈M〉.

3. The set of all languages over Σ is uncountable.

An undecidable language

ATM ={〈M,w〉

∣∣ M is a TM and M accepts w}.

TheoremATM is undecidable.

Proof (1)

Assume H is a decider for ATM. That is

H(〈M,w〉

)=

{accept if M accepts w

reject if M does not accept.

Proof (2)

D on 〈M〉, where M is a TM:

1. Run H on input⟨M, 〈M〉

⟩.

2. Output the opposite of what H outputs. That is, if H accepts, then reject;and if H rejects, then accept.

D(〈M〉

)=

{accept if M does not accept 〈M〉reject if M accepts 〈M〉.

Then

D(〈D〉)

=

{accept if D does not accept 〈D〉reject if D accepts 〈D〉.

Proof (3)

〈M1〉 〈M2〉 〈M3〉 〈M4〉 . . .

M1 accept acceptM2 accept accept accept acceptM3 . . .M4 accept accept

......

Entry i , j is accept if Mi accepts 〈Mj〉.

〈M1〉 〈M2〉 〈M3〉 〈M4〉 . . .

M1 accept reject accept rejectM2 accept accept accept acceptM3 reject reject reject reject . . .M4 accept accept reject reject

......

Entry i , j is the value of H on input⟨Mi , 〈Mj〉

⟩.

Proof (4)

〈M1〉 〈M2〉 〈M3〉 〈M4〉 . . . 〈D〉 . . .

M1 accept reject accept reject acceptM2 accept accept accept accept . . . acceptM3 reject reject reject reject rejectM4 accept accept reject reject accept

......

D reject reject accept accept ?...

...

If D is in the figure, then a contradiction occurs at “?”

co-Turing-recognizable

DefinitionA language is co-Turing-recognizable if it is the complement of aTuring-recognizable language.

TheoremA language is decidable if and only if it is Turing recognizable andco-Turing-cognizable.

Proof

If A is decidable, then both A and A are Turing-recognizable: Any decidablelanguage is Turing-recognizable, and the complement of a decidable languagealso is decidable.

Assume both A and A are Turing recognizable by M1 and M2 respectively.

The TM M on input w :

1. Run M1 and M2 on input w in parallel.

2. If M1 accepts, then accept; and if M2 accepts, then reject.

Clearly, M decides A.

Corollary

ATM is not Turing-recognizable.

Proof.ATM is Turing-recognizable but not decidable.

Reducibility

Undecidable Problems from Language Theory

The halting problem

HALTTM ={〈M,w〉

∣∣ M is a TM and M halts on input w}.

TheoremHALTTM is undecidable.

Proof

Assume R decides HALTTM. We will exhibit a TM S which decides ATM.

S on input 〈M,w〉:

1. Run R on 〈M,w〉.

2. If R rejects, then reject.

3. If R accepts, simulate M on w until it halts.

4. If M has accepts, then accept; if M has rejected, reject.

Testing emptiness

ETM ={〈M〉

∣∣ M is a TM and L(M) = ∅}.

TheoremETM is undecidable.

Proof (1)

For every TM M and string w we construct an M1:

M1 on input x :

1. If x 6= w , then reject.

2. If x = w , run M on w and accept if M does.

Then

M accepts w ⇐⇒ L(M1) 6= ∅.

Proof (2)

Assume R decides ETM. Then the following TM S decides ATM.

S on input 〈M,w〉:

1. Use the description of M and w to construct the TM M1.

2. Run R on input 〈M1〉.

3. If R accepts, then reject; if R rejects, then accept.

Testing regularity

REGULARTM ={〈M〉

∣∣ M is a TM and L(M) is a regular language}.

TheoremREGULARTM is undecidable.

Proof (1)

For every TM M and string w we construct an M2:

M2 on input x :

1. If x has the form 0n1n, then accept.

2. Otherwise, run M on w and accept if M does.

Then

M accepts w ⇐⇒ L(M2) is regular.

Proof (2)

Assume R decides REGULARTM. Then the following S decides ATM.

S on input 〈M,w〉:

1. Use the description of M and w to construct the TM M2.

2. Run R on input 〈M2〉.

3. If R accepts, then accept; if R rejects, then reject.

Testing equality

EQTM ={〈M1,M2〉

∣∣ M1 and M2 are TMs and L(M1) = L(M2)}.

TheoremEQTM is undecidable.

Proof

Assume R decides EQTM. Then we can decide ETM as follows.

S on input 〈M〉:

1. Run R on input 〈M,M1〉, where M1 is a TM that rejects all inputs.

2. If R accepts, then accept; if R rejects, then reject.

Reductions via computation histories

Computation histories

DefinitionLet M be a TM and w an input string. An accepting computation history forM on w is a sequence of configurations.

C1, . . . ,C`,

where C1 is the start configuration of M on w , C` is an accepting configurationof M, and each Ci legally follows from Ci−1 according to the rules of M.

A rejecting computation history for M on w is defined similarly, except that C`

is a rejecting configuration.

Linear bounded automata

DefinitionA linear bounded automaton (LBA) is a TM wherein the tape head isn’tpermitted to move off the portion of the tape containing the input.

If the machine tries to move its head off either end of the input, the head stayswhere it is.

ALBA ={〈M,w〉

∣∣ M is an LBA that accepts w}.

TheoremALBA is decidable.

LemmaLet M be an LBA with q states and g symbols in the tape alphabet. There areexactly qngn distinct configurations of M for a tape length n.

Proof

L on input 〈M,w〉:

1. Simulate M on w for qngn steps or until it halts.

2. If M has halted, accept if it has accepted and reject if it has rejected. If ithas not halted, reject.

If M on w has not halted within qngn steps, it must be repeating aconfiguration and therefore looping.

Testing emptiness

ELBA ={〈M〉

∣∣ M is an LBA and L(M) = ∅}.

TheoremELBA is undecidable.

An LBA recognizing computation histories

Let M be a TM and w an input string.

On input x , the LBA B works as follows:

1. breaks up x according to the delimiters into strings C1, . . . ,C`;

2. determines whether Ci ’s satisfy

2.1 C1 is the start configuration for M on w ,

2.2 each Ci+1 legally follows from Ci ,

2.3 C` is an accepting configuration.

Then

M accepts x ⇐⇒ L(B) 6= ∅.

Proof

Assume R decides ELBA. Then the following S decides ATM.

S on input 〈M,w〉:

1. Construct LBA B from M and w .

2. Run R on input 〈B〉.

3. If R rejects, then accept; if R accepts, then reject.

ALLCFG ={〈G〉

∣∣ G is a CFG and L(G) = Σ∗}.

TheoremALLCFG is undecidable.

Proof (1)

Let M be a TM and w a string. We will construct a CFG G such that

M accepts w ⇐⇒ L(G) 6= Σ∗

⇐⇒ G doesn’t generate

the accepting computation history for M on w .

Proof (2)

An accepting computation history for M on w appears as

#C1#C2# · · ·#C`#,

where Ci is the configuration of M on the ith step of the computation on w .

Then, G generates all strings

1. that do not start with C1,

2. that do not end with an accepting configuration, or

3. in which Ci does not properly yield Ci+1 under the rule of M.

Proof (3)

We construct a PDA D and then convert it to G .

1. D starts by nondeterministically branching to guess which of the threeconditions to check.

2. The first and the second are straightforward.

3. The third branch accepts if some Ci does not properly yield Ci+1.

3.1 It scans the input and nondeterministically decides that it has come to Ci .

3.2 It pushes Ci onto the stack until it reads #.

3.3 Then D pops the stack to compare with Ci+1: they are almost the sameexcept around the head position, where the difference is dictated by thetransition function of M.

3.4 D accepts if there is a mismatch or an improper update.

Proof (4)

A minor problem: when D pops Ci off the stack, it is in reverse order.

We write the accepting computation history as

# −→︸ ︷︷ ︸C1

# −→︸ ︷︷ ︸CR

2

# −→︸ ︷︷ ︸C3

# −→︸ ︷︷ ︸CR

4

# · · ·# −→︸ ︷︷ ︸C`

#

Mapping Reducibility

Computable functions

DefinitionA function f : Σ∗ → Σ∗ is computable function if some Turing machine M, onevery input w , halts with f (w) on its tape.

Formal definition of mapping reducibility

DefinitionLanguage A is mapping reducible to language B, written A ≤m B, if there is acomputable function f : Σ∗ → Σ∗, where for every w ∈ Σ∗

w ∈ A ⇐⇒ f (w) ∈ B.

The function f is called the reduction from A to B.

TheoremIf A ≤m B and B is decidable, then A is decidable.

Corollary

If A ≤m B and A is undecidable, then B is undecidable.

ATM ≤m HALTTM

F on input 〈M,w〉:

1. Construct the following machine M ′(x).

1.1 Run M on x .

1.2 If M accepts, then accept.1.3 If M rejects, then enter a loop.

2. Output 〈M ′,w〉.

TheoremIf A ≤m B and B is Turing-recognizable, then A is Turing-recognizable.

Corollary

If A ≤m B and A is not Turing-recognizable, then B is not Turing-recognizable.

TheoremEQTM is neither Turing-recognizable nor co-Turing-recognizable.

Proof (1)

To show EQTM is not Turing-recognizable, we prove ATM ≤m EQTM:

F on input 〈M,w〉:

1. Construct the following two machines M1 and M2.

1.1 M1 reject any input.

1.2 M2 accepts an input if M accepts w .

2. Output 〈M1,M2〉.

Proof (2)

To show EQTM is not Turing-recognizable, we prove ATM ≤m EQTM:

G on input 〈M,w〉:

1. Construct the following two machines M1 and M2.

1.1 M1 accepts any input.

1.2 M2 accepts an input if M accepts w .

2. Output 〈M1,M2〉.