Page 1
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computational Complexity I:From finite automata to Turing machines
Maria-Eirini Pegia
Seminar on Theoretical Computer Science and Discrete MathematicsAristotle University of Thessaloniki
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 2
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Context
1 Section 1: Decision and Optimization Problems
2 Section 2: Finite Automata - Recognizable Languages
3 Section 3: Context-free Grammars - Context-free Languages
4 Section 4: Turing Machines
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 3
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
What is a decision problem?
Definition
In computability theory andcomputational complexity theory,a decision problem is a questionin some formal system with ayes-or-no answer, depending onthe values of some inputparameters.
Figure: A decision problem has onlytwo possible outputs, yes or no
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 4
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computational Theory
Decision problems typically appear in mathematical questionsof decidability, that is, the question of the existence of aneffective method to determine the existence of some object orits membership in a set.
Can we solve all the problems?
� NO!!!
Why some problems are notsolving by computers?
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 5
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computational Theory
Decision problems typically appear in mathematical questionsof decidability, that is, the question of the existence of aneffective method to determine the existence of some object orits membership in a set.
Can we solve all the problems?
� NO!!!
Why some problems are notsolving by computers?
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 6
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computational Theory
Decision problems typically appear in mathematical questionsof decidability, that is, the question of the existence of aneffective method to determine the existence of some object orits membership in a set.
Can we solve all the problems?
� NO!!!
Why some problems are notsolving by computers?
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 7
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computational Theory
Decision problems typically appear in mathematical questionsof decidability, that is, the question of the existence of aneffective method to determine the existence of some object orits membership in a set.
Can we solve all the problems?
� NO!!!
Why some problems are notsolving by computers?
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 8
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Why some problems aren’t solving by computers?
Hilbert (1900): Completeness and automation of mathematics10th Problem: Algorithm for diophantine equation
Algorithm: Expression and Proof of correctness
Does an algorithm exist?
Definition of ”algorithm” with a computational model and theproof that the existence of an algorithm leads tocontradictions.
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 9
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Why some problems aren’t solving by computers?
Hilbert (1900): Completeness and automation of mathematics10th Problem: Algorithm for diophantine equation
Algorithm: Expression and Proof of correctness
Does an algorithm exist?
Definition of ”algorithm” with a computational model and theproof that the existence of an algorithm leads tocontradictions.
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 10
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Why some problems aren’t solving by computers?
Hilbert (1900): Completeness and automation of mathematics10th Problem: Algorithm for diophantine equation
Algorithm: Expression and Proof of correctness
Does an algorithm exist?
Definition of ”algorithm” with a computational model and theproof that the existence of an algorithm leads tocontradictions.
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 11
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Why some problems aren’t solving by computers?
Godel: Mathematics are not complete!
Turing: Mathematics are not automating!
There are problems that are not computable.
Some of the most importantproblems in mathematics areundecidable!!!,,,
Matijasevic (1970): There is no algorithm for the solution ofevery diophantine equation. For every algorithm A there is anequation that A answers false!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 12
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Why some problems aren’t solving by computers?
Godel: Mathematics are not complete!
Turing: Mathematics are not automating!
There are problems that are not computable.
Some of the most importantproblems in mathematics areundecidable!!!,,,
Matijasevic (1970): There is no algorithm for the solution ofevery diophantine equation. For every algorithm A there is anequation that A answers false!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 13
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Why some problems aren’t solving by computers?
Godel: Mathematics are not complete!
Turing: Mathematics are not automating!
There are problems that are not computable.
Some of the most importantproblems in mathematics areundecidable!!!,,,
Matijasevic (1970): There is no algorithm for the solution ofevery diophantine equation. For every algorithm A there is anequation that A answers false!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 14
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Why some problems aren’t solving by computers?
Godel: Mathematics are not complete!
Turing: Mathematics are not automating!
There are problems that are not computable.
Some of the most importantproblems in mathematics areundecidable!!!,,,
Matijasevic (1970): There is no algorithm for the solution ofevery diophantine equation. For every algorithm A there is anequation that A answers false!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 15
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Why some problems aren’t solving by computers?
Godel: Mathematics are not complete!
Turing: Mathematics are not automating!
There are problems that are not computable.
Some of the most importantproblems in mathematics areundecidable!!!,,,
Matijasevic (1970): There is no algorithm for the solution ofevery diophantine equation. For every algorithm A there is anequation that A answers false!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 16
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computable Problem and algorithm
(Computable) problem: defines a mapping from input data toexport data.
Intuitively: is defined by a question for entry snapshots
Snapshot: object that corresponds to entry data.We set a question and we wait the answer.Infinity set of snapshots.
Algorithm: clearly defined process for the solution of aproblem in finite time by a computational machine (Turing).
Ô� We see TM below!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 17
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computable Problem and algorithm
(Computable) problem: defines a mapping from input data toexport data.Intuitively: is defined by a question for entry snapshots
Snapshot: object that corresponds to entry data.We set a question and we wait the answer.Infinity set of snapshots.
Algorithm: clearly defined process for the solution of aproblem in finite time by a computational machine (Turing).
Ô� We see TM below!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 18
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computable Problem and algorithm
(Computable) problem: defines a mapping from input data toexport data.Intuitively: is defined by a question for entry snapshots
Snapshot: object that corresponds to entry data.
We set a question and we wait the answer.Infinity set of snapshots.
Algorithm: clearly defined process for the solution of aproblem in finite time by a computational machine (Turing).
Ô� We see TM below!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 19
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computable Problem and algorithm
(Computable) problem: defines a mapping from input data toexport data.Intuitively: is defined by a question for entry snapshots
Snapshot: object that corresponds to entry data.We set a question and we wait the answer.
Infinity set of snapshots.
Algorithm: clearly defined process for the solution of aproblem in finite time by a computational machine (Turing).
Ô� We see TM below!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 20
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computable Problem and algorithm
(Computable) problem: defines a mapping from input data toexport data.Intuitively: is defined by a question for entry snapshots
Snapshot: object that corresponds to entry data.We set a question and we wait the answer.Infinity set of snapshots.
Algorithm: clearly defined process for the solution of aproblem in finite time by a computational machine (Turing).
Ô� We see TM below!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 21
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computable Problem and algorithm
(Computable) problem: defines a mapping from input data toexport data.Intuitively: is defined by a question for entry snapshots
Snapshot: object that corresponds to entry data.We set a question and we wait the answer.Infinity set of snapshots.
Algorithm: clearly defined process for the solution of aproblem in finite time by a computational machine (Turing).
Ô� We see TM below!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 22
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Computable Problem and algorithm
(Computable) problem: defines a mapping from input data toexport data.Intuitively: is defined by a question for entry snapshots
Snapshot: object that corresponds to entry data.We set a question and we wait the answer.Infinity set of snapshots.
Algorithm: clearly defined process for the solution of aproblem in finite time by a computational machine (Turing).
Ô� We see TM below!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 23
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example: travelling salesman problem (TSP)
Decision Problem
Given a list of cities and the distances between each pair of cities,is there a possible route that visits each city exactly once andreturns to the origin city?
Optimization Problem
Given a list of cities and the distances between each pair of cities,find the shortest (minimum cost) possible route that visits eachcity exactly once and returns to the origin city.
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 24
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Problems and Formal Languages
Optimization ProblemreductionÐÐÐÐÐ� Decision Problem with bound B
Ì Minimization: § optimal solution with cost B B?
Ì Maximization: § optimal solution with gain C B?
Ì Optimization Problem is solved (in polynomial time)
iffthe corresponding Decision Problem is solved (in polynomial time).
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 25
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Problems and Formal Languages
Optimization ProblemreductionÐÐÐÐÐ� Decision Problem with bound B
Ì Minimization: § optimal solution with cost B B?
Ì Maximization: § optimal solution with gain C B?
Ì Optimization Problem is solved (in polynomial time)
iffthe corresponding Decision Problem is solved (in polynomial time).
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 26
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Problems and Formal Languages
Optimization ProblemreductionÐÐÐÐÐ� Decision Problem with bound B
Ì Minimization: § optimal solution with cost B B?
Ì Maximization: § optimal solution with gain C B?
Ì Optimization Problem is solved (in polynomial time)
iffthe corresponding Decision Problem is solved (in polynomial time).
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 27
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Problems and Formal Languages
Optimization ProblemreductionÐÐÐÐÐ� Decision Problem with bound B
Ì Minimization: § optimal solution with cost B B?
Ì Maximization: § optimal solution with gain C B?
Ì Optimization Problem is solved (in polynomial time)
iffthe corresponding Decision Problem is solved (in polynomial time).
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 28
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Finite Automata - Recognizable Languages
Examples:
Í L= �ab 8 aab��
Í L= ( �ab�� 8 �bc�� )ab
Í L= ��ab 8 aba��a��
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 29
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
How I can show that a language is not recognizable?
L=� anbn S nC0 � (is recognizable?)
(NO!!!)
Can we do something better with another computational model???
YES!!!
We will see it in the next section ,,,
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 30
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
How I can show that a language is not recognizable?
L=� anbn S nC0 � (is recognizable?)
(NO!!!)
Can we do something better with another computational model???
YES!!!
We will see it in the next section ,,,
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 31
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
How I can show that a language is not recognizable?
L=� anbn S nC0 � (is recognizable?)
(NO!!!)
Can we do something better with another computational model???
YES!!!
We will see it in the next section ,,,
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 32
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
How I can show that a language is not recognizable?
L=� anbn S nC0 � (is recognizable?)
(NO!!!)
Can we do something better with another computational model???
YES!!!
We will see it in the next section ,,,
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 33
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Context-free Grammars - Context-free Languages
Definition
G=(X,V,S,R): context-free grammar
X: terminals,
V: variables,
S: axiom,
R b V x �V 8X �� rules
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 34
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Examples
Í L = � anbn S nC0 � is context-free???
YES!!! ��Y�Y�/
Í L = � wwR S w > �a,b�� �
Í L = � w > �a,b�� S w = wR �
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 35
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Examples
Í L = � anbn S nC0 � is context-free??? YES!!! ��Y�Y�/
Í L = � wwR S w > �a,b�� �
Í L = � w > �a,b�� S w = wR �
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 36
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Examples
Í L = � anbn S nC0 � is context-free??? YES!!! ��Y�Y�/
Í L = � wwR S w > �a,b�� �
Í L = � w > �a,b�� S w = wR �
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 37
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
How I can show that a language is not context-free?
L = � ww S w > �a,b�� � (is context-free?)
(NO!!!)
Can we do something better with another computational model???
YES!!! We will see it in the next section ,,,
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 38
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
How I can show that a language is not context-free?
L = � ww S w > �a,b�� � (is context-free?) (NO!!!)
Can we do something better with another computational model???
YES!!! We will see it in the next section ,,,
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 39
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
How I can show that a language is not context-free?
L = � ww S w > �a,b�� � (is context-free?) (NO!!!)
Can we do something better with another computational model???
YES!!! We will see it in the next section ,,,
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 40
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Definition
M = (Q, Σ, Γ, δ, q0, B, F): Turing Machine (TM)
Q: set of states,
Σ: finite set of inputs,
Γ: complete set of tape symbols, Σ ` Γ,
δ: Q x Γ Ð� Q x Γ x �L,R�- L, R standing for ’left’ and ’right’ direction, respectively
q0 > Q: initial state
B > Γ: blank symbol
F ` Q: final states
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 41
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Definition
M = (Q, Σ, Γ, δ, q0, B, F): Turing Machine (TM)
Q: set of states,
Σ: finite set of inputs,
Γ: complete set of tape symbols, Σ ` Γ,
δ: Q x Γ Ð� Q x Γ x �L,R�- L, R standing for ’left’ and ’right’ direction, respectively
q0 > Q: initial state
B > Γ: blank symbol
F ` Q: final states
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 42
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Definition
M = (Q, Σ, Γ, δ, q0, B, F): Turing Machine (TM)
Q: set of states,
Σ: finite set of inputs,
Γ: complete set of tape symbols, Σ ` Γ,
δ: Q x Γ Ð� Q x Γ x �L,R�- L, R standing for ’left’ and ’right’ direction, respectively
q0 > Q: initial state
B > Γ: blank symbol
F ` Q: final states
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 43
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Definition
M = (Q, Σ, Γ, δ, q0, B, F): Turing Machine (TM)
Q: set of states,
Σ: finite set of inputs,
Γ: complete set of tape symbols, Σ ` Γ,
δ: Q x Γ Ð� Q x Γ x �L,R�- L, R standing for ’left’ and ’right’ direction, respectively
q0 > Q: initial state
B > Γ: blank symbol
F ` Q: final states
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 44
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Definition
M = (Q, Σ, Γ, δ, q0, B, F): Turing Machine (TM)
Q: set of states,
Σ: finite set of inputs,
Γ: complete set of tape symbols, Σ ` Γ,
δ: Q x Γ Ð� Q x Γ x �L,R�
- L, R standing for ’left’ and ’right’ direction, respectively
q0 > Q: initial state
B > Γ: blank symbol
F ` Q: final states
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 45
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Definition
M = (Q, Σ, Γ, δ, q0, B, F): Turing Machine (TM)
Q: set of states,
Σ: finite set of inputs,
Γ: complete set of tape symbols, Σ ` Γ,
δ: Q x Γ Ð� Q x Γ x �L,R�- L, R standing for ’left’ and ’right’ direction, respectively
q0 > Q: initial state
B > Γ: blank symbol
F ` Q: final states
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 46
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Definition
M = (Q, Σ, Γ, δ, q0, B, F): Turing Machine (TM)
Q: set of states,
Σ: finite set of inputs,
Γ: complete set of tape symbols, Σ ` Γ,
δ: Q x Γ Ð� Q x Γ x �L,R�- L, R standing for ’left’ and ’right’ direction, respectively
q0 > Q: initial state
B > Γ: blank symbol
F ` Q: final states
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 47
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Definition
M = (Q, Σ, Γ, δ, q0, B, F): Turing Machine (TM)
Q: set of states,
Σ: finite set of inputs,
Γ: complete set of tape symbols, Σ ` Γ,
δ: Q x Γ Ð� Q x Γ x �L,R�- L, R standing for ’left’ and ’right’ direction, respectively
q0 > Q: initial state
B > Γ: blank symbol
F ` Q: final states
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 48
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Definition
M = (Q, Σ, Γ, δ, q0, B, F): Turing Machine (TM)
Q: set of states,
Σ: finite set of inputs,
Γ: complete set of tape symbols, Σ ` Γ,
δ: Q x Γ Ð� Q x Γ x �L,R�- L, R standing for ’left’ and ’right’ direction, respectively
q0 > Q: initial state
B > Γ: blank symbol
F ` Q: final statesMaria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 49
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 1: printing 110
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 50
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 1: printing 110
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 51
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 1: printing 110
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 52
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 1: printing 110
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 53
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 1: printing 110
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 54
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 2: bit inversion of 110 , !!!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 55
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 2: bit inversion of 110 , !!!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 56
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 2: bit inversion of 110 , !!!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 57
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 2: bit inversion of 110 , !!!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 58
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 3: bit inversion of 001
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 59
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 3: bit inversion of 001
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 60
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 3: bit inversion of 001
Figure: Example 1 and 2 ’together’
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 61
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 3: bit inversion of 001
Figure: Example 1 and 2 ’together’
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 62
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 4
M = (Q, Σ, Γ, δ, q0, B, F)
Q = �q0,q1,q2,q3,q4�, Σ = �0,1�, Γ = �0,1,X ,Y ,B�, F = �q4�,
δ: Q x Γ Ð� Q x Γ x �L,R�
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 63
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 4
Figure: L =� 0n1n S n C 1 �
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 64
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 4
Í q00011 Ø� Xq1011 Ø� X0q111 Ø� Xq20Y1 Ø� q2X0Y1 Ø�
Xq00Y1 � XXq1Y1 � XXYq11 � XXq2YY � Xq2XYY �
XXq0YY � XXYq3Y � XXYYq3B � XXYYBq4B
Í q00010 Ø� Xq1010 Ø� X0q110 Ø� Xq20Y0 Ø� Xq00Y0 Ø�
XXq1Y0 � XXYq10 � XXY0q1B
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 65
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example 4
Í q00011 Ø� Xq1011 Ø� X0q111 Ø� Xq20Y1 Ø� q2X0Y1 Ø�
Xq00Y1 � XXq1Y1 � XXYq11 � XXq2YY � Xq2XYY �
XXq0YY � XXYq3Y � XXYYq3B � XXYYBq4B
Í q00010 Ø� Xq1010 Ø� X0q110 Ø� Xq20Y0 Ø� Xq00Y0 Ø�
XXq1Y0 � XXYq10 � XXY0q1B
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 66
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
L(M) = � w > Σ� S q0w Ø� upv for some p > F and any u,v > Γ �
recursively enumerable languages: the recognizable languages inTM
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 67
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
An equivalent model: Unrestricted Grammars
Definition
G=(X,V,S,R): unrestricted grammar
X: terminals
V: set of nonterminal symbols, X 9 V = g
S > V: axiom
R: rules of the form u Ô� w, u, w > �V 8X ��, w x ε
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 68
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
An equivalent model: Unrestricted Grammars
Definition
G=(X,V,S,R): unrestricted grammar
X: terminals
V: set of nonterminal symbols, X 9 V = g
S > V: axiom
R: rules of the form u Ô� w, u, w > �V 8X ��, w x ε
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 69
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
An equivalent model: Unrestricted Grammars
Definition
G=(X,V,S,R): unrestricted grammar
X: terminals
V: set of nonterminal symbols, X 9 V = g
S > V: axiom
R: rules of the form u Ô� w, u, w > �V 8X ��, w x ε
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 70
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
An equivalent model: Unrestricted Grammars
Definition
G=(X,V,S,R): unrestricted grammar
X: terminals
V: set of nonterminal symbols, X 9 V = g
S > V: axiom
R: rules of the form u Ô� w, u, w > �V 8X ��, w x ε
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 71
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
An equivalent model: Unrestricted Grammars
Definition
G=(X,V,S,R): unrestricted grammar
X: terminals
V: set of nonterminal symbols, X 9 V = g
S > V: axiom
R: rules of the form u Ô� w, u, w > �V 8X ��, w x ε
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 72
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example
L = � ww S w > �a,b�� � is recursively enumerable???
YES!!! ��Y�Y�/
G = (�S ,F ,M,A,B�, �a,b�, S, R): unrestricted grammar
S Ð� FM, F Ð� FaA, F Ð� FbB, Aa Ð� aA, Ab Ð� bA,Ba Ð� aB, Bb Ð� bB, AM Ð� Ma, BM Ð� Mb, F Ð� ε,M Ð� ε
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 73
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example
L = � ww S w > �a,b�� � is recursively enumerable???
YES!!! ��Y�Y�/
G = (�S ,F ,M,A,B�, �a,b�, S, R): unrestricted grammar
S Ð� FM, F Ð� FaA, F Ð� FbB, Aa Ð� aA, Ab Ð� bA,Ba Ð� aB, Bb Ð� bB, AM Ð� Ma, BM Ð� Mb, F Ð� ε,M Ð� ε
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 74
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example
L = � ww S w > �a,b�� � is recursively enumerable???
YES!!! ��Y�Y�/
G = (�S ,F ,M,A,B�, �a,b�, S, R): unrestricted grammar
S Ð� FM, F Ð� FaA, F Ð� FbB, Aa Ð� aA, Ab Ð� bA,Ba Ð� aB, Bb Ð� bB, AM Ð� Ma, BM Ð� Mb, F Ð� ε,M Ð� ε
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 75
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Example
Í S Ô� εM Ô� εε = ε
Í S Ô� (FaA)M Ô� Fa(Ma) Ô� εaMa Ô� aεa Ô� aa= a2
Í S Ô� (FaA)M Ô� F(aAM) Ô� (FbB)(aAM) Ô�Fb(Ba)AM Ô� Fb(aB)BAM Ô� FbaB(AM)Ô�FbaB(Ma) Ô� Fba(BM)a Ô� Fba(bM)a Ô�baba = �ba�2
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 76
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Thesis Church - Turing
Thesis Church - TuringIf a problem can be solved with an algorithm, then there is aTM which solves the problem.
extended models of TM- DTM- Nondeterministic TM- Restricted TM- Multitape TM
.
.
.
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 77
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Thesis Church - Turing
Thesis Church - TuringIf a problem can be solved with an algorithm, then there is aTM which solves the problem.
extended models of TM- DTM- Nondeterministic TM- Restricted TM- Multitape TM
.
.
.
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 78
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Halting Problem is undecidable
_\_( ") )_/
_
Í In computability theory, the halting problem is the problem ofdetermining, from a description of an arbitrary computer programand an input, whether the program will finish running or continueto run forever.
Í Alan Turing (1936) proved that a general algorithm to solve thehalting problem for all possible program-input pairs cannot exist.
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 79
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Halting Problem is undecidable
_\_( ") )_/
_
Í In computability theory, the halting problem is the problem ofdetermining, from a description of an arbitrary computer programand an input, whether the program will finish running or continueto run forever.
Í Alan Turing (1936) proved that a general algorithm to solve thehalting problem for all possible program-input pairs cannot exist.
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 80
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
References
J. E. Hopcroft and J. D. Ullman. Introduction to AutomataTheory, Languages, and Computation. Boston:Addison-Wesley, c2001.
C. H. Papadimitriou. Computational Complexity. Reading,Mass.: Addison-Wesley, 1994.
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 81
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Next
_\_( ") )_/
_
_\_( ") )_/
_
_\_( ") )_/
_Asymptotic Notation!!!_
\_( ") )_/_
_\_( ") )_/
_
_\_( ") )_/
_
_\_( ") )_/
_Classification Algorithms!!!_
\_( ") )_/_
_\_( ") )_/
_
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines
Page 82
Section 1: Decision and Optimization ProblemsSection 2: Finite Automata - Recognizable Languages
Section 3: Context-free Grammars - Context-free LanguagesSection 4: Turing Machines
Thank you!!!
Maria-Eirini Pegia Computational Complexity I: From finite automata to Turing machines