1 Theory of Computation 計計計計
Jan 17, 2016
1
Theory of Computation
計算理論
2
Instructor: 顏嗣鈞
E-mail: [email protected]
Web: http://www.ee.ntu.edu.tw/~yen
Time: 9:10-12:10 PM, Monday
Place: BL 103
Office hours: by appointment
Class web page: http://www.ee.ntu.edu.tw/~yen/courses/TOC-2009.html
3
TEXTBOOK
Introduction to Automata Theory, Languages, and Computation 3rd Edition
John E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman,
(Addison-Wesley, 2006)
4
Introduction to Automata Theory, Languages, and Computation
John E. Hopcroft, Rajeev Motwani,
Jeffrey D. Ullman,
(2nd Ed. Addison-Wesley, 2001)
2nd Edition
5
Introduction to Automata Theory, Languages, and Computation John E. Hopcroft,
Jeffrey D. Ullman,
(Addison-Wesley, 1979)
1st Edition
6
Grading
HW/Project : 20%
Midterm exam.: 40%
Final exam.: 40%
7
A simple computer
BATTERY
SWITCH
input: switch
output: light bulb
actions: flip switch
states: on, off
8
A simple “computer”
BATTERY
SWITCH
off onstart
f
f
input: switch
output: light bulb
actions: f for “flip switch”
states: on, off
bulb is on if and only if there was an odd number of flips
9
Another “computer”
BATTERY
off offstart
1
inputs: switches 1 and 2
actions: 1 for “flip switch 1”actions: 2 for “flip switch 2”
states: on, off
bulb is on if and only if both switches were flipped an odd number of times
1
2
1
off on
1
1
2 2 2 2
10
A design problem
Can you design a circuit where the light is on if and only if all the switches were flipped exactly the same number of times?
4
BATTERY
1
2
3
5
?
11
A design problem
• Such devices are difficult to reason about, because they can be designed in an infinite number of ways
• By representing them as abstract computational devices, or automata, we will learn how to answer such questions
12
These devices can model many things
• They can describe the operation of any “small computer”, like the control component of an alarm clock or a microwave
• They are also used in lexical analyzers to recognize well formed expressions in programming languages:
ab1 is a legal name of a variable in C5u= is not
13
Different kinds of automata
• This was only one example of a computational device, and there are others
• We will look at different devices, and look at the following questions:– What can a given type of device compute, and
what are its limitations?– Is one type of device more powerful than
another?
14
Some devices we will seefinite automata Devices with a finite amount of memory.
Used to model “small” computers.
push-down automata
Devices with infinite memory that can be accessed in a restricted way.
Used to model parsers, etc.
Turing Machines Devices with infinite memory.
Used to model any computer.
time-bounded Turing Machines
Infinite memory, but bounded running time.
Used to model any computer program that runs in a “reasonable” amount of time.
15
Why Study Automata Theory?
Finite automata are a useful model for important kinds of hardware and software:
• Software for designing and checking digital circuits.
• Lexical analyzer of compilers. • Finding words and patterns in large bodies of
text, e.g. in web pages.• Verification of systems with finite number of
states, e.g. communication protocols.
16
Why Study Automata Theory? (2)
The study of Finite Automata and Formal Languages are intimately connected. Methods for specifying formal languages are very important in many areas of CS, e.g.:
• Context Free Grammars are very useful when designing software that processes data with recursive structure, like the parser in a compiler.
• Regular Expressions are very useful for specifying lexical aspects of programming languages and search patterns.
17
Why Study Automata Theory? (3)
Automata are essential for the study of the limits of computation. Two issues:
• What can a computer do at all? (Decidability)
• What can a computer do efficiently? (Intractability)
18
Applications
Theoretical Computer ScienceAutomata Theory, Formal Languages,
Computability, Complexity …
Com
pile
r
Pro
g.
lan
gu
ag
es
Com
m.
pro
toco
ls
circuits
Patte
rn
reco
gn
ition
Sup
erv
isory
co
ntro
l
Qu
an
tum
co
mpu
ting
Com
pu
ter-
Aid
ed
V
erifi
catio
n ...
19
Aims of the Course• To familiarize you with key Computer Science
concepts in central areas like- Automata Theory- Formal Languages- Models of Computation- Complexity Theory
• To equip you with tools with wide applicability in the fields of CS and EE, e.g. for- Complier Construction- Text Processing- XML
20
Fundamental Theme
• What are the capabilities and limitations of computers and computer programs?– What can we do with
computers/programs?
– Are there things we cannot do with computers/programs?
21
Studying the Theme
• How do we prove something CAN be done by SOME program?
• How do we prove something CANNOT be done by ANY program?
22
Example: The Halting Problem (1)
Consider the following program. Does it terminate for all values of n 1?
while (n > 1) {if even(n) {
n = n / 2;} else {
n = n * 3 + 1;}
}
23
Example: The Halting Problem (2)
Not as easy to answer as it might first seem.
Say we start with n = 7, for example:
7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5,
16, 8, 4, 2, 1
In fact, for all numbers that have been tried
(a lot!), it does terminate . . .
. . . but in general?
24
Example: The Halting Problem (3)Then the following important undecidability
result should perhaps not come as a total surprise:
It is impossible to write a program that decides if another, arbitrary, program terminates (halts) or not.
What might be surprising is that it is possible to prove such a result. This was first done by the British mathematician Alan Turing.
25
Our focus
AutomataComputability
Complexity
26
Topics1. Finite automata, Regular languages, Regular
grammars: deterministic vs. nondeterministic, one-way vs. two-way finite automata, minimization, pumping lemma for regular sets, closure properties.
2. Pushdown automata, Context-free languages, Context-free grammars: deterministic vs. nondeterministic, one-way vs. two-way PDAs, reversal bounded PDAs, linear grammars, counter machines, pumping lemma for CFLs, Chomsky normal form, Greibach normal form, closure properties.
3.
27
Topics (cont’d)3. Linear bounded automata, Context-
sensitive languages, Context-sensitive grammars.
4. Turing machines, Recursively enumerable sets, Type 0 grammars: variants of Turing machines, halting problem, undecidability, Post correspondence problem, valid and invalid computations of TMs.
28
Topics (cont’d)5. Basic recursive function theory 6. Basic complexity theory: Various
resource bounded complexity classes, including NLOGSPACE, P, NP, PSPACE, EXPTIME, and many more. reducibility, completeness.
7. Advanced topics: Tree Automata, quantum automata, probabilistic automata, interactive proof systems, oracle computations, cryptography.
29
Who should take this course?
YOU
30
Languages
The terms language and word are used in a strict technical sense in this course:
A language is a set of words.
A word is a sequence (or string) of symbols.
(or ) denotes the empty word, the sequence of zero symbols.
31
Symbols and Alphabets
• What is a symbol, then?
• Anything, but it has to come from an alphabet which is a finite set.
• A common (and important) instance is = {0, 1}.
, the empty word, is never an symbol of an alphabet.
32
Computation
CPU memory
33
CPU
input memory
output memory
Program memory
temporary memory
34
CPU
input memory
output memoryProgram memory
temporary memory
3)( xxf
compute xx
compute xx 2
Example:
35
CPU
input memory
output memoryProgram memory
temporary memory
3)( xxf
compute xx
compute xx 2
2x
36
CPU
input memory
output memoryProgram memory
temporary memory3)( xxf
compute xx
compute xx 2
2x
42*2 z82*)( zxf
37
CPU
input memory
output memoryProgram memory
temporary memory3)( xxf
compute xx
compute xx 2
2x
42*2 z82*)( zxf
8)( xf
38
Automaton
CPU
input memory
output memory
Program memory
temporary memory
Automaton
39
Different Kinds of Automata
Automata are distinguished by the temporary memory
• Finite Automata: no temporary memory
• Pushdown Automata: stack
• Turing Machines: random access memory
40
input memory
output memory
temporary memory
Finite
Automaton
Finite Automaton
Example: Vending Machines
(small computing power)
41
input memory
output memory
Stack
Pushdown
Automaton
Pushdown Automaton
Example: Compilers for Programming Languages
(medium computing power)
Push, Pop
42
input memory
output memory
Random Access Memory
Turing
Machine
Turing Machine
Examples: Any Algorithm
(highest computing power)
43
Finite
Automata
Pushdown
Automata
Turing
Machine
Power of Automata
Less power More power
Solve more
computational problems