Computational Models - Lecture 11tau-cm2019.wdfiles.com/local--files/course-schedule/DFA1.pdf · Roded Sharan (TAU) Computational Models, Lecture 1 March, 2019 9 / 58. Computability

Computational Models - Lecture 11

Handout Mode

Roded Sharan.

Tel Aviv University.

March, 2019

1Based on frames by Benny Chor and Iftach Haitner, TAU; Maurice Herlihy, Brown University.

Roded Sharan (TAU) Computational Models, Lecture 1 March, 2019 1 / 58

Part I

Administrativia


The teaching team

Instructors:Roded Sharan, roded at tau.ac.ilBenny Chor, bchor at cs.tau.ac.il

Teaching assistants:Yuval Moskovitch, mosyuval at gmail.comMark Rozanov, markroza at tau.ac.ilNoam Mazor, noammaz at gmail.com


Administrativia

Course website: http://tau-cm2019.wikidot.com

Site (containing a forum) is our sole mean of disseminating information.

Course Requirements:I 6 problem sets (10% of grade).

Submission via MoodleI Readable, concise, correct answers expected.I Late submission will not be accepted. (You have between one and

two weeks, start working when you get them. Any excuse has tocover all the period.)

I See more instructions on the course website.

I Solving problems independently is highly recommended.I (variants of) questions from HW in exams!


AdministraTrivia II

I Midterm, covering first 5 lectures. We will discuss it when we getcloser.

I Midterm (magen) is scheduled to April 17, 2019.I Final exam, covering all course material.I You must pass the final exam to get a passing course grade.I Final Grade:

0.75 · Exam + 0.15 ·max{Midterm,Exam}+ 0.10 · HW .I Prerequisites (formally): Extended introduction to computer

scienceI But most importantly is “mathematical maturity”.

I Textbook: Sipser — Introduction to the theory of computation, firstor second editions.

I Other (excelent) book: Hopcroft, Motwani, and Ullman —Introduction to Automata Theory, Languages, and Computation.


Part II

Course overview


Why study theory?

I Basic computer science issuesI What is a computation?I Are computers omnipotent?I What are the fundamental capabilities and limitations of computers?

I Pragmatic reasonsI Avoid intractable or impossible problems.I Apply efficient algorithms when possible.I Learn to tell the difference.


Course Topics

I Automata Theory: Basic model of computation.I Re-invented in many other disciplines.

I Computability Theory: What can computers do?I True impossibility results.

I Complexity Theory: What makes some problems computationallyhard and others easy?


Automata theory – Simple models

I Finite automata.I Related to controllers and hardware design.I Useful in text processing and finding patterns in strings.I Probabilistic (Markov) versions useful in modeling various natural

phenomena (e.g. speech recognition).I Push down automata.

I Tightly related to a family of languages known as context freelanguages.

I Play important role in compilers, design of programming languages,and studies of natural languages.


Computability Theory

In the first half of the 20th century, mathematicians such as KurtGöedel, Alan Turing, and Alonzo Church discovered that somefundamental problems cannot be solved by computers.I Proof verification of statements can be automated.I It is natural to expect that determining validity can also be done by

a computer.I Theorem: A computer cannot determine if a mathematical

statement is true or false.I Results needed theoretical models for computers.I These theoretical models helped lead to the construction of real

computers.


Complexity Theory

Key notion: tractable vs. intractable problems.I A problem is a general computational question:

I description of parametersI description of solution

I An algorithm is a step-by-step procedureI a recipeI a computer programI a mathematical object

I We want the most efficient algorithmsI fastest (usually)I most economical with memory (sometimes)I expressed as a function of problem size


Example: Traveling Salesman Problem

10 9

36

9

a

b

c

d

5

Roger WilliamsZoo

Brown UniversityAl FornoRestaurant

StateCapitol

(not drawn to scale)

Input:I set of citiesI set of inter-city distances

Goal: want the shortest tour through the citiesRoded Sharan (TAU) Computational Models, Lecture 1 March, 2019 12 / 58

Example: Traveling Salesman Problem

Example: a,b,d , c,a has length 27


Problem Size

I What is an appropriate measure of problem size?I m nodes?I m(m + 1)/2 distances?

I Encoding of a problemI alphabet of symbolsI strings: a/b/c/d//10/5/9//6/9//3.

I MeasuresI Problem Size: length of encoding (above: 23 ascii characters).I Time Complexity: how long an algorithm runs, as function of

problem size?


Time Complexity - What is tractable?

I A function f (n) is O(g(n)), if there is a constant c such that forlarge enough n, |f (n)| ≤ c · |g(n)|.

I A polynomial-time algorithm is one whose time complexity isO(p(n)) for some polynomial p(n), where n denotes the length ofthe input.

I An exponential-time algorithm is one that runs in time 2cn for somec > 0 (thus, not polynomial time ).


Tractability: Polynomial is Good, Exponential is Bad

How would execution time for a fast, modern processor (1010 ops per second,say) vary for a task with the following time complexities andn = input sizes? (Modified from Garey and Johnson’s classical book.)

I Polynomial time = tractable (“good”).

I Exponential time = intractable (“bad”).

10 20 30 40 50 60

n 10−9 2 · 10−9 3 · 10−9 4 · 10−9 5 · 10−9 6 · 10−9

seconds seconds seconds seconds seconds seconds

n2 10−8 4 · 10−8 9 · 10−8 1.6 · 10−7 2.5 · 10−7 3.6 · 10−7


n3 10−7 8 · 10−7 2.7 · 10−6 6.4 · 10−6 1.2 · 10−5 2.2 · 10−5


n5 10−6 0.00032 0.00243 0.01024 0.03125 0.07776seconds seconds seconds seconds seconds seconds

2n 10−7 10−4 0.107 1 : 50 1.3 3.66seconds seconds seconds minutes days years

3n 5 · 10−6 0.34 2 2300 1.3 · 106 8 · 1010seconds minutes weeks years centuries centuries


NP-Completeness / NP-Hardness

I A set of important and interesting optimization problems.I Searching exhaustively for solution is exponential.I All known algorithms are exponential.I Actual complexity not proven.I Believed to require exponential time.


Part III

Lecture 1


Outline of today’s class

I Languages, words and alphabetsI Finite automata and regular languagesI Regular operations

I Sipser’s book, chapter 1.1


Part IV

Languages, words and alphabets


Languages, words and alphabets

Definition 1An alphabet Σ is a finite set of letters.

I Σ = {a,b, c, . . . , z} – the English alphabet.I Σ = {α, β, γ, . . . , ζ} – the Greek alphabet.I Σ = {0,1} – the binary alphabet.I Σ = {0,1, . . . ,9} – the digital alphabet.

Definition 2A word (i.e., string) over Σ, is a finite sequence of letters from Σ.

The collection of all strings over Σ is denoted by Σ∗.

For {0,1}, the binary alphabet, ε, 1, 0, 000000000, 1111111000 are allmembers of Σ∗.

Definition 3A language over Σ is a (possibly infinite) subset of Σ∗.


Language Examples

I Modern English.I Ancient Greek.I All prime numbers, written using digits.I A = {w ∈ {0,1}∗ : w has at most seventeen 0’s}.I B = {0n1n : n ≥ 0}.I C = {w ∈ {0,1}∗ : w has an equal number of 0’s and 1’s}.

Make sure you understand what the above notions stand for...


Part V

Finite Automata


Example: A One-Way Automatic Door

frontpad

rearpad

door

I open when person approachesI hold open until person clearsI don’t open when someone standing behind door


The Automatic Door as DFA

closed open

FRONT

NEITHER

FRONTREARBOTH

REARBOTH

NEITHER

I States:I OPENI CLOSED

I Sensor:I FRONT: someone on front padI REAR: someone on rear padI BOTH: someone(s) on both padsI NEITHER no one on either pad.


The Automatic Door as DFA

A DFA is Deterministic Finite Automata

closed open

FRONT

NEITHER

FRONTREARBOTH

REARBOTH

NEITHER

neither front rear bothclosed closed open closed closedopen closed open open open


DFA: Informal definition

The machine M1:

q1

0

q2 3q

11 0

0,1

I States: q1,q2, and q3.I Start state: q1 (arrow from “outside”).I Accept state: q2 (double circle).I State transitions: arrows tagged with letters.


DFA: Informal definition (cont.)

q1

0

q2 3q

11 0

0,1

I On an input stringI DFA begins in start state q1I after reading each symbol, DFA makes

state transition with matching label.I After reading last symbol, DFA “produces" output:

I accept if DFA is an accepting state.I reject otherwise.


DFA: Informal definition (cont..)

q1

0

q2 3q

11 0

0,1

What happens on the following input strings:I 1101I 0010I 01100I In general?!


DFA: Informal definition (cont...)

q1

0

q2 3q

11 0

0,1

This DFA acceptsI All input strings that end with a 1I All input strings that contain at least one 1, and end with an even

number of 0’sI No other strings

Proof: ?


DFA - Formal Definition

Definition 4A deterministic finite automaton (DFA) is a 5-tuple (Q,Σ, δ,q0,F ),where

I Q is a finite set called the statesI Σ is a finite set called the alphabetI δ : Q × Σ→ Q is the transition functionI q0 ∈ Q is the start stateI F ⊆ Q is the set of accept states


Back to M1

q1

0

q2 3q

11 0

0,1

M1 = (Q,Σ, δ,q1,F ) whereI Q = {q1,q2,q3}, Σ = {0,1},

I the transition function δ is

0 1q1 q1 q2q2 q3 q2q3 q2 q2

I q1 is the start stateI F = {q2}.


Another Example

q1

q2

a b

a

a a

a

b

b

b

b

r1

s

r2


Formal Model of Computation

Definition 5

M = (Q,Σ, δ,q0,F ) accepts w ∈ Σ∗ if δ̂M(q0,w) ∈ F .

Definition 6 (δ̂)

δ̂M : Q × Σ∗ 7→ Q is defined by

δ̂M(q,w) =

{δ(δ̂M(q,w1,...,n−1),wn), n = |w | ≥ 1q, w = ε.

.

I w1,...,k stands for the word w1, . . . ,wk — the k -letter prefix of w(w1,...,0 = w∅ = ε)

I Note that δ̂M(q, σ) = δ(q, σ) for σ ∈ Σ.I We write δ̂ when M is clear from the context.


An equivalent definition

Definition 7 (Equivalent definition)M = (Q,Σ, δ,q0,F ) accepts w = w1w2 . . .wn, if ∃r0, . . . , rn ∈ Q s.t.,I r0 = q0.I δ(ri ,wi+1) = ri+1, for all 0 ≤ i < n.I rn ∈ F .


The language of a DFA

Definition 8L(M), the language of a DFA M, is the set of strings that M accepts.

We sometime say that M accepts the language L(M).

I M may accept many stringsI M accepts only one language.

What language does M accept if it accepts no strings?

Definition 9A language is called regular, if some deterministic finite automatonaccepts it.


Example: DFA M2

I Q = {q1,q2}, Σ = {0,1}, F = {q2}, δ =?

I What is L(M2) := {w ∈ {0,1}∗ : δ̂(q1,w) = q2}?

Theorem 10L(M2) = {w ∈ {0,1}∗ : #1(w) is odd}. #1(w) — number of ones inw.

I Proof by induction on the word lengthI What does it mean?

The j ’th assumption: w ∈ {0,1}j is in L(M) iff #1(w) is odd


Proving L(M2) = L := {w ∈ Σ∗ : #1(w) is odd}

Basis (length 0):I δ̂(q1, ε) = q1 =⇒ ε /∈ L(M2).I Since ε /∈ L, we have that x ∈ L(M2)⇐⇒ x ∈ L

Step: assume hypothesis holds for words of length j ≥ 0.

I Let x = yσ ∈ {0,1}j+1 for σ ∈ {0,1} (hence, |y | = j).I If #1(y) is even. By assumption δ̂(q1, y) = q1.

I σ = 1 =⇒1. #1(x) is odd =⇒ x ∈ L2. δ̂(q1, x) := δ(δ̂(q1, y), σ) = δ(q1, 1) = q2 =⇒ x ∈ L(M2).

I σ = 0 =⇒1. #1(x) is even =⇒ x /∈ L2. δ̂(q1, x) := δ(δ̂(q1, y), σ) = δ(q1, 0) = q1 =⇒ x /∈ L(M2).

I If #1(y) is odd...I We proved for x ∈ {0,1}j+1: x ∈ L(M2)⇐⇒ x ∈ L


The language of M1

q1

0

q2 3q

11 0

0,1

Theorem 11

L(M1) = {w102k : k ≥ 0,w ∈ {0,1}∗}

Proof:

Claim 12 (implies the theorem)

Let L′i = {x ∈ {0,1}∗ : δ̂(q1, x) = qi} and letI L1 = {0k : k ≥ 0}I L2 = {w102k : k ≥ 0,w ∈ {0,1}∗}I L3 = {w102k+1 : k ≥ 0,w ∈ {0,1}∗}

Then, L′i = Li for every i ∈ {1,2,3}


Proving Claim 12

We need to prove that ∀i ∈ {1,2,3}:

x ∈ Li ⇐⇒ x ∈ L′i .

Recall that

x ∈ L′i ⇐⇒ δ̂(x) = qi

Proof by induction on word length.

I Basis: Same line as the previous proof, noting that ε ∈ L1 andε ∈ L′1.

I Step: Assume hypothesis holds for all words of length j ≥ 0.I Let x = yσ ∈ {0,1}j+1 for σ ∈ {0,1}.I We prove the hypothesis for x , for each i ∈ {1,2,3} separately


L1 = {0k : k ≥ 0}

Recall x = yσ ∈ {0,1}j+1 for σ ∈ {0,1}.

Proving: x ∈ L1 =⇒ δ̂(q1, x) = q1 ( =⇒ x ∈ L′1).

I x = 0j+1, y = 0j and σ = 0.I Since y ∈ L1, by i.h. δ̂(q1, y) = q1

I Therefore, δ̂(q1, x) = δ(δ̂(q, y), σ) = δ(q1,0) = q1.

Proving: δ̂(q1, x) = q1 =⇒ x ∈ L1.

I Let qy = δ̂(q1, y) (hence, δ̂(q1, x) = δ(qy , σ) = q1)I qy = q1 and σ = 0. (?)I By i.h. y = 0j .I Hence, x = yσ = 0j0 = 0j+1 ∈ L1.


L2 = {w102k : k ≥ 0,w ∈ {0,1}∗}

Recall x = yσ ∈ {0,1}j+1 for σ ∈ {0,1}.

Proving x ∈ L2 =⇒ δ̂(q1, x) = q2.

I Assume σ = 1I Since δ(qi ,1) = q2 for any i =⇒ δ̂(q1, x) = q2.

I Assume x = w102k for k > 0 ( =⇒ y = w102k−1 and σ = 0)I Hence, y ∈ L3.I By i.h. δ̂(q1, y) = q3I Thus, δ̂(q1, x) = δ(q3,0) = q2.

Proving δ̂(q1, x) = q2 =⇒ x ∈ L2.

I Assume σ = 1 =⇒ x ∈ L2. (?)I Assume σ = 0

I qy := δ̂(q1, y) = q3(?)I By i.h. y = w102k+1 for some k ≥ 0I Therefore x = yσ = w102k+10 ∈ L2.


L3 = {w102k+1 : k ≥ 0,w ∈ {0,1}∗}

Recall x = yσ ∈ {0,1}j+1 for σ ∈ {0,1}.

Proving x ∈ L3 =⇒ δ̂(q1, x) = q3.

I x = w102k+1, y = w102k and σ = 0I y ∈ L2

I By i.h. δ̂(q1, y) = q2.I Therefore, δ̂(q1, x) = δ(q2,0) = q3.

Proving δ̂(q1, x) = q3 =⇒ x ∈ L3.

I Let qy = δ̂(q1, y)

I Hence, qy = q2 and σ = 0 (?)I By i.h. y = w102k

I Therefore, x = yσ = w102k0 ∈ L3.


Part VI

Regular Operations


Additional examples of regular languages

Let Σ = {0,1}.I {w ∈ {0,1}∗ : #1(w) ≡ 0 mod 7}.I Sequence of 0 followed by sequence of 1, i.e., {0m1n : m,n ≥ 0}.I Any finite language.

All the above languages are regular

Is there a simple proof?


The regular operations

Let A and B be languages.

The union operation:

A∪B = {x : x ∈ A ∨ x ∈ B}

The concatenation operation:

A‖B = {xy : x ∈ A ∧ y ∈ B}

The star operation:

A∗ = {x1x2 . . . xk : k ≥ 0 and each xi ∈ A}


The regular operations – Examples

Let A= {good, bad} and B = {boy, girl}.

UnionA ∪ B = {good, bad, boy, girl}

Concatenation

A‖B = {goodboy, goodgirl, badboy, badgirl}

StarA∗ = {ε, good, bad, goodgood, goodbad, badbad, badgood, . . .}


Closure under union

Theorem 13If L1 and L2 are regular languages, then so is L1 ∪ L2.

Approach to proof:I Some DFA M1 accepts L1

I Some DFA M2 accepts L2

I Construct DFA M that accepts L1 ∪ L2.

Attempted proof idea:I first emulate M1, andI if M1 doesn’t accept, then emulate M2.

What’s wrong with this?Fix: Emulate both machines simultaneously.


Closure Under Union: Correct Proof

SupposeI M1 = (Q1,Σ, δ1,q1,F1) accepts L1,I M2 = (Q2,Σ, δ2,q2,F2) accepts L2.

Define M as follows (M will accept L1 ∪ L2):I Q = Q1 ×Q2.I Σ is the same.I For each (r1, r2) ∈ Q and a ∈ Σ,δ((r1, r2),a) = (δ1(r1,a), δ2(r2,a))

I q0 = (q1,q2)

I F = {(r1, r2) : r1 ∈ F1 or r2 ∈ F2}.

Why not choose F = F1 × F2?


Correctness of the construction

Claim 14

L(M) = L(M1) ∪ L(M2).

Follows by the next claim.

Claim 15

δ̂M((q1,q2), x) = (δ̂1(q1, x), δ̂2(q2, x)), for δ̂i = δ̂Mi .

Proof: By induction on word length. DIY ... ♣Proving Claim 14:I x ∈ L(M1) =⇒ δ̂1(q1, x) = r1 ∈ F1. (similar if x ∈ L(M2).)I By Claim 15, δ̂M((q1,q2), x) = (r1, ·) ∈ F =⇒ x ∈ L(M).

I x ∈ L(M) =⇒ δ̂M((q1,q2), x) = (r1, r2) ∈ F .I By Claim 15, (r1, r2) = (δ̂1(q1, x), δ̂2(q2, x)).I Hence, either r1 ∈ F1 or r2 ∈ F2 =⇒ x ∈ L(M1) ∪ L(M2).


What about concatenation?

Theorem 16If L1, L2 are regular languages, then so is L1‖L2.

Example: L1= {good, bad} and L2 = {boy, girl}.

L1‖L2 = {goodboy, goodgirl, badboy, badgirl}

This is much harder to prove.

Idea: Simulate M1 for a while, then switch to M2.

Problem: But when do you switch?

This leads us into non-determinism, wait for next class...


Part VII

Non-deterministic Finite Automata


NFA — non-deterministic Finite Automata

q4

q1

q2

q3

0

0,1 0,1

10

I May have more than one transition labeled with the same symbol,I May have no transitions labeled with a certain symbol,I May have transitions labeled with ε, the symbol of the empty

string. Will deal with this later

Every DFA “is” also an NFA.


Non-deterministic computation

q4

q1

q2

q3

0

0,1 0,1

10

What happens when more than one transition is possible?I The machine “splits” into multiple copiesI Each branch follows one possibilityI Together, branches follow all possibilities.I If the input doesn’t appear, that branch “dies”.I Automaton accepts if some branch accepts.


Computation on 1001

q4

q1

q2

q3

0

0,1 0,1

10

q1

q1

q2

q1

q2

q1

q3

q1

q4

1

0

0

1

symbol


Why non-determinism?

Theorem 17 (Informal, to be proved soon)Deterministic and non-deterministic finite automata, accept exactly thesame set of languages.

Q.: So why do we need NFA’s?

Design a finite automaton for the language L— all binary strings witha 1 in their third-to-the-last position?


NFA for L

q4

q1

q2

q3

1

0,1

0,10,1

I “Guesses” which symbol is third from the last, andI checks that indeed it is a 1.I If guess is premature, that branch “dies”, and no harm occurs.


DFA for LI Have 8 states, encoding the last three observed letters.

I A state for each string in {0,1}3.

I Add transitions on modifying the suffix, give the new letter.

I Mark as accepting, the strings 1 ∗ ∗

1

q000

q111

q001

q010

q011

q100

q101

q110

0

0 0

0

000

1 1

1

1 1 1

DFA has few bugs...Roded Sharan (TAU) Computational Models, Lecture 1 March, 2019 58 / 58

Computational Models - Lecture 11tau-cm2019.wdfiles.com/local--files/course-schedule/DFA1.pdf · Roded Sharan (TAU) Computational Models, Lecture 1 March, 2019 9 / 58. Computability

Documents