1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 3 Mälardalen University…

1

CD5560

FABER

Formal Languages, Automata and Models of Computation

Lecture 3

Mälardalen University2006

2

Content

Finite Automata, FADeterministic Finite Automata, DFANondeterministic Automata NFANFA DFA EquivalenceGrammatikLinjär grammatikReguljär grammatik

3

Finite Automata FA(Finite State Machines)

4

There is no formal general definition for "automaton". Instead, there are various kinds of automata, each with it's own formal definition.

• has some form of input

• has some form of output

• has internal states,

• may or may not have some form of storage

• is hard-wired rather than programmable

Generally, an automaton

5

Finite Automaton

Input

String

Output

String

FiniteAutomaton

6

Finite Accepter

Input

“Accept” or“Reject”

String

FiniteAutomaton

Output

7

Nodes = States Edges = Transitions

An edge with several symbols is a short-hand for several edges:

FA as Directed Graph

1qa

0q

1qba,0q1q

a

0qb

8

Deterministic Finite Automata DFA

9

• Deterministic there is no element of choice• Finite only a finite number of states and arcs • Acceptors produce only a yes/no answer

DFA

10

Transition Graph

initialstate

final state“accept”state

transition

abba -Finite Acceptor

0q 1q 2q 3q 4qa b b a

5q

a a bb

ba,

ba,

},{ baAlphabet =

11

Formal definitionsDeterministic Finite Accepter (DFA)

FqQM ,,,, 0

Q

0q

F

: set of states: input alphabet: transition function: initial state: set of final states

12

Input Aplhabet


5q

a a bb

ba,

ba,

ba,

13

Set of States

Q


5q

a a bb

ba,

543210 ,,,,, qqqqqqQ

ba,

14

Initial State

0q

1q 2q 3q 4qa b b a

5q

a a bb

ba,

ba,0q

15

Set of Final States

F

0q 1q 2q 3qa b b a

5q

a a bb

ba, 4qF

ba,

4q

16

Transition Function


5q

a a bb

ba,

QQ :

ba,

17

10 , qaq

2q 3q 4qa b b a

5q

a a bb

ba,

ba,0q 1q

18

50 , qbq

1q 2q 3q 4qa b b a

5q

a a bb

ba,

ba,0q

19


5q

a a bb

ba,

ba,

32 , qbq

20

Transition Function


5q

a a bb

ba,

ba,

a b0q

2q

5q

1q 5q

5q5q

3q

4q

1q 5q 2q5q 3q4q 5q5q 5q

21

Extended Transition Function

*

QQ *:*


5q

a a bb

ba,

ba,

22

20 ,* qabq

3q 4qa b b a

5q

a a bb

ba,

ba,0q 1q 2q

23

40 ,* qabbaq


5q

a a bb

ba,

ba,

24

Another Example


5q

a a bb

ba,

ba,

abbaabML ,, M

acceptacceptaccept

},{ baAlphabet =

25

Formally

For a DFA

Language accepted by :

FqQM ,,,, 0

M

FwqwML ,*:* 0

alphabet transitionfunction

initialstate

finalstates

26

Observation

Language accepted by

FwqwML ,*:* 0

M

FwqwML ,*:* 0

MLanguage rejected by

27

Regular Languages

All regular languages form a language family

LM MLL

A language is regular if there is

a DFA such that

28

Nondeterministic Automata NFA

29

• Nondeterministic there is an element of choice: in a given state NFA

can act on a given string in different ways. Several start/final states are allowed. -transitions are allowed.

• Finite only a finite number of states and arcs • Acceptors produce only a yes/no answer

NFA

30

1q 2q

3q

a

a

a

0q

Two choices

}{aAlphabet =

Nondeterministic Finite Accepter (NFA)

31

a a

0q

1q 2q

3q

a

a

First Choice

a

32

a a

0q

1q 2q

3q

a

a

a

First Choice

33

a a

0q

1q 2q

3q

a

a

First Choice

a

34

a a

0q

1q 2q

3q

a

a

a “accept”

First Choice

35

a a

0q

1q 2q

3q

a

a

Second Choice

a

36

a a

0q

1q 2qa

a

Second Choice

a

3q

37

a a

0q

1q 2qa

a

a

3q

Second Choice

No transition:the automaton hangs

38

a a

0q

1q 2qa

a

a

3q

Second Choice

“reject”

39

Observation

An NFA accepts a string ifthere is a computation of the NFAthat accepts the string

40

Exampleaa is accepted by the NFA:

0q

1q 2q

3q

a

a

a

41

Lambda Transitions

1q 3qa0q 1q a

42

a a

1q 3qa0q 2q a

43

a a

1q 3qa0q 2q a

44

a a

1q 3qa0q 2q a

(read head doesn’t move)

45

a a

1q 3qa0q 2q a

46

a a

1q 3qa0q 2q a

“accept”

String is acceptedaa

47

Language accepted: }{aaL

1q 3qa0q 2q a

48

Another NFA Example

0q 1q 2qa b

3q

},{ baAlphabet =

49

a b

0q 1q 2qa b

3q

50

a b

0q 2qa b

3q1q

51

a b

0q 1qa b

3q2q

52

a b

0q 1qa b

3q2q“accept”

53

a b

Another String

a b

0q a b

1q 2q 3q

},{ baAlphabet =

54

a b a b

0q a b

1q 2q 3q

55

a b a b

0q a b

1q 2q 3q

56

a b a b

0q a b

1q 2q 3q

57

a b a b

0q a b

1q 2q 3q

58

a b a b

0q a b

1q 2q 3q

59

a b a b

0q a b

1q 2q 3q

60

a b a b

0q a b

1q 2q 3q“accept”

61

ab

ababababababL ...,,,

Language accepted

0q 1q 2qa b

3q

},{ baAlphabet =

62

Another NFA Example

0q 1q 2q0

11,0

}1,0{Alphabet =

63

*10

...,101010,1010,10, L

0q 1q 2q0

11,0

Language accepted

64

Formal Definition of NFA FqQM ,,,, 0

:Q

::0q

:F

Set of states, i.e. 210 ,, qqq

: Input alphabet, i.e. ba,Transition function

Initial state

Final states

65

10 1, qq

Transition Function

0

11,0

0q 1q 2q

66

},{)0,( 201 qqq

0q0

11,0

1q 2q

67

0q0

11,0

1q 2q

},{),( 200 qqq

68

0q0

11,0

1q 2q

)1,( 2q

69

Extended Transition Function

*

10 ,* qaq

0q

5q4q

3q2q1qaaa

b

(Utvidgad övergångsfunktion)

70

540 ,,* qqaaq

0q

5q4q

3q2q1qaaa

b

71

0320 ,,,* qqqabq

0q

5q4q

3q2q1qaaa

b

72

Formally

wqq ij ,*

if and only if

there is a walk from towith label

iq jqw

73

The Language of an NFA

540 ,,* qqaaq

0q

5q4q

3q2q1qaaa

b

M

)(MLaa

50 ,qqF

},{ baAlphabet =

74

0320 ,,,* qqqabq

0q

5q4q

3q2q1qaaa

b

MLab

50 ,qqF

75

50 ,qqF

540 ,,* qqabaaq )(MLabaa

0q

5q4q

3q2q1qaaa

b

76

10 ,* qabaq MLaba

0q

5q4q

3q2q1qaaa

b

50 ,qqF

77

0q

5q4q

3q2q1qaaa

b

aaababaaML *

78

Formally

The language accepted by NFA M

,...,, 321 wwwML

,...},{),(* 0 jim qqwq

Fqk (final state)

where

and there is some (at least one)

is:

79

MLw

0q kq

w

w

w

),(* 0 wq

Fqk

iq

jq

80

NFA DFA Equivalence

81

Equivalence of NFAs and DFAs

Accept the same languages?

YES!

NFAs DFAs ?

Same power?

82

We will prove:

Languages acceptedby NFAs

Languages acceptedby DFAs

NFAs and DFAs have the same computation power!

83



Step 1

Proof Every DFA is also an NFA

A language accepted by a DFAis also accepted by an NFA

84



Step 2

Proof Any NFA can be converted to anequivalent DFA

A language accepted by an NFAis also accepted by a DFA

85

Procedure NFA to DFA

1. Initial state of NFA:

Initial state of DFA:

0q

0q

86

Example

a

b

a

0q 1q 2q

NFA

DFA 0q

Step 1

87

Procedure NFA to DFA 2. For every DFA’s state

Compute in the NFA

},...,,{ mji qqq

...

,,*,,*

aqaq

j

i

},...,,{},,...,,{ mjimji qqqaqqq

},...,,{ mji qqq

Add transition

88

Example NFA

DFA

a

b

a

0q 1q 2q

},{),(* 210 qqaq

0q 21,qqa

210 ,, qqaq

Step 2

89


Repeat Step 2 for all letters in alphabet, until

no more transitions can be added.

90

Example

a

b

a

0q 1q 2q

NFA

DFA 0q 21,qq

a

b

ab

ba,

Step 3

91


3. For any DFA state

If some is a final state in the NFA

Then is a final state in the DFA

},...,,{ mji qqq

jq

},...,,{ mji qqq

92

Example

a

b

a0q 1q 2q

NFA

DFA

0q 21,qqa

b

ab

ba,

Fq 1

Fqq 21,

Step 4

93

Theorem

Take NFA M

Apply procedure to obtain DFA M

Then and are equivalent :M M

MLML

94



We have proven (proof by construction):

Regular Languages

END OF PROOF

95

Nondeterministic vs.

Deterministic Automata

96

Formal Definition of NFA

FqQM ,,,, 0

:Q Set of states, i.e. 210 ,, qqq: Input alphabet, i.e. ba,: Transition function:0q Initial state

:F Final (accepting) states

NFA is a mathematical model defined as a quintuple:

97

Deterministic Finite Automata

A deterministic finite automaton (DFA) is a special case of a nondeterministic finite automaton in which1. no state has an -transition, i.e. a transition

on input , and2. for each state q and input symbol a, there

is at most one edge labeled a leaving q.

98

STATEINPUT SYMBOL

a b

012

{0, 1}--

{0}{2}{3}

Transition table for the finite automaton above

A nondeterministic finite automaton

b

0start 1a 2b b3

a

Example

99

NFA accepting aa* + bb*

0start

1

a2

a

3

b 4

b

Example

100

NFA accepting (a+b)*abb

0start 1a 2b b

b

a a

a

b

3

a

Example

101

NFA recognizing three different patterns.

(a) NFA for a, abb, and a*b+.

(b) Combined NFA.

Example

4

1start a 2

3start a 65b b

7start b 8

ba

4

1

start

a 2

3 a 65b b

7 b 8

ba0

102

Ways to think of nondeterminism

• always make the correct guess• “backtracking” (systematically try all possibilities)

For a particular string, imagine a tree of possiblestate transitions:

q0

q3

q0

q4

q2

q1a

a

aa

b

a

103

Advantages of nondeterminism

• an NFA can be smaller, easier to construct and easier to understand than a DFA that accepts the same language

• useful for proving some theorems• good introduction to nondeterminism in more

powerful computational models, where nondeterminism plays an important role

104

Space and time taken to recognize regular expressions:- NFA more compact but take time to backtrack all choices- DFA take place, but save time

AUTOMATON SPACE TIME

NFADFA

O(|r|)O(2|r|)

O(|r||x|)O(|x|)

Determinism vs. nondeterminism

(Where r is regular expression, and x is input string)

105

Equivalent automata

Two finite automata M1 and M2 are equivalent if

L(M1) = L(M2)

that is, if they both accept the same language.

106

Equivalence of NFAs and DFAs

To show that NFAs and DFAs accept the same class of languages, we show two things:

– Any language accepted by a DFA can also be accepted by some NFA (As DFA is a special case of NFA)

– Any language accepted by a NFA can also be accepted by some (corresponding, specially constructed) DFA

107

Proof strategy

To show that any language accepted by a NFA is also accepted by some DFA, we describe an algorithm that takes any NFA and converts it into a DFA that accepts the same language.

The algorithm is called the “subset construction algorithm”.

We can use mathematical induction (on the length of a string accepted by the automaton) to prove the DFA that is constructed accepts the same language as the NFA.

108

Converting NFA to DFA Subset Construction

109

Subset construction

Given a NFA constructs a DFA that accepts the same language

The equivalent DFA simulates the NFA by keeping track of the possible states it could be in. Each state of the DFA is a subset of the set of states of the NFA -hence, the name of the algorithm.

If the NFA has n states, the DFA can have as many as 2n states, although it usually has many less.

110

Steps of subset construction

The initial state of the DFA is the set of all states the NFA can be in without reading any input.

For any state {qi,qj,…,qk} of the DFA and any input a, the next state of the DFA is the set of all states of the NFA that can result as next states if the NFA is in any of the states qi,qj,…,qk when it reads a. This includes states that can be reached by reading a, followed by any number of -moves. Use this rule to keep adding new states and transitions until it is no longer possible to do so.

The accepting states of the DFA are those states that contain an accepting state of the NFA.

111

Example

Here is a NFA that we want to convert to an equivalent DFA.

a

bb

01

2

b

a

a

b

112

{0,1}

The start state of the DFA is the set of states the NFA can be in before reading any input. This includes the start state of the NFA and any states that can be reached by a -transition.

a

bb

b

a

a

01

2

b

NFA

DFA

113

{0,1}

a

b

{2}

For start state {0,1}, make transitions for each possible input, here a and b. Reading b from start {0,1}, we reach state {2}. Means from either {0}, or {1} we reach {2}.

a

bb

b

a

a

01

2

b

NFADFA

114

For state {2}, we create a transition for each possible input, a and b. From {2}, with b we are either back to {2} (loop) or we reach {1}- see the little framed original NFA. So from {2}, with b we end in state {1, 2}. Reading a leads us from {2} to {0} in the original NFA, which means state {0, 1} in the new DFA.

ba

a

b{0,1}{1,2}

{2}

a

bb

b

a

a

01

2

b

NFADFA

115

For state {1, 2}, we make again transition for eachpossible input, a and b. From {2} a leads us to {0}. From {1} with a we are back to {1}. So, we reach {0, 1} with a from {1,2}. With b we are back to {1,2}.

At this point, a transition is defined for every state-input pair.

ba

a

b{0,1}

{1,2}

{2}

b

a

DFA a

bb

b

a

a

01

2

b

NFA

116

The last step is to mark the final states of the DFA.As {1} was the accepting state in NFA, all states containing {1} in DFA will be accepting states: ({0, 1} and {1, 2}).

ba

a

b{0,1} {1,2}

{2}

b

a

DFA a

bb

b

a

a

01

2

b

NFA

117

Subset Construction Algorithm

118

Subset Construction

States of nondeterministic M´ will correspond to sets of states of deterministic M

Where q0 is start state of M, use {q0} as start state of M´.

Accepting states of M´ will be those state-sets containing at least one accepting state of M.

119

Subset Construction (cont.)

For each state-set S and for each s in alphabet of M, we draw an arc labeled s from state S to that state-set consisting of all and only the s-successors of members of S.

Eliminate any state-set, as well as all arcs incident upon it, such that there is no path leading to it from {q0}.

120

The power set of a finite set, Q, consists of 2|Q| elements

The DFA corresponding to a given NFA with Q states have a finite number of states, 2|Q|.

If |Q| = 0 then Q is the empty set, | P(Q)| = 1 = 20.

If |Q| = N and N 1, we construct subset of a given set so that for each element of the initial set there are two alternatives, either is the element member of a subset or not. So we have

2 · 2 · 2 · 2 · 2 · 2 · 2…. ·2 = 2N

N times

121

From an NFA to a DFASubset ConstructionOperation Description

- closure(s)

- closure(T)

Move(T,a)

Set of NFA states reachable from an NFA state s on -transitions along

Set of NFA states reachable from some NFA state s in T on -transitions along

Set of NFA states reachable from some NFA state set with a transition on input symbol a

122

From an NFA to a DFA

Subset Construction

Initially, -closure (s0) is the only states in D and it is unmarked

while there is an unmarked state T in D do mark T; for each input symbol a do U:= e-closure(move(T,a)); if U is not in D then add U as an unmarked state to D Dtran[T,a]:=U; end(for) end(while)

123

Grammars

124

GrammarsGrammars express languages

Example: the English language

verbpredicate

nounarticlephrasenoun

predicatephrasenounsentence

_

_

125

barksverbsingsverb

dognounbirdnoun

thearticleaarticle

126

A derivation of “the bird sings”:

birdtheverbbirdtheverbnounthe

verbnounarticlepredicatenounarticlepredicatephrasenounsentence

sings

_

127

A derivation of “a dog barks”:

barksdogaverbdoga

verbnounaverbnounarticleverbphrasenounpredicatephrasenounsentence

__

128

The language of the grammar:

"ingssogdhet","barksogdhet"

,"ingssogda","barksogda"

,"ingssbirdhet","barksbirdhet"

,"ingssbirda","barksbirda"{L

}

}

129

Notation

dognoun

birdnoun

Non-terminal (Variable)

TerminalProduction rule

130

Example

Derivation of sentence:

SaSbS

ab

abaSbS

aSbS S

Grammar:

131

aabbaaSbbaSbS

aSbS S

aabb

SaSbSGrammar:

Derivation of sentence

132

Other derivations

aaabbbaaaSbbbaaSbbaSbS

aaaabbbbaaaaSbbbbaaaSbbbaaSbbaSbS

133

The language of the grammar

SaSbS

}0:{ nbaL nn

134

Formal Definition

Grammar PSTVG ,,,

:V Set of variables

:T Set of terminal symbols

:S Start variable

:P Set of production rules

135

ExampleGrammar

SaSbS

G

}{SV },{ baT

},{ SaSbSP

PSTVG ,,,

136

Sentential Form A sentence that contains variables and terminals

Example


sentential forms Sentence(sats)

137

We write:

Instead of:

aaabbbS*


138

nww*

1

nwwww 321

In general we write

if

By default ww*( )

139

Example

SaSbS

aaabbbS

aabbS

abS

S

*

*

*

*

Grammar Derivations

140

baaaaaSbbbbaaSbb

aaSbbS

SaSbS

GrammarExample

Derivations

141

Another Grammar Example

AaAbAAbS

Derivations

aabbbaaAbbbaAbbSabbaAbbAbS

bAbS

GGrammar

142

More Derivations

aaaabbbbbaaaaAbbbbbaaaAbbbbaaAbbbaAbbAbS

bbaS

bbbaaaaaabbbbS

aaaabbbbbS

nn

143

The Language of a Grammar

For a grammar with start variable G S

}:{)( wSwGL

String of terminals

144

ExampleFor grammar

AaAbAAbS

}0:{)( nbbaGL nn

Since bbaS nn

G

145

Notation

AaAbA

|aAbA

thearticleaarticle

theaarticle |

146

Linear Grammars

147

Linear Grammars

Grammars with at most one variable (non-terminal) at the right side of a production

AaAbAAbS

SaSbS

Examples:

148

A Non-Linear Grammar

bSaSaSbS

SSSS

Grammar G

)}()(:{)( wnwnwGL ba

149

Another Linear Grammar

Grammar

AbBaBAAS

|

}0:{)( nbaGL nn

G

150

Right-Linear Grammars

All productions have form: xBA

xA or

aSabSS

Example

151

Left-Linear Grammars

All productions have form BxA

aBBAabA

AabS

|

xA or

Example

152

Regular Grammars

153

Regular Grammars Generate

Regular Languages

154

Theorem

LanguagesGenerated byRegular Grammars

RegularLanguages

155

Theorem - Part 1


RegularLanguages

Any regular grammar generatesa regular language

156

Theorem - Part 2

Any regular language is generated by a regular grammar


RegularLanguages

157

Proof – Part 1

The language generated by any regular grammar is regular

)(GLG


RegularLanguages

158

The case of Right-Linear Grammars

Let be a right-linear grammar

We will prove: is regular

Proof idea We will construct NFA with

G

)(GL

M)()( GLML

159

Grammar is right-linearG

Example

aBbBBaaABaAS

|

|

160

Construct NFA such that every state is a grammar variable:

M

aBbBBaaABaAS

|

|

A

B

S specialfinal stateFV

161

Add edges for each production:

S FV

A

B

a

aBbBBaaABaAS

|

|

162

S FV

A

B

a

aBbBBaaABaAS

|

|

163

S FV

A

B

a

a

a

aBbBBaaABaAS

|

|

164

S FV

A

B

a

a

a

baBbB

BaaABaAS

|

|

165

S FV

A

B

a

a

a

b

a

aBbBBaaABaAS

|

|

166

aaabaaaabBaaaBaAS

S FV

A

B

a

a

a

b

a

167

S FV

A

B

a

a

a

b

a

abBBBaaABaAS

|

|

GM GrammarNFA

abaaaabGLML

**)()(

168

In GeneralA right-linear grammar

has variables:

and productions:

G

,,, 210 VVV

jmi VaaaV 21

mi aaaV 21or

169

We construct the NFA such that:

each variable corresponds to a node:

M

iV

0V 1V 2V FV

specialfinal state

….

170

For each production:

we add transitions and intermediate nodes

jmi VaaaV 21

iV jV………

1a 2a ma

171

Example

M

)()( MLGL

04

5193

452

48433421

23110

|

||

aVaVaV

VaVVaaaVaaV

VaVaV

0VFV

1V

2V

3V

4V

1a

3a3a

4a

8a

2a 4a5a

9a5a

9a

172

The case of Left-Linear Grammars

Let be a left-linear grammar

We will prove: is regular

Proof idea We will construct a right-linear grammar with

G

)(GL

G RGLGL )()(

173

Since is left-linear grammarthe productions look like:

G

kaaBaA 21

kaaaA 21

174

Construct right-linear grammar G

In :G kaaBaA 21

In :G BaaaA k 12

vBA

BvA R

175

Construct right-linear grammar G

In :G kaaaA 21

In :G 12aaaA k

vA

RvA

176

It is easy to see that:

Since is right-linear, we have:

RGLGL )()(

)(GL RGL )(

G

)(GLRegularLanguage

RegularLanguage

RegularLanguage

177

Proof - Part 2

Any regular language is generated by some regular grammar

LG


RegularLanguages

178

Proof idea

Any regular language is generated by some regular grammar

LG

Construct from a regular grammar such that

M G)()( GLML

Since is regularthere is an NFA such that

LM )(MLL

179

Example

*)*(* abbababL )(MLL

a

b

a

b

M1q 2q

3q

0q

180

a

b

a

b

M0q 1q 2q

3q

3

13

32

21

11

10

qqqbqqaqqbqqaqq

G

LMLGL )()(

Convert to a right-linear grammarM

181

In General

For any transition:aq p

Add production: apq

variable terminal variable

182

For any final state: fq

Add production: fq

183

Since is right-linear grammar

is also a regular grammar with

G

LMLGL )()(

G

184

Regular GrammarsA regular grammar is any right-linear or left-linear grammar

aSabSS

aBBAabA

AabS

|

1G 2G

Examples

185

ObservationRegular grammars generate regular languages

aSabSS

aabGL *)()( 1

aBBAabA

AabS

|

*)()( 2 abaabGL

1G 2GExamples

186

Regular Languages

Chomsky’s Language Hierarchy

Non-regular languages

1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 3 Mälardalen University…

Documents