Class1

Mathematical Preliminaries

Hw (p.13) 1, 4, 7, 8, 9, 13, 23, 26, 30, 32

2

Mathematical Preliminaries

• Sets • Functions• Relations• Graphs• Proof Techniques

3

A set is a collection of elements

SETS

}3,2,1{A},,,{ airplanebicyclebustrainB

We writeA1

Bship

1 is a member (or element) of set A

ship is not a member (or element) of set B

Membership of a given set

4

Set Representations

C = { a, b, c, d, e, f, g, h, i, j, k }

C = { a, b, …, k }

S = { 2, 4, 6, … }

S = { j : j > 0, and j = 2k for some integer k > 0 }

S = { j : j is nonnegative and even }

finite set

infinite set

5

A = { 1, 2, 3, 4, 5 }

Universal Set: all possible elements

1 2 34 5

AU

67

8910

U = { 1 , … , 10 }

6

Set OperationsA = { 1, 2, 3 } B = { 2, 3, 4, 5}

• Union A U B = { 1, 2, 3, 4, 5 } • Intersection

A B = { 2, 3 }• Difference

A - B = { 1 }B - A = { 4, 5 }

U

A B231 4

5

23

1

Venn diagrams

7

A

• ComplementUniversal set = {1, …, 7} A = { 1, 2, 3 } A = { 4, 5, 6, 7}

1 2 34

56

7A

A = A

8

024

61

35

7even

{ even integers } = { odd integers }

odd

Integers

9

DeMorgan’s Laws

A U B = A BU

A B = A U BU

10

Empty, Null Set:= { }

S U = S

S =

S - = S

- S =

U = Universal Set

11

SubsetA = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }

A B

U

Proper Subset: A B

UAB

12

Disjoint SetsA = { 1, 2, 3 } B = { 5, 6}

A B = UA B

13

Set Cardinality• For finite setsA = { 2, 5, 7 }

|A| = 3

(set size)

14

PowersetsA powerset is a set of subsets

Powerset of S = the set of all the subsets of S

S = { a, b, c }

2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }

Observation: | 2S | = 2|S| ( 8 = 23 )

15

Cartesian ProductA = { 2, 4 } B = { 2, 3, 5 }

A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }

A X B is an ordered set, i.e. A X B ≠ B X A

|A X B| = |A|·|B|

Generalizes to more than two sets

A X B X … X Z

16

Relation from sets A to BRelation from sets A to B Ex., A = { 1, 2, …, 7 }, B={ 1, 2, …,

50}– {(x, y): xA, yB, and y= x2} – {(x, y): xA, yB, and x < y}Are relations from A to B.

A relation on set A Equivalence relation a partition on set A

17

FUNCTIONSdomain

123

ab

c

range f : A -> B

A B

If A = domain then f is a total function otherwise f is a partial function

f(1) = a4

5

In general, we mean this.

18

GRAPHSA directed graph

• Nodes (Vertices) V = { a, b, c, d, e }• Edges (Ordered pairs) E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }

node

edge

ab

c

d

e

19

Labeled Graph

ab

c

d

e

1 35 6

262

20

Walk

ab

c

d

e

Walk is a sequence of adjacent edges(e, d), (d, c), (c, a)(e, d), (d, c), (c, a) is a walk from e to a of length 3 (or denoted as e-d-c-ae-d-c-a )Length = # of edges

21

Path

ab

c

d

e

PathPath is a walk where no edge is repeated

Simple path: no node is repeated

22

Path

ab

c

d

e

PathPath is a walk where no edge is repeated

Simple path: no node is repeated(e, b), (b, e), (e, d), (d, c), (c, a) is a path from e to a but it is not a simple path.

23

Cycle

ab

c

d

e

123

Cycle: a walk from a node (base) to itself without repeated edgesSimple cycle: only the base node is repeated Loop: an edge from a node to itself

base

24

Find All Simple Paths starting from c

ab

c

d

e

origin

The longest simple path has at most length 4.Since every vertex can only be visited at most once, and there are 4 other vertices.

25

(c, a)(c, e)

Step 1

ab

c

d

e

originStarting from vertex c, list all outgoing edges as long as they do not lead to any vertex already used in the path. At this point, we have all paths of length one starting at c.

For all vertices a,e reached by c, we list all outgoing edges originating at a or e according the same way we did before.

26

(c, a)(c, a), (a, b)(c, e)(c, e), (e, b)(c, e), (e, d)

Step 2

ab

c

d

e

origin

27

Step 3

ab

c

d

e

origin(c, a)(c, a), (a, b)(c, a), (a, b), (b, e)(c, e)(c, e), (e, b)(c, e), (e, d)

28

Step 4

ab

c

d

e

origin

(c, a)(c, a), (a, b)(c, a), (a, b), (b, e)(c, a), (a, b), (b, e), (e,d)(c, e)(c, e), (e, b)(c, e), (e, d)

29

Treesroot

leaf

parent

child

are connected directed graphs without cycles such that there is a special vertex called “root” having exactly one path to every other vertices.

30

root

leaf

Level 0

Level 1

Level 2

Level 3

Height 3

The level associated with each vertex is the number of edges in the path form the root to the vertex.The height of the tree is the largest level number of any vertex.

31

Binary Trees

A binary tree is a tree in which no parent can have more than two children.

A binary tree is a tree in which no parent can have more than two children.(p.10) Example 1.5.

Prove that a binary tree of height n has at most 2n leaves.

32

PROOF TECHNIQUES

• Proof by induction

• Proof by contradiction

33

Induction

We have statements P1, P2, P3, …

If we know• for some m that P1, P2, …, Pm are true• for any k >= m that

P1, P2, …, Pk imply Pk+1

Then Every Pi is true

34

Proof by Induction• Inductive basis

Find P1, P2, …, Pm which are true• Inductive hypothesis

Let’s assume P1, P2, …, Pk are true, for any k >= m

• Inductive stepShow that Pk+1 is true

35

ExampleTheorem: A binary tree of height n has at most 2n leaves. (p.10)

Proof by induction: let L(i) be the maximum number of leaves of any subtree at height i

We want to show: L(n) ≦ 2n for n = 0, 1, 2,….

36

We want to show: L(n) ≦ 2n for n = 0, 1, 2,….

• Inductive basis L(0) =1 ≦ 20 (the root node : height=0)

• Inductive hypothesisLet’s assume L(i) ≦ 2i for all i = 0, 1, …, k

• Induction stepwe need to show that L(k + 1) ≦ 2k+1 , k≧0

37

Induction Step

From Inductive hypothesis:

height

k

k+1

Let’s assume L(i) ≦ 2i for all i = 0, 1, …, k

need to show that L(k + 1) ≦ 2k+1

0

…

L(k) ≦ 2k

38

L(k) ≦ 2k

L(k+1) ≦ 2 · L(k) ≦ 2· 2k = 2k+1

Induction Stepheight

k

k+1

(we add at most two nodes for every leaf of level k)

…

need to show that L(k + 1) ≦ 2k+1

To get a binary tree of height k+1 from one of height k, we can create at most 2 leaves in place of each previous one

39

RemarkRecursion is another thing

Example of recursive function:

f(n) = f(n-1) + f(n-2)

f(0) = 1, f(1) = 1

40

Proof by Contradiction

We want to prove that a statement P is true

• we assume that P is false• then we arrive at an incorrect conclusion• therefore, statement P must be true

41

ExampleTheorem: is not rational

Proof:Assume by contradiction that it is rational = n/m n, m are nonzero integers and without common factorsWe will show that this is impossible

2

2

42

= n/m 2 m2 = n2

Therefore, n2 is even n is evenn = 2 k

2 m2 = 4k2 m2 = 2k2m is evenm = 2 p

Thus, m and n have a common factor 2

Contradiction!

2

43

LanguagesLanguages

44

A language is a set of strings

String: A sequence of letters

Examples: “cat”, “dog”, “house”, …

Defined over an alphabet: zcba ,...,,,

Non-empty and finite

45

Alphabets and StringsAlphabets and Strings We will use small alphabets:

Strings

abbawbbbaaavabu

ba,

baaabbbaabababaabbaaba

46

Empty StringEmpty String

A string with no letters

47

String OperationsString Operations

m

n

bbbvaaaw

......

21

21

bbbaaaabba

mn bbbaaawv ...... 2121

Concatenation

abbabbbaaa

w = ? w = ?

48

Empty StringEmpty StringObservations:

abbaabbaabba

www

49

12... aaaw nR

naaaw ...21 ababaaabbb

Reverse

bbbaaababa

50

String LengthString Length

Length:

Examples:

naaaw ...21nw

12

4

aaaabba

| | = ?

51

Length of Length of ConcatenationConcatenation

Example:

vuuv

8538

5,3,

vuuvaababaabuv

vabaabvuaabu

52

Example 1.8 Example 1.8 (p.17) |uv| = |u| + |v|

A recursive definition of the length of a string: | a | =1,

| wa | = | w | + 1

For all a, w is any string from

Fix the string u and consider all possible strings v ( the length of v can be 1, 2, …. ( 0 is trivial) )

The proof is done by induction on the length of v ( for any given u )

53

Example 1.8 Example 1.8 (p.17) |uv| = |u| + |v|

For a string u, consider the length of uv, concatenation of u with a string v

Basis

Inductive Step

Induction Assumption

54

SubstringSubstring Substring of string: a

subsequence of consecutive letters

String Substring

bbabbabbaab

abbababbababbababbab

55

Prefix and SuffixPrefix and Suffix Prefixes Suffixes

abbab

abbababbaabbaba

babbabbbababbab uvw

prefixsuffix

56

Another Another OperationOperation

Example: abbaabbaabba 2

?0 w

n

n wwww

57

Another OperationAnother Operation

Definitions:

abbawwabba 11

0w

0abba

wwwwn ...

58

The * OperationThe * Operation : the set of all possible strings from alphabet

Example: = { a, b} then = ?

*

,,,,,,,,,*

,aabaaabbbaabaaba

ba

*

59

The + Operation : the set of all possible strings from alphabet except

,...,,,,,,,,*,

aabaaabbbaabaababa

}{*

,...,,,,,,, aabaaabbbaabaaba

Fall 2008 Automata 60

LanguagesLanguages*

,...,,,,,,,*,

aaabbbaabaababa

},:{

},,,,,{,,,,

nmevenisnba

aaaaaaabaababaabbaaabaaa

mn

61

Note that:

}{}{

0}{

1}{

0

Sets

Set size

Set size

String length

62

Another ExampleAnother Example An infinite language }0:{ nbaL nn

aaaaabbbbbaabbab

L Labb

63

Operations on LanguagesOperations on Languages The usual set operations

Complement: ??

aaaaaabbbaaaaaba

ababbbaaaaabaaaaabbabaabbbaaaaaba

,,,,}{,,,

},,,{,,,

LL * ,,,,,,, aaabbabaabbaa

64

When we talk about a language, we must know what ground does this language stands on …..

LanguagesLanguages

We should know the ALPHABETSALPHABETS that constitute the language

65

Complement ExampleComplement Example The complement

of},{ baaL

Universal Set?

66

ReverseReverse Definition:

Examples:

}:{ LwwL RR

ababbaabababaaabab R ,,,,

}0:{

}0:{

nabL

nbaLnnR

nn

67

ReverseReverse

}:{

})(:{}:{

Luu

LwwLwwLR

RRRRR

RRR LLLL )()()( 2121

RRR LLwthusLLw )(...,,: 2121

Hw # 10 (a) Prove or Disprove:

RRR LLwthusLLw 2121 ....,,)(:

i.e., wR L w LR

68

ConcatenatioConcatenationn

Definition:

Example:

2121 ,: LyLxxyLL

baaabababaaabbaaaab

aabbaaba,,,,,

,,,

21

2121

,:)(

)}(:{)(

LvLuuv

LLwwLLR

RR

69

ConcatenationConcatenation

RRRRR uvuvww )()(

RRR LLLL 1221 )( Hw 8. Prove

21

2121

..

,)(:

LvandLusomeforuvwei

LLwthenLLwR

RR

2121 ,: LyLxxyLL

70

Another OperationAnother Operation Definition:

Special case:

bbbbbababbaaabbabaaabaaa

babababa,,,,,,,

,,,, 3

0

0

,, aaabbaa

L

)timesitselfeconcatenat(... nLLLLLn

71

More ExamplesMore Examples}0:{ nbaL nn

}0,:{2 mnbabaL mmnn

2Laabbaaabbb

LL 1

72

Star-Closure (Kleene *)Star-Closure (Kleene *) Definition:

Example:

210* LLLL

,,,,,,,,

,,,

*,

abbbbabbaaabbaaabbbbbbaabbaa

bbabba

73

Positive ClosurePositive ClosureDefinition:

21 LLL

,,,,,,,,

,,,

abbbbabbaaabbaaabbbbbbaabbaa

bbabba

*LLIt is not necessary thatIf L then L+ L* - {}

74

True or FalseTrue or False

L *L

*L *)(L

75

True or FalseTrue or False

*L *)( L

How to prove your answer?

76

Try Hw# 9 & 10(b) on p.28

What does wL2 mean?What does wL* mean?

210* LLLL

.0..,,, 1010

*

ksomeforuuuwtsLuuu

Lw

kk

77

More ExamplesMore Examples Consider a language on = {a, b}

)}()(:{ wnwnwL ba

What is L2 ? L*?

e.g. 4)(&2)(, wnwnabbabbw ba

78

GrammarsGrammars

79

Another ExampleAnother Example Grammar:

Derivation of sentence :

SaSbS

abaSbS ab

aSbS S

80

Grammar:

Derivation of sentence :

aabbaaSbbaSbS

aSbS S

aabb

SaSbS

81

Other derivations for

aaabbbaaaSbbbaaSbbaSbS

aaaabbbbaaaaSbbbbaaaSbbbaaSbbaSbS

SaSbSGrammar:

82

Language of the grammar

SaSbS

}0:{ nbaL nn

83

More NotationMore Notation Grammar PSTVG ,,,

:V

:T

:S

:P

Set of variables

Set of terminal symbols

Start variable

Set of Production rules

p.21

84

ExampleExample Grammar :

SaSbSG

PSTVG ,,,

}{SV },{ baT

},{ SaSbSP

85

More NotationMore Notation Sentential Form: A sentence that contains variables

Example:aaabbbaaaSbbbaaSbbaSbS

sentenceSentential Forms

86

We write:

Instead of:

aaabbbS*

aaabbbaaaSbbbaaSbbaSbS

87

In general we write:

If:

nww*

1

nwwww 321

88

By default: ww*

89

ExampleExample

S

aSbSGrammar

aaabbbS

aabbS

abS

S

*

*

*

*

Derivations

90

Another Grammar ExampleAnother Grammar Example

Grammar :

bbaS nn

AaAbAAbS

Derivations:G

From A aAb and A , we know

A, ab, aabb, aaabbb, … *

aabbbAbbSabbAbSbAbS

*

*

*

S

aSbSbSaSbS

91

Language defined by a GrammarLanguage defined by a GrammarFor a grammar with start variable :

GS

}:*{)( wSTwGL

String of terminals

92

|aAbA

ExampleExample For grammar :

AaAbAAbS

}0:{)( 1 nbaGL nn

Since bbaS nn

G

Pf: show L(G) {anbn+1} & L(G) {anbn+1} from A aAb we get AanAbn when it is applied n times. Together with A, we get Aanbn for n = 0, 1, 2, …. : w L(G), i.e. S* w

93

A Convenient NotationA Convenient Notation

AaAbA

|aAbA

In general, we need to give a proof that a given language indeed generated by a certain grammar.

Back to last Example

94

ExampleExample For grammar :

}0:{ 1 nbaLLet nn

AaAbAAbS

}0:{)( 1 nbaGL nn

GTo show

L(G) L

&

L(G) L

Need to show?)( GLw

?LwPf: show L(G) {anbn+1} & L(G) {anbn+1} from A aAb we get AanAbn when it is applied n times. Together with A, we get Aanbn for n = 0, 1, 2, …. : w L(G), i.e. S* w

95

More Examples on GrammarsMore Examples on Grammars Find grammars for L on {a, b} and give brief

arguments to explain why they work.

L contains all strings with exactly one a

L contains all strings with at least one a

L3

}0:{ 2 nba nn

},0:{ 2 nmnba mn

}3:{ 32 nba nn

At Least: S BaB; BaB | bB |

96

1. Problems on p.27 1. Problems on p.27 You should be able to do 2 ~ 17, 21 Hand in: 9, 10, 11c, 14ef, 15c, 17

2. Read 2. Read P. 37~ 41P. 37~ 41, and try to describe , and try to describe LL((MM)) in Fig. 2.6. in Fig. 2.6.

Homework for next week.