Lecture 04[Math Preliminaries]

8/12/2019 Lecture 04[Math Preliminaries]

1/47


2/47

Mathematical Preliminaries


3/47

Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/2

Mathematical Preliminaries

Set Theory

Combinatorics

Mathematical Functions

Summations

Probability Theory

Topics


4/47


Set Theory


5/47


Set Theory

A Setis a collection of objects called members. If a, b, c, d are members of a set Swe write :S ={a, b, c, d}

where a, b, c, d S ( read a, b, c, d belong to S )

The set builder notation can be used to define a set. For example, the set

S={1, 3, 5, 7, 11, 13}

can be defined as

S = {x: where x is prime number and x


6/47Mathematical Preliminaries/IIU 2008/Dr.A.Sattar/5

Set Theory

A setA is called asubsetof setB if every element ofA is also a memberB. Symbolically:

A

B, ifx thenxA B

The setA isproper subsetofB, if A contains only some of the elements ofB. Symbolicallythe relationship is represented as:

A B

A

B

A=B

A B

A

Subsets

The relationship among sets is usually shown by a picture, which is called Venn diagram.

Figures (a), (b) show Venn diagrams for a subset and proper subset

B

(a) A is proper subset of B (b) A is subset of B


7/47



Set Operations

B, ifx or xA BA

Union The union of setA and setB is set of all elementsx such thatx is in A or x is in B.

Symbolically:

Example (1): A={ 1, 3, 4, 5},B={ 3, 4, 7, 8, 9}. The Union of sets A andB is the

set { 1, 3, 4, 5, 7,8, 9}.

U

Example(2): Union operation can be used to add elements to a set. Consider sets {a, b}, {c, d}

(i) Setting S to empty set S=

(ii) To insert elements of first set into S, we perform union of Swith {a, b}

S = S {a, b} = {a, b}

(iii) To insert elements of second set {b, c} into S, we perform union of Swith {c, d}S = S {c, d} = {a, b, c, d}U

U

The Union operation is pictorially represented by Venn diagram, as shown below:

A B A BU

Union set



Set OperationsIntersection

B, ifx and xA BA

The intersection of setA and setB is set of all elementsx,such that x is in A and

x is in B. Symbolically:

I

Example: Let A={1,2, 3, 4, 5} and B={2, 4, 7, 8} then

A B = {2, 4}

The intersection operation is pictorially represented by Venn Diagram, as shown below.

A B A BI

Set intersection

I

Two sets are called disjoint, if their intersection is empty set. Thus, ifA andB are disjoint,

A B=I



Set OperationsDifference Set

The difference of setA and setB is set of all elementsx, such thatx is not in B if x is in A.Symbolically:

B A , if x A then x B The difference operation is illustrated by Venn Diagram, as shown below.

Example(1): IfA={1,2,3,4,5} andB={4, 6, 8, 10} thenA B = {1, 2, 3, 5}

Example(2): We can use difference operation to delete elements from a set. Consider

S= {a, b, c, d, e, f }

(i) To delete element a we perform difference operation on S and set {a}.

S = S {a} = { b, c, d, e, f }

(ii) To delete element dwe perform difference operation on set S and set {d}.

S = S-{d} = { b, c, e, f }

A B B - ADifference set


11/47


12/47


Set TheoryCartesian Product

The Cartesian productof two sets A and B, denoted byAXB, is a set of all

ordered pairs such that the first element of the pair is in setA and the second element

is in setB. It is also called cross product. Mathematically,

Example: Let A = {a, b, c} andB = {g, h}, then

AXB = {(a, g), (a, h), (b, g), (b, h), (c, g), (c, h) }

AXB = { (a , b) : a A and b B }

The cardinality of Cartesian product is the product of cardinalities of sets

|AXB | = | A | .| B |


13/47


Set TheoryBinary Relation

A binary relation R on setsA andB is a subset of the Cartesian productAXB of the sets .In set notation, the relation is represented as:

R AXB

Example: Consider the sets A = {a, b, c}, B = {x, y}

Their cross product is set AXB = {(a, x), (a, y), (b, x), (b, y), (c, x), (c, y) }

(1) A binary relation R1 is

R1 = { (a, y), (b, y), (c, y) }

(2) Another binary relation R2 is

R2 = {(a, x), (a, y) }

Since a cross product set can have one or more subsets there can be many binary relations


14/47


Binary RelationExample

Agraph G is a pair (V, E ), where V is a finite set, called vertex set, andEis a binaryrelation on V, called edge set

Example: Consider the graph G = (V, E) , and V = { a, b, c }, vertex set for the graph

A binary relation is

E = {(a, a), (a, b), (a, c), (b, c), (c, b) }

The setErepresents edges of graph G. The graph is shown pictorially in the diagram below.


15/47


Combinatorics

P i


16/47


Permutations

Apermutation is an arrangement of n objects in some order, such that each object appearsexactly once.

Elements of a set of n objects can be arranged in the following ways

n.(n-1).(n-2)..3.2.1

because, the first place can be filled in n ways, and the second place in n-1 ways. Thus, first

and second places together can be filled in n(n-1) ways. The first, second and third places can

be filled in n.(n-1).(n-2) ways, and so on

The product n(n-1)(n-2).3.2.1 is called n-factorial, which is denoted as n!

Example: The elements of Set S={ a, b, c } can be permuted in six ways because

3! = 3.2.1 = 6

The six arrangements are: abc, acb, bac, bca, cab, cba

It is assumed that 0! = 1 (factorial of zero is one ). The factorial of a negative integeris

not defined

Definition

i l i


17/47


Factorial FunctionStirlings Approximation

It can be shown that 1000! is an integer consisting of 2,500 digits.

The analysis of algorithms sometimes involves large inputs. For large values of n, the

n! is approximated by the following formula,:

n! ( 2 n ) ( n / e) n , where e = 2.718

The above formula is referred to as Stirlings approximation

2.6 x 10 3532

20,922,789,888,00016

40,3208

244

22

n!n

The n! factorial increases rapidly with the increase of n. The table below shows the

growth.


18/47


CombinationsDefinition

The arrangement of n objects taken k ( k n ) at a time, without regard to the order,is called k-combination.

Example: The objects a, b, c, dhave following 2-combinations:

ab, ac, ad, bc, bd, cd

Observe that the combination ab ba i.e, order does not matter

The number of k-combinations of n objects is denoted by the symbol

Another notation is C(n ,k).

It can be shown that :

The k-combinations are called binomial coefficientsbecause they occur in

binomial theorem for the expansion of the expression:

C(n,k) =


19/47




20/47



Several mathematical functions are used to analyze algorithms and study their behavior.We briefly examine the properties of some these functions From analysis perspective, the

following functions are particularly useful:

Use in Analysis

i. y = x (Floor function)

ii. y = x (Ceil function )

iii. y = log ax (Logarithm function)

iv. y = x (Linear function )

v. y = x2 (Quadratic function)

vi. y = x3 (Cubic function)

vii. y = 2x (Exponential function with base 2)

viii. y =nn (Exponential)

When a mathematical function is used to represent the running time of an algorithm, the

function is referred to asgrowth function. The behavior (growth rate) of a function can berepresented pictorially by plotting agraph of (x,y)points given by the functional relation


21/47


Floor Function

Thefloor function y(x) is defined as:

y = x wherex is a real numberandy is the largestinteger which issmaller than or equal to

numberx.

Example: 4.9= 4,

4.0 = 4

It follows from the definition : x -1


22/47


Ceil Function

The ceil function y(x) is defined as:

y = x

where x is real number andy is thesmallestinteger larger than or equal tox.

Example: 4.9= 5

4.0 = 4

It follows from the definition : x x < x +1

Definition

The ceil function is used for mapping a real valuedfunctions into integerfunction


23/47


The logarithm function y= logax is the inverse of exponential function ay, where a is some

positive constant.

Thus, if y = log a x ,

then

x = a y

Example: Consider, y= log 5 25 . Since 25= 52, it follows y=2 .

Logarithm FunctionDefinition

The figure shows a plot of logarithm function to the base 2

The logarithm function will frequently arise in the analysis of algorithms

i h i


24/47


Logarithm FunctionBases

The choice of base depends on the nature of application and data type.

The base 10 is normally used in computations involving decimal numbers. Here are some

examples

log10

1 =0 because 1= 10 0 by definition

log 10 10 =1 10= 10 1log 10 100 = 2 100=10

2

log 10 1000000 = 6 1000000=106

The base 2 is often used in the analysis of algorithms. Some examples of base 2 are

log 2 1=0 because 1= 20

by definitionlog 2 2= 1 2=21

log 2 16= 4 16= 24

log 2 64= 6 64= 26

The logarithm to base 2 are referred to as binary logarithm. A special notation lg is used

to represent a binary algorithm, as followslg (n) log 2 (n)

The base e 2.718 is used in calculus for purposes of differentiation and integration It iscalled natural logarithm. A special notation ln is used to represent natural logarithms.

ln (n) log e (n)

L ith F ti


25/47


Logarithm FunctionProperties

log bx . y = logbx + logby (Sum rule)

logbx / y = logbx logby (Difference rule)

logbxn = n logbx (Exponentiation rule)

x logby = y logbx ( Symmetry rule )

The following properties of logarithms are often used in several applications

The last property is particularly useful forinterchanging logarithmic exponentialand

base of exponentiation. Here are some examples.

2log 2n = n log 2

2 = n

8log 2n = n log 2

8 = n3

3log 2 n = n log 2 3 = n1.585 ( Interchanging 3 and n and using lg 3 = 1.585)

(Interchanging 2 and n)

(Interchanging 8 and n)

L i h F i


26/47


Logarithm FunctionChange of Base

Example : lg(binary logarithm) can be converted to ln (natural logarithm):

lg x = log2x

= log ex / loge2

= ln x / ln 2

loga x = logbx / logb a

Sometimes it is necessary to change the base of a logarithm. The formula for thetransformation from base a to b is as follows:

( ln 2 = 0.6931)


27/47



3.856 x 10 215

1.269 x 10 89

2.631 x 10 35

20,922,789,888,000

40,320

24

2

n!

3.4 x 10 38

1.8 x 10 19

4,294,967,296

65,536

256

16

4

2n

5.283 x 10 269

3.940 x 10 115

1.461 x 10 48

1.845 x 1019

16,777,216

256

2

nn

7

6

5

4

3

2

1

lg n

128

64

32

16

8

4

2

n

2,097,15216,38489611

262,1444,0963848

32,7681,0241605.7

4,096256644

51264242.8

641682

8421.4

n3n2n lg nn

The table shows the growth rate of common mathematical functions, which are useful in theanalysis of algorithms

Growth Rates

The logarithm lg n has the lowest growth rate , and the exponential function n n

the highest

The relationship of functions in terms of their growth rates is symbolically represented as

lg n < n < n < n lgn < n2

< n3

< 2n

< n! < nn


28/47


Growth FunctionsGraph

n

f(n)

The plot of mathematical functions lg n, n, n lg n n2, n3, 2n is shown below.

Because of extremely rapid growth of n! and nn, these functions are not shown in theranges shown in the graph scale


29/47


Summations

Summation


30/47


Summation

The arithmetic summation consist of sum natural numbers (1, 2, 3.n)Arithmetic

= 1 + 2 + 3+..+ n = n ( n+1 ) / 2

The sum can be simply evaluated, as under.

Let Sbe sum of first n terms of the series.Therefore,

S= 1+ 2 + 3 + + (n-2)+(n-1)+ n [ n terms] (1)

Writing the summation in reverse order

S= n + (n-1)+(n-2)++3 + 2 + 1 [n terms] . .(2)

Adding (1) and (2)

2S= (n+1)+(n+1)+(n+1)+.(n+1) + (n+1) + (n+1) [ n terms] ...(3)

= n(n+1)

Thus, S = n ( n + 1) / 2

Another summation sometimes used in analysis is sum of squares of natural numbers. It has

the sum:

S= 12 + 22 + 32+ +n2 = n(n+1)(2n+1)/6

=

n

k

k1

S ti


31/47


Summation

The geometric summation consists of sum ofpowers of some fixed number.

The fixed number is referred to as geometric ratio. Hers is a common form of exponential

summation:

Geometric

The summation is evaluated as follows. Let S be the sum of first n terms of the series. Then,

S= r 0 + r 1 + r 2+r3 ..+ r n-1 + r n .(1)

Multiplying both sides of (1) with r

r .S = r 1 + r 2+r 3+ +rn-1 + rn +rn+1 ..(2)

Subtracting (1) from (2)(r-1).S = rn+1-r0

= rn+1 -1

Or S= (rn+1-1) / ( r -1) (3)

= r 0 + r 1+ r 2 + ..+ rn-1 + r n = ( rn+1 - 1) / (r - 1)

where r is thegeometric ratio.

Example(1) : Let r = 2, the sum of geometric series is as under

20 + 21+ 22+..+2n =2n+1 - 1

Example (2): Let r = 2/3, the sum is as under

(2/3)0 + (2/3)1 + +(2/3)n = ( (2/3)n+1 -1)/( 2/3 - 1 ) = 3 3(2/3)n+1

=

n

k

r0

k


32/47


Summation

=

n

k

k0

2.

Arithogeometric

Another useful series, referred to as arithogeometric . It consists of sum of the product ofnatural numbers with exponentials of a constant. :

= 1.21 + 2.22 + 3.23+ .+ n.2n = (n-1) 2n+1 + 2k

= 1.m1 + 2.m2 + 3.m3+ .+ n.mnk

=

=

nk

k

km0

where m is some constant

The following summation of arithogeomatric series, with base 2, arises in the analysis of

some algorithms.


33/47


SummationHarmonic

The harmonic summation consists of reciprocals of natural numbers (1,2,..n).

The Harmonic series does not have an exact formula for the sum . The approximate

sum is obtained by converting thesummation into an integral

By using mathematical analysis it can be shown that upperand lowerbounds of

harmonic series are given by the relation

= 1/1 + 1/2 + 1/3 +.+1/n ln n + + 1 / 2n - 1 / 12n

=0.57721 is called Eulers constant

=+

n

k

ln(n)1k11

/ln(n+1)

=

n

k

k1

/1


34/47


Summation

The logarithmic summation includes logarithms of natural numbers (1,2,,n ). Assumingbase 2 for the logarithm, the series is expressed as follows:

= lg(1) + lg(2) + lg(3)+..+lg(n) . = lg(1.2.3..n) =lg( n! )

Logarithmic

An estimate for the above summation is obtained, by using the Stirling s approximation

for large factorial. Since n! ( 2 n ) ( n / e) n where e = 2.718, itfollows that

lg n! n lg( n) ( ignoring lower order terms and constants)

Therefore, the approximate summation for logarithm series for large n is given by :

lg(1) + lg(2) + lg(3)+..+lg(n) n lg( n )

=

n

k

k1

)lg(


35/47

Sample Space


36/47


Sample SpaceDefinition

The set of all possible outcomes of an experiment are calledsample space orpopulation..

Let e1, e2, e3, en are the probable n outcomes of an experiment. Then the sample space

is the set = { e1, e2, e3, en}

The Probability Theory is concerned with the study of experiments whose outcome is notknown in advance ( a priori). However, all likely outcomes are assumed to be finite and

known before the conduct of the experiment

For example, tossing of a coin is an experiment. The outcome of tossing cannot be

predicted in advance, but outcome is likely be Head or Tail

Sample Space


37/47


Sample SpaceExamples

Example (1): Afair coin is tossed. The result can beHead( H ) or Tail( T ). The samplespace for the tossing experiment is:

= {H, T}.

Example (2) : Afair coin is tossed twice ( or two coins are tossed simultaneously) the sample

space for the toss is: = { HH, HT, TH, TT}

Example (3): Asix-faced die is thrown. The sample space for the event is

= {1, 2, 3, 4, 5, 6 }. where digits represent number of dots on each face of the die.

Example(4): Two six-faced dice are tossed. The sample space consists of 36 pairs of

integers:

={(1,1),(1,2)(1,3),(1,4),(1,5), (1,6), (2,1),(2,2)(2,3,),(2,4),(2,5), (2,6),(3,1),(3,2)(3,3),(3,4),(3,5), (3,6), (4,1),(4,2)(4,3,),(4,4),(4,5), (4,6),

(5,1),(5,2)(5,3),(5,4),(5,5), (5,6), (6,1),(6,2)(6,3),(6,4),(6,5), (6,6) }

Example(5): The sample space for weather forecast can be = (fair, rainy, cloudy, partly cloudy, hailstorm, snow}

E t S


38/47


Events SpaceDefinition

An event set consisting of asingle element is called elementary event.

The empty set is referred to as impossible event

A six-faced die is thrown. The sample space consists of all numbers that are likely tobe shown i. e,

={1, 2, 3, 4, 5, 6 }

Example(1): The event space that an even numberis shown:

E1={2, 4, 6}

Example(2): The event space that an of odd number is shown:

E2=(1, 3, 5}

Example(3) : The event space that a numbergreater than 4 is shown

E3={ 5, 6}

A subset E of the sample space is called event space . Symbolically,

E

P b biliti


39/47


ProbabilitiesDefinition

LetEbe an event space consisting of events e1, e2, e3..en. Each of the events in the is

assigned a real value (number) denoted byP(e1), P(e2) , P(e3)..P(en), calledprobabilities, such that

P(e1) + P(e2) + P(e3)+.+ P(en) = 1

The functionP(e) is calledprobability function.

The probabilities are generally assigned on the basis of experience orjudgment. Some

times the probabilities are expressed as percentages, or in terms frequencies of happening of

different events.

Example(1): Chance that a flight will be on time is 90% and would be late is 10% . Thus,

P(on time)= 90/100=0.9 andP(late)=10/100=0.1

Example(2): Based on experience, we say that chance of finding a document on Yahoo is twice that of

finding on MSN, and four times on Google. Ifp is probability of finding the document on MSN then,

we have :P (msn ) = p ; P(yahoo ) = 2p ; P(google)) = 4p

By probability axiom, p + 2p + 4p =1 or p=1/7

Thus, we have P(msn)= 1 / 7, P(yahoo)= 2 / 7, andP(google)=4 / 7

Probability Theory


40/47


Probability TheoryEquiprobable Events

In several cases theprobabilities are not known,but it is expected that all events are equally

likely to occur . Therefore, all the elementary events are assigned equal probabilities.

Let E be event space for equiprobable events

E ={e1, e2, .en}

Then, P( e1) = P( e2) = P(e3) =.= P( en) = p , say, wherep is some constant

Since by definition

P(e1)+P(e2)+P(e3)++P(en)=1

it follows that

p +p + p+..+ p ( n terms ) = 1. i.e n.p= 1 or p = 1/n

Thus, for equiprobable n possible events, P(ej) = 1 / n ( 1 j n)

The functionP(n) is called uniform probability function.

E i b bl E t


41/47


Equiprobable EventsExamples

Example (1) In tossing a fair coin heads and tails have equal chance. The event set is {H, T}.,whereH refers to occurrence ofheadand T to occurrence oftail. Therefore,

P(H)=1/2;

P(T)=1/2

Example (2) When six-faced die is thrown, all numbers, 1 to 6, have equal chance of being

shown. Therefore, each event has probability 1/6. Thus, probabilities of showing different

numbers are

. P(1)=1/6, P(2)=1/6, P(3)=1/6, P(4)=1/6, P(5)=1/6, P(6)=1/6

Example(3): An array stores n random keys. Assuming that a search key has equal chance

of being in any of the array cells, the probability of finding the key in a given cell is 1/n

Example(4): A six-faced die is tossed. The event set that a odd number is shown consists ofthe elements 1, 3, 5. The probability that an odd number would be displayed is 3/6=1/2

P b bilit Th


42/47


Probability TheoryLaw of Addition

According toLaw of Addition of Probabilities, If A andB are two mutually exclusive

events with probabilitiesP(A) andP(B) ,then probability thatA occurs ORB occurs is given

by thesum of probabilities P(A) + P(B). Simply stated ,

P(A ORB)= P(A)+ P(B)

In general, ifA1, A2, A3Akare mutually exclusive events then

P(A1

ORA2

ORA3..ORA

k)=P(A

1)+ P(A

2)+P(A

3)+.+P(A

k)

Two events A, B are said to be mutually exclusive provided that if A occurs thenB cannotoccur; conversely, if B occurs thenA cannot occur. For example, if a six-faced die is tossed

then showing of 1 and 6are mutually exclusive events because if 1 is shown then 6will not be

shown and vice versa.

Law of Addition


43/47


Law of AdditionExamples

Example(2): A die is thrown. What is probability of showing aprime number or a number

greater than 5 ?

(1) Events set that aprime number would be shown is A= {2,3,5}. Each event has the

probability of 1/6. Since these are mutually exclusive events, probability that a prime

number would be shown is given byP( Prime Number) =P( 2 OR 3 OR 5) = 1/6+ 1/6+ 1/6 =1/2

(2)Events set that a number than greater 5 would be shown isB={6} . The Probability that

a number greater than 5 would be shown is

P( Number greater than 5) =1/6(3) Combining the above results, the probability that outcome would be aprime number or a

number greater than 5 is

P(Prime Number OR Number greater than 5)=P(Prime Number) + P(Number greater than 5)

=1/2 + 1/6

= 2/3

Example (1) : A six-faced die is tossed. What is the probability that an even number would

be show.?The likely even numbers are 2,4,6. Each has the probability of 1/6. Thus, according law

of addition

P( Even Number)= P(2 OR 4 OR 6)= 1/6 + 1/6+ 1/6 =1/2

Probability Theory


44/47


Probability TheoryLaw of Multiplication

Two eventsA , B are said to be stochastically independentif occurrence ofA is notinfluenced by the occurrence ofB. Conversely, occurrence ofB is not influenced by

occurrence ofA.

For example, if a coin is tossed several times the result of previous toss does affect the

result of current toss, and result of current toss would not influence the result of next toss.

We can say that the events of successive tossing are stochastically independent events.

According toLaw of Multiplication, ifA andB are stochastically independent events then

the probability of occurrence A AND occurrence ofB is given by the product of theprobabilities P(A).P(B). Mathematically,

P( A AND B) = P(A).P(B)

In general, if A1, A2, A3, .Akare stochastically independent events then

P( A1 ANDA2 AND A3 AND Ak) = P(A1) .P(A2).P(A3). P(Ak)


45/47

Probability Theory


46/47


Probability TheoryExpected Cost

Let c1

, c2

, ..cn

be the costs associated with events e1

, e2

, en

. Let the probabilities for

the occurrence of events be P(e1), P(e2)..P(en) , then the expected cost (EC) is defined as

EC = c1.P( e1) + c2.P( e2)+cn.P (en)

Example: Suppose a coin is tossed 2 times. Assume that the costs associated with events

are :

if two heads appear cost is 2

if one head appears cost is 1

if no heads appear (equivalently only tails show) cost is 0 The event space Sfor the experiment is = { HH, HT, TH, TT}

Probability that event HH occurs = 1/2.1/2= 1/4

Probability that event HT occurs =1/2.1/2=1/4

Probability that event TH occurs = 1/2.1/2=1/4Probability that event TT occurs =1/2.1/2=1/4

The four events have associated costs of 2, 1,1, 0 with probabilities 1/4, 1/4. 1/4, 1/4.

Thus, the expected cost of the tossing experiment = 2.(1/4) + 1.(1/4) + 1.(1/4) + 0.(1/4)

= 1/ 2 + 1/ 4 + 1/ 4 = 1

By law of multiplication


47/47

Lecture 04[Math Preliminaries]

Documents