Lecture 8 - Random Number Generation

8/13/2019 Lecture 8 - Random Number Generation

1/42

Random Number Generation

Dr. Akram Ibrahim Aly

Copyright Akram I. Aly 2012

Lecture (8)


2/42


3/42

Applications of Random Numbers Random numbers have a lot of real world

applications; e.g.: sampling

Simulation: generate event times and randomvariables in simulation statistical inference

random search optimization lotteries, gambling


4/42

Properties of random numbers A sequence of random numbers R1, R2, , Rn

must have 2 important statistical properties: Uniformity Independence


5/42

Uniformity Each random number is an independent sample

drawn from a continuous uniform distribution between 0 and 1: R i [0,1]

otherwise ,0

10 ,1)(

x x f

21

2)(

1

0

1

0

2 x xdx R E

12

1

4

1

3

1

2

1

3)(

21

0

1

0

32

xdx x RV

x

f ( x

)

0

1

PDF:


6/42

Independence Independence : the probability of observing a

value in a particular interval is independent ofthe previous value drawn.


7/42

Generation of Pseudo-RandomNumbers

Pseudo means fake

Pseudo, because we utilize known methods togenerate numbers which removes the potential fortrue randomness .

Pseudo-random numbers: Deterministic sequence ofnumbers with a repeat period but with the appearanceof randomness (if you dont know the algorithm).


8/42

Generation of Random Numbers Goal: To produce a sequence of numbers in [ 0,1 ]

that simulates, or imitates, the ideal properties ofrandom numbers (RN). Important considerations in RN routines:

Fast Portable to different computers Have sufficiently long cycle

Replicable Closely approximate the ideal statistical properties of

uniformity and independence.


9/42

Techniques for Generating RandomNumbers

Mid-square method Linear Congruential Method Combined Linear Congruential Method

Tausworthe generators (self study) Extended Fibonacci generators (self study)


10/42

Mid-Square Method Proposed by Von Neumann and Metropolis in

the 1940s1. Start with an initial d-digit integer seed X 0 2. Square X

0 3. Take the middle d- digits as the next four-digit

number, X 1

4. Place a decimal point at the left of X 1 to get firstrandom number R 15. Square X 1 and continue process


11/42

SEED Random number seed -initial random number

used to generate the next random numberwhich is in turn transformed into the new seedvalue.


12/42

Example Let X 0 = 5497

X1 = (5497) 2= 30 2170 09 X 1 = 2170 R 1 = 0.2170 X2 = (2170 )2= 04708900 X 2 = 7089 R 2 = 0.7089 X3 = (7089) 2= 50253921 X 3 = 2539 R 3 = 0.2539

...and so on...


13/42

Example 2: Poor Mid-SquareGenerator

Let X 0 = 5197

X1 = (5197) 2= 27008809 X1 = 0088 R1 = 0.0088 X2 = (0088) 2= 00007744 X2 = 0077 R2 = 0.0077 X3 = (0077) 2= 00005929 X3 = 0059 R3 = 0.0059

X4 = (0059) 2= 00003481 X4 = 0034 R4 = 0.0034 Once zeros appear, they are carried in subsequent

numbers.


14/42

Example 3: Poor Mid-SquareGenerator

Let X 0 = 6500

X 1 = (6500) 2= 42250000 X1 = 2500 R1 = 0.2500 X2 = (2500) 2= 06250000 X2 = 2500 R2 = 0.25000

Cannot choose a seed that guarantees that the

sequence will not degenerate and will have a long period.


15/42

Advantages and disadvantages ofthe MID-SQUARE method

Advantages of the mid-square method Rather simple to implement

Disadvantages of the mid-square method

Difficult to choose initial seed that will give goodsequence Strong tendency to degenerate fairly rapidly to

zero


16/42

More Examples on the mid-squareMethod


17/42

17

The Mid-product Method Improved Variation of Mid-square method

Example: Start with two arbitrary 3-digit numbers

Z0 = 123 Z 1 = 456

Z0Z1 = (123)(456) = 56088 U 1 = 0.608 Z 2 = 608Z1Z2 = (456)(608) = 277248 U 2 = 0.772 Z 3 = 772

Z2Z3 = (608)(772) = 469376 U 3 = 0.693 Z 4 = 693


18/42

Linear Congruential Methodby D. H. Lehmer, 1951

To produce a sequence of integers, X 1 , X 2 , between 0 and

(m-1) by following a recursive relationship:

If c 0, the form is called the mixed congruentialmethod.

If c=0, the form is known as the multiplicativecongruential method


19/42

Linear Congruential Method The random integers are being generated in the

interval [0, m-1], and to convert the integers torandom numbers: For [0,1[

For [0,1]

,...2,1 im

X R ii

,...2,1 1

im X

R ii


20/42

Example Use X0= 27 , a = 17 , c = 43 , and m = 100 .

The Xi and Ri values are: X 1= (17*27+43) mod 100 = 502 mod 100 = 2,R1= 0.02; X 2= (17*2+32) mod 100 = 77, R2= 0.77 ; X 3= (17*77+32) mod 100 = 52, R3= 0.52;


21/42

21

Example For example,

Starting with x 0=5:

The first 32 numbers obtained by the above procedure10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5;10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5.

By dividing x's by 16:

0.6250, 0.1875, 0.0000, 0.0625, 0.3750, 0.9375, 0.7500, 0.8125, 0.1250,0.6875, 0.5000, 0.5625, 0.8750, 0.4375, 0.2500, 0.3125;0.6250, 0.1875, 0.0000, 0.0625, 0.3750, 0.9375, 0.7500, 0.8125, 0.1250,0.6875, 0.5000, 0.5625, 0.8750, 0.4375, 0.2500, 0.3125.


22/42

Selection of LCG Parameters The selection of the values for a , c , m , and X 0

affects the statistical properties and the cyclelength. Most natural choice for m is one that equals to

the capacity of a computer word. m = 2 b (binary machine), where b is the number of

bits in the computer word.

m = 10 d (decimal machine), where d is the numberof digits in the computer word.


23/42

23

Selection of LCG Parameters The modulus m should be large

All generated numbers, X i , are between 0 and m-1 ,the period can never be more than m

For mod m computation to be efficient, m must be a power of 2 Mod m can beobtained by dropping the leftmost binary

digits in X i+1 and using only the b rightmost binary digits


24/42


25/42

Example Using the multiplicative congruential method, find

the period of the generator for a = 13, m = 26, and X 0

= 1, 2, 3, and 4. The solution is given in next slide. When the seed is 1 and 3, the sequence has period 16.

However, a period of length eight is achieved whenthe seed is 2 and a period of length four occurs whenthe seed is 4.


26/42

Random Numbers 26

SolutionPeriod Determination Using Various seeds

i X i X i X i X i

0 1 2 3 41 13 26 39 522 41 18 59 363 21 42 63 204 17 34 51 45 29 58 236 57 50 437 37 10 478 33 2 359 45 7

10 9 2711 53 3112 49 1913 61 5514 25 1115 5 1516 1 3


27/42

27

Selection of LCG Parameters forMaximum Period

If c 0, the maximum possible period m

is obtained if and only if: Integers m and c are relatively prime, that is, have nocommon factors other than 1

Every prime number that is a factor of m is also afactor of ( a-1)

If integer m is a multiple of 4, a-1 should be amultiple of 4

Notice that all of these conditions are met if m=2 b ,a = 4k + 1 , and c is odd.


28/42

Selection of LCG Parameters forMaximum Period

For m a power of 2, say m = 2 b , and c = 0 , the

longest possible period is P = m / 4 = 2 b-2

, which isachieved provided that the seed X 0 is odd and themultiplier, a , is given by a = 3 + 8k or a = 5 + 8k , for

some k = 0, 1,...


29/42


30/42

30

Example

Using a seed of x 0=1 5, 25, 29, 17, 21, 9, 13, 1, 5,

Period = 8 = 32/4 With x 0=2, the sequence is

10, 18, 26, 2, 10,

Here, the period is only 4


31/42

31

Example Multiplier not of the form 8i 3:

Using a seed of x 0=1, we get the sequence 7, 17, 23, 1, 7, The period is only 4


32/42


33/42

33

Multiplicative LCG with m 2k

Modulus m = prime number With a proper multiplier a , period = m- 1 Maximum possible period = m If and only if the multiplier a is a primitive root of the modulus m

a is a primitive root of m if and only if a n mod m 1

for n = 1, 2, , m- 2


34/42


35/42

35

Full period generators A generator that has the maximum possible period, m,

is called a full-period generator Examples:


36/42

Combined Linear CongruentialGenerators,LEcuyer[1988]

Reason: Longer period generator is needed because of the increasing complexity ofstimulated systems.

Approach: Combine two or more

multiplicative congruential generators.


37/42


38/42

Combined Linear CongruentialGenerators

The maximum possible period is:


39/42


40/42

Example The algorithm (contd)

Step 4:Return

Step 5: Set j = j+1 , go back to step 2

Combined generator has period: (m11)(m21)/2 ~2x10 18


41/42


42/42

Random-Numbers Streams A single random-number generator with k

streams can act like k distinct virtual random-number generators

To compare two or more alternative systems itis advantageous to dedicate portions of the

pseudo-random number sequence to the same

purpose in each of the simulated systems.

Lecture 8 - Random Number Generation

Documents