A li dN i l A li dN i l Applied Numerical Applied Numerical Analysis Analysis Analysis Analysis Pseudo Random Number Generator Pseudo Random Number Generator Lecturer: Emad Fatemizadeh Lecturer: Emad Fatemizadeh Lecturer: Emad Fatemizadeh Lecturer: Emad Fatemizadeh Applied Numerical Methods Applied Numerical Methods E. Fatemizadeh E. Fatemizadeh
47
Embed
AlidN i lApplied Numerical Analysis - Sharifsharif.edu/~fatemizadeh/Courses/ANA/10-PRNG-View.pdfPseudo Random Number GeneratorPseudo Random Number Generator What is random number:
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
A li d N i lA li d N i lApplied Numerical Applied Numerical AnalysisAnalysisAnalysisAnalysis
Pseudo Random Number GeneratorPseudo Random Number GeneratorLecturer: Emad FatemizadehLecturer: Emad FatemizadehLecturer: Emad FatemizadehLecturer: Emad Fatemizadeh
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
What is random number:What is random number:What is random number:What is random number:•• A sequence in which each term is unpredictableA sequence in which each term is unpredictable
29, 95, 11, 60, 2229, 95, 11, 60, 22
Application:Application:•• Monte Carlo SimulationsMonte Carlo Simulations•• Generation of Cryptographic KeysGeneration of Cryptographic Keys
Password generationPassword generation
•• Many Combinatorial Optimization AlgorithmsMany Combinatorial Optimization Algorithms•• Many Combinatorial Optimization AlgorithmsMany Combinatorial Optimization Algorithms•• GamesGames
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
RN’s Types:RN’s Types:RN s Types:RN s Types:•• True random:True random:
Generated in nonGenerated in non--deterministic waysdeterministic waysGenerated in nonGenerated in non--deterministic ways.deterministic ways.•• Not predictable Not predictable •• Not repeatable.Not repeatable.
•• Pseudo random:Pseudo random:Numbers that Numbers that appearappear random, but are random, but are bt i d i d t i i ti bt i d i d t i i ti obtained in a deterministic, obtained in a deterministic, •• RepeatableRepeatable•• PredictablePredictable
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
RN’s Generation:RN’s Generation:RN s Generation:RN s Generation:•• True Random:True Random:
Physical Phenomenon:Physical Phenomenon:Physical Phenomenon:Physical Phenomenon:•• Decay times of radioactive materialDecay times of radioactive material•• Electrical noise from a resistor or semiconductorElectrical noise from a resistor or semiconductor•• Radio channel or audible noiseRadio channel or audible noise•• Keyboard timingsKeyboard timings
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Pseudo Random Number Generator:Pseudo Random Number Generator:Pseudo Random Number Generator:Pseudo Random Number Generator:•• Uniform distributionUniform distribution
Theoretically all others distribution can be be Theoretically all others distribution can be be f ff fgenerate from Uniform.generate from Uniform.
•• Ideal: Between [0 1]Ideal: Between [0 1]In computer we generate random integer [0,2In computer we generate random integer [0,2NN--1] or 1] or In computer we generate random integer [0,2In computer we generate random integer [0,2 1] or 1] or [0,2[0,2NN] then convert to [0,1]] then convert to [0,1]
•• Desired Properties:Desired Properties:L P i dL P i dLong PeriodLong PeriodUncorrelatedUncorrelatedFastFast
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Linear Congruential Generators:Linear Congruential Generators:Linear Congruential Generators:Linear Congruential Generators:
th d l 0 th d l 0( )1 mod 0n nX aX c m n+ = + ≥
m: the modulus m>0m: the modulus m>0a: the multiplier 0≤a<ma: the multiplier 0≤a<mc: the increment 0≤c<mc: the increment 0≤c<mXX00 the seed point 0≤Xthe seed point 0≤X00<m<mXX0:0: the seed point 0≤Xthe seed point 0≤X00<m<mmod: Integer remindermod: Integer reminder
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Linear Congruential Generators:Linear Congruential Generators:Linear Congruential Generators:Linear Congruential Generators:
Three Classes:Three Classes:( )1 mod 0n nX aX c m n+ = + ≥
Three Classes:Three Classes:•• m=2m=2NN, c>0, c>0•• m=2m=2NN c=0 c=0•• m=2m=2 , c=0, c=0•• m=prime, c=0m=prime, c=0Two first are fast and easy Third has Two first are fast and easy Third has Two first are fast and easy. Third has Two first are fast and easy. Third has high randomness.high randomness.Choice of m a and c are critical!Choice of m a and c are critical!
Choice of m, a, and c are critical!Choice of m, a, and c are critical!
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Linear Congruential Generators:Linear Congruential Generators:Linear Congruential Generators:Linear Congruential Generators:•• Let a=1,c=5,m=16 and xLet a=1,c=5,m=16 and x00=1. =1. •• The sequence of pseudoThe sequence of pseudo random random •• The sequence of pseudoThe sequence of pseudo--random random
integers generated by this algorithm is: integers generated by this algorithm is: 1,6,15,12,13,2,11,8,9,14,7,4,5,10,3,01,6,15,12,13,2,11,8,9,14,7,4,5,10,3,0,,111,6,15,12,13,2,11,8,9,14,7,4,5,10,3,01,6,15,12,13,2,11,8,9,14,7,4,5,10,3,0,,11,6,15,12,13,2,11,8,9,14,6,15,12,13,2,11,8,9,14,,……..
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Improvement on LCG Method:Improvement on LCG Method:Improvement on LCG Method:Improvement on LCG Method:•• Multiple Recursive GeneratorMultiple Recursive Generator
1 -1 2 -2 -( ) mod i i i k i kX a X a X a X c m= + + + +
random seed required!Popular Value of (r,s): (17,5), (55,24), (127,97), (607,273), (1279,418)r
•
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
How to generate other distributions:How to generate other distributions:How to generate other distributions:How to generate other distributions:•• A mathematical theory:A mathematical theory:
If we pass a Uni[0 1] random variables from If we pass a Uni[0 1] random variables from If we pass a Uni[0,1] random variables from If we pass a Uni[0,1] random variables from inverse of desired cumulative density inverse of desired cumulative density function, then output will have distribution function, then output will have distribution f b bilit d it f tif b bilit d it f tiof probability density function:of probability density function:
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
How to generate other distributions:How to generate other distributions:How to generate other distributions:How to generate other distributions:•• Example:Example:
We want 10000 random variable with We want 10000 random variable with We want 10000 random variable with We want 10000 random variable with distribution of distribution of
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Method is good but Method is good but NOTNOT practical:practical:Method is good but Method is good but NOTNOT practical:practical:•• For example consider normal For example consider normal
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
BoxBox--Muller Method for Normal:Muller Method for Normal:BoxBox Muller Method for Normal:Muller Method for Normal:•• Suppose uSuppose u11 and uand u2 2 are two independent are two independent
uniform random variable then:uniform random variable then:uniform random variable, then:uniform random variable, then:
π= − 1 22log( ) cos(2 )x u u
are jointly normal random variable;are jointly normal random variable;
π= − 1 22log( ) sin(2 )y u u
are jointly normal random variable;are jointly normal random variable;
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Central Limit Theorem for Normal:Central Limit Theorem for Normal:Central Limit Theorem for Normal:Central Limit Theorem for Normal:•• Average of N independent random Average of N independent random
variable (with zero mean) from any variable (with zero mean) from any variable (with zero mean) from any variable (with zero mean) from any distribution converge in limits to distribution converge in limits to normal!normal!
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Matlab Command:Matlab Command:Matlab Command:Matlab Command:•• rand(m,n): A mrand(m,n): A m××n matrix with Uni[0,1]n matrix with Uni[0,1]•• s = rand('state') : s = rand('state') : Return Current state.Return Current state.s rand( state ) : s rand( state ) : Return Current state.Return Current state.•• rand('state',s): rand('state',s): Resets the state to s.Resets the state to s.•• rand('state',0): rand('state',0): Resets the generator to Resets the generator to ( , )( , ) gg
its initial state. its initial state. •• rand('state',j): rand('state',j): For integer j, resets the For integer j, resets the
t t it jt t it j th t tth t tgenerator to its jgenerator to its j--th state.th state.•• rand('state',sum(100*clock)): rand('state',sum(100*clock)): Resets it Resets it
to a different state each timeto a different state each time
to a different state each time.to a different state each time.
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Matlab Command:Matlab Command:Matlab Command:Matlab Command:•• rand(m,n): A mrand(m,n): A m××n matrix with Uni[0,1]n matrix with Uni[0,1]•• How to u[a b]?How to u[a b]?•• How to u[a,b]?How to u[a,b]?
a = 10; b=20;a = 10; b=20;u=rand(1 1000);u=rand(1 1000);u=rand(1,1000);u=rand(1,1000);x = a + (bx = a + (b--a) * u;a) * u;
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Matlab Command:Matlab Command:Matlab Command:Matlab Command:•• randrandnn(m,n): A m(m,n): A m××n matrix with N[0,1]n matrix with N[0,1]•• s = rand('state') : s = rand('state') : Return Current state.Return Current state.s rand( state ) : s rand( state ) : Return Current state.Return Current state.•• rand('state',s): rand('state',s): Resets the state to s.Resets the state to s.•• rand('state',0): rand('state',0): Resets the generator to Resets the generator to ( , )( , ) gg
its initial state. its initial state. •• rand('state',j): rand('state',j): For integer j, resets the For integer j, resets the
t t it jt t it j th t tth t tgenerator to its jgenerator to its j--th state.th state.•• rand('state',sum(100*clock)): rand('state',sum(100*clock)): Resets it Resets it
to a different state each timeto a different state each time
to a different state each time.to a different state each time.
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Matlab Command:Matlab Command:Matlab Command:Matlab Command:•• rand(m,n): A mrand(m,n): A m××n matrix with Uni[0,1]n matrix with Uni[0,1]•• How to N[mHow to N[m δδ22]?]?•• How to N[m,How to N[m,δδ22]?]?
m=10; s=16;m=10; s=16;u=randn(1 1000);u=randn(1 1000);u=randn(1,1000);u=randn(1,1000);x = m + sqrt(s) * u;x = m + sqrt(s) * u;
Pseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number GeneratorPseudo Random Number Generator
Matlab Command:Matlab Command:Matlab Command:Matlab Command:•• Other distribution:Other distribution:
exprndexprnd Exponential random numbersExponential random numbersexprndexprnd Exponential random numbersExponential random numberstrndtrnd Student's t random numbersStudent's t random numbersraylrndraylrnd Rayleigh random numbersRayleigh random numbersyy y gy gSee See StatisticsStatistics Toolbox for others.Toolbox for others.
Monte Carlo MethodMonte Carlo MethodMonte Carlo MethodMonte Carlo Method
Monte Carlo IntegrationMonte Carlo IntegrationMonte Carlo IntegrationMonte Carlo Integration••ππ estimation as a simple example:estimation as a simple example:
l fl fGenerate N couples of RVs Generate N couples of RVs
Function MinimizationFunction MinimizationFunction MinimizationFunction Minimization
Multivariable function F(Multivariable function F(xx):):Multivariable function F(Multivariable function F(xx):):•• Gradient: A vector:Gradient: A vector:
Function MinimizationFunction MinimizationFunction MinimizationFunction Minimization
Steepest Descent Rule:Steepest Descent Rule:Steepest Descent Rule:Steepest Descent Rule:•• Change function in direction of Change function in direction of ––G(G(xx))
Example in 1DExample in 1DExample in 1DExample in 1D