What is cross-entropy? From Riemann to Monte-Carlo Cross-Entropy techniques Cross-Entropy tricks Questions Using cross-entropy techniques for rare event simulation and optimization Arthur Breitman NYC Machine learning meetup August 18, 2011 Arthur Breitman crossentropy for rare event simulation and optimization
45
Embed
Simulation of rare events and optimisation with the cross-entropy method
The cross-entropy method is a parametric technique performing adaptive importance sampling in Monte-Carlo methods. Information about the distribution being integrated is collected as samples are generated from a parametric sampling distribution. The parameters of this distribution are iteratively updated to minimize the cross-entropy between the sample of the distribution of interest and the sampling distribution. This versatile adaptive technique has found many applications in rare even simulation, combinatorial optimization and optimization of functions with multiple extrema. Through a series of use cases, I'll present a quick practioner's guide to using the cross-entropy method and will discuss common tricks and pitfalls.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
What is cross-entropy?From Riemann to Monte-Carlo
Cross-Entropy techniquesCross-Entropy tricks
Questions
Using cross-entropy techniques for rare event
simulation and optimization
Arthur Breitman
NYC Machine learning meetup
August 18, 2011
Arthur Breitman crossentropy for rare event simulation and optimization
What is cross-entropy?From Riemann to Monte-Carlo
Cross-Entropy techniquesCross-Entropy tricks
Questions
EntropyKullback-Leibler divergence
OutlineWhat is cross-entropy?
EntropyKullback-Leibler divergence
From Riemann to Monte-CarloRiemann integrationMonte-Carlo integrationImportance sampling
Cross-Entropy techniquesAnalytical expressionsSimulation of rare eventsOptimizationFitting parameters
Arthur Breitman crossentropy for rare event simulation and optimization
What is cross-entropy?From Riemann to Monte-Carlo
Cross-Entropy techniquesCross-Entropy tricks
Questions
Analytical expressionsSimulation of rare eventsOptimizationFitting parameters
Beta distribution
Not analytical! To fit, start with approximate values from themoment’s method
α = X
(
X (1− X
S2− 1
)
, β = (1− X )
(
X (1− X
S2− 1
)
The likelihood is given by
n(ln(Γ(α+β)−ln(Γ(α)−ln(Γ(β))+(α−1)n∑
i=0
ln(Xi )+(β−1)n∑
i=0
ln(1−Xi )
The first and second derivatives are the digamma and trigammafunction, available in the gsl. Newton’s method using the Jacobianconverges in a couple iterations. Very useful to model boundedvariables.
Arthur Breitman crossentropy for rare event simulation and optimization
What is cross-entropy?From Riemann to Monte-Carlo
Cross-Entropy techniquesCross-Entropy tricks
Questions
Analytical expressionsSimulation of rare eventsOptimizationFitting parameters
OutlineWhat is cross-entropy?
EntropyKullback-Leibler divergence
From Riemann to Monte-CarloRiemann integrationMonte-Carlo integrationImportance sampling
Cross-Entropy techniquesAnalytical expressionsSimulation of rare eventsOptimizationFitting parameters
Questions Arthur Breitman crossentropy for rare event simulation and optimization
What is cross-entropy?From Riemann to Monte-Carlo
Cross-Entropy techniquesCross-Entropy tricks
Questions
Analytical expressionsSimulation of rare eventsOptimizationFitting parameters
From integration to optimization
Using an elite sample to help convergence is a trick that does aform of hill climbing of a smooth function approximating theindicator function of the rare event.
Interesting even if not interested in integrating f .
Keep iterating based on an elite sample to converge towardsone global maximum.
variance of the sampling distribution follows the curvature off .
e.g. using a multivariate normal allows the covariance toreflect the differential
Arthur Breitman crossentropy for rare event simulation and optimization
What is cross-entropy?From Riemann to Monte-Carlo
Cross-Entropy techniquesCross-Entropy tricks
Questions
Analytical expressionsSimulation of rare eventsOptimizationFitting parameters
Combinatorial optimization
One classical example if combinatorial optimization. To solve aTSP with Cross-Entropy:
Assume the travel is a Markov chain on the graph nodes.
Generate travels by coercing them to be permutations.
Update transition probabilities from the elite sample.
Arthur Breitman crossentropy for rare event simulation and optimization
What is cross-entropy?From Riemann to Monte-Carlo
Cross-Entropy techniquesCross-Entropy tricks
Questions
Analytical expressionsSimulation of rare eventsOptimizationFitting parameters
Clustering
CE does clustering too!
Assign probabilities of membership to classes for each point(the sampling distribution).
Sample random membership assignments.
Use average distance to centroids to find an elite sample.
Slower than K-means but much less sensitive to initial choiceof centroids.
Arthur Breitman crossentropy for rare event simulation and optimization
What is cross-entropy?From Riemann to Monte-Carlo
Cross-Entropy techniquesCross-Entropy tricks
Questions
Analytical expressionsSimulation of rare eventsOptimizationFitting parameters
A form of global optimization
Is it global optimization?
If the sampling distribution is bounded below by a distributionthat covers the global maximum, yes, with probability 1!
In practice we may never see one maximum and converge toanother local maximum.
Arthur Breitman crossentropy for rare event simulation and optimization
What is cross-entropy?From Riemann to Monte-Carlo
Cross-Entropy techniquesCross-Entropy tricks
Questions
Analytical expressionsSimulation of rare eventsOptimizationFitting parameters
OutlineWhat is cross-entropy?
EntropyKullback-Leibler divergence
From Riemann to Monte-CarloRiemann integrationMonte-Carlo integrationImportance sampling
Cross-Entropy techniquesAnalytical expressionsSimulation of rare eventsOptimizationFitting parameters