Stochastic Constraint Programming: a seamless modeling ... · A Stochastic Combinatorial Optimization Problem", Journal of Combinatorial Optimization Applications R. Rossi, S. A.

Introduction Modeling Framework Ongoing Research SCP and LDI Conclusions

Stochastic Constraint Programming: aseamless modeling framework for decision

making under uncertainty

Dr Roberto Rossi1

1LDI, Wageningen UR, the Netherlands

Mansholt Lecture

Research

A Complete Overview

Theoretical Results

Applications

Decision Support Systems

Modeling Frameworks

Research

Theoretical Results

Applications

Modeling Frameworks

Theoretical Results

B. Hnich, R. Rossi, S. A. Tarim and S. Prestwich,"Synthesizing Filtering Algorithms for Global Chance-Constraints",submitted for possible publication to the 15th International Conferenceon Principles and Practice of Constraint Programming (CP-09) Lisbon,Portugal, September 21-24, 2009

Research

Modeling Frameworks

Theoretical Results

Applications

Modeling Frameworks

S. A. Tarim, S. Manandhar and T. Walsh,"Stochastic Constraint Programming:A Scenario-Based Approach",Constraints, Vol.11, pp.53-80, 2006.

Research

Applications

Theoretical Results

Modeling Frameworks

R. Rossi, S. A. Tarim, B. Hnich, S. Prestwich andS. Karacaer, "Scheduling Internal Audit Activities:A Stochastic Combinatorial OptimizationProblem", Journal of Combinatorial Optimization

Applications

R. Rossi, S. A. Tarim, B. Hnich and S. Prestwich,"Replenishment Planning for Stochastic InventorySystems with Shortage Cost", In proceedings ofThe Fourth International Conference on Integrationof AI and OR Techniques in Constraint Programmingfor Combinatorial Optimization Problems (CP-AI-OR 07)May 23-26, 2007, Brussels, Belgium, Lecture Notes inComputer Science, Springer-Verlag, LNCS 4510, pp.229-243, 2007

Research

Theoretical Results

Applications

Modeling Frameworks

S. A. Tarim, B. Hnich, R. Rossi and S. Prestwich,"A Decision Support System for Computing Optimal(R,S) Policy Parameters", In proceedings of the 19thIrish Conference on Artificial Intelligence and CognitiveScience (AICS 2008) Aug. 27, 2008, Cork, Ireland

Why going stochastic?

Decision Making Under Uncertainty

UNCERTAINTY

Theoretical Results

Applications

Modeling Frameworks

Why going stochastic?

Decision Making Under Uncertainty: A Pervasive Issue

UNCERTAINTY

Theoretical Results

Applications

Modeling Frameworks

Production Planning

Inventory Control

Land-Crop Allocation

Sustainable Energy Production

Financial Planning

Food Quality Control

What is available

Decision Making in a Deterministic Setting

DETERMINISTIC

Theoretical Results

Applications

Modeling Frameworks

What is available

DETERMINISTIC

Theoretical Results

Applications

Modeling Frameworks

LPSimplex Shortest Path

ERPInventory Control

Production Planning

Transport Scheduling

What is available

DETERMINISTIC

Theoretical Results

Applications

Modeling Frameworks

Production Planning

CPLEXCPLEX

What is available

DETERMINISTIC

Theoretical Results

Applications

Modeling Frameworks

Production Planning

OPL STUDIOOPL STUDIO

What is missing

Decision Making under Uncertainty

UNCERTAINTY

Theoretical Results

Applications

Modeling Frameworks

Stochastic Dynamic Programming

Stochastic Programming

Simplex Convex Analysis

Production Planning

What is missing

UNCERTAINTY

Theoretical Results

Applications

Modeling Frameworks

Stochastic Dynamic Programming

Stochastic Programming

Simplex Convex Analysis

Production Planning

STOCHASTIC OPLSTOCHASTIC OPL

An example

0-1 Knapsack Problem

Problem : we have k kinds of items , 1 through k . Each kind ofitem i has

a value ri

a weight wi .

We usually assume that all values and weights arenonnegative. The maximum weight that we can carry in thebag is W .Objective : find a set of objects that provides the maximumvalue and that fits in the given capacity.

An example

0-1 KP: MIP Formulation

Objective:max

∑ki=1 rixi

Constraints:∑ki=1 wixi ≤ W

Decision variables:xi ∈ {0, 1} ∀i ∈ {1, . . . , k}

An example

Static Stochastic Knapsack ProblemProblem : we have k kinds of items and a knapsack of size c into which to fit them.

Each item i , if included in the knapsack, brings a deterministic profit ri .

The size Wi of each item is not known at the time the decision has to be made,but we assume that the decision maker knows the probability distribution ofW = (W1, . . . ,Wk ).

A per unit penalty cost p has to be paid for exceeding the capacity of theknapsack.

Furthermore, the probability of not exceeding the capacity of the knapsackshould be greater or equal to a given threshold θ.

Objective : find the knapsack that maximizes the expected profit.

An example

SSKP: Stochastic Programming Formulation

Objective:

Pki=1 riXi − pE

Pki=1 Wi Xi − c

Subject to:

Pki=1 WiXi ≤ c

≥ θ

Decision variables:Xi ∈ {0, 1} ∀i ∈ 1, . . . , k

Stochastic variables:Wi → item i weight ∀i ∈ 1, . . . , k

Stage structure:V1 = {X1, . . . , Xk}S1 = {W1, . . . ,Wk}L = [〈V1, S1〉]

Seamless stochastic optimization

A First Step Towards Seamless Stochastic Optimization

UNCERTAINTY

Theoretical Results

Applications

Modeling Frameworks

Stochastic Constraint ProgrammingScenario-Based Formulation

Scenario-based Compilation toa classic (deterministic) Constraint Program

Stochastic OPL

A First Step Towards Seamless Stochastic Optimization

Advantages

Seamless Modeling under Uncertainty!

Stochastic OPL not necessarily linked to CP

DrawbacksSize of the compiled model

Propagation not fully supported

A Viable Approach for Seamless Stochastic Optimization

UNCERTAINTY

Theoretical Results

Applications

Modeling Frameworks

Stochastic Constraint ProgrammingFiltering Algorithms forGlobal Chance-Constraints

Compilation using Global Chance-Constraints

Stochastic OPL

B. Hnich, R. Rossi, S. A. Tarim and S. Prestwich,"Synthesizing Filtering Algorithms for Global Chance-Constraints",submitted for possible publication to the 15th International Conferenceon Principles and Practice of Constraint Programming (CP-09) Lisbon,Portugal, September 21-24, 2009

T. Walsh,"Stochastic Constraint Programming",Proceedings of the European Conference on AI, 2002.

Rossi, S. A. Tarim, B. Hnich and S. Prestwich,"A Global Chance-Constraint for Stochastic Inventory Systemsunder Service Level Constraints", Constraints, Springer-Verlag,Vol. 13(4):490-517, 2008

Constraint Programming Solversupporting Global Chance-Constraints

stoch myrand[onestage]=...;int nbItems=...;float c = ...;float q = ...;range Items 1..nbItems;range onestage 1..1;float W[Items,onestage]^myrand = ...;float r[Items] = ...;dvar float+ z;dvar int x[Items] in 0..1;

maximizesum(i in Items) x[i]*r[i] - expected(c*z)

subject to{z >= sum(i in Items) W[i]*x[i] - q;prob(sum(i in Items) W[i]*x[i] <= q) >= 0.6;};

Solution

Stochastic Constraint Program Stochastic OPL Model

Filtering Algorithms forGlobal Chance-Constraints

In what follows we shall discuss:

Constraint Programming

Stochastic Constraint Programming

Global Chance-Constraints

Stochastic OPL.

During the discussion the SSKP will be employed as runningexample.

Introduction

“Constraint programming (CP) represents one of the closestapproaches computer science has yet made to the Holy Grail ofprogramming: the user states the problem, the computersolves it”.Eugene C. Freuder, Constraints, April 1997

Formal Background

A slightly formal definition

A Constraint Satisfaction Problem (CSP) is a triple 〈V , D, C〉.

V = {v1, . . . , vn} is a set of variables

D is a function mapping each variable vi to a domain D(vi)of values

C is a set of constraints.

A Constraint Optimization Problem (COP) consists of a CSPand objective function f (V ) defined on a subset V of thedecision variables in V . The aim in a COP is to find a feasiblesolution that minimizes (maximizes) the objective function.

An Example

Sample COP: 0-1 KP

V = {x1, . . . , x3}

D(xi) = {0, 1} ∀i ∈ {1, . . . , 3}

C = {8x1 + 5x2 + 4x3 ≤ 10}

f (x1, . . . , x3) = 8x1 + 15x2 + 10x3

The optimal solution for the COP has a value of 25 and corre-sponds to the assignment is x2 = x3 = 1 and x1 = 0.

An Example

Sample COP: 0-1 KP

r=8w=8

r=15w=5

r=10w=4

Capacity: 10

Objects:

Knapsack:

An Example

Sample COP: 0-1 KP

r=8w=8

r=15w=5

r=10w=4

Capacity: 10Required: 9

Decision:

Knapsack:

Solution Method

Strategy

Constraint Programming proposes to solve CSPs/COPsby associating with each constraint a filtering algorithm .

A filtering algorithm removes from decision variabledomains values that cannot belong to any solution of theCSP/COP.

Constraint Propagation is the process that repeatedlycalls filtering algorithms until no new deduction can bemade.

Constraint Solving interleaves filtering algorithms and asearch procedure (for instance a backtracking algorithm).

An Example

Sample COP: 0-1 KP

V = {x1, . . . , x3}

D(xi) = {0, 1} ∀i ∈ {1, . . . , 3}

C = {8x1 + 5x2 + 4x3 ≤ 10}

f (x1, . . . , x3) = 8x1 + 15x2 + 10x3

Constraint Propagation initially does not produce any effect .

An Example

Sample COP: 0-1 KP

V = {x1, . . . , x3}

D(xi) = {0, 1} ∀i ∈ {1, . . . , 3}

C = {8x1 + 5x2 + 4x3 ≤ 10}

f (x1, . . . , x3) = 8x1 + 15x2 + 10x3

To build a search tree, we apply the lexicographic variableand value ordering heuristics and use labeling as domainsplitting procedure .

An Example

Sample COP: 0-1 KPV = {x1, . . . , x3}D(xi) = {0, 1} ∀i ∈ {1, . . . , 3}C = {8x1 + 5x2 + 4x3 ≤ 10}f (x1, . . . , x3) = 8x1 + 15x2 + 10x3

Filtered domains at P0

D(x1) = {0, 1} D(x2) = {0, 1} D(x3) = {0, 1}D(z) = {0, . . . , 33}

An Example

x1 ∈ {0}

D(x1) = {0} D(x2) = {0, 1} D(x3) = {0, 1}D(z) = {0, . . . , 25}

An Example

x1 ∈ {0}

Optimal solution associated with P1

D(x1) = {0} D(x2) = {1} D(x3) = {1} D(z) = {25}

An Example

x1 ∈ {0} x1 ∈ {1}

D(x1) = {1} D(x2) = {0} D(x3) = {0} D(z) = {8}

We backtrack to P0 and we return the optimal solution withvalue 25 found in the subtree associated with node P1.

Global Constraints

Efficiency

Filtering algorithms

detect inconsistencies in a proactive fashion

speed up the search

provided that the time spent in filtering is less then the timesaved in terms of search efforts.A challenging research topic is the design of efficient filteringstrategies.

Global Constraints

Not only binary relations

In constraint programming is common to find constraints over anon-predefined number of variables

alldifferent

element

cumulative

These constraints are called global constraints

they can be used in a variety of situations

they are associated with powerful filtering strategies

new custom global constraints can be defined

A slightly formal definition

Stochastic Constraint Satisfaction ProblemA Stochastic Constraint Satisfaction Problem (SCSP) is a7-tuple

〈V , S, D, P, C, θ, L〉.

V = {v1, . . . , vn} is a set of decision variables

S = {s1, . . . , sn} is a set of stochastic variables

D is a function mapping each variable to a domain of potential values

P is a function mapping each variable in S to a probability distribution for itsassociated domain

C is a set of (chance)-constraints, possibly involving stochastic variables

θh is a threshold probability associated to chance-constraint h

L = [〈V1, S1〉, . . . , 〈Vi , Si〉, . . . , 〈Vm, Sm〉] is a list of decision stages.

By considering an objective function f (V , S) we obtain a SCOP.

An Example

Sample SCOP: SSKP

V = {x1, . . . , x3}

D(xi) = {0, 1} ∀i ∈ {1, . . . , 3}

S = {w1, . . . , w3}

D(w1) = {5(0.5), 8(0.5)}, D(w2) ={3(0.5), 9(0.5)}, D(w3) = {15(0.5), 4(0.5)}

C = {Pr(w1x1 + w2x2 + w3x3 ≤ 10) ≥ 0.2}

L = [〈V , S〉]

f (x1, . . . , x3) =

8x1 + 15x2 + 10x3 − 2E max[0,

∑3i=1 wixi − 10

Sample SCOP: SSKP

r=8w=5 OR 8

r=15w=3 OR 9

r=10w=15 OR 4

Capacity: 10

Objects:

Knapsack:

Sample SCOP: SSKP

r=8w=5 OR 8

r=15w=3 OR 9

r=10w=15 OR 4

Decision:

Knapsack:

Observation:

r=8w=8

r=15w=3

r=10w=15

Sample SCOP: SSKP

x1=1x2=1

x3=0w1=5

Obj: 17

An Example

Sample SCOP: DSKP

V = {x1, . . . , x3}

D(xi) = {0, 1} ∀i ∈ {1, . . . , 3}

S = {w1, . . . , w3}

D(w1) = {5(0.5), 8(0.5)}, D(w2) ={3(0.5), 9(0.5)}, D(w3) = {15(0.5), 4(0.5)}

C = {Pr(w1x1 + w2x2 + w3x3 ≤ 10) ≥ 0.2}

L = [〈{x1}, {w1}〉, 〈{x2}, {w2}〉, 〈{x3}, {w3}〉]

f (x1, . . . , x3) =

E[8x1 + 15x2 + 10x3] − 2E max[0,

∑3i=1 wixi − 10

Sample SCOP: DSKP

r=8w=5 OR 8

r=15w=3 OR 9

r=10w=15 OR 4

Capacity: 10

Objects:

Knapsack:

Sample SCOP: DSKP

r=8w=5 OR 8

Decision stage 1:

Observation:

r=8w=8

r=15w=3 OR 9

r=10w=15 OR 4

Knapsack:

Sample SCOP: DSKP

r=8w=8

r=15w=3 OR 9

Decision stage 2:

Observation:

r=15w=3

r=10w=15 OR 4

Knapsack:

Sample SCOP: DSKP

r=8w=8

r=15w=3

r=10w=15 OR 4

Decision stage 3:

Knapsack:

Observation:

r=18w=8

r=10w=15

Sample SCOP: DSKP

Obj: 27.9687

Scenario-based Compilation

By using the approach discussed in

S. A. Tarim, S. Manandhar and T. Walsh,Stochastic Constraint Programming: A Scenario-Based Approach,Constraints, Vol.11, pp.53-80, 2006

it is possible to compile any SCSP/SCOP down to a determinis-tic equivalent CSP.

Scenario-based Compilation

SSKP: Compiled Deterministic Equivalent CSP

An Alternative Compilation Strategy Employing GCCs

SSKP: Compiled Deterministic Equivalent CSP withGlobal Chance-Constraints

Filtering in SCSPs

Also in Stochastic Constraint Programming (SCP) we have

constraints

filtering algorithms

In contrast to CP, in SCP constraints divide into

hard constraints

chance-constraints

Global Chance-ConstraintsPerhaps the most interesting aspect of SCP is that the conceptof global constraint can be also adopted in a stochasticenvironment, thus leading to

Global Chance-Constraints (Rossi et al., 2008)

Filtering in SCSPs

represent relations among a non-predefined number ofdecision and random variablesimplement dedicated filtering algorithms based on

feasibility reasoningoptimality reasoning

Global Chance-Constraints performing optimality reasoning arecalled Optimization-Oriented Global Chance-Constraints(Rossi et al., 2008).

Filtering in SCSPs

“Synthesizing Filtering Algorithms for Global Chance-Constraints” (Hnich et al., 2009)

Filtering Algorithms for GCCs

Stochastic Programming Model

Pr{∑k

i=1 WiXi ≤ c}≥ θ

Stochastic Constraint Programming Model

C = {Pr(w1x1 + w2x2 + w3x3 ≤ 10) ≥ 0.2}

Global Chance-ConstraintstochLinIneq(x,W,Pr,q,0.2);

x1={0,1}x2={0,1}

x3={0,1}w1=5

Pr(w1x1+w2x2+w3x3²10)>0.5

x1={0,1}x2={0,1}

x3={0,1}w1=5

Pr(w1x1+w2x2+w3x3²10)>0.5

x1={0,1}x2={0,1}

x3={0,1}

x1={0,1}w1=5

Pr(w1x1+w2x2+w3x3²10)³0.5

x3={0,1}

x1={0,1}

Pr(w1x1+w2x2+w3x3²10)³0.5

x3={0,1}

x1={0,1}

Pr(w1x1+w2x2+w3x3²10)³0.5

Stochastic OPL

A language specifically introduced by Tarim et al. (Tarim et al.,2006) for modeling decision problems under uncertainty .It captures several high level concepts that facilitate theprocess of modeling uncertainty:

stochastic variables (independent or conditionaldistributions)

several probabilistic measures for the objective function(expectation, variance, etc.)

chance-constraints

decision stages

Stochastic OPL

SSKPint N = 3;int c = 10;int p = 2;float θ = 0.2range Object [1..3];int value[Object] = [8,15,10];stoch int weight[Object] = [<5(0.5),8(0.5)>,

<3(0.5),9(0.5)>,<15(0.5),4(0.5)>];var int+ X[Object] in 0..1;stages = [<X,weight>];var int+ z;

maximize sum(i in Object) X[i]*value[i] - p*zsubject to{z = max(0,expected(sum(i in Object) X[i]*weight[i] - c));prob(sum(i in Object) X[i]*weight[i] - c ≤ 0) ≥ θ;};

Stochastic OPL

DSKPint N = 3;int c = 10;int p = 2;float θ = 0.2range Object [1..3];int value[Object] = [8,15,10];stoch int weight[Object] = [<5(0.5),8(0.5)>,

<3(0.5),9(0.5)>,<15(0.5),4(0.5)>];var int+ X[Object] in 0..1;stages = [<X[1],weight[1]>,<X[2],weight[2]>,<X[3],weight[3]>];var int+ z;

maximize sum(i in Object) X[i]*value[i] - p*zsubject to{z = max(0,expected(sum(i in Object) X[i]*weight[i] - c));prob(sum(i in Object) X[i]*weight[i] - c ≤ 0) ≥ θ;};

Overview of the Framework

Constraint Programming Solversupporting Global Chance-Constraints

stoch myrand[onestage]=...;int nbItems=...;float c = ...;float q = ...;range Items 1..nbItems;range onestage 1..1;float W[Items,onestage]^myrand = ...;float r[Items] = ...;dvar float+ z;dvar int x[Items] in 0..1;

maximizesum(i in Items) x[i]*r[i] - expected(c*z)

subject to{z >= sum(i in Items) W[i]*x[i] - q;prob(sum(i in Items) W[i]*x[i] <= q) >= 0.6;};

Solution

Stochastic Constraint Program Stochastic OPL Model

Filtering Algorithms forGlobal Chance-Constraints

Filtering Algorithms

B. Hnich, R. Rossi, S. A. Tarim and S. Prestwich,Synthesizing Filtering Algorithms for GlobalChance-Constraints ,submitted for possible publication to the 15th InternationalConference on Principles and Practice of ConstraintProgramming (CP-09) Lisbon, Portugal, September 21-24,2009

ContributionA generic approach for constraint reasoning underuncertainty .Works with any existing propagation algorithm!

DrawbackOnly implemented for linear inequalities/equalities:stochLinIneq(x,W,Pr,q,0.2);

i.e. SSKP → Pr(w1x1 + w2x2 + w3x3 ≤ 10) ≥ 0.2

Future workConsidering more global constraints :allDifferent()NValue()Cumulative()...

Language Compiler

S. A. Tarim, S. Manandhar and T. Walsh,Stochastic Constraint Programming: A Scenario-Based Approach,Constraints, Vol.11, pp.53-80, 2006

Language Compiler

Sampling Strategies

Scenario Reduction

Inspired by:

A. J. Kleywegt, A. Shapiro, T. Homem-De-MelloThe Sample Average Approximation Method for StochasticDiscrete OptimizationSIAM Journal of Optimization, 12:479–502

Sampling Strategies

Scenario Reduction

Pr(w1x1+w2x2+w3x3²10)³0.5

x1={0,1}x2={0,1}

x3={0,1}

Sampling Strategies

Scenario Reduction

x1={0,1}x2={0,1}

x3={0,1}w1=5

Pr(w1x1+w2x2+w3x3²10)³0.5 w3=15

Sampling Strategies

Scenario Reduction

x1={0,1}x2={0,1}

x3={0,1}w1=5

Pr(w1x1+w2x2+w3x3²10)³0.5 0

Sampling Strategies

Scenario Reduction

x1={0,1}x2={0,1}

x3={0,1}w1=5

Pr(w1x1+w2x2+w3x3²10)³0.5

w3=15 5

Policy Compression

Neuro-evolutionary Stochastic Constraint Programming

S. Prestwich, S. A. Tarim, R. Rossi, B. Hnich,Evolving Parameterised Policies for Stochastic Constrain t Programming ,submitted for possible publication to the 15th International Conference on Principlesand Practice of Constraint Programming (CP-09) Lisbon, Portugal, September 21-24,2009

Applications

Food Supply Chain Networks

Farm - Slaugherhouse allocation (with W. Rijpkema)

Inventory Control for Perishable Items (with K.Pauls-Worms)

Agrifood Domain

Land Allocation

Pest Control

Environmental Domain...

EducationDecision Science II ???

The LDI research framework

StochasticConstraintProgramming

IDE &Compiler

Final remarks

Summary

We discussed a Framework for Modeling DecisionProblems under Uncertainty

Stochastic Constraint ProgrammingGlobal Chance-constraintsStochastic OPL

We presented some current and possible futureapplication areas

We positioned our Framework within the LDI researchframework

Final remarks

Questions

Stochastic Constraint Programming: a seamless modeling ... · A Stochastic Combinatorial Optimization Problem", Journal of Combinatorial Optimization Applications R. Rossi, S. A.

Documents

Combinatorial Optimization in Clustering

A Novel Sampling Approach to Combinatorial Optimization...

Vienna Workshop on Computational Optimization€¦ ·...

Approximation Algorithms for Stochastic … Algorithms for.....

Inapproximability of Combinatorial Optimization...

A Model Reference Adaptive Search Method for Stochastic...

1 On the Cost and Benefits of Procrastination: Approximation...

Introduction to Stochastic Combinatorial...

Modelling and solving complex combinatorial optimization...

Combinatorial Stochastic Processes - SelectedWorks - Bepress

Combinatorial Optimization and Combinatorial Structure in...

Amsaa: A Multistep Anticipatory Algorithm for Online...

Probabilistic Methods in Combinatorial and Stochastic...

Approximation Algorithms for Stochastic Combinatorial...

Combinatorial Optimization - GBV

Stochastic Combinatorial Optimization with Risk