Past, Present, and Future · 2020-07-09 · • Internet/web • Users hardly ... • Gams Data eXchange (GDX): • Complements the ASCII text data input • Advantages: – Fast
Post on 14-Jul-2020
1 Views
Preview:
Transcript
Past, Present, and Future
Workshop on Optimization/Modeling/Applications
Washington, DC September 19, 2003
GAMS
Alex Meeraus
GAMS Development Corporation
2
Model Structure
3
Model Data
4
Matrix Generator
5
Matrix Generator Input
6
MPS File – Column Section
7
MPS Revision File
8
MPS Output
9
WB Old Slide 1
10
WB Old Slide 2
11
WB Old Slide 3
12
WB Old Slide 4
13
WB Old Slide 5
14
WB Old Slide 6
15
WB Old Slide 7
16
Basic Principles
• Separation of model and solution methods
•Models is a data base operator and/or object
• Balanced mix of declarative and procedural approaches
• Computing platform independence
•Multiple model types, solvers and platforms
17
Computation Past
• Algorithm limits applications
• Problem representation is low priority
• Large costly projects
• Long development times
• Centralized expert groups
• High computational cost, mainframes
• Users left out
Model Present
• Modeling skill limitsapplications
• Algebraic model representation
• Smaller projects
• Rapid development
• Decentralized modeling teams
• Low computational cost, workstations
• Machine independence
• Users involved
ApplicationFuture
• Domain expertise limits application
• Off-the-shelf graphical user interfaces
• Links to other types of models
• Models embedded in business applications
• New computing environments
• Internet/web
• Users hardly aware of model
Change in Focus
18
Dinner Table Assignment
• ‘Couples” on the same table
• Mix backgrounds and interest (algorithms,applications,World Bank, GAMS staff and family)
• Allow for last minute changes
• Use a model to shed responsibility for ‘poor’ assignments
19
List of Participants
20
Problem Domains
21
New Excel Input
22
Model Definition
23
Solve Summary
24
Output Report
25
Group Balance
26
Excel Report
27
Data Connectivity
• Data Import/Export from Standard Applications– MS Office, Database, Text files, …
• Capture an Instance of a Model Failure– Reproducibility of Model/System Bugs
– Problems: Life Database/different Platforms
• Definition of Data Interface– Gams Data eXchange (GDX)
– Separation of Responsibility for Data and Model
28
Gams Data eXchange
• Gams Data eXchange (GDX):
• Complements the ASCII text data input
• Advantages:– Fast exchange of data
– Syntactical check on data before model starts
– Compile-time and Run-time Data Exchange
ApplicationGDX
GAMS
29
GDX Tools
GDX
gdxxrw(MS Office)
gams
IDE GDX Viewer
gdxdiff
gdxdump
gdxmerge
gdxsplitGDXAPI
30
Data Contract in GAMS
31
Data Contract in Excel
32
Data Contract in Excel – cont.
33
GAMS/MapInfo
Less Than 0
0 - 199
200-1000
1000-3000
Increase in Ktons Per Year
34
GAMS/MATLAB
35
36
Three Examples of GDX
• VEDA : post processing and ad-hoc information restructuring
• LOGMIP: experimental language extensions, algorithmic design studies
• Branch & Cut & Heuristic: automation of complex solver enhancements
37
The VErsatile Data Analyst (VEDA)
Amit Kanudia, KanORS Consulting Inc.
Gary A. Goldstein, IRG
38
VEDA Description• Manages GAMS Sets, Parameters and Model
Results for the purpose of dynamic viewing and re-organizing, and presentation to others.
• Allows for cross-cutting access to parameters and results according to index structure and element values.
• Supports construction of user-defined sets, definition of new tables, and manipulation of tables.
• Provides powerful filtering capabilities on both elements names and descriptions.
39
VEDA Description• Employs customizable n-dimensional Data
Cube component for displaying tables.• Full-blown graphing facilities, as well as
exporting to Excel, Word, PPT, HTML.• Web-enabled, allowing model results to be
made available over the web in user-friendly, reconfigurable tables.
• Convenient GDX2VEDA data definition file for identification of information to be ported from GAMS to VEDA.
40
VEDA Table Form – Attribute List
41
VEDA Table Form – Sets (Model&User)/Elements
42
VEDA Table Form – Search & Select Engine
43
ProcessSet/Process/Commodity/Scenario
44
VEDA Cube – Scenario/ProcessSetTotals
45
VEDA Cube – Analysis Graph
46
GDX2VEDA Specification
47
Characterization and Formulation of Disjunctions and their Relaxations
Aldo Vecchietti (*), Sangbum Lee(+) and Ignacio Grossmann(+)
(+) Chemical Engineering Department - Carnegie Mellon University(*) INGAR - Instituto de Desarrollo y Diseño
UTN - Facultad Regional Santa Fe
LOGMIP
48
Accepted Formulations
MILP and MINLP formulations are the most accepted byacademia and the industry for problems involving discretedecisions
Recent formulations
Constrained Logic Programming (CLP): general logic constraints, applied mainly for highlycombinatoric problems
Generalized Disjunctive Programming (GDP) :disjunctions and logic propositions for discretechoices
Problem Formulations
49
min Z = Σk ck + f(x) + dTysujeto a:
g(x) ≤ 0
r(x) + Dy ≤ 0Ay ≥ a
Yik
∨ hik (x) ≤ 0 k∈ SDi∈Dk ck = γik
Ω (Y) = Truex∈ Rn, y∈ 0,1q
Y∈ True, Falsem , ck ≥ 0
q y binary variables (0-1)q x y ck continuous variablesq Yik Boolean variables to establish if a
disjunction term is true or false q f(x) objective function (linear or not linear)q dTy Linear cost term q g(x) constraint (linear or not linear)
independent of discrete choicesq r(x)+Dy≤ 0 mixed integer constraint
(linear or not linear) q Ay≥ a integer constraintqΩ (Y) propositional logic relating Boolean
variables (disjunction terms)
Disjunctions
50
Big-M
Linear
[ ] ni
Ti
DiR x b x a F ∈≤∨=
∈
)y-(1M b x a iiiTi +≤
1 yi
i =∑
Mi = max aiT x – bi | xlo≤ x≤ xup
Non Linear
[ ] ni
DiR x 0 (x)h F ∈≤∨=
∈
Beaumont Surrogate
1 -N Mb
M
xa
i i
i
i i
Ti +∑≤∑ 1 -N
M)x(h
i i
i ≤∑
hi(x)≤ Mi(1-yi)
Mi = max hi(x) | xlo≤ x≤ xup
1 yi
i =∑
Big-M Relaxation
51
iupi
i
iDi
i
iiii
ni
Dii
y v v 0
D i 1, y 0 , 1 y
0 )y / v(h y
Rv x, 0 vx -
≤≤
∈≤≤∑ =
≤
∈=∑
∈
∈
y v v 0
D i 1, y 0 , 1 y
0 y b -v a
Rv x, 0 vx -
iupii
iDi
i
iiiTi
ni
Dii
≤≤
∈≤≤∑ =
≤
∈=∑
∈
∈
[ ] ni
Ti
DiR x b x a F ∈≤∨=
∈
Linear Non linear
[ ] ni
DiR x 0 (x)h F ∈≤∨=
∈
Convex Hull Relaxation
52
The x space
false,trueY
c;x,x
c
xx
Y
c
xx
Y
.t.s
c).x().x(minZ
∈
≤≤≤
=
==¬
∨
=
≤+
+−+−=
1
121
1
21
1
1
22
21
1
12
22
1
010
0
0
1
1
1111
Improper disjunction ofspecial interest in Process
Engineering
Both relaxations have the same feasible region
X1
X2
1
1
Relaxation Comparison
53
X1
X2
yConvex hull
X1
X2
y Big-M
10
0
0
1111
1
12
11
21
22
21
12
22
1
≤≤≤≤
≤≤≤+
+−+−=
y
yx
yx
yxx
.t.s
y).x().x(minZ
1010
1111
121
122
21
12
22
1
≤≤≤≤≤+
+−+−=
y;x,x
yxx
.t.s
y).x().x(minZ
X – Y Space
54
Modeling two terms disjunction
∨
2Constraint
se Fal
1 Constraint
True
IF (condition1) THENConstraints to be considered when condition1 is True
ELSEConstraints to be considered when condition1 is False
END IF
Two Term Disjunction
55
∨∨
∨
N sConstraint
N ..
2sConstraint
2
1 sConstraint
1
IF (condition1) THENConstraints to be considered when condition1 is True
ELSIF (condition2) THENConstraints to be considered when condition1 is True
ELSIF (condition3) THEN...
ELSIF (conditionN) THENConstraints to be considered when condition1 is True
END IF
N Term Disjunction
56
.3,2,1,,
0 ,5050
01
7
02
5
02
2min
21
32
321
1
3
21
3
2
2
21
1
21
=∈≥≤≤≤≤
∧¬¬⇒¬∧
=
¬∨
≤−
=≤−∨
=≤++−
++
jfalsetrueY
cx, x
)Y (Y
Y Y Y
x
Y
xx
Y
c
x
Y
c
xx
Y
x x c
j
Example
57
$ONTEXT BEGIN LOGMIP$ONTEXT BEGIN LOGMIP
IF Y('1') THENIF Y('1') THEN
EQUAT1;EQUAT1;
EQUAT2;EQUAT2;
ELSIF Y('2') THENELSIF Y('2') THEN
EQUAT3;EQUAT3;
EQUAT4;EQUAT4;
ENDIF;ENDIF;
IF Y('3') THENIF Y('3') THEN
EQUAT5;EQUAT5;
ELSEELSE
EQUAT6;EQUAT6;
ENDIF;ENDIF;
$OFFTEXT END LOGMIP$OFFTEXT END LOGMIP
OPTION MIP=LOGMIPC;OPTION MIP=LOGMIPC;
MODEL PEQUE2 /ALL/;MODEL PEQUE2 /ALL/;
SOLVE PEQUE2 USING MIP MINIMIZING Z;SOLVE PEQUE2 USING MIP MINIMIZING Z;
EQUAT1.. X('2')EQUAT1.. X('2')-- X('1') + 2 =L= 0;X('1') + 2 =L= 0;
EQUAT2.. C =E= 5;EQUAT2.. C =E= 5;
EQUAT3.. 2 EQUAT3.. 2 -- X('2') =L= 0;X('2') =L= 0;
EQUAT4.. C =E= 7;EQUAT4.. C =E= 7;
EQUAT5.. X('1')EQUAT5.. X('1')--X('2') =L= 1;X('2') =L= 1;
EQUAT6.. X('1') =E= 0;EQUAT6.. X('1') =E= 0;
INT1.. Y('1')+ Y('3') =L= 1;INT1.. Y('1')+ Y('3') =L= 1;
INT2.. Y('2')+ (1INT2.. Y('2')+ (1--Y('3')) =G= 1;Y('3')) =G= 1;
INT3.. Y('2')+ Y('3') =L= 1;INT3.. Y('2')+ Y('3') =L= 1;
FICT.. SUM(I, Y(I)) =G= 0;FICT.. SUM(I, Y(I)) =G= 0;
OBJECTIVE.. Z =E= C + 2*X('1') + OBJECTIVE.. Z =E= C + 2*X('1') + X('2');X('2');
X.UP(J)=20;X.UP(J)=20;
C.UP=7;C.UP=7;
GAMS Implementation
Branch-and-Cut & Heuristics in GAMS for MIP Problems
Hua (Edward) NiGeorge Washington University
59
Branch-and-Cut (B&C)
• B&C is an established algorithm to improve the B&B search.
• Implementation facilities:– MIP solver callback functions (CPLEX, XPRESS, …)– B&C framework (ABACUS, COIN BCP, …)
• Required Knowledge for B&C– IT knowledge (C/C++/JAVA, Solver APIs)– Mathematical programming knowledge– Application specific knowledge
• Supply GAMS users with an easy access to B&C
Design Principle
GAMS SYSTEM
GAMSSolverLink
MIP Solver(e.g. CPLEX)
B&B Callback
UserCut
Generator&
Heuristics(e.g. GAMS)
A Steiner Tree Problem
• Berlin52 – from SteinLib– 52 nodes (1 source, n’=15 sinks), 1326 edges
• Flow formulation:
• Dicut: if S⊂V and b(S)>0.
• Separation:
1,0,'0,..,min)(),()(),(),(
∈≤≤=− ∑∑∑+− ∈∈∈
yynxbxxtsyf ijijiijiij
iijji
Ajiijij
δδ
62
Cut Generator in GAMS I
63
Cut Generator in GAMS II
Computational Results
• Overhead– Time spent within the callback functions minus
MIP computation on cuts and heuristics.– 20% ~ 25%
• Performance Improvements (B&C vs. regular GAMS/CPLEX)– Steiner: 6 hours vs. 2+ days (unsolvable)– Pipeline Design: 20 minutes vs. 450 minutes
65
Convergence – Steiner Tree
626.4
835.2
1044
1252.8
0 300 600 900 1200 1500 1800
Compuation Time (seconds)
B&C&H_Dual
B&C&H_Primal
B&C_Dual
B&C_Primal
CPLEX_Dual
CPLEX_Primal
66
Convergence – Pipeline Design
1280.7
1351.85
1423
1494.15
1565.3
1636.45
0 300 600 900 1200 1500 1800
Computation Time (seconds)
B&C&H_Dual
B&C&H_Primal
CPLEX_Dual
CPLEX_Primal
B&C_Dual
B&C_Primal
67
What is a Model?
• List of Equations – Mathematical Programming (MP) Model
• Collection of several intertwined (MP) Models– Data Preparation and Calibration– “Solution” Module– Reporting Module
• Categorization of Models by answering:– Who is the User of a Model?
68
Who is the User of a Model?
• (Academic) Researcher– One time use (Research Paper)
• Domain&Model Expert– Model Results used for Consulting
• Black Box User– Model integrated in (Optimization) Application
• Each Category has its own needs– Development & Deployment
69
Research Models
• 95% of Model Source is (Equation) Algebra
• Declarative Modeling
• Set of (Benchmark) Problem Instances
• Modeling Language Differences: Few
• Taste (Syntax, Development Environment,…)
• Platform, Model Type, Solver
70
Supported Solvers
71
Available Solvers
• Growing number of MP Solvers (often out of Academic Labs)
• NEOS: >40 Solvers• Impractical to have
Interface to allModeling Languages
• “Solution”: Model Translation
72
Model Translation
73
Consulting Models
• Model is Tool for Problem Analysis
• 10% of Model Source is (Equation) Algebra
• User: Domain & Modeling Expert (not necessary the same person)
• Living Model (changes with the problem)– Lifecycle: At least 10 years
– Technology Change (Platform, Solver, …)
International CoalTrade Model
Coal MarginalCost Models
National Power Model™• Dispatch Economics• 90+ Control Areas
• Seasonal/TOD Prices, Flow & Gen.
DemandModel
GDPWeatherElectric Intensity
Transmission• Bi-Directional Simultaneous Flows• Seasonal Limits• Time-of-Day Rates
Regional Emission Limits• SO2, CO2, NOx
GenerationDatabase(all units)
GenerationCost Supply
Models
Hill and Associates, Inc.Electric Generation, Coal and Emissions Forecasting System
Utility FuelEconomics Model
• Fuel Switching• Clean-up Equip Choices
• Allowance Trading
GenerationCompliance
Models
Strategies for• SO2• NOx• Particulates
Demand• Industrial• Commercial• Residential
GenerationDatabase
(coal units)Coal Supply Curves• Cash Cost by Mine• All Regions• By Coal Type
• New Build• Gas & Oil Forecast
Coal Plant Costs
&EmissionsForecast
Plant/Area by TOD• Generation• Power Flows• Marginal Prices• Emissions
• Unit Info• Transp. Rates• Compliance Costs
Coal Plant Energy Demand
SeasonalDispatch Costs,Emission Rates
I-2
Figure I-1
75
Modeling System Requirements
• Survive in such a diverse Environment
• Compatibility – 15 year application lifecycle
• Data Connectivity/Exchange– Programs and People
• Support for Analysis/Reporting– Modeling System Tools and external Program
76
Black Box Model
• Innocent User
• Bulletproof Optimization Application– No failures: e.g. No Infeasible Models
• Model embedded in larger System:
• Optimization
• Takes Longer than one is willing to wait
• It will eventually fail
• Application
• Real Time
• Always need a Solution to Problem
77
Scheduling US Military Academy West Point
“ … each student’s daily activities are a carefully regimented balance of academic, military, and physical requirements.”
78
Modeling System Requirements
• Reliable System• Using Third Party Software (Solver)
– Keep Resource Usage of Solver in check– Solver will eventually crash
• Possibility to Implement Simple Heuristics• Platform Choice• Less important: Data Import/Export
– IT does not want Modeling System to mess with DB– Simple/Thin Interfaces (text files, XML)
79
Future Directions
• Value Added Applications
• Solution Service Providers
• Distributed System Architectures
• New Solution Approaches
• Continued Changes in the Modeling ‘Industry’
top related