Control Techniques for Complex Systems Department of Electrical & Computer Engineering University of Florida Sean P. Meyn Coordinated Science Laboratory and the Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign, USA April 21, 2011 1 / 26
The systems & control research community has developed a range of tools for understanding and controlling complex systems. Some of these techniques are model-based: Using a simple model we obtain insight regarding the structure of effective policies for control. The talk will survey how this point of view can be applied to approach resource allocation problems, such as those that will arise in the next-generation energy grid. We also show how insight from this kind of analysis can be used to construct architectures for reinforcement learning algorithms used in a broad range of applications.
Much of the talk is a survey from a recent book by the author with a similar title, Control Techniques for Complex Networks. Cambridge University Press, 2007. https://netfiles.uiuc.edu/meyn/www/spm_files/CTCN/CTCN.html
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Control Techniques for Complex SystemsDepartment of Electrical & Computer Engineering
University of Florida
Sean P. Meyn
Coordinated Science Laboratoryand the Department of Electrical and Computer Engineering
. Complex systems: Model reduction specialized to tomorrow’s gridShort term operations and long-term planning
. Resource allocation: Controlling supply, storage, and demandResource allocation with shared constraints.
. Statistics and learning: For planning and forecastingBoth rare and common events
. Economics for an Entropic Grid: Incorporate dynamics and uncertaintyin a strategic setting.
How to create policies to protect participants on both sides of themarket, while creating incentives for R&D on renewable energy?
23 / 26
Next Steps
Complex SystemsMainly energy
Entropic Grid : Advances in systems theory...
. Complex systems: Model reduction specialized to tomorrow’s gridShort term operations and long-term planning
. Resource allocation: Controlling supply, storage, and demandResource allocation with shared constraints.
. Statistics and learning: For planning and forecastingBoth rare and common events
. Economics for an Entropic Grid: Incorporate dynamics and uncertaintyin a strategic setting.
How to create policies to protect participants on both sides of themarket, while creating incentives for R&D on renewable energy?
23 / 26
Next Steps
Complex SystemsMainly energy
Entropic Grid : Advances in systems theory...
. Complex systems: Model reduction specialized to tomorrow’s gridShort term operations and long-term planning
. Resource allocation: Controlling supply, storage, and demandResource allocation with shared constraints.
. Statistics and learning: For planning and forecastingBoth rare and common events
. Economics for an Entropic Grid: Incorporate dynamics and uncertaintyin a strategic setting.
How to create policies to protect participants on both sides of themarket, while creating incentives for R&D on renewable energy?
23 / 26
Next Steps
Complex SystemsMainly energy
How to create policies to protect participants on both sides of the market,while creating incentives for R&D on renewable energy?
Our community must consider long-term planning and policy, along withtraditional systems operations
Planning and Policy, includes Markets & Competition
Evolution? Too slow!What we need is Intelligent Design
24 / 26
Next Steps
Complex SystemsMainly energy
How to create policies to protect participants on both sides of the market,while creating incentives for R&D on renewable energy?
Our community must consider long-term planning and policy, along withtraditional systems operations
Planning and Policy, includes Markets & Competition
Evolution? Too slow!What we need is Intelligent Design
24 / 26
Next Steps
Complex SystemsMainly energy
How to create policies to protect participants on both sides of the market,while creating incentives for R&D on renewable energy?
Our community must consider long-term planning and policy, along withtraditional systems operations
Planning and Policy, includes Markets & Competition
Evolution?
Too slow!What we need is Intelligent Design
24 / 26
Next Steps
Complex SystemsMainly energy
How to create policies to protect participants on both sides of the market,while creating incentives for R&D on renewable energy?
Our community must consider long-term planning and policy, along withtraditional systems operations
Planning and Policy, includes Markets & Competition
Evolution? Too slow!
What we need is Intelligent Design
24 / 26
Next Steps
Complex SystemsMainly energy
How to create policies to protect participants on both sides of the market,while creating incentives for R&D on renewable energy?
Our community must consider long-term planning and policy, along withtraditional systems operations
Planning and Policy, includes Markets & Competition
Evolution? Too slow!What we need is Intelligent Design
24 / 26
Next Steps
Conclusions
The control community has created many techniques for understandingcomplex systems, and a valuable philosophy for thinking about controldesign
In particular, stylized models can have great value:
. Insight in formulation of control policies
. Analysis of closed loop behavior, such as stability via ODE methods
. Architectures for learning algorithms
. Building bridges between OR, CS, and control disciplinesThe ideas surveyed here arose from partnerships with researchers in
mathematics, economics, computer science, and operations research.
Besides the many technical open questions, my hope is to extend theapplication of these ideas to long-range planning, especially in applicationsto sustainable energy.
25 / 26
Next Steps
Conclusions
The control community has created many techniques for understandingcomplex systems, and a valuable philosophy for thinking about controldesign
In particular, stylized models can have great value:
. Insight in formulation of control policies
. Analysis of closed loop behavior, such as stability via ODE methods
. Architectures for learning algorithms
. Building bridges between OR, CS, and control disciplinesThe ideas surveyed here arose from partnerships with researchers in
mathematics, economics, computer science, and operations research.
Besides the many technical open questions, my hope is to extend theapplication of these ideas to long-range planning, especially in applicationsto sustainable energy.
25 / 26
Next Steps
Conclusions
The control community has created many techniques for understandingcomplex systems, and a valuable philosophy for thinking about controldesign
In particular, stylized models can have great value:
. Insight in formulation of control policies
. Analysis of closed loop behavior, such as stability via ODE methods
. Architectures for learning algorithms
. Building bridges between OR, CS, and control disciplinesThe ideas surveyed here arose from partnerships with researchers in
mathematics, economics, computer science, and operations research.
Besides the many technical open questions, my hope is to extend theapplication of these ideas to long-range planning, especially in applicationsto sustainable energy.
25 / 26
Next Steps
References
S. P. Meyn. Control Techniques for Complex Networks. Cambridge University Press,Cambridge, 2007.
S. P. Meyn and R. L. Tweedie. Markov chains and stochastic stability. Second edition,Cambridge University Press – Cambridge Mathematical Library, 2009.
S. Meyn. Stability and asymptotic optimality of generalized MaxWeight policies. SIAM J.Control Optim., 47(6):3259–3294, 2009.
V. S. Borkar and S. P. Meyn. The ODE method for convergence of stochasticapproximation and reinforcement learning. SIAM J. Control Optim., 38(2):447–469, 2000.
S. P. Meyn. Sequencing and routing in multiclass queueing networks. Part II: Workloadrelaxations. SIAM J. Control Optim., 42(1):178–217, 2003.
P. G. Mehta and S. P. Meyn. Q-learning and Pontryagin’s minimum principle. In Proc. ofthe 48th IEEE Conf. on Dec. and Control, pp. 3598–3605, Dec. 2009.
W. Chen, D. Huang, A. A. Kulkarni, J. Unnikrishnan, Q. Zhu, P. Mehta, S. Meyn, andA. Wierman. Approximate dynamic programming using fluid and diffusion approximationswith applications to power management. In Proc. of the 48th IEEE Conf. on Dec. andControl, pp. 3575–3580, Dec. 2009.