1 The Role of Organizational The Role of Organizational Control in Scaling AI Control in Scaling AI Systems Systems Victor R. Lesser Victor R. Lesser Computer Science Department Computer Science Department University of Massachusetts, Amherst University of Massachusetts, Amherst IJCAI 2009 IJCAI 2009 Pasadena, California Pasadena, California July 16, 2009 July 16, 2009
46
Embed
1 The Role of Organizational Control in Scaling AI Systems Victor R. Lesser Computer Science Department University of Massachusetts, Amherst IJCAI 2009.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
The Role of Organizational The Role of Organizational Control in Scaling AI Control in Scaling AI
SystemsSystems
Victor R. LesserVictor R. Lesser
Computer Science DepartmentComputer Science DepartmentUniversity of Massachusetts, AmherstUniversity of Massachusetts, Amherst
IJCAI 2009IJCAI 2009Pasadena, California Pasadena, California
July 16, 2009July 16, 2009
2
ThanksThanks Raj Reddy – for his support, encouragement Raj Reddy – for his support, encouragement
and mentoringand mentoring Lee Erman – my early colleague and closest Lee Erman – my early colleague and closest
friend for over 40 yearsfriend for over 40 years My wonderful graduate students – for their My wonderful graduate students – for their
creativity, hard work and collegialitycreativity, hard work and collegiality A special thanks to my first graduate student, A special thanks to my first graduate student,
Dan CorkillDan Corkill Multi-Agent Systems community – who have Multi-Agent Systems community – who have
been a welcoming homebeen a welcoming home My wife and children – who have created a My wife and children – who have created a
richness in my personal liferichness in my personal life
3
OutlineOutline BackgroundBackground Examples of Organizational ControlExamples of Organizational Control
Distributed sensor networksDistributed sensor networks Distributed search in a peer-to-peer IRDistributed search in a peer-to-peer IR Multi-agent reinforcement learning for Multi-agent reinforcement learning for
Why This Model for Building Why This Model for Building Intelligent Systems vs. Intelligent Systems vs. A Monolithic Approach?A Monolithic Approach?
Geographical Distribution of Information, Geographical Distribution of Information, Resources, Expertise Resources, Expertise Privacy in sharing information, fee-based servicesPrivacy in sharing information, fee-based services
Modularity for Ease of Development, Modularity for Ease of Development, Debugging, Modification, EvolutionDebugging, Modification, Evolution
Do we need to add in some picture of Do we need to add in some picture of an example system -- Coordinatorsan example system -- Coordinators
6
7
What is the Control Problem What is the Control Problem
Managing Interdependencies among Managing Interdependencies among
Agent ActivitiesAgent Activities What tasks to do, when, where, howWhat tasks to do, when, where, how
What information to communicate, What information to communicate, when, to whomwhen, to whom
How to do this in a globally optimal wayHow to do this in a globally optimal way
Difficulty
Limited communication and computational resources
Ubiquity of uncertainty – uncertain, out-of-date, incomplete information
8
A Model for A Model for Computation in the 21Computation in the 21stst
CenturyCentury
Scaling to 100’s to 1000’s of agentsScaling to 100’s to 1000’s of agents Complex organizational relationships Complex organizational relationships
among agentsamong agents Organizationally situated agentsOrganizationally situated agents
Operate in a “satisficing” modeOperate in a “satisficing” mode Managing uncertainty as an integral Managing uncertainty as an integral
part of network problem solvingpart of network problem solving Highly adaptive and reliableHighly adaptive and reliable
Network of cooperating, intelligent agents Network of cooperating, intelligent agents (people/machines)(people/machines)
9
What is the Lecture What is the Lecture AboutAbout
Organizational control as one way to Organizational control as one way to approach the scaling of AI Systemsapproach the scaling of AI Systems Organizational control is a multi-level Organizational control is a multi-level
approach in which long-term approach in which long-term organizational goals and roles are used as organizational goals and roles are used as guidelines for agents’ detailed operational guidelines for agents’ detailed operational decisionsdecisions. .
Presenting interesting research topics Presenting interesting research topics associated with organizational controlassociated with organizational control
10
Multi-Layer Control Multi-Layer Control Approach Approach
Organizational ControlOrganizational Control Global and long-term perspective Global and long-term perspective
on system performance on system performance Long-term (a-temporal) directivesLong-term (a-temporal) directives
Operational ControlOperational Control Limited and dynamic perspectiveLimited and dynamic perspective Short-term (temporal) decision in Short-term (temporal) decision in
the context of organizational the context of organizational directivesdirectives
ORGANIZATIONAL
GROUP
LOCAL
Approximate, Distributed Optimization for the Global Control Problem
Organizational ControlOrganizational Control
Organizational Design
Guidelines, Roles, Goals...etc.
• Communication and action spaces limited by roles
• Reward functions modified to prefer certain actions aids in coordination
• Global reward function broken into limited-locality reward components
MDP- based operational decisions
Organization-Aware Operational Control
MDP Agent
= Group-coordinatio
n action
Why Does Organizational Why Does Organizational Control WorkControl Work
Exploits Repetitive and Nearly-Exploits Repetitive and Nearly-Decomposable Nature of Problem SolvingDecomposable Nature of Problem Solving
Exploits Knowledge of the EnvironmentExploits Knowledge of the Environment Expectations of Task Arrivals, Problem-Solving Expectations of Task Arrivals, Problem-Solving
Behavior and OutcomesBehavior and Outcomes
13
Efficiency through Assumptions
14
Drivers for Organization FocusDrivers for Organization Focus“Bounded Rationality”“Bounded Rationality”
Organizational Control provides a framework Organizational Control provides a framework for dealing with computational issues of scalefor dealing with computational issues of scale
Decrease non-local information and reasoning Decrease non-local information and reasoning
necessarynecessary
Acting in accordance with guidelines leads to Acting in accordance with guidelines leads to effective coordination decisionseffective coordination decisions
Shift from an Agent-Centric, Operational Shift from an Agent-Centric, Operational View of Coordination to an Organization-View of Coordination to an Organization-
Doppler radar Doppler radar units (30’s)units (30’s)– Scan Scan one of one of
threethree 120 120 sectors at a sectors at a timetime
• Commodity Commodity processor processor associated associated with each with each radarradar
•Communicate Communicate short short messages messages using one of 8 using one of 8 radio channelsradio channels
•Triangulate Triangulate radars to do radars to do trackingtracking
19
CASA - Monitoring for CASA - Monitoring for Severe Weather (2008)Severe Weather (2008)
Network of short-range (30 Network of short-range (30 km), overlapping, adaptive km), overlapping, adaptive weather-sensing radarsweather-sensing radars SSmall fielded system in mall fielded system in
OklahomaOklahoma
Goal: Detect low-lying weather Goal: Detect low-lying weather phenomena such as tornadoes phenomena such as tornadoes within 60 secondwithin 60 second
20
How to Control the DSNHow to Control the DSN
ScalabilityScalability: Hundreds of sensors, multiple : Hundreds of sensors, multiple targets, constrained communicationtargets, constrained communication
What if there were no (formal) organization?What if there were no (formal) organization? Who decides if a target is new?Who decides if a target is new? Who tracks a target?Who tracks a target? How do trackers obtain sensor information?How do trackers obtain sensor information?
These operational control decisions could be These operational control decisions could be made individually by each agent, but through made individually by each agent, but through organization can be made easierorganization can be made easier
Limits sources of Limits sources of informationinformation
Facilitates data Facilitates data retrievalretrieval
22
Partitioning of NodesPartitioning of Nodes
• The environment is first partitioned into sectors.The environment is first partitioned into sectors.• Sector managers are then assignedSector managers are then assigned..
23
Competition for Sensor Competition for Sensor AgentsAgents
• Sector members send their capabilities to their managers.Sector members send their capabilities to their managers.• Each manager then generates and disseminates a scan schedule.Each manager then generates and disseminates a scan schedule.
Group agents of similar contentGroup agents of similar content Limit subset of agents to be probedLimit subset of agents to be probed Add lateral links to quickly locate diverse Add lateral links to quickly locate diverse
contentcontent
Incremental construction of the organization Incremental construction of the organization as new agents join networkas new agents join network
A two-phase search algorithmA two-phase search algorithm Locate relevant hierarchical agent clusters Locate relevant hierarchical agent clusters Perform searches in clustersPerform searches in clusters
Partial view and no Partial view and no global reward global reward signalsignal
Communication Communication delaydelay
40
Organization-Based Organization-Based Control FrameworkControl Framework
5
41
Integrate Supervisory Integrate Supervisory Information into Multi-Information into Multi-
Agent LearningAgent Learning
Policy Update
Action Selection
Policy
Reward Action State Policy Update
Action Selection
Policy
Reward Action State
Supervisory Policy
Adaptation
ReportGenerator
Rules andSuggestions
Adapted Policy
Abstracted state and reward
(a) Multi-Agent Reinforcement Learning (MARL) (b) MARL under Supervision
Reward
42
Experiments: Distributed Experiments: Distributed Task Allocation Problem Task Allocation Problem
(729 agents)(729 agents)
27 X 27 Agent Network
43
What Do These Examples What Do These Examples Tell UsTell Us
Organizational Control can be used in Organizational Control can be used in scaling of very different types of AI scaling of very different types of AI problem solvingproblem solving
Flexibility and adaptability of control Flexibility and adaptability of control decisions at all levels is importantdecisions at all levels is important
Very early in our understanding of how to Very early in our understanding of how to effectively exploit this approacheffectively exploit this approach
44
How to Create an How to Create an OrganizationOrganization
What Constitutes an What Constitutes an OrganizationOrganization
What is the Role of Institutional What is the Role of Institutional MechanismsMechanisms Computational artifacts for controlComputational artifacts for control
What Type of AgentsWhat Type of Agents CooperativeCooperative Self-interestedSelf-interested Semi-cooperativeSemi-cooperative
What is an Organizationally Situated AgentWhat is an Organizationally Situated Agent
46
Relationship between MAS and Relationship between MAS and Organizational Structuring from a Organizational Structuring from a business/sociological perspective?business/sociological perspective?
Are emotions effective computational Are emotions effective computational
mechanisms?mechanisms? Skepticism – limits effect of info distraction Skepticism – limits effect of info distraction Boredom – avoid over-learning of routine Boredom – avoid over-learning of routine
taskstasks Self-interest – decision making without Self-interest – decision making without
global impact global impact
MAS and Human MAS and Human OrganizationsOrganizations
47
Can you Automate the Can you Automate the Organizational Design Organizational Design
Process?Process? Theory behind organizational designTheory behind organizational design
The nature of sub-problem interdependenciesThe nature of sub-problem interdependencies
Designing for multi-attributed nature of Designing for multi-attributed nature of organizational performanceorganizational performance Reliability, fail-softness, adaptabilityReliability, fail-softness, adaptability
Predicting the performance of a Predicting the performance of a computational organizationcomputational organization
Specialness of the search process for Specialness of the search process for finding a good organizationfinding a good organization Repetitiveness of structureRepetitiveness of structure
48
The Human in the LoopThe Human in the Loop
How can computational organizations How can computational organizations be controlled by peoplebe controlled by people
How can human and computational How can human and computational organizations interactorganizations interact
What is the implication for how we see What is the implication for how we see ourselves and others ourselves and others
49
SummarySummary
Organizational Control is Organizational Control is important in how we think about important in how we think about scaling AI systemsscaling AI systems
Organizational Control is an Organizational Control is an intrinsically interesting problem intrinsically interesting problem that deserves our intellectual that deserves our intellectual attentionattention