MULTI-AGENT BASED SCHEDULING D. Ouelhadj ASAP (Automated Scheduling Optimisation and Planning) Research Group School of Computer Science and IT University.

MULTI-AGENT BASED MULTI-AGENT BASED SCHEDULINGSCHEDULING

D. OuelhadjD. Ouelhadj

ASAP (Automated Scheduling Optimisation and Planning) ASAP (Automated Scheduling Optimisation and Planning) Research Group Research Group

School of Computer Science and IT School of Computer Science and IT

University of Nottingham, UKUniversity of Nottingham, UK

Open Issues in Grid Scheduling, 2003

Contents

1. Introduction1. Introduction

2. Multi-agent systems 2. Multi-agent systems

3. Multi-agent based scheduling3. Multi-agent based scheduling

4. Multi-agent systems for integrated and4. Multi-agent systems for integrated and

dynamic scheduling of steel production dynamic scheduling of steel production

5. Conclusion 5. Conclusion


Introduction


Characteristics of most scheduling systems developed in Characteristics of most scheduling systems developed in manufacturing environments:manufacturing environments:

Centralised or hierarchical.Centralised or hierarchical.

Tractable.Tractable.

Stochastic.Stochastic.

Resourcelevel

Supervisor level

Intermediate levels

Supervisor level

Resourcelevel

Centralised and hierarchical scheduling

Introduction


Classical scheduling techniques:Classical scheduling techniques:

Operational research-based techniques: branch and bound, etc.

Artificial intelligence-based techniques: heuristics, meta-heuristics, hyper-heuristics, knowledge-based systems, case-based reasoning, fuzzy logic, etc.

Distributed Scheduling systems using MULTI-AGENTS

Motivations


Real-life scheduling problems are usually physically or functionally distributed (air traffic control, manufacturing systems, health care, etc.).

Complex systems are beyond direct control. They operate through the cooperation of many interacting subsystems, which may have their independent interest, and modes of operation.

Complexity of real-life scheduling problems dictates a local point of view. When the problems are too extensive to be analysed as a whole, solutions based on local approaches are more efficient.

Centralised structures are difficult to maintain and reconfigure, inflexible, inefficient to satisfy real-world needs, costly in the presence of failures, and the amount of knowledge to manage is very large.

Motivations


Need for integration of multiple legacy systems and expertise.

Heterogeneity. Heterogeneous environments may use different data and models, and operate in different modes.

Robustness and reliability against failures.

Scalability and flexibility.

Computational efficiency. Agents can operate asynchronously and in parallel, which can result in increased overall speed.

Clarity of design and reusability.

Costs. It may be much more cost-effective than a centralised system, since it could be composed of simple subsystems of low unit cost.

What is a multi-agent system


communication

action

perception

Agent

environment

autonomy

goal-driven

reactivity and proactivity

social ability

persistent

mobility

adaptability

autonomy

goal-driven

reactivity and proactivity

social ability

persistent

mobility

adaptability

An agent is an intelligent entity that is situated in some environment, and that is capable of flexible and autonomous action in this environment in order to meet its design objectives. By flexible we mean that the system must be responsive, proactive, and social Wooldrige and Jennings (1995).

A Multi-Agent System is a system composed of a population of autonomous agents, which interact with each other to reach common objectives, while simultaneously

each agent pursues individual objectives Ferber (1997).

Cooperation in multi-agent systems


Contract Net Protocol

The contract net protocol is a high level protocol for achieving efficient cooperation introduced by Smith (1980) based on a market-like protocol.

Task announcement

Contract

Bid

agent agentTask announcement

Multi-agent-based scheduling


Resource agent: Local Scheduling

Local autonomy. An agent has the responsibility for carrying out local scheduling for one or more (functional or physical) components, such as machines and jobs.

Agents have the ability to observe their environment and to communicate and cooperate with other agents in order to ensure that local scheduling leads to a globally desirable schedule.

Autonomy allows the agents to respond to local variations, increasing the flexibility of the system.

Concurrency. Negotiation-based decision making instead of totally pre-planned scheduling.

Robustness: fast detection of and recovery from the failures.

Open and dynamic scheduling structures.

AnnouncemenAnnouncemen

tt

of production of production requirementsrequirements

Local Local schedulschedul

inging

Local Local schedulischeduli

ngng

I am free I am free in that in that periodperiod

brokbroken en

downdown

Resource Resource agentsagents

Multi-agent-based scheduling architectures


Autonomous.

Mediator.



Autonomous architectures

Manufacturing entities

Physical or functional agents (resources, parts,

tasks, etc)

Agents representing manufacturing entities (resources, tasks, etc.) have the ability to define their local schedules, react locally to local changes, and cooperate directly with each other to generate the global optimal and robust schedules.



Mediator architectures

Mediator agentCoordinator for global

scheduling

Mediator agent

Physical or functional agents (resources, parts,

tasks, etc)

Manufacturing entities

A mediator architecture has a basic structure of autonomous cooperating local agents that are capable of negotiation with each other in order to achieve production targets.That basic structure is extended with mediator agents to coordinate the behaviour of the local agents to generate the global optimal and robust schedules.

A multi-agent system for integrated and dynamic scheduling of steel production


IntegrationIntegration: how to integrate the scheduling systems of : how to integrate the scheduling systems of

the continuous caster and the hot strip mill ?the continuous caster and the hot strip mill ?

Dynamic scheduling: Dynamic scheduling: Robustness against failuresRobustness against failures ? ?

Use Of MULTI-AGENT SYSTEMS

Steel production scheduling


Multi-agent architecture proposed

`User agent

HSM AgentSY Agent

CC-1 Agent CC-3 AgentCC-2 Agent

user

Continuous Casters Slabs

Hot Strip MillSlabyard

coils

Ladle


Dynamic scheduling of the HSM and CC agents

Presence of real-time events

On the CC agent: steel with wrong chemical compositions.

On the HSM agent: non availability of slabs.

Robust predictive-reactive scheduling

first constructs a predictive schedule and then modifies the schedule in response to real-time events so as to minimise deviation between the performance measure values of the realised and predictive schedule.



Predictive schedules are generated using tabu search.

Robust predictive-reactive schedules are generated using:

Utility, stability, and robustness measures.

Rescheduling strategies: complete rescheduling and schedule repair.



Utility, stability and robustness measure the effect of real-time events, and are used to select the best rescheduling strategy (schedule repair or complete rescheduling) to react to real-time events.

Utility measures the change in the value of the schedule objective function following the schedule revision.

Stability measures the deviation from the original predictive schedule caused by schedule revision.

Robustness combines the maximisation of utility and the minimisation of stability.


Rescheduling strategies

Schedule repair and complete rescheduling strategies

On the CC agent

• Insert- at- end Schedule repair (IESR)

• Insert-Heat Schedule Repair (IHSR)

• Shift Schedule Repair (SHSR)

• Swap Schedule Repair (SWSR)

• Hybrid Schedule Repair (HBSR)

• Complete Rescheduling (CR)

On the HSM agent

•Do-nothing (DON)

• Simple Replacement (SR)

• Closed Schedule Repair (CSR)

• Open Schedule Repair (OSR)

• Hybrid Closed Schedule Repair (HCSR)

• Hybrid Open Schedule Repair (HOSR)

• Partial Reschedule (PR)

• Complete Rescheduling (CR)


Negotiation protocol for inter-agent cooperation

The negotiation protocol is a two-level bidding mechanism based on the contract net protocol involving negotiation at HSMA-SYA and SYA-CCA(s) levels.

At the HSMA-SYA negotiation level, the HSMA requests the supply of slabs from the SYA.

At the SYA-CCA (s) negotiation level, the SYA requests the production of slabs not available in the slabyard from the CCA(s).

A commitment duration is attached to the the negotiation messages to specify the time windows by which the agents must respond to a given negotiation message.



The negotiation protocol incorporates a decommitment mechanism to allow the agents to decommit by specifying appropriate contract’s alternatives in response to future real-time events.



Steps of the negotiation protocol

HSM agent

SY agent

1.HSMA-announcement for the supply of the slabs for the current turn.

CC-1 agent

2. SYA-Announcement for the production of slabs not

available in the SY.

HSMagent

SYagent

4. SYA-bid

3. CCA-bid(s)

6. Forward of the contract, or renegotiation of the non-satisfied slabs.

CC-1agent

CC-nagent

HSMagent

SYagent

CC-1agent

CC-nagent

An

nou

nci

ng

Bid

din

g

Con

trac

tin

g or

re

neg

otia

tin

g

5. Establishment of a contract, or renegotiation of the non-satisfied slabs.

CC-nagent


Prototype developed in simulation

• A prototype has been developed in Microsoft Visual C++/MFC.

• Cooperation between the agents is done with the exchange of asynchronous messages formatted in XML using MSMQ.


Prototype developed in simulation


Evaluation of the performance of the local and global predictive schedules

0

100

200

300

400

500

600

700

800

Ob

jec

tiv

e f

un

cti

on

va

lue

Turn 1

/143 (9

3 slabs)

Turn 1

/300 (1

07slabs)

Turn 2

/300 (1

17slabs)

Turn 1

/1000 (1

20 slabs)

Turn 2

/1000 (1

17 slabs)

Turn 3

/1000 (1

14 slabs)

Turn 4

/1000 (1

74 slabs)

Turn 5

/1000 (8

4 slabs)

Turn 6

/1000 (1

01 slabs)

Scheduled slabs

Objective function value of the initial local schedule

Objective function value of the global schedule afternegotiation/renegotiation


Average frequency of schedule repairand complete rescheduling strategies

0.00 0.

01 0.25 0.

50 0.75 0.

95 1.00

IES

R

IHS

R

SH

SR

SW

SR

HB

SRC

R

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

Frequency

0.0

0 0.0

1 0.2

5 0.5

0 0.7

5 0.9

5 1.0

0

NO

TSRCS

ROS

R

HC

SR

HO

SRP

RCR

0.00

0.10

0.20

0.30

0.40

0.50

0.60

Fre

qu

en

cy

Value of RRescheduling strategies

On the CC agent On the HSM agent


Performance of the utility and stability measures

30.0

50.0

70.0

90.0

110.0

130.0

150.0

170.0

-130.0 -120.0 -110.0 -100.0 -90.0 -80.0 -70.0 -60.0 -50.0 -40.0 -30.0

Utility

Sta

bil

ity

PESR IHSR SHSR SWSR HBSR CR

0

5

10

15

20

25

30

35

40

45

50

-180 -165 -150 -135 -120 -105 -90 -75 -60 -45 -30 -15

Utility

Sta

bilit

y

NOT SR CSR OSR HCSR HOSR PR CR

On the CC agent On the HSM agent


Conclusion

• Dynamic and autonomous distributed scheduling. The dynamic scheduling problem is distributed across a set of agents.

• Local autonomy allows the agents to respond to local variations and self-adaptation to real-time events , increasing the robustness and flexibility of the system.

• The cooperation protocol allows the agents to cooperate and coordinate their local tasks in order to generate desirable globally predictive and robust schedules.

• Dynamic task allocation.


Conclusion

• Natural load-balancing as busy agents do not need to bid.

• Increased Flexibility.

• Robustness against failures.

• Heterogeneity.

• Open and extensible scheduling architectures: Agents can be introduced and removed dynamically.

• Reduced complexity.

• Reduced costs.


MULTI-AGENT BASED SCHEDULING D. Ouelhadj ASAP (Automated Scheduling Optimisation and Planning) Research Group School of Computer Science and IT University.

Documents

grid scheduling

hierarchical scheduling

distributed scheduling

local scheduling leads

multiagent system open

dynamic scheduling structures

preplanned scheduling

introduction open issues