By Dr Simon Martin CHORDS Group Division of Computing Science and Mathematics School of Natural Sciences University of Stirling, Stirling FK9 4LA. [email protected].

ByDr Simon Martin

CHORDS GroupDivision of Computing Science and Mathematics

School of Natural Sciences University of Stirling, Stirling FK9 4LA.

[email protected]

A Multi-agent based Cooperative Approach to Scheduling and Routing

ContentsIntroduction – What are multi or intelligent agents?

Multi/Intelligent -agentsIEEE FIPA agent standard

MACS Agent-based platform

Case StudiesVRPPFSP

FairnessFairness in nurse rosteringFairness in requirements assignments for the next release problem

Conclusions

• Future Work

Thank you

Multi/Intelligent -agents

Agents maintain an internal representation of their environment.

They communicate by Asynchronous messaging.

They are autonomous, no one process is in overall control

They are capable of completing a task on their own or can cooperate

This means they can execute distributed algorithms where no agent is in overall control

IEEE FIPA standard

There is an IEEE standard called the Foundation for Intelligent Physical Agents (FIPA)

There are are number of Open source FIPA platforms:

FIPA-OShttp://fipa-os.sourceforge.net/index.htm

Jadex Agentshttp://www.activecomponents.org/bin/view/About/New+Home

JIAC Intellient Agentshttp://www.jiac.de/

JADEhttp://en.wikipedia.org/wiki/Java_Agent_Development_Framework

FIPA compliant Multi-Agent Platform

AMS

DF

AMS

DF

The DF (Directory Facilitator) provides a directory which announces which agents are available on the platform.

The AMS (Agent Management System) controls the platform. Is the only one who can create and destroy other agents, destroy containers and stop the platform.

Inter platform communication

Multi-Agent Cooperative System(MACS)

Meta-heuristics require careful tuning to a specific problemThey require parameter tuningBalancing intensification and diversification

Some meta-heuristics are better at some problems than others

They have different strengths and weaknesses

But what if there was a of combining these strengths and weaknesses in one system?

This might be achieved if different meta-heuristics cooperated with each other

MACS

Problem definition

Launcher Agent

Cooperating Agent

Cooperating Agent

Cooperating Agent

Cooperating Agent The Launcher Agent (LA) sends the same problem to each agent

MACS

Launcher Agent

Cooperating Agent

Cooperating Agent

Cooperating AgentAgents cooperate by passing Best edges

MACS -again ....

Problem definition

Launcher Agent

Cooperating Agent

Cooperating Agent

Cooperating Agent

Cooperating Agent Each agent sends its best overall solution to the launcher agent. The LA takes the bestAnd writes it to file

MACS – just to ram it home

Multi/Intelligent -agents

Image: Wikipedia by Utkarshraj Atmaram. http://en.wikipedia.org/wiki/Intelligent_agent#mediaviewer/File:IntelligentAgent-Learning.png

Inside a Multi/Intelligent -agent

Ontology for Scheduling and Routing

Graph

Edge

Constraints

Vertices

Cities Jobs Assignments

The Vertex objectIs the interface between theframework and specific Problem instance

Problem specific data

interface

Objects of the agent-based framework

Problem specific objects inheriting from the abstract vertex object

Subgraph

Customers & Depots

Cooperation protocol

Case Studies

Permutation Flowshop SchedulingMeta-heuristic Randomised NEHA Juan et. al

Capacitated Vehicle RoutingRandomised Clarke Wright Savings AlgorithmA Juan et. al

Fairness In Nurse RosteringVNS, Simulated Annealing and Tabu SearchMartin, Smet, Ouelhadj, Vanden Berghe, Özcan.

The platform has been applied to three case studies

Permutation Flowshop Scheduling

Permutation Flowshop SchedulingTaillard benckmark instance tai_051_50_20

Capacitated Vehicle Routing

Capacitated Vehicle Routing

Augerat Benchmark instance A-n63-k9

The Nurse Rostering problem

The Scheduling of hospital personnel is Particularly challenging because:

There are different staffing needs on different days and shifts

Staff work in shifts

Healthcare institutions work around the clockThe need for day and night shifts

The correct staff mix for each ward

Many different employment contractsPart-time Special arrangements

Fairness so that staff are happy

The standard objective function

Let C be the set of constraints. Wc is weight associated with a given constraint N

is the number of violations of that constraint.

Is the number of roster constraints

MinWS = minimise the sum of the sum of all nurses violations

Models of Fairness

Models of Fairness

New Fairness objective functions

MinMax = minimise the number of nurses × worst nurse violation

MinDev = minimise the sum of deviations from the average + the numbers of nurses × the mean roster quality

MinError = minimise the sum of the differences of max roster value – min roster value a + the mean roster quality

MinSS = minimise the sum of the squared violations associated with assigning a nurse to a given roster

Models of Fairness

Models of FairnessMeasuring fairness is done with the Jains Fairness function (Jain et al., 1984; Muhlenthaler and Wanka, 2012).

It is the sum of the squared violations in assigning a nurse to a given roster divided by the number of nurses times the squared value of assigning a nurse to a roster.

Its values range from the worse case 1/N to 1 where N is the number of nurses to 1 where the roster is completely fair.

Fairness Results

Fairness Results

Tensor Online learning

The agent system has been updated:

A new learning system has been developed based on tensors

It still uses the same conversation structure as before

Instead of sharing edges the agents now share tensors made from incumbent solutions.

Cooperation Protocol with Tensors

Tensor Online learning

Agents are 20 best incumbent solutions.

The initiator agent, for that conversation, collects all the incumbent solutions.

The initiator then builds an tensor where n is the length of problem instance and m is the number of incumbent solutions.

The tensor build from adjacent matrices of each incumbent solution.

The initiator factorises the matrix. The result is an matrix called a basic frame.

The basic frame is treated as an adjacent matrix and converted back to a list of good edges. This list is shared with all the agents.

The agents update their short-term memories.

The agents then use the list of edges in short-term memory in conjunction with their metaheuristic to build new incumbent solutions.

Conclusions

Distributed asynchronous agent platform

Modular

Ontologies

Peer to Peer

Scalable

Future Work

Fairness in requirements assignments for the next release problem

Model each customers requirements on an agent Compare multi-objective approach to single objective approach

Improve the ontology to work on more problems

Improve the tensor learning system

Papers

Simon Martin, Djamila Ouelhadj, Pieter Smet, Greet Vanden Berghe, and Ender Ozcan. Cooperative search for fair nurse rosters. Expert Systems with Applications, 40(16):6674-6683, 2013.

Simon Martin, Djamila Ouelhadj, Patrick Beullens,Ender Ozcan,Angel A. Juan,Edmund.K.Burke. A MULTI-AGENT BASED COOPERATIVE APPROACH TO SCHEDULING AND ROUTING. under review European Journal of Operational Research October 2015.

Shahriar Asta, Simon Martin, Ender Ozcan, Edmund Burke. A Multi-agent System Embedding Online Tensor Learning for flow shop Scheduling. Submitted to Information Sciences, July 2015.

Thank you