A Game Theoretic Approach for Managing Multi-Modal Urban Mobility Systems

A Game Theoretic Approach for Managing Multi-Modal Urban

Mobility Systems Christos Nikolaou

Marina BitsakiAlina PsycharakiGeorge Koutras

Transformation Services LabComputer Science Department

University of Crete

13/01/2015 nikolau @ tsl.gr 1

Table of Contents

• Collective Adaptive Systems

• The ALLOW ENSEMBLES project

• The Strategic Ensemble Concept

• The FlexiBus Scenario

• Game Theory: Non-cooperative and cooperative games

• Future Work - Publications


Collective Adaptive Systems (CAS)

• “Collective adaptive systems consist of many autonomous units that interact in a variety of ways over multiple scales”, https://www.elec.york.ac.uk/research/intSys/cas.html

• Concept similar to: complex adaptive systems, cyber-physical systems, service systems.

• Autonomous units could be (cyber and/or physical) systems and/or humans.

• The Internet of Things (IoT) helps their proliferation.• Collections of autonomous units (networks, hierarchies) are formed, often

with competing interests (for example for use of resources, of services, etc.).

• Various concerns (for example smarter energy use, empowering the patient to improve quality of life, environmentally friendly easy-to-use urban mobility) could be addressed if:– autonomous units (systems and/or humans) of CAS learn to cooperate and/or

compete, negotiate, develop strategies to achieve goals.


https://www.elec.york.ac.uk/research/intSys/cas.html

Examples of Collective Adaptive Systems


Smart Grid Smart Urban Mobility

Smart Health Services

The Urban Mobility Scenario (Buses, FlexiBuses, Carpools, Taxis, Passengers)


Table of Contents








The ALLOW ENSEMBLES project(Funded by FET Proactive, 7FP, EC)

• http://www.allow-ensembles.eu/


Allow Ensembles develops new models, theories and algorithms that can:• Autonomously form large-scale collective adaptive units (ensembles) that can flexibly satisfy arbitrary goals in the real world environments.• Improve the utility of ensembles by adapting the individual cells within an ensemble such that the overall system learns to get better at accomplishing goals.• Make ensembles robust• Evolve ensembles in order to promote beneficial emergent properties and suppress detrimental emergent properties.• Make ensembles secure and protect sensitive data by evolving security policies in unison with ensemble evolution.

http://www.allow-ensembles.eu/

Basic Concepts - Cells

• Cell: a unique identifiable building block representing a concrete functionality in a larger, multicellular system.

• Implementing the functionality may involve interacting with other cells through predefined protocols.

• Each cell is therefore defined in terms of its protocol (exposed process fragments) and behavior (flow).

• Cells can be created either by instantiating cell archetypes1, or by other cells through the process of differentiation.

(from “Terminology”, Internal Document, ALLOW ENSEMBLES consortium).


Basic Concepts – Passenger Trip Booking Cell Archetype Example


(from “Terminology”, Internal Document, ALLOW ENSEMBLES consortium).

Basic Concepts - Entities

• Entity: a physical or virtual organizational unit aggregating a set of cells.

• Cells can either be unique in an entity, or they can be replicated by the entity by instantiating the cell archetype as many times as necessary. However, each cell belongs to exactly one entity.

• Each entity has a context in which it operates, which is accessible by its cells.

• Furthermore, an entity has a set of goals that it attempts to fulfill by initiating or participating in one or more ensembles.

• Entity Examples: Bus Driver, Passenger, Route Manager and FlexiBus Manager.

• (from “Terminology”, Internal Document, ALLOW ENSEMBLES consortium).


Basic Concepts - Ensembles

• Ensemble: a set of cells from different entities collaborating with each other to fulfill some of the goals of the entities.

• Each ensemble is initiated and terminated by an entity, but more than one entities are expected and allowed to join and leave through the ensemble’s lifetime.

• Ensemble Example: The Bus Driver, Route Manager and the various Passenger entities are participating in one Route ensemble

• (from “Terminology”, Internal Document, ALLOW ENSEMBLES consortium).


12

Conceptual Integration of the Project

Entities Interactions

• Entities interact in order to achieve – Individual objectives

– Group objectives

• Interactions may:– Help entities make decisions to achieve goals (gather info, learn, negotiate, optimize,

participate in games, form coaltions, etc.)

– Implement decisions made by entities (make payments, step in a bus, schedule a bus, etc.)

• Interactions result in the creation of – (execution) ensembles in order to fulfill specific goals initiated by the entities

– Strategic ensembles in order to handle decision making and increase entities’ satisfaction expressed in utility terms


Table of Contents








The Strategic Ensembles Model

• A strategic ensemble model uses the entities’ utility-cells

– It is created before the execution ensemble

– It runs in parallel to the execution ensemble

– It affects the operations of the execution ensemble

• The objectives of a strategic ensemble include the following

– Impose constraints according to entity goals and preferences in order to reduce the various choices of entities

– Assign utility to each entity when participating in an ensemble in order to make the optimal choice

– Manage the negotiation among entities


Utility-cells

• Represent utility and game theoretic characteristics of entities

• The utility-cell of an entity has the following functionalities– calculates the utility of an entity when participating in a given ensemble

– communicates with other cells and makes decisions/computes strategies of the entity

– collects data from measurements (resource consumption, satisfaction, costs, delays, prices, …) and passes them to the Evo(lutionary) Knowledge Data Base

– Consults EvoKnowledge (learns) about utility function parameters, external conditions (for example changes in traffic patterns, special events in city, etc.).

– runs optimization algorithms to maximize entities’ utility or improve the performance of ensembles


Strategic Ensemble

Exec-cell

Utility-cell

Utility-cell

Exec-cell

Exec-cell

Exec-cell

Utility-cell

Exec-cell

Exec cell

Utility-cell

Exec-cell

Utility-cell

Entity 2

Entity 1

Entity 3

Entity 4

Entity 5

Entity 6


Passenger

Passengers

Utility-CellUtility-Cell

Cell

Flexibus Manager

Utility-Cell

Example Strategic Ensemble

Route Planner

Cell

Cell

Utility-Cell

Cell

Cell

Route Manager

Utility-Cell

Cell

FlexiBus Driver

rou

teInfo

Interactions

Exec cell

Utility-cellUtility-cell

Exec-cell

Evo-cell

Route Planner

Strategic Ensemble

Passenger

Evo-cell

Prior Art

• An efficient transportation system utilizes mass transit alternatives to the automobile in order to reduce congestion and support ecological solutions.

• Travelers make decisions based on timing, cost, comfort, safety and mode of trips, while planners face policy questions such as frequency of routes, itineraries, size, cost, environmental impact, etc.

• (Lam, Small, 2001): a method to value travel time and its reliability. People had to choose between two parallel routes, one free but congested and the other with time-varying tolls by maximizing a utility function (a function of travel time, variability in travel time, cost, characteristics such as time-of-day and car occupancy, and a random component).

• (Johansson et al., 2003): the labor market commuter behavior is analyzed taking into account the observation that the willingness of an individual to commute is different for short, medium and long time distances.


Prior Art (cont.)

• (Li, Huang, 2005): reliability of morning commuting in congested and uncertain transport networks

• Other studies analyze the interactions between commuters and planners or transport managers and examine how commuters choose their optimal routes and trip modes using non-cooperative games.

• (Sun, Gao, 2007): a non-cooperative, perfect information, static game to describe how travelers adjust their route choices and trip modes.

• (Anas, Berliant, 2010): the authors consider a commuting network consisting of a finite set of nodes at which the commuters live or to which they commute or through which they commute and a finite set of transport links between the nodes (there exists only one mode of transportation). A non-cooperative game is formulated consisting of a set of commuters who compete for routes.

• In our work, we investigate a dual problem facing both the commuters and the transportation authority; the commuters choose their trip mode, while at the same time the transportation company that provides a bus for example, makes decisions on accepting or not travel requests dynamically.


Table of Contents








The FlexiBus Scenario: Assumptions

• A route is a set of predefined pick-up points

• We consider two phases in the lifecycle of a route: – The pre-booking phase: a route is going to be executed if a

certain number of requests is reached until a certain deadline

– The execution phase: the route is bound to start or it has already started

• The pick-up points of a route are bound to change at the execution phase – Add a pick-up point due to a new request

– Remove a pick-up point after a cancellation

Time-line

End of Route

Pre-booking phase

Preparation phaseInitialization

Running phase

Execution Ensemble

Strategy Ensemble

Booking phase

Entity’s Utility

• Entity’s utility– Utility accrued when participating in a specific

ensemble

– Calculated by the entity according to• Her preferences (that are publicly known)

• Private information

• According to the entity’s utility the winning route may be different from the one resulted by the evaluation of the other members of the ensemble (e.g. the route manager).

Entity Utility (Example)

• Consider Peter sending the request (Destination: Piazza Duomo, arrival time: 21.30) to Urban Mobility System with preferences

– Pay with credit card (desired)

– Non-smoking bus (demanded)

– Window seat (desired)

• Consider the following candidate routes

– Route A (at a cost of 10 Euros): non-smoking bus, pay with credit card, window seat

– Route B (at a cost of 12 Euros): non-smoking bus, pay with credit card, aisle seat

– Route C (at a cost of 7 Euros): non-smoking bus, pay with credit card, aisle seat

• Route A has higher value to Peter than Route B but not clear for Route C (it depends on how Peter values money)

Table of Contents








Game theoretic models

• Analyze models that:

– show the strategic interactions among various components

• my decisions affect others’ decisions

– derive equilibria so that utility/profit is maximized

• Nash equilibrium

Nash Equilibrium

• Nash equilibrium is an outcome of a game in which each player is assumed to know the equilibrium strategies of the other players, and no player has anything to gain by changing only his own strategy unilaterally– In a game of two players A and B, the pair of strategies (s, g) is a Nash

equilibrium if s is optimal for A given g and g is optimal for B given s

Nash Equilibrium (Example)

• Consider two players A and B

• Strategies for player A {Top, Left}

• Strategies for player B {Left, Right}

• Nash equilibria: (Top, Left), (Bottom, Right)

B

A

Left Right

Top 2,1 0,0

Bottom 0,0 1,2

Non-cooperative Games

• Static games– all players' decisions are made simultaneously

– players receive payoffs that depend on the actions just chosen

• Dynamic games– each player can consider his plan of action not only at the beginning of the

game but also whenever he has to make a decision

– perfect/imperfect information• At each move the player with the move knows the full history of the play thus

far

• Games of complete/incomplete information– Each player’s payoff function is common knowledge among all players

A Non-cooperative Game of Complete Information

• Objective: model the interactions of entities when a new request arrives in a route that is being executed and compute their optimal choices (strategies) based on utilities

• Problem description– We consider a route to be a set of fixed pick-up points and the

respective estimated travel times between pick-up points– When a new request arrives, the various entities make

decisions:• route planner: accept or reject the request (in case of accept send

conditions (travel time, cost)• new passenger: accept or reject offer

– We formulate these interactions as a game • Are there equilibrium strategies?

Assumptions

• All passengers have the same destination

• Current passengers do not negotiate for the new conditions (resulted by the new passenger) of the route• The route planner has to take into account that any violation

of his past commitments will affect negatively his utility

• We consider a game of complete information• The route planner provides private information of current

passengers to the new passenger

• The utility functions of all actors are common knowledge

Assumptions

• We consider a dynamic game

• Decision points: request arrivals • new strategies have to de derived each time a new request arrives

(thus a new game is formulated)

• All these games across the route have to be synchronized in order to derive optimal profits for the FlexiBus company and the passengers

• Each game is sequential

• The new passenger makes a request

• The route planner calculates his strategy (accept or reject) based on request

• The new passenger calculates his strategy based on planner’s strategy

Game Formulation

• Set of players: – current passengers (1,…,n)

– new passenger (np)

– route planner (rp)

• Strategy profile:

(ti : travel time)

ti , i=1,…n are fixed and known

𝑠 = (𝑡1, 𝑡2,… , 𝑡𝑛 , 𝑡𝑛𝑝 , 𝑡𝑟𝑝 )

Game Formulation

Profits:

– Current passengers:

– New passenger:

G: the profit gained by alternative solutionf: function that incorporates the risk of adding future passengers

– Route Planner:

g: function that incorporates the risk of losing future passengers

𝑝 = (𝑝1,𝑝2,… ,𝑝𝑛 ,𝑝𝑛𝑝 ,𝑝𝑟𝑝 )

𝑝𝑖 = 0, 𝑡𝑖 < 𝑡𝑟𝑝

𝑢𝑖 𝑡𝑟𝑝 , 𝑡𝑖 ≥ 𝑡𝑟𝑝

𝑝𝑛𝑝 = 𝐺, 𝑡𝑛𝑝 < 𝑡𝑟𝑝

𝑢𝑛𝑝 𝑡𝑟𝑝 − 𝑓(𝑡𝑛𝑝 ), 𝑡𝑛𝑝 ≥ 𝑡𝑟𝑝

𝑝𝑟𝑝 = 𝑢𝑟𝑝 𝑡𝑟𝑝 + 𝑝𝑖𝑖≠𝑟𝑝

− 𝑔(𝑡𝑟𝑝 )

37

Set of

Processes

Subsystem architecture

Process/Flow Engine

Utility Module

Set of

Processes

services.tsl.gr

Set of

Processes

Fragments

(Parts of

Meta-cells)

Set of

ProcessesSet of

ProcessesSet of

Services

Design Time

Run Time

Entity Hierarchies

• Entities can form hierarchies (of entities):– For example: people and systems participate in a

department entity, which in turn may participate in a division entity, which in turn may participate in a corporation entity.

• Utility of higher-level entity could be defined as the sum of the utilities of the entities one level below (and so on recursively),

• But, optimal utility of a higher-level entity is NOT necessarily the sum of the optimal values of the utilities of the immediate lower-level entities.

• … a number of interesting research questions…


Role of utility in Evolution

• Utility function with incorrect (or partially correct) parameters

– may result in wrong selection of ensembles

– e.g., use of incorrect travel duration etc.

Execution

Context

EvoKnowledge

Monitor Utility

Evolve

Utility function selects the proposedsolution with highest utility takinginto account goal, context and userpreferences (measured utility)

1

Ensemble with high utility is executed

2

Parameters of utility function areadjusted according to past executions andthe calculated utility and monitored utilityin a given context

3

Utility Functions

Specialization tree

Goal

……... specialization ………


Ensemble N

Utility Functions


Example

Goal: Reach a destination

Car sharing

Non-Public Transportation

Public Transportation

Rent car

U1 (t,c)

U2 (t,c,r,d)

t: travel timec: costr: reliabilityd: walking distance

Example (in Collaboration with IPVS)

t: travel timec: costr: reliabilityd: walking distance

Goal: Reach a destination

Non-Public Transportation

Public Transportation

Train Flexibus

U1 (t,c)

U2(t,c, r,d)

Cooperative Games

• In cooperative game theory, interest is on outcomes of coalitions of players rather than actions of individual players – focus in cooperation games is on coalitions that will be formed and on

the sharing of value or cost incurred among members of the coalition.

• Cost allocation problem in which players perform a joint task and allocate its cost among them

• Why use cooperative games– Helpful tool if performance of an intelligent system and its entities can

be improved when several players cooperate

• Dynamic cooperative game (??)• Implementation issues:

– get information on game parameters and data from EvoKnowledge (?),– test game and obtain results in ALLOW Ensembles platform, ….


The General Idea of an Algorithm for Cooperative Games

• Game (𝑁, 𝑐)• Specify set 𝑁 = {1, … , 𝑛} of potential users involved (𝑛 = |𝑁|).• Specify cost function 𝑐: 2𝑁 → ℝ of a route, where 𝑐(𝑆) is the joint cost of

the route used by the set 𝑆 ⊆ 𝑁 of users (𝑐 ∅ = 0).

• A required property for 𝒄:• Subadditive cost: For every two disjoint sets of users the cost of the route

if they merge is smaller than or equal to the costs of the route they would incur if used separately (𝑐 𝑆1 ∪ 𝑆2 ≤ 𝑐 𝑆1 + 𝑐 𝑆2 , 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑆1, 𝑆2 ⊂ 𝑁).

• In a subadditive game: 𝑐(𝑁) ≤ 𝑖𝜖𝑁 𝑐( 𝑖 ). If this condition holds with strict inequality then the game is called essential (each player gains from cooperation).

• If subadditivity property does not hold, coalitions other than the grand coalition might be possible.

• Cost should reflect:– Services required by the operator, Infrastructure costs, Payments, …


Algorithm (cont.)

• Allocate 𝑐(𝑁) among the users in 𝑁.• Specify an allocation rule 𝑥: 𝑥 𝑐 =𝑥1, … , 𝑥𝑛 𝑠. 𝑡. 𝑖𝜖𝑁 𝑥𝑖 = 𝑐(𝑁) (such as Shapley

value for fairness or core for stability). • 𝑥 is an imputation of game (𝑁, 𝑐) if the following

hold:– 𝑖𝜖𝑁 𝑥𝑖 = 𝑐 𝑁 1 feasibility of the grand coalition

(costs are reimbursed) and– Pareto efficiency: 𝑥𝑖 ≤ 𝑐 𝑖 ∀𝑖𝜖𝑁 2 no player pays

a higher price in the grand coalition than he would do independently


Table of Contents








Future Work

• In the case of one route– Permit actions such as adding pick-up points in order

to serve new passenger requests or removing pick-up points in case of cancelations

– Perform negotiations with current passengers when a new request arrives

• Consider multiple routes for one request as part of the decision mechanism

• Design routes according to a decision making mechanism that takes into account traffic demand

Some recent related publications

• “A Game Theoretic Approach for Managing Multi-Modal Urban Mobility Systems”, Vasilios Andrikopoulos, Marina Bitsaki, Antonio Bucchiarone, Santiago Gómez Sáez, Dimka Karastoyanova, Frank Leymann, Christos Nikolaou, Marco Pistore, 2th International Conference on the Human Side of Service Engineering Human Factors and Ergonomics, 2014/7/19, Kraków, Poland: CRC Press/Taylor & Francis

• "Utility-based Decision Making in Collective Adaptive Systems.“, Proceedings of the 4th International Conference on Cloud Computing and Services Science (CLOSER’14), Andrikopoulos Vasilios, Marina Bitsaki, Santiago Gómez Sáez, Dimka Karastoyanova, Christos Nikolaou, and Alina Psycharaki. (2014).

• “Towards Modelling and Execution of Collective Adaptive Systems”. A. V. Andrikopoulos, A. Bucchiarone, S. Gomez Saez, D. Karastoyanova, and C. Antares Mezzina. 9th International Workshop on Engineering Service-Oriented Applications (WESOA 2013), In conjunction with ICSOC 2013, December 2nd 2013, Berlin, Germany.


A Game Theoretic Approach for Managing Multi-Modal Urban Mobility Systems

Technology

ensembles secure

ensembles projectfunded

utility of ensembles

complex adaptive systems

service systems

cyberphysical systems

smart health http

smart cities http