Top Banner
The use of agents in the control of fixed and radio networks John Bigham Department of Electronic Engineering Queen Mary, University of London email: [email protected]
72
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: jb.ppt

The use of agents in the control of fixed and radio networks

John Bigham

Department of Electronic Engineering

Queen Mary, University of London

email: [email protected]

Page 2: jb.ppt

Structure of talk Look at examples of the application of

agent technology to resource management in telecoms networks ATM networks AMPS networks 3G networks

Show some common threads architecture relation between autonomy &

communication suggest where we need to go further

Page 3: jb.ppt

What are agents ? - Definition used here:

Distributed software that exhibits features of: Autonomy: the ability to have control over its own

actions and states. An agent is able to make decisions and complete actions based on its internal representation of the world without direct intervention.

Social Ability: the ability to interact with other agents via some kind of communication language in a co-ordinated manner. Agents may co-operate or compete.

Reactivity: the ability to perceive changes in the environment and react timely and appropriately.

All of this depends on the code!

Page 4: jb.ppt

Types of agents

Intelligent Agents “Intelligent” pieces of software that communicate

through a standardised message-passing language (Agent Communication Language)

Like traditional signalling Mobile Agents

Agents that move – i.e. code that moves More reluctance to use mobile agents because of

the risk of viruses and rogue agents This talk will restrict itself to intelligent agents

controlling networks

Page 5: jb.ppt

For the real time systems described

All have to make some kind of decision in real time.

All need to be ready to make this decision All agents we have built can be viewed as

consisting of Reactive and Planning layers This applies to

user agents, resource management agents, negotiation agents….

5

Page 6: jb.ppt

Reactive and planning layers Reactive layer copes with immediate problems:

Equivalent to control plane actions Needs to be fast acting Scope for decision making limited View of environment limited

Otherwise large communications overhead Planning layers

Equivalent to management plane actions Aims to balance resources over a longer timescale

Works in the background Successful planning minimises need for reactive action and

reduces communication overheads But cannot be 100% correct

Can have multiple reactive layers Can have multiple planning layers

The boundary is grey

6

Page 7: jb.ppt

Kinds of Agent The different kinds of agent are determined by

their roles establishing an SLA, managing the radio

resource their ownership

e.g. SP or NP their location

e.g. at a RNC, at a point of presence Some roles are closely related to business structure

So agent logic depends on the business model used Some roles are related to the details of the resource

management So agent logic depends on the physical model

Different if ATM, IP, radio

Page 8: jb.ppt

Example: Controlling a fixed network

Intelligent resource management in ATM networks using a society of interacting agents – Impact project.

real time control of a real physical network Implemented on test bed at TeleDanmark in Århus Concentrated on setting up and charging for

connections Management of ATM was an issue at the time

much applicable to IP About using agents in a telecommunications

environment, not about research into agents per se.

8

Page 9: jb.ppt

Context : Value chain

customer user

Service provider

Network provider

Transport service

Higher-layer services$

Content provider

$

$

$ Value Value chainchain

Content can be Content can be paid for directly paid for directly

or via SPor via SP

Page 10: jb.ppt

Business model

Customer

NP 2

SP1

$

SP2 SP3 SPM

$$$

•Customer can access any Service Provider (SP)Customer can access any Service Provider (SP)

$

Page 11: jb.ppt

Modelling the business layers allows…

Service providers: to relate offer price to spot costs of bandwidth to build into the offer, information related to the

customer: tariff profile & volume discount Customers

to consider future use when accepting offer: e.g. if one SP offers a better volume discount, it

might be better to use that even if current price is higher.

11

Page 12: jb.ppt

Transport network of a service provider

Consist of bandwidth reservations through a core network

Consist of bandwidth reservations through a core network

RA

RA communicates with SPA to change resources for its source-

destination pair

Service provider agent communicates with NPA to change resources available

CA

PUA

Connection Agent gets bids from RAs for source-destination pair

Proxy User Agent requests connection

SPA

Control of ATM network

Architecture

Page 13: jb.ppt

Connection set-up PUA requests connection with parameters CA asks RAs for bid on that request RAs respond:

OK Wait while capacity is transferred Wait while connections are transferred NO

CA chooses (on some criterion) and connection notifies winning RA

Connection is set up. This is all at the reactive layer - FAST

These options do not exist with conventional

signalling

Page 14: jb.ppt

PC2

PC5

PC1PC4

PC3

42000

25000

44000

24000

Physical link

S-D pair 4-2 (3 VPs)

S-D pair 5-2

Example of the relaxation in the reactive layers

Free

Used

Page 15: jb.ppt

Biggestcapacity PC2

PC5

PC1PC4

PC3

25000

44000

24000

24000

20000

42000

First connection PC4-PC2 goes to VPwith the biggest capacity

Free capacity reduced

Allocation of connection to VPsmaximise minimum residualNow the biggest capacity

Page 16: jb.ppt

PC2

PC5

PC1PC4

PC3

22000

20000

22000

20000

24000

4000

20000

Total spare capacity 26000

250005000

20000

This is the load scenario that is used for the demonstrations

Adding more gives:

Page 17: jb.ppt

Now we wish to add another connection from PC5 to PC2 of 20,000 units

There is not enough capacity With no capacity transfer we get

rejection Capacity transfer between the two VPs

on the route between PC5 and PC2 allows the connection to be made.

Page 18: jb.ppt

PC2

PC5

PC1PC4

PC3

22000

20000

22000

20000

24000

4000

20000

Slow connection - allows capacity transfer

Total spare capacity 26000

5000

20000

Connection request occurs

Not enough capacity - connection blocked

But there is sufficient overall capacity

Page 19: jb.ppt

PC2

PC5

PC1PC4

PC3

Re-routing allows another connection

Not enough capacity for another connection PC5-PC2Re-route “blue” connectionNow capacity for the other connection

6000

20000

5000

20000

20000

24000

0

40000

4000

40000

Page 20: jb.ppt

But …. planning carried on the background might have been re-adjusting the capacities to avoid this reactive layer action Indeed it does Could demonstrate this……….

Page 21: jb.ppt

Architecture

Connection Allocation

SP Capacity Transfer

SP Connection Transfer

SP Tactical Planning

SP Strategic Planning

NP Strategic Planning

purely reactive

purely planning

capacity cannot be readily transferred

capacity is not immediately available

call allocation policies for RA

e.g. Selfish or Cooperative

Queen Mary – EPSRC support

Done within IMPACT

Can enrich the reactive layer by allowing time for optimisation

planning

Page 22: jb.ppt

Managing mobile networks

Initially on AMPS networks – more work in the literature there.

fixed set of frequencies, fixed shapes Extended to 3G Problem with mobile is that unlike fixed networks

the resources at access point (the air interface) are severely limited

More difficult problem Objectives

Optimise network capacity Maintain network coverage Guarantee Quality of Service

Page 23: jb.ppt

Conventional channel assignment : spacing out the frequencies

Channel Allocation Schemes Fixed Channel Assignment -

FCA Modified FCA schemes

channel borrowing with/without locking

load sharing Dynamic Channel Allocation

Centralised versions provide better results as they have a wider view, but:

Scalability problems Lack of robustness Heavy signalling load Not flexible

A

A

A

A

A

A

A

Page 24: jb.ppt

Current schemes are reactive

Reactive algorithms reach a point of saturation Signalling load becomes enormous Resources are there but cannot be

reached No real flexibility Investigated negotiation and co-operation

with other BSs to improve the efficiency of channel allocation.

Page 25: jb.ppt

Agent manages a cluster: Co-operation Strategy

B’

B’

B’

B’

A

A’

A’

A’

A’

A’

B’

B’

A’

Hot spot in one cluster For capacity, agent

managing cluster wants to negotiate moving frequency from neighbour

Into cells which will interfere with neighbouring cluster

Removing frequency from neighbour reduces its capacity

Negotiate with other clusters to see which can be borrowed with least effect Clusters can refuse Charge a price

Page 26: jb.ppt

Internal Architecture

Reactive Layer Responsible for fast response to incoming

traffic Implements a channel allocation algorithm

Local Planning Layer Responsible for channel re-assignment Takes into account efficiency of channel

usage Monitors efficiency of reactive layer

Co-operative Planning Layer Load balancing implemented by agent co-

operation Utility function to choose best region Market-based-control for management

handoffs

Co-operative planninglayer

Local planninglayer

Reactivelayer

Page 27: jb.ppt

Results

LoadLoad

Blocking probability)Blocking probability)

Stylised formStylised form

FCA

reactive layer

cooperative layer invoked

In this region, planning layer copes and co-operative layer is not invoked.

Page 28: jb.ppt

Challenges in 3G

New types of service Increasing numbers of

Service Providers Any-to-any marketplace Radio access technology

--- WCDMA all cells (of same type)

use same frequency interference is a

limiting factor needs accurate power

control cell breathing

Resource management is more complicated than in FDMA/TDMA systems

Work described has grown out of the SHUFFLE IST Project applying agent

technology to controlling 3G

this approach is sketched

Page 29: jb.ppt

Agents in SHUFFLE architecture lie in two kinds of plane

Negotiation plane holds agents concerned with making agreements between SPs & NPs on SLAs Primarily related to the business model

Each resource plane holds agents concerned with resource network management to meet the SLAs for each NP

UA

SPNA

NPNA1

NPRA1 NPRA2Handover & cell selection

QoS ManagementRadio Resource Management

NPNA2

Negotiation planeNPNA1 NPNA2

NPA1

NPA2

Negotiation plane

Resource planes

SPA

SPNA

Resource Agents

plane=agents with related roles

Page 30: jb.ppt

Three main groups of actors, each represented by agents:

Users

• Subscribe to service providers

• Make requests for connections

Service Providers• Brokers

• Offer services such as traffic information, videoconferencing

Network Operators• Own and control the physical networks

• Carry the connections requested by users

Page 31: jb.ppt

Connection Requests in a Free Market

UA

SPRA1

SPRA2

NPRAa

NPRAb

NPRAc

1. User makes a connection request

2. User’s UA contacts a SPRA to make the request

3. SPRA chooses a NPRA and relays the request

4. NPRA agrees to admit connection and informs SPRA

5. SPRA informs UA to which NPRA to connect

6. UA contacts NPRA - connection is set up

Use reputation

reinforcement learning

Page 32: jb.ppt

Layered control structure of resource agent

Co-operative planning layer Change antenna

radiation pattern Local planning layer

Prioritise Forcibly change QoS

Reactive layer Assignment to a Node

B Connection Admission

Control Possible change in

QoS

Local Planning Layer – acts on changing QOS (policy) within cell

Reactive Layer FAST

po

licy

po

licy

po

licy

assignment

OK?

set up connection

Exceptionhandler

reject

Sta

tus

rep

ort

ing

Yes

No

Modified request

No

YesCACOK?

Co-operative Planning Layer – changes radiation patterns between cells to meet wider goals

Request

Sta

tus r

ep

ort

ing

Sta

tus r

ep

ort

ing

Assignedrequest

Page 33: jb.ppt

Assignment: A hot spot occurs

Page 34: jb.ppt

Assignment Cost =f(distance from base station, power in direction of mobile)+ g(interference in base station if connected).

+ h(impact on non-assigned base stations if connected). ’

Assignment

lightly loaded

moderately loadedhighly loaded

Bid by each base station

Page 35: jb.ppt

Both normal and CBB graphs did not utilise social blocking. Theta is made to vary between 0.2 and above with no upper limit for CBB. Kuri’s prediction is calculated as 0.7918.

Page 36: jb.ppt

Simple Planner

Measure SIR statistics in a cell If minimum QoS threshold is not reached,

increase interference penalty for cell weight in g(interference).

requests neighbour cells to decrease their interference penalties (their ) if they want to

this tends to make new connections go to neighbours rather than overloaded cell

If maximum QoS threshold is exceeded, decrease interference penalty request neighbour cells to increase their interference

penalties MODIFYING POLICY AT THE LOWER LAYER

Page 37: jb.ppt

Other actions of the planner:if radio resources become scarce what are sensible resource plane actions ?

To meet SLA we can Prioritise calls ……. Reduce QoS ……..

All local planning actions Change radio configuration

Requires more cooperation

Page 38: jb.ppt

Behaviour of a conventional 3G network

Page 39: jb.ppt

Conventional 3G network

Cell structure blind to Call Traffic Changing

Low Flexibility

Page 40: jb.ppt

Dynamic reconfiguration

Page 41: jb.ppt

Dynamic reconfiguration

Page 42: jb.ppt

Dynamic reconfiguration

Reactive to Call Traffic Changing

Higher Flexibility Higher Capacity Low additional

cost Can use

Distributed Control

Page 43: jb.ppt

configuration achieved using

smart antenna technology + coordination

The basic idea of a smart antenna is to use antenna patterns that are not fixed but adapt to current radio conditions.

Smart antennas are usually categorised as either fully adaptive or switched-beam

Page 44: jb.ppt

Fully adaptive smart antenna

i n te r f e r e r

d e s i r e d u s e rVisualised as pointing a narrow beam in the direction of the wanted mobile and nulls in the directions of interferers

The complexity and cost of the adaptive beam former is still seen as a major disadvantage.

Page 45: jb.ppt

simple beam formers are used to make sector patterns

Do not require a fully adaptive antenna. Instead employ a fairly simple beam-former

The cooperative ‘intelligence’ is provided by the the resource agents.

Multiple beams from a sector combined to provide uniform and shaped coverage

Page 46: jb.ppt

A coverage snapshot achieved through cooperation between cells

reduced coverage

increased coverage

Scenario 1

Page 47: jb.ppt

Simulation ResultsScenario 2

Page 48: jb.ppt

Relative merits of AgentTel and fully adaptive smart antennas

Higher potential gain (X2 instead of X1.3)

Less complex and

cheaperSimpler – can be

mounted on the top of the

tower or uses fewer

connectors main points of failure

Beam forming easier

patented approach

Fully adaptiveAgentTel

Page 49: jb.ppt

How can it be achieved physically?

Tapered amplitude illumination provides

vertical pattern

Individual element excitation is fixed by a network within the antenna so only one RF input signal is required.

RF in

The simple linear array used in conventional base stations

Page 50: jb.ppt

Basic smart antenna

Amp&

Phasecontrol

Amp&

Phasecontrol

Amp&

Phasecontrol

Amp&

Phasecontrol

RF drive signal

A1 P1

A2 P2

A3 P3

A4 P4

Array of conventional base-station antennas gives the basis of a smart antenna

Page 51: jb.ppt

RESULTS:Scenario Descriptions: Scenario 2 (for example)

3000 traffic units in the whole area. 100 base stations, each has the capacity

to serve 36 traffic units 20% of traffic is uniformly distributed in

the whole area. Other 80% of traffic is distributed in 40

hot spots with normal distribution

Page 52: jb.ppt

Simulation Results

Generation

Sys

tem

Uplin

kC

apaci

ty

0 250 500 750 10001700

1800

1900

2000

2100

2200

2300

2400

2500

2600

2700

2800

Fixed (circular) pattern

First method (2.8)

Second method (2.9)

Scenario 1Scenario 2

shows benefit from shaping can be > 20%

faster convergence

slow

convergence

Page 53: jb.ppt

Allocation by service type possible

Gold

Silver

Bronze

Users Served:12% Gold17% Silver35% Bronze

Users Served:23% Gold29% Silver6% Bronze

Time

Page 54: jb.ppt

Real time change: base stations agree patterns by negotiation

Message type• Request for help (RFH)• Request for pattern (RFP)• Return from antenna agent (RFA)– Return from base station agent (RFB) • Request for commitment (RFC)• Acknowledgment of commitment (AOC)• Scheme expires (SE)• Cancel Transaction (CT)• Change Your Coverage (CYC)• Change Your Pattern (CYP)

Ask

Respond

Ask for commitment

Respond with

commitment

Perform change in pattern

Page 55: jb.ppt

Negotiation Process34 35 36

44 45 46 47

65 66

54 55 56

BS45 BS55BS46 BS66Ant45RFP(1,2,3,4,5)

RFA(3,4)

RFH(4)

RFH(3)

RFA(322)

Ant46 Ant55 Ant66

RFP(41,42,43)

RFA(41)

RFP(31,32,33)

RFA(32)

RFH(32)

RFP(321,322,323)

RFA(323)

RFB(323, price323)

RFA(33)

RFA(31)RFB(41, price41)

RFA(43)

RFA(42)

RFA(5)

RFA(321)RFB(32, price32+price323)

RFC(41)

AOC(41)

CYC(41)

CYP(4)

RFA(5)

CYP(41)

Ask

Respond

Ask for commitment

Respond with

commitment

Perform change in pattern

Page 56: jb.ppt

A RFH issued in two cases

When U>T, i.e. the observed utilisation U at a base station exceeds a chosen threshold T, or

during hypothetical reasoning in response to another RFH when helping would cause the utilisation threshold at the helping base station also to be exceeded so further requests have to be made. There is a HTK

Triggering on U>T could be formulated in terms of an interference constraint But not too bad a model if assume perfect power

control

Page 57: jb.ppt

If U>T then a reduction in coverage area is required

New hypothesised coverage shapes are generated from the current shape by removing locations.

Candidate locations for removal are the locations on the boundary of the

existing shape, and the next levels in These locations, are given a value

with respect to a base station that is a weighted sum of two selfish criteria and one social criterion

location

Page 58: jb.ppt

The value of a location to a BS is……

proportional to the profit at the location inversely proportional to a power of the distance. The capability of neighbouring base stations to help.

here the utilisation of the nearest immediately neighbouring base station.

Depends on the type of user…….. Also used to find the price for helping

' ( ( ))( )

( ) ( )z

kk U NN i

d iV i P i

Page 59: jb.ppt

Generating hypotheses

Locations are repeatedly removed, in a “randomised” way, until the threshold T is no longer exceeded.

This is repeated 5-10 times in the current implementation

to generate hypothesised patterns  LOTS of detail missing!

Key aspects are the hypothesis generation Value functions Need to reason with patterns the physical system

can generate

Page 60: jb.ppt

Rotating antenna test tower

Not just simulation

Fixed transmitantenna

Antennapattern

measurementreceiver

stop simulator and connect a selected base station to real antenna is pattern generate what we expected!

Alan Dick & Co outdoor range

Page 61: jb.ppt

Summary Tried to show that distributed resource

management is feasible and effective mechanism for resource management in fixed and radio networks.

Shown three examples where Performance is improved Increased flexibility is created

Design is not a monolithic hierarchy, rather coordinated hierarchical pillars of control

Coordination between pillar layers increases a planning becomes more dominant

Page 62: jb.ppt

Each layer helps to find more resource

LoadLoad

Blocking probabilityBlocking probability

Localised reactive algorithms

Local planning layer

cooperative layer

In this region, planning layer copes and co-operative layer is not invoked.

global optimisation

reactive layer

Page 63: jb.ppt

Each layer creates components of policy applied in lower layers

Cooperative

Local planning

Reactive

time for response

policy prescribed & uses little knowledge of neighbours

creates policy of lower layer + asks neighbours to modify their policy (voluntary)

e.g. maxmin/fragmentation

e.g. CBB + reservation

try to reach mutually advantageous agreements on policy of lower layers (policy +meta policy)

SLAs & other business drivers

create aspect of policy

autonomy

amount of inter component communication

Page 64: jb.ppt

Is this very different from distributed control?

No much of it is similar to hierarchical and distributed

control Of course there are a variety of approaches for this

also, e.g. subsumption architecture Even ideas like reputation, tit for tat, tit-for-tat with

forgiveness,…… have close analogues in reinforcement learning ewma

So a large part of what is often called agent technology is not really different But still extremely useful!

Page 65: jb.ppt

Higher layers

However agent work has extended modelling to higher layers to include social interactions In competitive as well as cooperative environments

In competitive models some people argue against optimisation of an agent’s resource, but for finding e.g. Nash Equilibria A game theory approach Some negotiation mechanisms try to find equilibria

Others argue that optimisation is still relevant, but we need to use recursive modelling E.g. a NP should model a SP’s behaviour in order

for the NP to make optimal resource allocationsAnd vice versa

Page 66: jb.ppt

Example:Negotiating SLAs (including micro SLAs)

SPRA1

SPRA2

NPRAa

NPRAb

NPRAc

SPRA chooses a NPRA

SPNA1

SPNA2

NPNAa

NPNAb

NPNAc

Model each other and use these models in the individual optimisations laurissa

Chose not to model the NPRAs, just to select “optimal” – but reward and punish over time

n.b.

Page 67: jb.ppt

Does the agent metaphor give advantages?

Yes It works down through all the levels of the

business Including a competitive environment Supports a distributed mix of reactive and

planning elements Allows the modeller to place their components

into context An increasing number of tools available

But it is only a very general idea

Page 68: jb.ppt

other technologies beginning to offering similar infrastructure capabilities

and these are becoming ubiquitous Some based on client server and peer to

peer ideas developed for the internet XML, SOAP messages….ACL JavaSpaces ….. Distributed BB systems Servers, EJB … “business” agents Jini……

In fact agent systems may well be built on this technology, rather than on “agent platforms”

Page 69: jb.ppt

But people working with these technologies will face the same problems that researchers in agent technologies have been looking at

So concepts supporting communication when there is not a global

unifying ontology adjustable autonomy negotiation strategies property/concept/code re-use when

functionality has to change …….

Are key in making the system work

Page 70: jb.ppt

Can be implemented/installed bottom up

Start of with the reactive layer

Add local planning functionality

Add cooperation and integration with the business layer

Plan to build a real system at Athen’s new airportSLAs for high QoS customers (i.e. high profit) customers

Tracking these customers around the airportEnsuring emergency and priority users are served

Without starvation to all others– which can itself cause panic

It is becoming real!

Page 71: jb.ppt

But just as we feel we are understanding one area…..

new pressing requirements include coping with: Flexible business models and rapid provision of

new services Operator mistakes Fraud a daily reality

Malicious attacks from insiders and outsiders Increased vulnerability as moves to IP

Voice->IP , high bw unsecure connections-> DOS Need to manage interactions between

“safeguard” agents managing the management layers and resource agents managing the resource layers a very difficult problem…… help appreciated!

Page 72: jb.ppt