V-Formation as Optimal Control - Salk Institute for ...navlakha/BDA2016/Material/... · V-Formation as Optimal Control Ashish Tiwari SRI International, Menlo Park, CA, USA BDA, July

V-FormationasOptimalControl

AshishTiwari

SRIInternational,MenloPark,CA,USA

BDA,July25th,2016

JointworkwithJunxingYang,RaduGrosu,andScottA.Smolka

Outline

• Introduction

• TheV-FormationProblem

• ModelPredictiveControlforV-Formation

• ExperimentalResults

• Conclusions&FutureWork

Outline

• Introduction

• TheV-FormationProblem

• ModelPredictiveControlforV-Formation

• ExperimentalResults

• Conclusions&FutureWork

V-Formation

• FlocksofbirdsorganizethemselvesintoV-formations

Eurasian Cranes migrating in a V-formation (HamidHajihusseini, Wikipedia)

Reason:Savesenergyasbirdsbenefit fromupwashregion;providesclearvisualfieldwithvisibilityoflateralneighbors

ReachingaV-Formation

• Rule-basedApproach:ØCombinationsofdynamicalflightrules asdrivingforcesØNotcompletelysatisfying

• ViewasaDistributedControlProblem:ØFlockwantstogetintoanoptimalconfigurationthatprovidesbestview,energybenefit,andstability

• OurApproach:ØUsesModel-Predictive Control(MPC)ØWhichusesParticle-SwarmOptimization(PSO)

Reynolds’Rules

Reynolds (1987)presentedthreerulesforgeneratingV-formations:

Alignment Cohesion SeparationAlignment:steertowardstheaverageheadingoflocalflockmatesCohesion:steertomovetowardtheaveragepositionoflocalflockmatesSeparation:steertoavoidcrowdinglocalflockmates

ExtendedReynoldsModel

Reynolds’modelwasextendedbyadditionalrules:• Arulethatforcesabirdtomovelaterallyawayfromanybirdthatblocksitsview(Flake(1998)).

• Dragreductionrule:computingthe induceddraggradientandsteeringalongthisgradient(Dimock&Selig(2003)).

Nathan&Barbosa’smodel(2008):• Coalescing:seekproximityofnearestbird• Gap-seeking:seeknearestpositionaffordingclearview• Stationingrule:movetoupwashofaleadingbird

ARule-basedAttempt

DesignedrulesthatgenerateaV-formation• Drivebirdstowardstheoptimalupwashpositionw.r.t.thenearestbirdinfront;unsatisfactory solution

Outline

• Introduction

• TheV-FormationProblem

• ModelPredictiveControlforV-Formation

• ExperimentalResults

• Conclusions&FutureWork

TheV-FormationProblem

Assumeageneric2-ddynamicmodelofaflockof

birds

xi(t+1)=xi(t)+vi(t+1)

vi(t+1)=vi(t)+ai(t)

Goal:findbestaccelerationsai(t) ateachtimestep

thatwillfinallyleadtoaV-formation.

Thisisadistributedcontrolproblem

WhatisaV-Formation?

Wewantaformationthatachievestheoptimumvaluesforthefollowingthreefitnessmetrics:1. VelocityMatching2. ClearView3. UpwashBenefit

VelocityMatching(VM)

s=stateofthen-birds=npositions,nvelocitiesVM(s)=normalizedsumofpairwisevelocitydifferenceVM(s)=0ifallbirdshavethesamevelocityVM(s)increasesasthevelocitiesgetmoremismatched

VMisminimizedwhenallbirdshaveequalvelocity.

Velocitynotmatched Velocitymatched

ClearView(CV)

• Accumulatethepercentageofthebird’sviewthatisblocked• CV(s)=0ifeverybirdhasa100%clearview• CV(s)increasesasmoreoftheviewofanybirdisblocked

(b) i’s view is completelyblocked by j and k.Clear view: 1

UpwashBenefit(UB)

• AGaussian-likemodelofupwash anddownwash• UB(s)=sumofupwash benefiteachbirdgetsfromeveryother• UB(s)=1ifn-1birdsgetsmaxpossibleUBbenefit• UB(s)increasesasbirdsgetlesserupwash benefit

FitnessFunction

Fitnessofastateisasum-of-squares

combinationofVM,CVandUB

F(s)=(VM(s)-VM(s*))2+(CV(s)-CV(s*))2+(UB(s)-UB(s*))2

• stateachieving optimal fitnessvalue (i.e.,aV-

formation)

TheV-FormationProblem

Assumeageneric2-ddynamicmodelofaflockof

birds

xi(t+1)=xi(t)+vi(t+1)

vi(t+1)=vi(t)+ai(t)

Goal:findbestaccelerationsai(t) ateachtimestep

thatwillfinallyleadtoastatewithminimumF(s)

Thisisadistributedcontrolproblem

Outline

• Introduction

• TheV-FormationProblem

• ModelPredictiveControlforV-Formation

• ExperimentalResults

• Conclusions&FutureWork

ModelPredictiveControl(1)

Ateachtimet,considerhowthemodelwillbehaveinthe

nextTstepsunderdifferentchoicesforthecontrolinputs

• Useamodel thatrepresentsthebehavioroftheplant

Useanoptimizationsolvertofindthebestcontrolinputs

overthisfinitepredictionhorizon

Onlyapplythefirstoptimalcontrolaction

Repeatatt+1

ModelPredictiveControl(2)

• Attimet+1,updatemodelstatewithnewmeasurementsoftheplant.• Repeattheoptimizationwithnewstates.

AdiscreteMPCscheme(Wikipedia):horizon=p,currenttime=k

ModelPredictiveControlforV-Formation(1)

Birdi attimetsolvesthefollowingoptimizationproblem:

a*i(t),…,a*i(t+T-1)=argmin ai(t),…,ai(t+T-1)F(sNi(t+T-1))

• sNi(t) :stateattimet consistingofpositionsandvelocitiesofbird’sneighbors

• Centralized controlifNi includesallbirds• F:fitness function.• T:predictionhorizon.

ModelPredictiveControlforV-Formation(2)

• Subjecttoconstraints:• Modeldynamics:Stateupdatesofeachbirdare

governedbythemodeldynamics

• Boundedvelocitiesandaccelerations:Thevelocitiesareupper-bounded byaconstant,andtheaccelerationsare

upper-bounded byafactorofthevelocities

• Finally,birdi usestheoptimalaccelerationforbird

itfoundfortime.

ParticleSwarmOptimization(1)

TheoptimizationproblemissolvedusingPSO• Inspiredbysocialbehaviorofbirdflockingorfishschooling.

• Initializeapopulation(swarm)ofcandidatesolutions(particles)thatmovearoundinthesearch-space.

• Eachparticlekeepstrackofthebestsolutionithasachievedsofar(pbest)andthebestsolutionobtainedsofarbyanyparticleintheneighborsoftheparticle(gbest).

ParticleSwarmOptimization(2)

• Repeatedlyupdatetheparticle’svelocityandpositionby:vi(t+1)=wvi(t)+c1 r1 (pbesti– xi(t))+c2 r2 (gbesti – xi(t))

xi(t+1)=xi(t)+vi(t+1)

where

w:inertiaweightr1,r2 :randomnumbersin(0,1)sampledeveryiterationc1,c2 :constantlearningfactors

• Terminatewhenmaximumiterationsordesiredfitnesscriteriaisattained.

DistributedMPCProcedure

Ateverytimestep:• EachbirdlooksatitsneighborsØPlaysseveralscenariosinitsheadtofindthebestconfiguration thattheneighborhoodcanreachin3 steps

ØThebirdthenappliesthefirstmove ofthatsolutiontoupdateitsposition

Inthenexttimestep,eachbirdupdatesitsknowledgeoftheneighbors (positionsandvelocities),whichmaynotbethesameofwhatthatbirdpredictedforitsneighbors

Outline

• Introduction

• TheV-FormationProblem

• ModelPredictiveControlforV-Formation

• ExperimentalResults

• Conclusions&FutureWork

ExperimentalResults(1)

ExperimentalResults(2)

Outline

• Introduction

• TheV-FormationProblem

• ModelPredictiveControlforV-Formation

• ExperimentalResults

• Conclusions&FutureWork

Conclusions

• Usedistributedcontrolinsteadofbehavioralrulesto

achieveV-formation.

• IntegrateMPCwithPSOtosolvetheoptimization

problem.

OngoingandFutureWork

• Deploytheapproachtoactualplants(drones).

• Collisionavoidance.

• ImprovesuccessrateofconvergingtoV-formation.

• UseSMCtoquantifytheprobabilityofsuccess.

• Energyconsumptionandleaderselection.

Thankyou!