Page 1
SEDA:AnArchitectureforWell-Conditioned,ScalableInternetServices
MattWelsh,DavidCuller,andEricBrewerComputerScienceDivision
UniversityofCalifornia,Berkeley
OperatingSystemsPrinciples(SOSP-18),ChateauLakeLouise,Canada,October21-24,2001.
Page 2
Motivation
• MillionsofInternetusers• DemandforInternetservicesgrows• Highvariationsinserviceload.Loadspikesareexpected• Webservicesgettingmorecomplex• Notstaticcontentanymore,dynamiccontentthatrequireextensivecomputationandI/O
Page 3
Problemstatement
• Servicesthatsupportmillionsofusers(massiveconcurrency)• Responsive• Robust• Highlyavailable
• Wellconditionedservice:• Requestratescaleswiththeresponserate• Excessivedemanddoesnotdegradethroughputandallclientsexperienceanequalresponsetimepenaltylineartothelengthofthequeue(gracefuldegradation).
• Anotionoffairness
Page 4
Thread-basedconcurrency
• Contentionforresourcesandcontextswitchescausehighoverhead• Highnumberofthreadsdegradesthethroughputandresponsetimeisgreatlyincreased• PartialSolution:Boundnumberofthreads
• Throughputmaintained• Butwhataboutmaxresponsetime?Someclientsexperiencelongwaitingtimes
• Overcommittingresources• Transparentresourcevirtualizationpreventsapplicationfromadaptingtoloadchangesandspotting
bottlenecks
Page 5
Event-drivenconcurrency
• Efficientandscalableconcurrency• Butdifficulttoengineerandtune• Howordertheprocessingofevents.Schedulingchallenges• Difficultytofollowtheflowofevents• LittlesupportfromOS
Page 6
StagedEvent-DrivenArchitecture(SEDA)
• Hybridapproach• Thread-basedconcurrencymodelsforeaseofprogramming• Event-basedmodelsforextensiveconcurrency
• MainIdea:Decomposeserviceintostagesseparatedbyqueues• Eachstageperformsasubsetofrequestprocessing
Page 7
SEDA- Stage
• Eventqueuescanposevariouscontrolpolicies
• Modularity.Eachstageimplementedandmanagedindependently
• Expliciteventdeliveryfacilitatestracingflowofeventsandthusspottingbottlenecksanddebugging
Page 8
Controllers
• Threadpoolcontroller:idealdegreeofconcurrencyforastage• Adjustnumberofthreadsbyobservingtheincomingqueuelength• Idlethreadsareremoved
• Batchingcontroller:aimsatlowresponsetimeandhighthroughput• Batchingfactor:numberofeventsconsumedateachiterationoftheeventhandler
• Largebatchingfactor:morelocality,higherthroughput• Smallbatchingfactor:lowerresponsetime
Page 9
SEDAPrototype:Sandstorm
• ImplementedinJava• Javaprovidessoftwareengineeringbenefits
• Built-inthreading,automaticmemorymanagement
• APIsareprovidedfornaming,creatinganddestroyingstages,performingqueueoperations,controllingqueuethresholdsandprofilinganddebugging• AsynchronousI/OprimitivesareimplementedusingexistingOSprimitives.• Thesocketsinterfaceconsistsofthreestages:read,writeandlisten• AsynchronousI/Ofileoperations.
Page 10
Evaluation
• UsedthestaticfileloadfromSpecWEB99benchmark,arealistic,industry-standardbenchmark• 1to1024clientsmakingrepeatedrequests• Filessizesrangefrom102to921600Bytes• Totalfilesetsizeis3.31GB• MemoryCacheof200Mb• Serverrunningon4-waySMP500MHzPentiumIIIsystemwith2GBofRAM• 32machinesofasimilarconfigurationwereusedforloadgeneration
EvaluatedHaboobahigh-performanceSEDA-basedHTTPserver
Page 11
Otherdesigns• ApacheWebserver• Thread-basedconcurrency• Fixed-sizeprocesspoolof150processes
• FlashWebserver• Event-basedconcurrency• Singleprocesshandlingmostrequest-processingtasks.• Upto506simultaneousconnections(duetolimitationsofselectsystemcall).
• Haboob• Hybridapproach.Event-basedandthread-basedconcurrency• Upto1024concurrentrequests
Page 13
Evaluation
• SEDAprovidessomefairness
• TheothertechniquessufferfromlongTCPretransmitbackoff times.Requestsrejectedandre-submitted
Page 14
Evaluation
• Underoverload• Requestswithhighcomputation
andI/Oneeds from1024clients
• Admissioncontrolpolicybyqueues• Canperformprioritizationorfilteringduringheavyload
• Adjustsizeofqueueaccordingtotheresponsetime
• Maintainlowresponsetime
Page 15
Summary
• Newdesignsneededfortheeverincreasingdemandsofwebservices• SEDAisthusproposedtoreachthedesiredperformance• Combinesthread-basedandevent-basedconcurrencymodels• Splitsanapplicationintoanetworkofstageswitheventqueuesinbetween• Dynamicresourcecontrollersforeachstage• Simplifiedbuildinghigh-concurrentservicesbydecouplingloadmanagementfromcoreapplicationlogic
Page 16
Strengths
• Highconcurrency.Abilitytoscaletolargenumbersofconcurrentrequests• Easeofengineering.Simplifyconstructionofwell-conditionedservices• Modularity.Eachstageimplementedandmanagedindependently• Adaptiontoloadvariations.Resourcemanagementadjusteddynamically.• Lowvarianceinresponsetime
Page 17
Weaknesses
• Increasedlatency.Arequesttraversemanystagesandexperiencesmultiplecontextswitchesandadditionaldelaysduetoqueuing.• Onalightlyloadedserver,theworstcasecontextswitchingoverheadcandominate
• Whatabouttheaveragecaseperformance?Istheworstresponsetimethemostimportantmetrictoconsider?
• Programmingstillharderthanthread-basedconcurrencymodels