An overview of designing microservices based applications on AWS Peter Dalbhanjan, Solutions Architect, AWS
Apr 11, 2017
An overview of designing microservices based applications on
AWSPeter Dalbhanjan, Solutions Architect, AWS
EvolutionfromMonolithstoMicroservices
ApproachesforBuildingMicroservicesApplicationsonAWS
CorePrinciplesofMicroservices
Demo
AccountSupport
Support
ManagedServices
ProfessionalServices
PartnerEcosystem
Training&Certification
SolutionArchitects
AccountManagement
Security&PricingReports
TechnicalAcct.Management
MarketplaceBusiness
Applications
DevOpsTools
BusinessIntelligence
Security
Networking
Database&Storage
SaaSSubscriptions
OperatingSystems
Mobile
Build,Test,MonitorApps
PushNotifications
Build,Deploy,ManageAPIs
DeviceTesting
Identity
EnterpriseApplications
DocumentSharing
Email&Calendaring
HostedDesktops
ApplicationStreaming
Backup
GameDevelopment
3DGameEngine
Multi-playerBackends
Mgmt.Tools
Monitoring
Auditing
ServiceCatalog
ServerManagement
ConfigurationTracking
Optimization
ResourceTemplates
Automation
AnalyticsQueryLargeDataSets
Elasticsearch
BusinessAnalytics
Hadoop/Spark
Real-timeDataStreaming
OrchestrationWorkflows
ManagedSearch
ManagedETL
ArtificialIntelligence
Voice&TextChatbots
MachineLearning
Text-to-Speech
ImageAnalysis
IoT
RulesEngine
LocalComputeandSync
DeviceShadows
DeviceGateway
Registry
Hybrid Devices&EdgeSystems
DataIntegration
IntegratedNetworking
ResourceManagement
VMwareonAWS
IdentityFederation
Migration ApplicationDiscovery
ApplicationMigration
DatabaseMigration
ServerMigrationDataMigration
Infrastructure Regions AvailabilityZones
PointsofPresence
Compute Containers Event-drivenComputing
VirtualMachines SimpleServers AutoScaling BatchWeb
Applications
Storage ObjectStorage Archive BlockStorage ManagedFileStorage
Exabyte-scaleDataTransport
Database MariaDB DataWarehousing NoSQLAurora MySQL Oracle SQLServerPostgreSQL
ApplicationServices Transcoding StepFunctions Messaging
Security CertificateManagement
WebApp.Firewall
Identity&Access
KeyStorage&Management
DDoSProtection
ApplicationAnalysis
ActiveDirectory
DevTools
PrivateGitRepositories
ContinuousDelivery
Build,Test,andDebug
Deployment
Networking IsolatedResources
DedicatedConnections LoadBalancing ScalableDNSGlobalCDN
TheAWSPlatform
*Asof1March2017
2010
61
1,017
159
2012 2014 2016
516
“The Monolith”
Challenges with monolithic software
LongBuild/Test/ReleaseCycles(whobrokethebuild?)
Operationsisanightmare(moduleXisfailing,who’stheowner?)
Difficulttoscale
Newreleasestakemonths
Longtimetoaddnewfeatures
Architectureishardtomaintainandevolve
Lackofinnovation
Frustratedcustomers
Lackofagility
Challenges with monolithic software
LongBuild/Test/ReleaseCycles(whobrokethebuild?)
Operationsisanightmare(moduleXisfailing,who’stheowner?)
Difficulttoscale
Newreleasestakemonths
Longtimetoaddnewfeatures
Architectureishardtomaintainandevolve
Lackofinnovation
Frustratedcustomers
Lackofagility
Challenges with monolithic software
LongBuild/Test/ReleaseCycles(whobrokethebuild?)
Operationsisanightmare(moduleXisfailing,who’stheowner?)
Difficulttoscale
Newreleasestakemonths
Longtimetoaddnewfeatures
Architectureishardtomaintainandevolve
Lackofinnovation
Frustratedcustomers
Lackofagility
“20080219BonMorningDSC_0022B”bySunphol Sorakul .Noalterationsotherthancropping.https://www.flickr.com/photos/83424882@N00/3483881705/ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
Monolith development lifecycle
releasetestbuild
deliverypipeline
app(akathe“monolith”)developers
PhotobySageRoss.Noalterationsotherthancropping.https://www.flickr.com/photos/ragesoss/2931770125/ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
“IMG_1760”byRobertCouse-Baker.Noalterationsotherthancropping.https://www.flickr.com/photos/29233640@N07/14859431605/ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
“service-orientedarchitecturecomposed ofloosely coupled elementsthat havebounded contexts”
AdrianCockcroft(VP,CloudArchitectureStrategyatAWS)
Servicescommunicatewitheachotheroverthenetwork
“service-orientedarchitecturecomposed ofloosely coupled elementsthat havebounded contexts”
AdrianCockcroft(VP,CloudArchitectureStrategyatAWS)
“service-orientedarchitecturecomposed ofloosely coupled elementsthat havebounded contexts”
AdrianCockcroft(VP,CloudArchitectureStrategyatAWS)
You canupdatetheservicesindependently;updatingoneservicedoesn’trequirechanginganyotherservices.
“service-orientedarchitecturecomposed ofloosely coupled elementsthat havebounded contexts”
AdrianCockcroft(VP,CloudArchitectureStrategyatAWS)
Self-contained;youcanupdatethecodewithoutknowinganythingabouttheinternalsofothermicroservices
“Do one thing, and do it well”
“SwissArmy”bybyJimPennucci.Noalterationsotherthancropping.https://www.flickr.com/photos/pennuja/5363518281/ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
“Tools”byTonyWalmsley:Noalterationsotherthancropping.https://www.flickr.com/photos/twalmsley/6825340663/ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
“Do one thing, and do it well”
Anatomy of a Micro-service
DataStore(eg,RDS,DynamoDB
ElastiCache,ElasticSearch)
Anatomy of a Micro-service
Application/Logic(code,libraries,etc)
Anatomy of a Micro-service
DataStore(eg,RDS,DynamoDB
ElastiCache,ElasticSearch)
Application/Logic(code,libraries,etc)
Anatomy of a Micro-service
DataStore(eg,RDS,DynamoDB
ElastiCache,ElasticSearch)
PublicAPIPOST/micro-serviceGET/micro-service
AvoidSoftwareCoupling
Driversmicro-service
Paymentsmicro-service Location
micro-service
Orderingmicro-service
Restaurantmicro-service
Ecosystem of microservices
= 50 million deployments a year
Thousands of teams× Microservice architecture
× Continuousdelivery
× Multipleenvironments
(5708 per hour, or every 0.63 second)
Driversmicro-service
Paymentsmicro-service Location
micro-service
Orderingmicro-service
Restaurantmicro-service
Typical microservices application
Micro-serviceDesign
Approach#1EC2
Micro-serviceDesign
EC2
Micro-serviceDesign
EC2
Micro-serviceDesign
EC2EC2 EC2 EC2
Micro-serviceDesign
EC2EC2 EC2 EC2
ElasticLoadBalancer
Micro-serviceDesign
EC2EC2 EC2 EC2
ElasticLoadBalancer
AWSElasticBeanstalk
Elastic Beanstalk vs. DIY
Yourcode
HTTPserver
Applicationserver
Languageinterpreter
Operatingsystem
Host
ElasticBeanstalkconfigureseachEC2instanceinyourenvironmentwiththecomponentsnecessarytorunapplicationsfortheselectedplatform.Nomoreworryingaboutloggingintoinstancestoinstallandconfigureyourapplicationstack.
Focusonbuildingyourapplication
Providedbyyou
ProvidedandmanagedbyElasticBeanstalk
On-instanceconfiguration
Micro-serviceDesign
Approach#2ContainersUsingECS
AmazonEC2Container Service(ECS)
istheclustermanagementsystemtorunyourDockercontainers
Cluster Management Made Easy• Nothing to run
• Complete state
• Control and monitoring
• Scale
UseAmazonEC2ContainerServiceforcontainerworkloads
AmazonEC2ContainerService(ECS)
Flexible Scheduling• Applications
• Batch jobs
• Multiple schedulers
UseAmazonEC2ContainerServiceforcontainerworkloads
AmazonEC2ContainerService(ECS)
Designed for Use with Other AWS Services• Elastic Load Balancing• Amazon Elastic Block Store• Amazon Virtual Private Cloud• AWS Identity and Access Management• AWS CloudTrail
UseAmazonEC2ContainerServiceforcontainerworkloads
AmazonEC2ContainerService(ECS)
Micro-serviceDesign
EC2EC2 EC2 EC2
ElasticLoadBalancer
Micro-serviceDesign
EC2EC2 EC2 EC2
ElasticLoadBalancer
Micro-serviceDesign
EC2EC2 EC2 EC2
ElasticLoadBalancer
AmazonEC2ContainerService(ECS)tomanagecontainers
Micro-serviceDesign
Approach#3APIGateway+Lambda
AWSLambdaletsyouruncode
withoutmanagingservers
Lambdaautomatically
scales
Uploadyourcode(Java,JavaScript,
Python)
Payforonlythecomputetime
youuse(sub-secondmetering)
SetupyourcodetotriggerfromotherAWSservices,webservicecalls,orappactivity
AWSAPIGatewayistheeasiestwaytodeploymicro-services
CreateaunifiedAPIfrontendfor
multiplemicro-services
…aswellasmonitoring,
logging,rollbacks,clientSDK
generation…
Authenticateandauthorizerequests
HandlesDDoSprotectionandAPIthrottling
Principle1
Micro-servicesonlyrelyoneachother’spublicAPI
“Contracts”byNobMouse.Noalterationsotherthancropping.https://www.flickr.com/photos/nobmouse/4052848608/
ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
Micro-serviceA Micro-serviceB
publicAPI publicAPI
Principle1: Microservicesonlyrelyoneachother’spublicAPI
DynamoDB
Micro-serviceA Micro-serviceB
publicAPI publicAPI
Principle1: Microservicesonlyrelyoneachother’spublicAPI(HideYourData)
DynamoDB
Micro-serviceA Micro-serviceB
publicAPI publicAPI
Principle1: Microservicesonlyrelyoneachother’spublicAPI(HideYourData)
Nope!
DynamoDB
Micro-serviceA Micro-serviceB
publicAPI publicAPI
Principle1: Microservicesonlyrelyoneachother’spublicAPI(HideYourData)
DynamoDB
Micro-serviceA
publicAPI
Principle1: Microservicesonlyrelyoneachother’spublicAPI(EvolveAPIinbackward-compatibleway…and
document!)
storeRestaurant (id,name,cuisine)Version1.0.0
Micro-serviceA
publicAPI
Principle1: Microservicesonlyrelyoneachother’spublicAPI(EvolveAPIinbackward-compatibleway…and
document!)
storeRestaurant (id,name,cuisine)Version1.0.0
storeRestaurant (id,name,cuisine)storeRestaurant (id,name,arbitrary_metadata)addReview (restaurantId,rating,comments)
Version1.1.0
Micro-serviceA
publicAPI
Principle1: Microservicesonlyrelyoneachother’spublicAPI(EvolveAPIinbackward-compatibleway…and
document!)
storeRestaurant (id,name,cuisine)Version1.0.0
storeRestaurant (id,name,cuisine)storeRestaurant (id,name,arbitrary_metadata)addReview (restaurantId,rating,comments)
Version1.1.0
storeRestaurant (id,name,arbitrary_metadata)addReview (restaurantId,rating,comments)
Version2.0.0
Principle2
Usetherighttoolforthejob
“Tools#2”byJuanPabloOlmo.Noalterationsotherthancropping.https://www.flickr.com/photos/juanpol/1562101472/
ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
Principle2:Usetherighttoolforthejob(Embracepolyglotpersistence)
Micro-serviceA Micro-serviceB
publicAPI publicAPI
DynamoDB
Principle2:Usetherighttoolforthejob(Embracepolyglotpersistence)
Micro-serviceA Micro-serviceB
publicAPI publicAPI
DynamoDB
AmazonElasticsearchService
Principle2:Usetherighttoolforthejob(Embracepolyglotpersistence)
Micro-serviceA Micro-serviceB
publicAPI publicAPIAmazonElasticsearchService
RDSAurora
Principle2:Usetherighttoolforthejob(Embracepolyglotprogrammingframeworks)
Micro-serviceA Micro-serviceB
publicAPI publicAPIAmazonElasticsearchService
RDSAurora
Principle2:Usetherighttoolforthejob(Embracepolyglotprogrammingframeworks)
Micro-serviceA Micro-serviceB
publicAPI publicAPIAmazonElasticsearchService
RDSAurora
Principle3
SecureYourServices
“security”byDaveBleasdale.Noalterationsotherthancropping.https://www.flickr.com/photos/sidelong/3878741556/
ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
Principle 3: Secure Your Services• Defense-in-depth
• Networklevel(e.g.VPC,SecurityGroups,TLS)• Server/container-level• App-level• IAMpolicies• IAMrolesonECStasks• CloudTrail logs
• Authentication&Authorization• Client-to-service,aswellasservice-to-service• IAM-basedAuthentication
• Secretsmanagement• ParameterStore• S3bucketpolicies+KMS+IAM• Open-sourcetools(e.g.Vault,Keywhiz)
AmazonEC2ContainerService(ECS)
Principle 3: Secure Your Services• Defense-in-depth
• Networklevel(e.g.VPC,SecurityGroups,TLS)• Server/container-level• App-level• IAMpolicies
• Gateway (“Frontdoor”)
• APIThrottling• Stage-levelandMethod-levelthrottling
• Authentication&Authorization• Client-to-service,aswellasservice-to-service• AWSCognito:userpools,federatedidentities• APIGateway:customLambdaauthorizers• IAM-basedAuthentication• Token-basedauth (JWTtokens,OAuth 2.0)
• Secretsmanagement• S3bucketpolicies+KMS+IAM• Open-sourcetools(e.g.Vault,Keywhiz)
APIGateway
Principle4
Beagoodcitizenwithintheecosystem
“LamingtonNationalPark,rainforest”byJussarian.Noalterationsotherthancropping.https://www.flickr.com/photos/kerr_at_large/87771074/
ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
HeySally,weneedtocallyourmicro-servicetofetch
restaurantsdetails.
SurePaul.WhichAPIsyouneedtocall?OnceIknow
betteryourusecasesI’llgiveyoupermissiontoregisteryourserviceasaclienton
ourservice’sdirectoryentry.
Micro-serviceA Micro-serviceB
publicAPI publicAPI
Principle 4: Be a good citizen within the ecosystem
Principle 4: Be a good citizen within the ecosystem(Have clear SLAs)
RestaurantMicro-service
15TPS100TPS5TPS20TPS
Beforeweletyoucallourmicro-serviceweneedtounderstand
yourusecase,expectedload(TPS)andaccepted
latency
…andmany,manyothers!
Distributedmonitoringandtracing• “IstheservicemeetingitsSLA?”• “Whichserviceswereinvolvedinarequest?”• “Howdiddownstreamdependenciesperform?”
Sharedmetrics• e.g.servicedependency,request/responsetime
Distributedtracing• AWSX-Ray• 3rd party:Zipkin,OpenTracing
User-experiencemetrics• Statuscodes,latency,errorcounts,timetofirstbyte
Principle4:Beagoodcitizenwithintheecosystem(Distributedmonitoring,loggingandtracing)
AWSX-RayDistributedtracingservicethat
enablesdeveloperstoanalyzethebehavioroftheirapplications
AWSX-Raytracesrequestsmadetoyourapplication
X-Ray service
X-Raycombinesthedata
gatheredfromeachserviceintosingularunitscalled
traces
Viewtheservicemaptosee
tracedatasuchaslatencies,HTTPstatuses,andmetadataforeachservice
Drillintotheserviceshowing
unusualbehaviorto
identifytherootissue
X-Raycollectsdataabouttherequestfromeachofthe
underlyingapplicationsservicesitpassesthrough
Principle5
Morethanjusttechnologytransformation
“rowingontheriverinBedford”byMatthewHunt.Noalterationsotherthancropping.https://www.flickr.com/photos/mattphotos/19189529/
ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
“Anyorganizationthatdesignsasystemwillinevitablyproduceadesignwhosestructureisacopyoftheorganization’scommunicationstructure.”
MelvinE.Conway,1967
Conway’sLaw
Silo’d functional teams à silo’d application architectures
ImagefromMartinFowler’sarticleonmicroservices,athttp://martinfowler.com/articles/microservices.html
Noalterationsotherthancropping.Permissiontoreproduce:http://martinfowler.com/faq.html
Cross functional teams à self-contained services
ImagefromMartinFowler’sarticleonmicroservices,athttp://martinfowler.com/articles/microservices.html
Noalterationsotherthancropping.Permissiontoreproduce:http://martinfowler.com/faq.html
Fullownership
Fullaccountability
Alignedincentives
“DevOps”
Non-pizzaimagefromMartinFowler’sarticleonmicroservices,athttp://martinfowler.com/articles/microservices.html
Noalterationsotherthancropping.Permissiontoreproduce:http://martinfowler.com/faq.html
Cross functional teams à self-contained services(“Two-pizza teams” at Amazon)
Principle6
AutomateEverything
“Robot”byRobinZebrowski.Noalterationsotherthancropping.https://www.flickr.com/photos/firepile/438134733/
ImageusedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)
releasetestbuild
Focused agile teams
2-pizzateam deliverypipeline service
releasetestbuild
releasetestbuild
Focused agile teams
2-pizzateam deliverypipeline service
releasetestbuild
releasetestbuild
Focused agile teams
2-pizzateam deliverypipeline service
releasetestbuild
releasetestbuild
releasetestbuild
Focused agile teams
2-pizzateam deliverypipeline service
releasetestbuild
releasetestbuild
releasetestbuild
releasetestbuild
Focused agile teams
2-pizzateam deliverypipeline service
releasetestbuild
releasetestbuild
releasetestbuild
releasetestbuild
releasetestbuild
Focused agile teams
2-pizzateam deliverypipeline service
releasetestbuild
releasetestbuild
releasetestbuild
releasetestbuild
Principle 6: Automate everything
AWSCodeCommit AWSCodePipeline AWSCodeDeploy
ELBAutoScalingEC2 LambdaECS
DynamoDBRDS ElastiCache
SQS SWF SESSNS
APIGatewayCloudWatch CloudTrail
KinesisElastic
Beanstalk
It’s a journey…
Expectchallengesalongtheway…
• Understandingofbusinessdomains• EventualConsistency• Servicediscovery• Lotsofmovingpartsrequiresincreased
coordination• Complexityoftesting/deploying/
operatingadistributedsystem• Culturaltransformation
Principles of Microservices
1.RelyonlyonthepublicAPI� Hideyourdata� DocumentyourAPIs� Defineaversioningstrategy
2.Usetherighttoolforthejob� Containerjourney?(useECS)� Polyglotpersistence(datalayer)� Polyglotframeworks(applayer)
3.Secureyourservices� Defense-in-depth� Authentication/authorization
6.Automateeverything� AdoptDevOps
4.Beagoodcitizenwithintheecosystem� HaveSLAs� Distributedmonitoring,logging,tracing
5.Morethanjusttechnologytransformation� Embraceorganizationalchange� Favorsmallfocuseddev teams
Benefits of microservices
RapidBuild/Test/ReleaseCycles
Clearownershipandaccountability
Easiertoscaleeachindividualmicro-service
Benefits of microservices
RapidBuild/Test/ReleaseCycles
Clearownershipandaccountability
Easiertoscaleeachindividualmicro-service
Newreleasestakeminutes
Shorttimetoaddnewfeatures
Easiertomaintainandevolvesystem
Benefits of microservices
RapidBuild/Test/ReleaseCycles
Clearownershipandaccountability
Easiertoscaleeachindividualmicro-service
Newreleasestakeminutes
Shorttimetoaddnewfeatures
Easiertomaintainandevolvesystem
Fasterinnovation
Delightedcustomers
Increasedagility
AdditionalAWSresources:• MicroservicesonAWSwhitepaper:
https://d0.awsstatic.com/whitepapers/microservices-on-aws.pdf
• ServerlessWebapp - ReferenceArchitecture:https://github.com/awslabs/lambda-refarch-webapp
• MicroserviceswithECS:https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-with-microservices-deployed-on-amazon-ecs/
• MicroserviceswithouttheServershttps://aws.amazon.com/blogs/compute/microservices-without-the-servers
Popularopen-sourcetools:• Serverless – http://serverless.com• Apex – http://apex.run/
https://aws.amazon.com/devops/
Additional resources
Thank you!Peter Dalbhanjan