11 Projects, 88 Question Marks and '0' Problems – A Yellow ... Marketing/Summit-Berlin...SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 11880 –The

S U M M I TB E R L I N

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

11 Projects, 88 Question Marks and "0" Problems –A Yellow Pages Switches to the Cloud

Andreas BayerFrederic BiermannFlorian in der Beek11880 Internet Services AG

S e s s i o n I D


Speaker details

• Andreas Bayer• M. Sc. in Systems Engineering

• Webdeveloper and DevOp

• Team Portals & SEO

• Frederic Biermann• Webdeveloper and DevOp


• Florian in der Beek• Webdeveloper and DevOp



11880 – The Story BehindOverview

1996 Foundation 1999 IPO

31.000* 617**

Essen

RostockNeubrandenburg


11880 – Metrics

More than

65 MUser / Year

More than

92 MSessions / Year

More than

158 MPages / Total

More than

850 MSearches / Year

More than

6 BRequests / Year

More than

156 TBTraffic / Year


Why Cloud?

Business side

• Restructure/re-thinking of company

• More flexibility for our customers

• Increase profitability

Technical side

• In-house data center runs out of maintenance

• Consolidation and space reduction of hardware


Why Cloud?

Business side

• Restructure/re-thinking of company

• More flexibility for our customers

• Increase profitability

Technical side

• In-house data center runs out of maintenance

• Consolidation and space reduction of hardware


“we are not in this luxury situation”


“but we can handle the dragon!!!”


Timeline


• AWS for us the best solution• Huge amount of services

• High performance

• Easy to use

• High availability

• High scalability

• Flexibility

• StabilityServices Performance Costs

Preparation Migration Optimization

Services Performance Costs


Move fast

Move a lot

Improve performance

Shut-down legacy

Migration goals Preparation Migration Optimization

Photo by rawpixel.com from Pexels

“we want to live the DevOps approach”


How we started Preparation Migration Optimization

“we just started without a big planning phase”

Illustrate infrastructure

Define POC components

Performance analysis

AWS and partner ramp-up support


PoC setup Preparation Migration Optimization

Involved teams

• 3 developer teams

• IT operations

• External service provider

“PoC gets bigger than expected”


AWS services we learned to

• AWS Lambda as allround helper• Lambda solves (almost) every problem

• Amazon SQS as queuing service• Using RabbitMQ before

• Easy to use, but limitations in message size

• Centralized logging with Amazon Elasticsearch Service and Amazon CloudWatch• Using ELK-Stack with Logstash before (now using CloudWatch with Lambda instead of Logstash)

• Less overhead, better performance and cheaper

“it makes fun to try outnew AWS services”

“the more AWS services you use, the more you have to learn”



• Automated infrastructure scaling with AWS CloudFormation

• Systems already based on Docker – easy to use Amazon ECS

• Blue-Green deployment in 5 Steps

• Future solutions• AWS Cloud Development Kit [1]

• Blue-Green with AWS Lambda [2]

Preparation Migration OptimizationECS & CloudFormation

Version 1

Version 1 Version 1

Version 2 Version 1

Version 2 Version 1

Version 2


• Automated infrastructure scaling with AWS CloudFormation

• Systems already based on Docker – easy to use Amazon ECS

• Blue-Green deployment in 5 Steps

• Future solutions• AWS Cloud Development Kit [1]

• Blue-Green with AWS Lambda [2]

Preparation Migration OptimizationECS & CloudFormation

Version 1

Version 1 Version 1

Version 2 Version 1

Version 2 Version 1

Version 2

“CloudFormation gives us better insights about our

infrastructure”


Preparation Migration OptimizationStorage opportunities

• Need fast data storage for minimal costs

• Central data store for less overhead

• Tested several AWS storages• Amazon ElastiCache (Redis)

• Amazon EFS

• Amazon S3

• Amazon DynamoDB (limited object size)

Solution

Combination of ElastiCache + EFS + S3Performance Costs

ElastiCache EFS S3 DynamoDB


Timeline

EvaluationPreparation

ConsumtionOptimizations

10 months 2 months

“PoC works better than expected, so we

directly put it live”


Migrate step by step

Lift and shift for fast results

Go-Live at one weekend

Optimize afterwards

Migration strategy Preparation Migration Optimization



AWS DMS Database Migration

• Current database situation: Multiple MySQL databases with different versions

• Using AWS Database Migration Service✓ Allows transfer of multiple sources

x Resync problems with on-prem database

x Missing support for ENUM values

• Using MySQL standard solutionx AWS Aurora was not able to handle Multi-master replication

✓ Master-Slave-Slave replication



AWS DMS Database Migration

• Current database situation: Multiple MySQL databases with different versions

• Using AWS Database Migration Service✓ Allows transfer of multiple sources

x Resync problems with on-prem database

x Missing support for ENUM values

• Using MySQL standard solutionx AWS Aurora was not able to handle Multi-master replication

✓ Master-Slave-Slave replication



just one weekend with pizza…


…we were LIVE


Results

• One year until cloud go-live

• Migration without bigger downtime

• Better performance than on-prem• 20% better TTFB

• Less infrastructure errors

Learnings

• Fast migration resulted initially in higher costs

• Fast migration allowed us to startearlier with optimizations

• On-prem and cloud are fully integrateable

Migration results and learnings Preparation Migration Optimization


Timeline

EvaluationPreparation

MigrationGo-Live

10 months 2 months ongoing


Right sizing

Increaseelasticity

Use reservedinstances

Match storage toyour need

Design forcost


“Be aware of your workload level to reduce costs”



our cloud cost flow

lift and shiftRight sizing30% costs reduction

Increase elasticityjust start staging environments when needed

Use reserved instances70% reservation of machines and services

Design for costUsing AWS services

AWS optimized

“we are on the right way, but still a long

way to go”


• Payment of most AWS services: Pay-as-you-go

• AWS Lambda• Number of executions increases (e.g. when using AWS Lambda for logging events)

• Execution time increases (e.g. AWS Lambda blocks by waiting of other services)

• AWS Application Load Balancer• Rule executions and traffic increases (e.g. website crawler bot)

Solutions

Using AWS-Budgets, Amazon CloudWatch Monitoring and Alarms, Optimize code and logs



• Payment of most AWS services: Pay-as-you-go

• AWS Lambda• Number of executions increases (e.g. when using AWS Lambda for logging events)

• Execution time increases (e.g. AWS Lambda blocks by waiting of other services)

• AWS Application Load Balancer• Rule executions and traffic increases (e.g. website crawler bot)

Solutions

Using AWS-Budgets, Amazon CloudWatch Monitoring and Alarms, Optimize code and logs


“everybody should focus on costs”

“especially the developer :-)”


Next steps

[1] https://aws.amazon.com/de/well-architected-tool/


Future workshops with AWS Solutions Architect

AWS Well-Architected Tool [1]

Increase performance by optimizing instance and AWS service usage

Extend service usage with Amazon WorkSpacesas call center agent solution

Migrate other on-prem systems into AWS

Photo by Pixabay from Pexels


Just start migrating, optimize later

On-prem and cloud works side by side

Use AWS services for easy administration

Read the AWS service docs twice

Never stop optimizing

Our takeaways

Photo by Pixabay from Pexels

“reduce costs to have more money for new fancy shit!!!”


Thank you!

S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Andreas [email protected]

Andreas_Bayer39

Frederic [email protected]

Frederic_Biermann

Florian in der [email protected]

Florian_inderBeek

11 Projects, 88 Question Marks and '0' Problems – A Yellow ... Marketing/Summit-Berlin...SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 11880 –The

Documents