Migrate the Mission Critical Application to AWS Cloud

Shuen-Huei Guan, KKBOX

2015-12-01

AWS re:Invnet 2015 Recap Migrate the Mission Critical Application to

AWS Cloud

Who am I?

Shuen-Huei (Drake) Guan / @drakeguan Sr. Principal Engineer, Video.KKBOX Volunteer, PyCon Taiwan

What to Expect from the Session

•  How we migrate a video-on-demand service. •  What we benefited from AWS. •  What we saw in AWS re:Invent 2015. •  What’s next then.

VideoPass

•  Video-on-demand (OTT) service, built as on-premises platform.

•  1M subscription users. •  The scale of commercial contents is huge.

•  More than 10k videos, ~= 2,000,000 minutes.

•  Content encoding capacity is constrained.

VideoPass Migration Plan

•  1yr Development. •  Integration of 4+2 vendors. •  Nearly seamless. No obvious service shutdown. •  2-staged migration.

What We Pursue…

•  Built on cloud from 1st day. •  Just one AWS DevOps Engineer. •  4 dynamic environments:

develop, testing, staging, and production. •  Scalable infrastructure, storage and bandwidth. •  Powerful content encoding capability. •  Standardized format: MPEG-DASH, HLS, CENC, …

Typical Architecture

•  Entry points: EC2 + ELB + Auto Scaling group.

•  Valuable Data: Multi-AZ RDS + ElasticSearch.

•  Media warehouse: S3 + CDN.

•  Future-proof: wrapped API access to AWS.

Content Encoding Workflow

Orchestrating the execution of workflows for media streaming service and even more

- PyCon APAC 2015 - https://github.com/KKBOX/mass�

Gains from AWS

•  Cooperation with AWS team to survive the migration. - IEM (Infrastructure Event Management) - Enterprise Support.

•  Really insane encoding capability. - 3,000 videos in just one day. - done by 800 encoders.

•  We just hired one AWS DevOps!

Rumor has it that AWS re:Invent is awesome!

Data Center Migration

•  GAM303 - Riot Games: Migrating Mountains of Big Data to AWS

•  DAT205 - Introduction to AWS Database Migration Service

•  ISM201 - Preparing for Migration: Developing a Roadmap to AWS for 7,700 Systems and 5 Global Data Centers

•  … (Data center of KKBOX music streaming?)

Real-time Streaming Data

•  BDT403 - Best Practices for Building Real-time Streaming Applications with Amazon Kinesis

•  ARC310-APAC - Amazon.com: Solving Amazon's Catalog Contention and Cost with Amazon Kinesis

•  BDT205 - Your First Big Data Application on AWS •  … (Real-time user behaviors analysis?)

Microservices

•  ARC201 - Microservices Architecture for Digital Platforms with AWS Lambda, Amazon CloudFront and Amazon DynamoDB

•  ARC303 - Pure Play Video OTT: A Microservices Architecture in the Cloud

•  ARC309 - From Monolithic to Microservices: Evolving Architecture Patterns in the Cloud

•  ... (Developers >> Researchers >> DevOps)

Cost Optimization

•  ARC302 - Running Lean Architectures: How to Optimize for Cost Efficiency

•  ISM402 - Cost Optimization at Scale •  STG311 - AWS Storage Gateway: Simple, Secure, Cost-

Effective Backup and Archive in the Cloud •  … (Cost optimization is always that hot even after

re:Play party!)

The best server is no server at all

AWS Lambda (2.0)

•  Python++ •  1 minute -> 5 minutes •  Scheduling •  Events

•  S3 •  Kinesis •  SWF •  CloudWatch Log

Before AWS re:Invent, We Thought

•  Cost optimization: - lower IT labor - flexible infrastructure

•  Fully-managed scalability (IaaS): - Auto-Scaling group - ELB - VPC

•  Well support: - Enterprise support

After AWS re:Invent, We Thought

•  Cost optimization: - development faster, then deployment - quit/change even faster if necessary

•  Brainless scalability (PaaS): - serverless microservices - Lambda + API Gateway

•  Deeper support: - continuous mutual cooperation - more KKBOX staffs to AWS re:Invent!

KKBOX Video

•  Initiated in 2014. •  Team built from 10 staff. •  1yr development for the whole system. •  Preparing for next challenges.

Thank you!

Have I mentioned that we are hiring? Try this: [email protected]