CONFIDENCE AT EVERY TURN Prakriteswar Santikary, PhD Vice President and Global Chief Data Officer MODERN DATA ARCHITECTURE AT SCALE IN THE CLOUD : BEST PRACTICES OF SERVERLESS, LAMBDA AND MICROSERVICES ARCHITECTURE
CONFIDENCEAT EVERY TURNCONFIDENCEAT EVERY TURN
Prakriteswar Santikary, PhDVice President and Global Chief Data Officer
MODERN DATA ARCHITECTURE AT SCALE IN THE CLOUD : BEST PRACTICES OF SERVERLESS, LAMBDA AND MICROSERVICES ARCHITECTURE
ABOUT ERT
│ © Copyright ERT 20182
Founded in 1977; privately held
Supporting Pharmacos, Biotechs & CROs around the world
Operations in 12 countries
2500+ employeesMINIMIZING RISK &
UNCERTAINTY, SO
YOU CAN MOVE
AHEAD QUICKLY
WITH CONFIDENCE
Agenda
• Definition• Characteristics• Advantages• Disadvantages
Microservices Architecture
• Definition• Characteristics• Advantages• Disadvantages
Serverless Architecture
• Definition• Characteristics• Advantages• Disadvantages
Monolithic Architecture
| Copyright ERT 20183
01
Monolithic Architecture – a refresher
TIPS ; Fresh business idea, needs quicker turn around, small engineering team – start with monolith – build simple and lightweight application
Monolithic Architecture – Weaknesses and challenges
6
SOFTWARE COMPLEXITY HAS BEEN GROWING STEADILY
│ © Copyright ERT 20186
Forces behindMicroservicesArchitecture
§ Multiple data sources§ Multiple channels§ Multiple data types§ Scale requirements§ Real time insight§ Big Data (unstructured)
§ Small incremental release§ Continuous deployment§ Continuous delivery§ Quick time to market§ Competitive pressure
§ New Programming language§ New Programming framework§ NoSQL databases§ API-first design§ Cloud-first design
§ Focus on code, not on infrastructure – less IT
§ Serverless deployment§ Pay per use§ Cost reduction
World of Big Data
Cloud/Serverless computing
DevOps and Automation01 02
0304 Emerging Technology
Monolith to Microservices Transition – loosely coupled, independent services communicating via lightweight protocol
TIPS : Microservices architecture is not a silver bullet for a bad software design. Design discipline must be exercised as to when to use monolith Vs Microservices.
Monolith is hard to scale; Microservices enable y-axis scaling by breaking the application into smaller independent services. Each service is indepedently scaled based on need, enabling horizonal scaling.
Microservices Logical Architecture
Let’s take a n-tier e-commerce application to explain monolith and microservices
11
BREAK IT DOWN INTO MICROSERVICES
│ © Copyright ERT 201811
12
MICROSERVICES ARCHITECTURE – A PROCESS VIEW
│ © Copyright ERT 201812
13
MICROSERVICES ARCHITECTURE- ADVANTAGES
│ © Copyright ERT 201813
14
MICROSERVICES ARCHITECTURE- DISADVANTAGES
│ © Copyright ERT 201814
15
MICROSERVICES ARCHITECTURE DESIGN PATTERNS
│ © Copyright ERT 201815
MicroservicesBest
Practice
§ By Business Capability§ By business domain§ Single responsibility principle§ Independently deployable§ Small service/small team§ You build it, you own it§ Domain-driven design
§ Service per host/VM§ Multiple services per host/VM§ Serverless Deployment§ On-premise§ Hybrid (with Cloud)
§ Polyglot persistence§ Database per service§ Data consistency§ Event sourcing§ Eventual consistency§ CAP theorem
§ Log Aggregation§ Audit logging§ Distributed Tracing§ Exception Tracking§ Health-check API
Decomposition Pattern
Monitoring/Tracing Pattern
Deployment Pattern01 02
0304 Data Management Pattern
16
MICROSERVICES ARCHITECTURE – BEST PRACTICES
│ © Copyright ERT 201816
Microservices Architecture – a summary view
• Decentralized• Independent• Do one thing well• Polyglot• API-first Design• You build it; You own it
Benefits
• Agility• Innovation• Quality• Scalability• Availability
Challenges
• Distributed systems• Monolith->Microservices
transition not easy• Organizational issues
(DevOps)• Skillsets
Characteristics
| Copyright ERT 201817
TIPS : Microservices is not a silver bullet. You still need to follow sound software design principles.
Serverless ArchitectureFocus on your application, not on infrastructure
• No server to maintain• No server to provision• Focus on code• Ephemeral; Stateless• Event-driven• Horizontal scaling (auto)• Pay per use; no idle time
Benefits
• Faster innovation• Lower operational cost• Reduce operational
management• Speed to market• Fosters innovation
Challenges
• Skillsets• Vendor lock-in concerns• Lack of operational tools• Architectural complexity• Restriction on execution time• Restriction on memory
Characteristics
| Copyright ERT 201818
TIPS : Build scalable platform and applications using a combination of serverless and microservices in the cloud
SERVERLESS ARCHITECTURE – FEW EXAMPLES
| Confidential – Internal Use Only
SERVERLESS ARCHITECTURE USING MICROSERVICES
| Confidential – Internal Use Only
Lambda Architecture – execution of real time and batch together
• Real time data processing• Batch data processing• Merging of real time and
batch in single architecture
Benefits
• Serve both real and batch use cases simultaneously
Challenges
• Architectural complexity• skillsets
Characteristics
| Copyright ERT 201821
TIPS : One architecture supports both real time and batch
22
LAMBDA ARCHITECTURE- PHYSICAL VIEW ON AWS (SERVERLESS)
│ © Copyright ERT 201822
KEY TAKEAWAYS FROM TODAY’S DISCUSSION
│ © Copyright ERT 201823
Monolithic architecture
Simple application; quick turn around – good for simple lightweight application – validate ideas quickly. Not ideal for complex and growing applications – scalability is the biggest issue.
MicroservicesComplex application; scalability need is high; strong engineering skillsets available; distributed computing is complex; Microservices is no silver bullet – still depends on sound software design.
Serverless ArchitectureCloud-native; no HW to maintain; focus on code, not on infrastructure; works well with microservices;