@AnimeshSingh How to build an event-driven, polyglot serverless microservices framework on OpenStack Animesh Singh, STSM IBM Cloud Platform OpenStack Summit Barcelona, October 2016 Serverless - Functions as a Service Ease of getting started Full stack Control Functions PaaS Containers IaaS
35
Embed
How to build an event-driven, polyglot serverless microservices framework on OpenStack
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
@AnimeshSingh
How to build an event-driven, polyglot serverless microservices framework on OpenStack Animesh Singh, STSM IBM Cloud Platform OpenStack Summit Barcelona, October 2016
Serverless - Functions as a Service
Ease of getting started Full stack Control
Functions PaaS Containers IaaS
@AnimeshSingh
Cloud Computing Evolution
Bare Metal
IaaS
Container Orchestrators PaaS
@AnimeshSingh
Problem: It’s expensive to scale microservices, even on a PaaS
Explosion in number of containers / processes:
1. Increase of infrastructure cost footprint
2. Increase of operational management cost and complexity
Region B Region A
Break-down into microservices
Make each micro service HA
Protect against regional outages
Monolithic application
@AnimeshSingh
Problem: the programming and cost model doesn’t help
• Continuous polling needed in the absence of an event driven programming model.
• Charged for resources, even when idle.
• Worries persist about capacity management.
Swift
Application
Container VM CF
2
Polling
1b
Request
1a
@AnimeshSingh
Bare Metal
IaaS
Container Orchestrators PaaS
Enter Serverless!
Serverless
Introducing OpenWhisk, a fabric and platform for the serverless, event driven programming model
@AnimeshSingh
OpenWhisk- A distributed compute service to execute application logic in response to events.
@AnimeshSingh
OpenWhisk: How does it work?
} 1
Event Providers
OpenWhisk
Cloudant
Git
Weather
…
…
Data event occurs, e.g. - Commit on a Git Repository - CRUD operation on Cloudant - ….
Trigger execution of associated OpenWhisk action
2
… JS Swift ! Docker …
@AnimeshSingh
OpenWhisk: How does it work?
OpenWhisk
JS Swift ! Docker …
Incoming HTTP request, e.g. HTTP GET mynewcoolapp.com/customers
Actions: In any other language by packaging with Docker A Programming model
@AnimeshSingh
Rules: An association of a trigger and an action R
R := T A
Programming model
@AnimeshSingh
Actions: Can be chained to create sequences to increase flexibility and foster reuse A
AA := A1 + A2 + A3
AB := A2 + A1 + A3
AC := A3 + A1 + A2
Programming model
@AnimeshSingh
Packages: A shared collection of triggers and actions P
A
A read
write T changes A translate A forecast
A post T topic
Open Source A myAction
T myFeed
Yours
T commit
Third Party
Programming model
OpenWhisk backend architecture
@AnimeshSingh
OpenWhisk Architecture
Controller
Kafka
Invoker Invoker … Invoker
Router (nginx)
NodeJs Java Swift
Consul Couch DB
22
Controller
… Invoker Invoker Invoker
1!
2!3,4!
5!
6!
7!
8!
OpenWhisk Topology
OpenWhisk on Openstack
@AnimeshSingh
Architecture: Distributed OpenWhisk on OpenStack
!
Bootstrapper Machine !!!!!!
@AnimeshSingh
OpenStack Configuration • Services: Tested on OpenStack Kilo and Mitaka releases. We expect at a minimum
following OpenStack Services available: Identity (Keystone), Compute (Nova), Image (Glance) and Network (Neutron).
• Capacity: For each component VM, we would recommend 4 vCPU, 8 GB RAM and 60 GB disk. So if you deploy just 1 Invoker VM, you can expect to have a minimum capacity of around 36 vCPUs, 64 GB of RAM and 480 GB local disk. For 25 concurrent invocations/sec, we would recommend around 10 Invoker VMs.
• Network: We would recommend creating a virtual router, network and subnet for OpenWhisk, ideally in its own tenant. Also except for the ‘Router’ none of the VMs provisioned would need an external IP, so once the deployment is complete we can assign a floating ip to the ‘Router’ from the external network.
@AnimeshSingh
External Shared Customer Network
OpenWhisk Tenant Private Neutron Network
Neutron OpenWhisk
Tenant Router
OpenWhisk VMs Other Tenant VMs
Other VMs
Other VMs
Other VMs
Tenant Private Network
Physical Router
OpenWhisk - VM Networking View
Other Tenant Router
@AnimeshSingh
Deployment Overview: • To kickoff things, spin a VM which we call ‘bootstrapper’, and it’s essentially using Gradle and
Ansible under the covers to do the build and deployment respectively. • Setup the bootstrapper using the instructions listed here
• Once bootstrapper is ready jump to the following link to build and deploy OpenWhisk on OpenStack https://github.com/openwhisk/openwhisk/blob/master/ansible/README_DISTRIBUTED.md
• In addition to the default Ansible modules, we add Ansible Cloud Modules for OpenStack, and then based on a set of environment variables, we build and deploy OpenWhisk on your OpenStack. http://docs.ansible.com/ansible/list_of_cloud_modules.html
@AnimeshSingh
Deployment Parameters: • For instances and hosts file to be generated and managed by Ansible, set values for the following keys using
environment variables.
• OS_WSK_DB_VOLUME is optional. If not specified, local disk will be used instead of persistent disk for CouchDB.