AUTONOMIC COMPUTING MEETUP MARCH 6, 2017 LOS ANGELES
Autonomic Computing - Dataflow Programming and Reactive State Machines
Mar 20, 2017
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
AUTONOMIC COMPUTING MEETUP MARCH 6, 2017
LOS ANGELES
DATAFLOW PROGRAMMING AND REACTIVE STATE MACHINES
Peter Lee @ Corenova Technologies
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
INTELLIGENCE IS THE ABILITY TO ADAPT TO CHANGE. Stephen Hawking
WHAT IS AN AUTONOMIC SYSTEM?
▸ An approach to enable systems to manage themselves without direct human intervention
▸ Inspired by human autonomic nervous system which controls important bodily functions without conscious intervention
▸ A methodology for reducing the complexity of maintaining distributed systems
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
THE AUTONOMIC COMPUTING MODEL
▸ A fundamental building block of an autonomic system is the sensing capability (Sensors Si), which enables the system to observe its external operational context.
▸ Inherent to an autonomic system is the knowledge of the Purpose (intention) and the Know-how to operate itself (e.g., bootstrapping, configuration knowledge, interpretation of sensory data, etc.) without external intervention.
▸ The actual operation of the autonomic system is dictated by the Logic, which is responsible for making the right decisions to serve its Purpose, and influence by the observation of the operational context (based on the sensor input).
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
WHAT IS DATAFLOW PROGRAMMING?
▸ A programming paradigm that models a program as a directed graph of the data flowing between operations, thus implementing dataflow principles and architecture.
▸ Emphasizes the movement of data and models programs as a series of connections. Explicitly defined inputs and outputs connect operations, which function like black boxes. An operation runs as soon as all of its inputs become valid. Thus, dataflow languages are inherently parallel and can work well in large, decentralized systems.
▸ Dataflow Programming - Concept, Languages and Applications
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
BASIC DATAFLOW MODELS
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
OUTPUT BINPUT A f
OUTPUT DINPUT B
fINPUT C
INPUT D fOUTPUT E
OUTPUT F
REACTOR FUNCTIONS
ONE-TO-ONE
MANY-TO-ONE
ONE-TO-MANY
INPUT Ff
INPUT G
OUTPUT H
OUTPUT IMANY-TO-MANY
IMPERATIVE VS. DATAFLOW PROGRAMMING
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
MODULE/SUPERAGENT
HTTP/REQUEST/GET/URLHTTP/RESPONSE
REQUIRE f
var request = require(“superagent”); request.get(“http://www.google.com") request.end((err, response) => { // logic to deal with response })
f
IMPERATIVE JAVASCRIPT CODE
require “superagent" http/request/get/url “http://www.google.com"
DATAFLOW TRIGGERS
synchronous, must return expected module
asynchronous, must supply callback function asynchronous, sequence of
triggers doesn’t matter
IMPERATIVE EXAMPLE: REACTING TO ENVIRONMENT
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
try { var request = require(“superagent”); makeRequest(request, “http://www.google.com") } catch (err) { var npm = require(“npm”); npm.load({}, (err, loaded) => { npm.install(“superagent”, (err, installed) => { // deal with error, otherwise continue let request = require(“superagent”) makeRequest(request, “http://www.google.com") } }) }
IMPERATIVE JAVASCRIPT CODE
function makeRequest(request, url) { request.get(url) request.end((err, response) => { // logic to deal with response }) }
nested async callbacks, still need to deal with errors at each level
break out common function to deal with
multiple entry points for execution
Logic to deal with missing library dependency, “superagent”
DATAFLOW EXAMPLE: REACTING TO ENVIRONMENT
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
require “npm” require “superagent” http/request/get/url “http://www.google.com"
DATAFLOW TRIGGERS
MODULE/NPM
NPM/INSTALL
REQUIRE
MODULE/SUPERAGENT
HTTP/REQUEST/GET/URLHTTP/RESPONSEf
f f
NPM/INSTALLEDf
NPM/LOADED
additional dataflow to support NPM workflows
WHAT IS A DATA PIPELINE?
▸ In computing, a pipeline is a set of data processing elements connected in series, where the output of one element is the input of the next one.
▸ Software pipelines, where commands can be written where the output of one operation is automatically fed to the next, following operation. The Unix system call pipe is a classic example of this concept, although other operating systems do support pipes as well.
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
TRADITIONAL DATA PIPELINE
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
FORMAT BFORMAT A f FORMAT Cf
▸ All data chunks of format A runs through a processing node (which internally buffers incoming/outgoing chunks) and generates stream of format B chunks
▸ All data chunks of format B then runs through another processing node which generates a stream of format C chunks
▸ You can create a sequence of transforms to achieve desired final outcome
DATAFLOW PIPELINE
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
OUTPUT BINPUT A
OUTPUT Cf
f OUTPUT EINPUT C
OUTPUT FINPUT D
f
INPUT F OUTPUT D
INPUT B
f
CIRCULAR FEEDBACK FLOW!
INPUT OUTPUT
DISTRIBUTED DATAFLOW PIPELINE: MICRO-SERVICES
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
OUTPUT BINPUT A
OUTPUT Cf
f OUTPUT EINPUT C
OUTPUT FINPUT D
fINPUT B
ANY DATA f SOCKET ANY DATAfSOCKET
▸ You can build data pipelines across any instance of the Flow running anywhere!
DISTRIBUTED DATAFLOW PIPELINE
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
CREATIVITY IS JUST CONNECTING THINGS.
Steve Jobs
KOS: A DATAFLOW STREAMING FRAMEWORK FOR CREATING AWESOME DATA PIPELINES
LA AUTONOMIC COMPUTING MEETUP #3 - MARCH 6, 2017
▸ JavaScript (Node.js and the web browser)
▸ Open-Source License
▸ https://github.com/corenova/kos
LET’S CONNECT THINGS!
THANKS FOR COMING!
SEE YOU NEXT TIME
LA Autonomic Computing Meetup
Related Documents
