What, How and Why BigML
Predictions and other adventures
observations
theory
predictions
problem
conjecture
explanation
observations
predictions
models
observations
theory
predictions
datasets
datasources
predictions
Large DatasetsLarge Datasets
ParallelismParallelism
AnytimeAnytime
DATA STREAMING
Large DatasetsLarge Datasets
ParallelismParallelism
AnytimeAnytime
Streaming Streaming HistogramsHistograms
Streaming Streaming TreesTrees
Accuracy = (TP + TN) / (TP + TN + FP + FN)
Recall = TP / (TP + FN)
Precision = TP / (TP + FP)
F1 = 2 Precison Recall / (Precision + Recall)
I program, therefore I am ...
REST
POSTGETDELETEPUT
application/json
REST+
queued finishedin-progress
REST-
queued errorin-progress
nginxnginx nginxnginx
apianapian apianapian apianapian
API service
nginxnginx
RESTish JSON over HTTPS
. . .. . .
nginxnginx nginxnginx
apianapian apianapian apianapian
nginxnginx
RESTish JSON over HTTPS
. . .. . .
nginxnginx nginxnginx
apianapian apianapian apianapian
nginxnginx
RESTish JSON over HTTPS
. . .. . .
pymongo
pymongo
nginxnginx nginxnginx
apianapian apianapian apianapian
nginxnginx
RESTish JSON over HTTPS
. . .. . .
pymongo
pymongo
RESTish JSON over HTTPS
nginxnginx nginxnginx
skysky skysky skysky
nginxnginx
HTTPS + AJAX/JSON
. . .. . .
Browser
curious yellow
RESTish JSON over HTTP
wintermutewintermute
. . .
curious yellow(front-end)
apian(middle-end, API)
wintermute(back-end)
apian(middle-end, API)
apian(middle-end, API)
apian(middle-end, API)
NoNoSQL?
wintermute wintermute
transactional eventual consistency
wintermute
eventual consistency
“It's better to ask for
forgivenessthan for
permission.”
“Better safethan sorry.”
wintermute(back-end)
storm
Not quite real-time
Not quite map-reduce-able
Too rigid
Too complex
HTTP/JSONHTTPCache-Control: max-age=300, must-revalidateLast-Modified: Tue, 2 Oct 2012 11:00:00 GMT
store
schema freeexternal JSON schema
The datasets saga
a b c d e f
columns: { a: [ ] b: [ ] c: [ ] d: [ ] e: [ ] f: [ ]}
MR