Top Banner
15

Flightcaster_HUG

Jan 17, 2015

Download

Documents

guest3537ac

Flightcaster and Hadoop
Hadoop Bay Area User Group, March 24
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Flightcaster_HUG
Page 2: Flightcaster_HUG
Page 3: Flightcaster_HUG

75% of major delays     

less than 30 min. before departure

Page 4: Flightcaster_HUG
Page 5: Flightcaster_HUG
Page 6: Flightcaster_HUG

hadoopcascading     ->  clojure

infer cascading-clojurecrane

Open Source

Page 7: Flightcaster_HUG
Page 8: Flightcaster_HUG
Page 9: Flightcaster_HUG

Discrete Event Simulation?

Fuzzy Join with funky predicate?

Page 10: Flightcaster_HUG

group-by with secondary sort.

accumulator to simulate a discrete event simultor.

Page 11: Flightcaster_HUG
Page 12: Flightcaster_HUG

cascading-clojure: motivation

FP

dev

deploy

Page 13: Flightcaster_HUG

cascading-clojure: lessons learned

planner

serialization

data structures for wide data

Page 14: Flightcaster_HUG

cascading-clojure: future direction

functions + data structures

abstractions for planner and M/R

Page 15: Flightcaster_HUG