Preventing Bugs Putting Accumulo To Work In The Field Russ Weeks, PHEMI Orchard
Jul 15, 2015
Preventing BugsPutting Accumulo To Work In The Field
Russ Weeks, PHEMI
Orchard
Precision Agriculture
Sensors
Gateways
Why We ❤ Accumulo
• Continuous Aggregations
• Time-series Data
• Spatiotemporal Data
Farming is Risky!
Adrian Baggaley, FruitForum
Horticultural Sciences Department, University of Florida
USDA
Aggregation Schema
Time-Series Data
• Sensor messages arriving in constant, high-velocity stream
• Common use case: “Show me the last 2 weeks of data for sensor X”
No problem!
Row CF CQ Val
M0 msg timestamp 2014/07/01T19:30
M0 msg sensor_id 1138
M0 msg msg_id 1138:2014/07/01T1930
M0 dev battery 3.3v
M0 env temp 20.3c
M0 env windspd 2.96m/s
Lexicode + Transpose…
Row CF CQ Val
msg:msg_id:1138:2014/07/01T2020 M842 1
msg:msg_id:1138:2014/07/01T2010 M767 1
msg:msg_id:1138:2014/07/01T2000 M684 1
msg:msg_id:1138:2014/07/01T1950 M592 1
msg:msg_id:1138:2014/07/01T1940 M356 1
msg:msg_id:1138:2014/07/01T1930 M0 1
• But… my old database could dereference those message IDs automatically!
• All I had to do was this join, see…
Time-Series Index
Row CF CQ Val(sensor_id:1138,2014/07/01T2020,battery,3.3v) […] M842 1(sensor_id:1138,2014/07/01T2020,windspd,1.83m/s) […] M842 1(sensor_id:1138,2014/07/01T2020,temp,18.6c) […] M842 1(sensor_id:1138,2014/07/01T2010,battery,3.2v) […] M767 1(sensor_id:1138,2014/07/01T2010,windspd,1.51m/s) […] M767 1(sensor_id:1138,2014/07/01T2010,temp,18.8c) […] M767 1
Spatiotemporal
Map Tiles: OpenStreetMap
Pos. 0 Pos. 212-1
Spatiotemporal
OK, But Why Not HBase?
• Iterators are awesome
• Customer is interested in cell-level security
• Future applications may involve controlled data-sharing
Thanks!