Consulting Manager, 10gen Inc Richard Kreuter Machine to Machine: Managing Mechanically Generated Data with MongoDB
May 12, 2015
Consulting Manager, 10gen Inc
Richard Kreuter
Machine to Machine: Managing Mechanically
Generated Data with MongoDB
Agenda
• What's Machine to Machine Data?• What's MongoDB?• Some use cases• Next steps
What’s Machine to Machine About?
Why this is game-changing
• Resource tracking• Process optimization• Market analysis• Real-time decision making• etc.– (… including things we haven’t thought of yet…)
Why this is technically challenging
• Massive volumes of data• High data ingestion rates• Complex data analysis requirements• Evolving data modeling needs
What’s MongoDB About?
MongoDB is a ___________ database
• Open source• High performance• Full featured• Document-oriented• Horizontally scalable
Full Featured
• Dynamic (ad-hoc) queries• Built-in online aggregation• Rich query capabilities• Traditionally consistent• Many advanced features• Support for many programming languages
Document-Oriented Database
• A document is a nestable associative array
• Document schemas are flexible
• Documents can contain various data types (numbers, text, timestamps, blobs, etc)
Horizontally Scalable (Sharding)
Replication within a shard
Use Case #1: Keep on Trucking
Suppose you’ve got a lot of trucks
• You probably care about where they are
• You probably care if they’re on schedule
• You might also care what they’re carrying (cargo and/or fuel)
What are these data?
• Vehicle tracking data is positional info available via GPS
• Cargo/fuel might be continuous sensor data (e.g., volume or weight) or inventory (e.g., RFID)
What MongoDB can do here
• For GPS type data, MongoDB has long had powerful geospatial query facilities:
– “Find all trucks within a region (within a certain time range, perhaps)”
– “Find all trucks within 100km from the warehouse/customer”
Recent additions to GeoSpatial
• MongoDB version 2.4 introduces support for indexing and querying on various GeoJSON data types (polygons, line strings)
– “Find trucking routes that intersect”
– “Find routes that pass nearby to a customer”
What about the cargo/fuel?
• Sensor data is an easy fit for MongoDB
• Cf. MMS, the MongoDB Monitoring Service– Cloud service hosted by 10gen since 2011– On the order of 1M measurements written per
second (i.e., 10B writes/day)– Continuous data rendering/alerting/analysis of
ingested data, 24x7
Use Case #2: Keeping Cool
Suppose you’ve got a building
• You probably keep it climate controlled…
• … and lit …
• … and perhaps secured with entry cards.
So you measure things
• Temperature readings and/or HVAC utilization reports
• Lights on/off
• Swipe-ins/swipe-outs through secured doors
This is “just” sensor data
• Straightforward to store in MongoDB documents
• With strategic document design, a single server can save hundreds of thousands of sensor reads per second
But how do you use this data?
• MongoDB has a built-in Aggregation Framework that supports ad-hoc analysis tasks over data sets
– “What rooms had the highest average Air Conditioning utilization bracketed daily?”
– “Which secured doors have the most ‘pass-back’ problems?”
Replication supports analytics
• E • Queries can be parceled out to different replicas
• In different DCs, say,• Or to segregate
competing workloads
Use Case #3: Who are the clients in your neighborhood?
Suppose you’ve mobile customers
• You might know where they are during their day
• You might want to pitch them offers while they’re there– Or perhaps notify partners in real-time
• And you might evolve your model progressively
Tracking locations is straightforward
• We’ve already discussed GPS data– Recording positional information– Geospatial queries for proximity etc.
• Though mobile customers might have more interesting locational data than just GPS– e.g., user U is in retail store S now– or, user U1 is near to U2 now
Evolving your customer model
• One interesting thing about people is that they differ
• And you might choose to pay attention to changing things as your business evolves
• MongoDB makes this easy
Flexible Schema supports Agility
• As you become interested in new data, you include that data in your documents– No laborious data migration processes are
necessary – just adapt the application models to record new information
– Queries and indexes work over polymorphic data models, too
What’s next?
Going further
• Where do you build your next warehouse/place your next office?
• How should you staff your retail spaces/factory floors?
• What will next year’s smart home products enable?
To recap
• MongoDB is widely used for machine-generated sensor, GPS, inventory, and other kinds of data
• Dynamic schema, rich query facilities, built-in analytic features, replication and horizontal scaling simplify M2M architectures
Consulting Manager, 10gen
Richard Kreuter
Questions?Next webinar:
How Financial Firms Create a Single Customer View with MongoDB
http://bit.ly/mgoSCV