Rick Negrin, Director of Product Management, MemSQL March 3, 2017 Enabling Real-Time Analytics for IoT Building Real-Time Data Pipelines with Kafka and MemSQL
Rick Negrin, Director of Product Management, MemSQL
March 3, 2017
Enabling Real-Time Analytics for IoTBuilding Real-Time Data Pipelines with Kafka and MemSQL
Data Producers(simulating
sensor activity)
User Interface
Architecting for Real-Time Analytics
Database
...
Data Transformation
Message Queue
7
Real-Time Architecture and Ecosystem
Real-Time Data Warehouse
Streaming IngestReal-time
data pipelines
Live DataMemory optimized
tables
Historical DataDisk optimized
tables
ApplicationsEmbed custom visualizations and reports for analytic applications
BI DashboardsTableau, Looker, Zoomdata,
MicroStrategy, and Qlik
On-Premises or CloudFreely deploy on- premises, Amazon AWS, Microsoft Azure, Google Cloud Platform and beyond
Containers and VMsDeploy on virtualization and container services like Docker,Mesosphere, VMware, Kubernetes, Cisco, Dell and HP
Historical DataHDFS, Amazon S3, SQL Server,
SAP, Oracle, MySQL
Real-Time DataOperational Appsand web services
Messaging andTransforms
Kafka and Spark
MemSQL is the SQL database capable of ingesting millions of events per secondwhile analyzing petabytes of data for industry-leading time to insight results
8
CommunityFree Forever
EnterpriseFree 30-day Trial
CloudManaged Service
▪ Unlimited capacity and scale
▪ Comprehensive SQL features
▪ 24/7 support▪ Enterprise functionality
▪ High availability▪ Cluster replication▪ Granular access
controls▪ Security features
▪ Fully managed▪ Enterprise functionality▪ Currently in
Pre-Release(GA in 2017)
MemSQL Editions
Data Producers(simulating
sensor activity)
User Interface
Architecting for Real-Time Analytics
MemSQL
...
Data Transformation
Message Queue
MemEx: IoT Showcase Application
- Combines MemSQL, Apache Kafka, and Spark for global supply chain management
- Enables enterprises to predict throughput of supply warehouses
- Processes 2 million data points, based on 2,000 sensors across 1,000 warehouses
Data Producers(simulating
sensor activity)
MemEx UI (OpenMaps)
MemEx Architecture
...
Data Transformation
Apache Spark
Spark MLlib Predictive Model
Raw Sensor 1 + Predictive Score 1
S1 P1
1