Bigdata analytics and our IoT gateway

2 May 2023 1

Aemro Amare, Freek van Gool

Bigdata Analytics and our IoT Gateway :Toon

Agenda• A little bit about us• Who is Quby & what is Toon?• Our big data journey• Data collection & services• Architecture & technologies• Demo• Questions

@javafreekNL @twoxey

A little bit about us

Freek Aemro

@javafreekNL @twoxey

First attemptHome control box

@javafreekNL @twoxey

PivotEnergy stick

What is Toon?

Meter Adapter

Gas Sensor

Electricity Sensor

Z-Wave

Phillips Hue

WiFi Router

Z-wave

Smart plugand smoke detectors

Central heating boiler

Boiler adaptor

Toon®

Toon connections - Data sources

The service center

Service Center

Toon Displays

Mobile devices

ProductApplication

s

Back OfficeApplication

s

Mobile Backend

Internet

Internet

VPNAccess

Data collector

Firstpartnership

Exponential growth

2013 30.0002014 90.0002015 190.0002016 300.0002017 (1.000.000)

2013 2014 2015 2016 2017

2 May 2023 12

Our journey towards bigdata 2014

• 30K customers • 40 types of

sensor data from 3K customers

• up to 1 minutes resolution

2012

• 4K customers • Aggregated

Energy data collected(DAY,WEEK,MONTH)

• Benchmarking functionality for customers

2013

• 90K customers • Data collection

from 30K customers

• Event and Measurement data

2 May 2023 13

Steps moving towards bigdata(cont’d)2017

• 300K customers • Near Real Time

aggregation with segmentation

• 400 signal/5 minutes

• Sharing Measurement data with tenants

2015

• 190K customers • Apache storm for

Near Real Time Energy consumption (3K customers)

• Data analysis using Cloudera stack

• Up to 10 seconds resolution

2016

• 330K customers• Collect data

from 230K customers anonymously

• Data as a service

• Analytics platform as a service

Toon®

Types of data

More than 300 types of sensor and user interaction data. If you want to know more please contact us : [email protected]

Energy CH-boiler (OT) Thermostat

Gas consumption

Electricity consumption

Water temperature

Service messages

Burner information

Room temperatureThermostat

programManual settings

Plug-dataElectricity

consumptionOn/off

Water pressure

Energy production

Energy feed-in

Boiler status

Touch eventsThermostat

Mobile

1 2 3 4 5

Thermostat setting

Hue data

2 May 2023 15

Anonymous Data Collection Service Center Customer

Always anonymous data collected

Customer may opt in for services and

send keys+ =

>Customer may opt out and

change keysData can’t be de-anonymized

again

2 May 2023 16

Sample Services with Toon’s Data?1. Benchmarking with daily energy meter data

2 May 2023 17

Sample Services with Toon’s Data? ( cont’d)2. Nearly Real Time Forecasting : Energy data Aggregation

2 May 2023 18

Sample Services with Toon’s Data? ( cont’d)3. Home Appliances Energy Consumption Detection: Energy Data disaggregation

2 May 2023 19

Sample Services with Toon’s Data ?(cont’d) 4. Boiler Maintenance Prediction

(KetelIQ)

2 May 2023 20

Sample Services with Toon’s Data? ( cont’d)5. Customer’s Behavioral studies

Click data Thermostat programs and co2 emission

21

What are we collecting? 24K measurements / day / customer 600KB (zipped) 230K customers

22

Arch. Principle : lambda Arch.

2 May 2023 23

Messaging

Data warehouse

Tools and Technologies

Analytic Clusters

Tools and Frameworks

Containers & micro services

Mon

itorin

g an

d al

ert

24

Data flow (partial view)

2 May 2023 27

Things to mention • ssl mutual authentication• raw-data stored partitioned in better way• Customers can control access to their own

data

2 May 2023 29

What are we doing with all that data?• Firehose – data as a service for B2B (and

researchers)• Analytic platform as a service• New data driven services for Toon users• Improve existing services• Marketing strategy (understand your customers)

2 May 2023 30

2 May 2023 31