Dell Customer Communication - Confidential Unstructured Storage Management im Scale-out Petabyte-Zeitalter Matthias Meppen Advisory SE, Unstructured Data Solutions 07.06.2019, Gut Ullrichshusen
Dell Customer Communication - Confidential
Unstructured Storage Management im Scale-out Petabyte-Zeitalter
Matthias Meppen
Advisory SE, Unstructured Data Solutions
07.06.2019, Gut Ullrichshusen
Dell - Internal Use - Confidential
2 of Y
Dell Customer Communication - Confidential
Key Digital Transformation Initiatives
D I G I T A L E X P E R I E N C E S
I N T E R N E T O F T H I N G S
M L / A I B I G
D A T A
D AT A P O W E R S T H E M AL L
Dell - Internal Use - Confidential
3 of Y
Dell Customer Communication - Confidential
What is Data Capital?
D A T A
Human Capital Intellectual Property
Operations Infrastructure
T R A D I T I O N A L A S S E T S
Data is rapidly becoming organizations’ most valuable asset
Dell Customer Communication - Confidential
CHALLENGE:
Transforming Data Into Value
C R E AT I N G B U S I N E S S
I M PA C T F R O M D ATA
K E E P I N G U P W I T H
D ATA G R O W T H
U N L O C K I N G D ATA
T R A P P E D I N S I L O S
Dell Customer Communication - Confidential
D I G I TA L
E X P E R I E N C E S
I N T E R N E T
O F T H I N G S
• Smart sensors
• Connected homes
• Intelligent factories
• Self-service kiosks
• Logistics and supply chain
• Mobile/Web Apps
• E-commerce
• Social Media
• AR/VR
• Sharing economy
Where Data Capital Impacts your Business
B I G D ATA
• Targeted advertising
• Real-time analytics
• Business Intelligence
• Predictive analytics
• Performance monitoring
M L / A I
• Natural language processing
• Robotic automation
• Intelligent security
• Chatbots
• Autonomous vehicles
Dell Customer Communication - Confidential
Transformation of
Your Data Environment
Dell Customer Communication - Confidential
N O W
U N S T R U C T U R E D
E X P O N E N T I A L G R O W T H
D I S T R I B U T E D
R E A L - T I M E
T R A D I T I O N A L
S T R U C T U R E D
L I N E A R G R O W T H
C O N TA I N E D
B AT C H
Evolving Data Landscape
Dell Customer Communication - Confidential
3 data types that have different characteristics
80% of Data is Unstructured
F I L E
O B J E C T
S T R E A M
D A T A C A P I T A L
Dell Customer Communication - Confidential
Unstructured Data Vision
U N I F I E D D A T A L A K E
S I M P L I C I T Y A T S C A L E
E X T R A C T V A L U E F R O M D A T A
F I L E S T R E A M
O B J E C T
© Copyright 2017 Dell Inc. 10
Dell Customer Communication - Confidential
File H I G H P E R F O R M A N C E
S C A L I N G C A PA C I T Y &
P E R F O R M A N C E
F I L E
© Copyright 2017 Dell Inc. 11
Dell Customer Communication - Confidential
© Copyright 2017 Dell Inc. 11
11
Dell EMC Scale-Out NAS
S i n g l e F i l e S y s t e m &
O n e N a m e s p a c e
G l o b a l A v a i l a b i l i t y
& P r o t e c t i o n
U n m a t c h e d
S t o r a g e E f f i c i e n c y
S i m p l i c i t y &
E a s e o f U s e
L i n e a r
S c a l a b i l i t y
A u t o m a t e d P o l i c y
B a s e d T i e r i n g
© Copyright 2017 Dell Inc. 12
Dell Customer Communication - Confidential
Consume Scale-Out NAS on Your Own Terms
T U R N K E Y A P P L I A N C E
S O F T W A R E I S I L O N S D
E D G E
D E D I C A T E D C L O U D
( G C P , A W S , A z u r e )
Cloud
Dell Customer Communication - Confidential
Benefits of Scale-Out NAS for Your Data Capital
E A S E O F M A N A G E M E N T
I N V E S T M E N T P R O T E C T I O N
D R A M A T I C O P E X R E D U C T I O N
U N M A T C H E D R E L I A B I L I T Y
& A V A I L A B I L I T Y
One FS
IDC/Gartner: Isilon is THE Scale-Out NAS Leader
Dell - Internal Use - Confidential
Dell Customer Communication - Confidential
Unstructured Data Vision
U N I F I E D D A T A L A K E
S I M P L I C I T Y A T S C A L E
E X T R A C T V A L U E F R O M D A T A
F I L E S T R E A M
O B J E C T
© Copyright 2017 Dell Inc. 15
Dell Customer Communication - Confidential
Object C L O U D S C A L E
S T O R A G E
G L O B A L D ATA A C C E S S
C L O U D - N AT I V E /
M O D E R N A P P S
O B J E C T
© Copyright 2017 Dell Inc. 16
Dell Customer Communication - Confidential
© Copyright 2017 Dell Inc. 16
16
U n l i m i t e d O b j e c t S t o r a g e
E n t e r p r i s e G r a d e
S e c u r i t y a n d C o m p l i a n c e
4 8 % L o w e r T C O
t h a n P u b l i c C l o u d
Dell EMC Object Storage
C o m p r e h e n s i v e
A P I - b a s e d D a t a A c c e s s
N a t i v e M u l t i - t e n a n t
A r c h i t e c t u r e
B u i l t - i n
M e t a d a t a S e a r c h
ECS
Dell - Internal Use - Confidential
Dell Customer Communication - Confidential
Consume ECS on Your Own Terms
T U R N K E Y A P P L I A N C E
S O F T W A R E D E D I C A T E D
C L O U D M U L T I - T E N A N T
C L O U D
Dell - Internal Use - Confidential
Dell Customer Communication - Confidential
Cloud Native Applications
ECS Supports Your Data Driven Workloads
Tiered Archive Sync & Share Cloud Gateway
Cloud Native Apps
(web/mobile) Analytics IoT Cloud Backup
ECS
Traditional Workloads
Consolidate Archive to Lower Costs
Modernize Existing Applications
Accelerate Cloud Native Apps
Dell - Internal Use - Confidential
Dell Customer Communication - Confidential
Unstructured Data Vision
U N I F I E D D A T A L A K E
S I M P L I C I T Y A T S C A L E
E X T R A C T V A L U E F R O M D A T A
F I L E S T R E A M
O B J E C T
© Copyright 2017 Dell Inc. 20
Dell Customer Communication - Confidential
Stream S T R E A M I N G D ATA
I N G E S T
R E A L - T I M E I N S I G H T S
S T R E A M
© Copyright 2017 Dell Inc. 21
Dell Customer Communication - Confidential
Unstructured Storage
Management
Industry Example - ADAS
Data Lifecycle for
ADAS
© Copyright 2017 Dell Inc. 22
Dell Customer Communication - Confidential
Disk Load Station
Data Enrichment & Labeling
Test Cases
Test Results
Analysis, Reporting
& Management
Deep Learning Environment
HiL Server Farm Physical devices
HiL/SiL Test Environments
SiL Server Farm
Sensor Sensor Sensor
ECU ECU ECU
Ingest
File-Object
Archive
1
2 3
4
Legend:
1. Ingest Vehicle Data
2. Data Enrichment
3. Data Management
4. Simulation / AI
5. Archive Data
5
ADAS development lifecycle
Test Vehicle Data: Video
Radar
Lidar
GPS
Ultrasonic
Vehicle Data
And More
© Copyright 2017 Dell Inc. 23
Dell Customer Communication - Confidential
Single Sensor - SAE LEVEL 2 - (200,000 km)
Storage Requirements:
Scalability
Predictable Performance at Scale
Data Management
EXAMPLE OF DATA CAPACITY REQUIREMENTS
Example: A single Front Looking Radar, operating at 2800 MBit/s (raw)
• 2800 Mbits/s 1260GBytes/hr
• 200,000km, 60km/h ave 3333.3 hours
• 1260 GB/h * 3333.3h = 4.2 PB (raw), 0.9 PB (at ECU)
Multi Sensor - SAE LEVEL 2 vs 3 - (200k vs 1M km)
Sensor Fusion Example:
• Multiple sensors (Lidar, GPS, Ultrasonic, Radar, telemetry, etc)
SAE Level 2
4-10 PB 50-100
PB
SAE Level 3
© Copyright 2017 Dell Inc. 24
Dell Customer Communication - Confidential
Potentially hundreds of HIL servers read streams from an Isilon system in parallel
Each stream typically 150-400MB/s (growing with larger resolutions)
Example: 80 HIL servers reading at 200-400 MB/s results in up to 32 GB/s throughput
At same time, new data must also be ingested (multiple GB/s)
Read
Results
32 GB/s Ingest
Single radar sensor
350MB/s
HIL-Server farm Physical Devices
Real time stream
Results
Sensor
ECU
Sensor
ECU
Sensor
ECU
Data performance requirements
© Copyright 2017 Dell Inc. 25
Dell Customer Communication - Confidential
Ho
t T
ier
Warm
Tie
r C
ool T
ier
A2000
A2000
H500
H500
H500
F800
TCO optimization: Policy-based automated tiering
Highest performance
Lowest tco
• Tier Between Performance,
Enterprise and Archive Nodes
• Transparent to Users and
Applications
Just Enough Flash
© Copyright 2017 Dell Inc. 26
Dell Customer Communication - Confidential
Zero Re-Sim SLA
F800 for DL,
H500 for Simulation Data
No Archiving
Medium Re-Sim SLA
F800 for DL,
H500 for Simulation Data
Archiving to A2000 Tier
Long Re-Sim SLA
F800 for DL,
H500 for Simulation Data
Archiving to ECS Object
Ho
t
Tie
r W
arm
Tie
r C
old
Tie
r
ECS
ECS
ADAS SLA Options – 1 day / 7 days / 30 days
Ho
t T
ier
Warm
Tie
r C
ool T
ier
A2000
A2000
H500
H500
H500
F800 F800
Ho
t T
ier
Warm
Tie
r
H500
H500
H500
F800
H500
H500
H500
H500
H500
© Copyright 2017 Dell Inc. 27
Dell Customer Communication - Confidential
Use-Case “ADAS Development”
© Copyright 2017 Dell Inc. 28
Dell Customer Communication - Confidential
Unstructured Data Vision
U N I F I E D D A T A L A K E
S I M P L I C I T Y A T S C A L E
E X T R A C T V A L U E F R O M D A T A
F I L E S T R E A M
O B J E C T
© Copyright 2017 Dell Inc. 29
Dell Customer Communication - Confidential
DataFrameworks - ClarityNow
Data
Visibility
FILE OBJECT
Data
Mobility
Data
Access
Data
Search
© Copyright 2017 Dell Inc. 30
Dell Customer Communication - Confidential
SITE 1 SITE 2 SITE 3
Cloud
FILE Access OBJECT Access
FILE OBJECT
Active file-object access
© Copyright 2017 Dell Inc. 31
Dell Customer Communication - Confidential
CORE Data Centre Hosted Data Lake
CLOUD Expansion
CloudPools - Policy based
automated cloud tiering
SITE 1 SITE 2 SITE 3
Cloud
Archive tiering to cloud