RozoFS: a fault tolerant I/O intensive distributed file system based on Mojette erasure code Workshop Autonomic Oct. 16-17 2014 Laas, Europe, Toulouse Benoît Parrein, Dimitri Pertin*, Nicolas Normand, Didier Féron° Université de Nantes, IRCCyN Lab, UMR 6597 * Université de Nantes - Fizians SAS °Fizians SAS
24
Embed
RozoFS: a fault tolerant I/O intensive distributed file ...projects.laas.fr/.../parrein_slides.pdf · RozoFS: a fault tolerant I/O intensive distributed file system based on Mojette
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
RozoFS: a fault tolerant I/O intensive distributed file system based on Mojette erasure code
Workshop Autonomic Oct. 16-17 2014
Laas, Europe, Toulouse
Benoît Parrein, Dimitri Pertin*, Nicolas Normand, Didier Féron°
Université de Nantes, IRCCyN Lab, UMR 6597
* Université de Nantes - Fizians SAS
°Fizians SAS
2
Outline
Storage and file systems
FEC4Cloud project
Mojette Erasure code
RozoFS global architecture
Performances
Workshop Autonomique, Oct. 16-17 2014, Toulouse, France - Benoît Parrein, Université de Nantes, IRCCyN Lab
3
The storage in the world
40 Exabytes (1018 bytes) stored in 2020
15 EB (37%) in the Cloud(s)
7,5 EB (50%) video, images, ...
4
High availability means...
99.999999...% reachable
Copies and copies and copies... (up to 7 times)
Hard disks and hard disks and hard disks...
High consumption of energy
Privacy problems
Workshop Autonomique, Oct. 16-17 2014, Toulouse, France - Benoît Parrein, Université de Nantes, IRCCyN Lab
5
6
Requirements
Convergence archiving, big data, virtualisation, hpc,...
=
Convergence of cold and hot data
Workshop Autonomique, Oct. 16-17 2014, Toulouse, France - Benoît Parrein, Université de Nantes, IRCCyN Lab
7
Distributed File Systems
HDFS
CephFS, GlusterFS,...
Scality (based on Chord)
Facebook file system (f4)
...
Mix of replicas and erasure coding (sometime): – none uses erasure codes always
Workshop Autonomique, Oct. 16-17 2014, Toulouse, France - Benoît Parrein, Université de Nantes, IRCCyN Lab