Exo-clones Instantaneous Change Tracking for the Whole File System Richard P. Spillane, Wenguang Wang, Luke Lu, Maxime Austruy, Christos Karmanolis, Rawlinson Rivera May 10 th , 2016 VDFS (POSIX) VSAN VDFS (POSIX) Developer Laptop Backups, Documents, Applications Exo Support Exo Support 1 Hot Storage 2016
22
Embed
Exo-clones - USENIX · Network Overhead of Docker FS Layers vs. Exo-clones Transfer size of Docker FS layers compared to Exo-clones using whole-file dedup and Exo-clones using either
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Exo-clonesInstantaneous Change Tracking for the Whole File SystemRichard P. Spillane, Wenguang Wang, Luke Lu, Maxime Austruy, Christos Karmanolis, Rawlinson Rivera
May 10th, 2016
VDFS (POSIX)
VSAN
VDFS (POSIX)
DeveloperLaptop
Backups, Documents,Applications
Exo Support Exo Support
1Hot Storage 2016
rsync/diff -R
Exo-clone
What is an Exo-clone?
2
A
FS SnapshotUpdatedSnapshot
A’
File Data
Meta Data A
FS SnapshotUpdatedSnapshot
A’
File Data
Meta Data
VDFS on VSAN VDFS on Developer Laptop
Copy of A’
Scan whole FS
• “free” to store• “free” to create• “free” to import
skipped
Hot Storage 2016
Benefits of Exo-clone
3
No duplication
No FS Scanning
diff -R ✖ ✖
git pack-objects ✔ ✖
Docker FS layers ✖ ✔
ZFS/BtrFS send/receive ✖ ✔
Exo-clones ✔ ✔
Hot Storage 2016
Exo-clone Use Cases
4
Changes in last hour
Application changes
Backup/replication
Application Revision Control
Improve with Exo-clones?
Hot Storage 2016
A Common Problem…
5
MyApp 1.6
Sources…git
Images, 3D models,icons, sounds, …?100sMB—10sGB
OS & Libraries…Docker
Lots of changes to large andsmall files
Backup MyApp
Hot Storage 2016
git? Sadly, no…
6
WORKING OBJECT STORE
A B
Redundant copy, 5-10x bigger thancurrent working state
Network Overhead of Docker FS Layers vs. Exo-clonesTransfer size of Docker FS layers compared to Exo-clones using whole-file dedup and Exo-clones using either block or 6-9KB variable length chunking
Summary• Instantly create and share diffs between any two FSes with
– No space overhead– No sending the same block twice over the wire
• Better backend for Docker, rsync, git• Easy but efficient backup/replication between any storage environment• What becomes simpler when we make revision control on the whole FS cheap and standard?
– What if every FS (e.g., ZFS, BtrFS, LVM, NASes) supported exo-clones?