PostgreSQL on ZFS Lightning Talk

PostgreSQL on ZFSBits Hopefully Structured the way you Left Them: The scary parts followed by a Happy Ending™.

“All good lightning talks have a little FUD.”

… except it’s not FUD.

… and few care or are aware (thank you cloud providers!)

Cloud providers have duped a generation of operators into thinking bitrot does not exist.

It does. The reality has been hidden from plain sight, but it’s there… lurking… silently, waiting…

–Many (myself included)

"If you are not using ZFS, you are losing data."

This has been studied recently for SSDs…

ECC on Flash works until it doesn’t.

Probability will rip your face if you do not acknowledge its existence and its control over your life.

TL;DR: 4.2% -> 34% of SSDs have one UBER per year

TL;DR: 4.2% -> 34% of SSDs have one UBER per year

How many SSDs in that shiny box 'ya got there?

How many boxes are running?

TL;DR: 4.2% -> 34% of SSDs have one UBER per year

How many SSDs in that shiny box 'ya got there?

How many boxes are running?

(1-(1-uberRate)^(numDisks)) = Probability of UBER/server/year

(1-(1-0.042)^(20)) = 58%(1-(1-0.34)^(20)) = 99.975%

External Factors for UBER on SSDs:

• Temperature • Bus Power Consumption • Data Written by the System Software • Workload changes due to SSD failure

In a Datacenter…

…no-one can hear your bits scream.

Well, maybe they can…

Take care of your bits. Your bits are taking care of you.

Take care of your bits. Your bits are taking care of you.

Answer their cry for help.

Similar studies and research exist for:

• Fibre Channel • SAS • SATA • Tape • SANs • Cloud Object Stores

"Groupon’s Deal on FreeBSD" FreeBSD Journal, July/August 2015

https://www.freebsdfoundation.org/past-issues/freebsd-in-the-enterprise/ https://www.freebsdfoundation.org/wp-content/uploads/2015/12/vol2_no4_groupon.pdf

PostgreSQL and ZFS were meant for each other

PostgreSQL and ZFS were meant for each other

• Because bitrot happens

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast

2-4µs/pwrite(2)!!

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast

P.S. This was observed on 10K RPM spinning rust.

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast • Because restoring from backups is AWFUL!

# zfsnap snapshot -rv -a 25h tank/pgdata # zfs list -r -t snapshot# zfs rollback -r tank/pgdata@hourly-2016-09-14_14.52.00—25h

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast • Because restoring from backups is AWFUL!

# zfsnap snapshot -rv -a 25h tank/pgdata # zfs list -r -t snapshot# zfs rollback -r tank/pgdata@hourly-2016-09-14_14.52.00—25h

This happens in seconds! It’s YUGE people, absolutely YUGE!

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast • Because restoring from backups is AWFUL! • Because compression is a performance win,

even on SSDs

# zfs set compression=lz4 tank/pgdata

(wtb publishing of benchmarks any year now… you know who you are…)

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast • Because restoring from backups is AWFUL! • Because compression is a performance win,

even on SSDs • Because compression is a space win (2.2:1

compression for most PG data)

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast • Because restoring from backups is AWFUL! • Because compression is a performance win,

even on SSDs • Because compression is a space win (2.2:1

compression for most PG data) • Because zfs snap; zfs send; ssh… ; zfs recv

PostgreSQL and ZFS were meant for each other

• Because bitrot happens • Because its fast • Because restoring from backups is AWFUL! • Because compression is a performance win,

even on SSDs • Because compression is a space win (2.2:1

compression for most PG data) • Because zfs snap; zfs send; ssh… ; zfs recv • Because caching compressed records is win

https://www.illumos.org/issues/6950

It even runs on Linux!

It even runs on Linux!

Don’t become cranky and jaded and relearn the

perils of data sadness.

Winning Strategy

Winning Strategy

Be aware. But take the Blue Pill.

+

Not running PostgreSQL on ZFS is tantamount to professional negligence. Discuss.