© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
(SDD416) Amazon EBS Deep Dive | AWS re:Invent 2014
Jun 30, 2015
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Amazon Elastic Block Store Deep Dive
• Larger and faster volumes
EBS !=
A few definitions…
• IOPS: Input/output operations per second (#)
• Throughput: Read/write rate to storage (MB/s)
• Latency: Delay between request and completion (ms)
• Capacity: Volume of data that can be stored (GB)
• Block size: Size of each I/O (k)
When performance matters, use SSD-backed volumes
• New default volume type for Amazon EBS
• Every volume can burst up to 3,000 IOPS
- Larger volumes can burst for longer periods
• 3 IOPS per GB baseline performance
- Maximum of 3,000 on a 1TB volume
• 99% performance consistency
(2) Max IO credit per bucket is 5.4M
(1) Always accumulating 3 IOPS per GB per
second
(3) You can spend up to
3000 IOPS per second
13
14
15
16
Volume Type Boot Time Access Time OS
GP2 3:31 4:33 Windows Server
2012
Standard 4:30 7:16 Windows Server
2012
GP2 0:36 0:45 CentOS6
Standard 0:57 1:16 CentOS6
40% Reduction in boot times by using General Purpose (SSD)
80% Cost Savings. 50% more peak I/O with
General Purpose (SSD)
19
• Best for I/O intensive databases that require highest
consistency
• Provision up to 4,000 IOPS per volume
• Supports IOPS to GB ratio of 30
• Designed to offer 99.9% consistency
• Best for cold workloads
• Rarely accessed data that needs always on access
• IOPS: ~100 IOPS steady-state, with best-effort bursts
• Throughput: variable by workload, best effort to 10s of MB/s
• Latency: Varies, reads typically ~20-40 ms, writes typically ~2-10ms
General Purpose (SSD) Provisioned IOPS (SSD) Magnetic
Recommend use casesBoot volumes Small to med DBs Dev and test
I/O intensive Large DBs
Cold storage
Storage media SSD-backed SSD-backed Magnetic-backed
Volume size 1GB- 1TB 1GB- 1TB 1GB- 1TB
Max IOPS per volume3 IOPS/GB Burst up to 3000
IOPS4,000 IOPS ~100 IOPS
Read and write peak throughput 128 MB/s 128 MB/s ~50-90 MBps
Max I/O per node(16k) 48,000 48,000 48,000
Peak throughput node 800 MB/s 800 MB/s 800MB/s
Latency (random read) 1-2ms 1-2 ms 20-40 ms
API name gp2 io1 standard
Price* $.10/GB-Month$.125/GB-Month
$.065/provisioned IOPS$.05/GB-Month $.05/ 1M I/O
Always use General Purpose (SSD) for boot volumes
• L A W– L
– A
– W
1. IOPS: Read/write I/O rate (IOPS)
2. Latency: Time between I/O submission and completion (ms)
3. Throughput: Read/write transfer rate (MB/s)
– Throughput = IOPS X I/O Size
(1) Amazon EC2
(2) I/O
(4) Amazon
EBS
(3) Network
link
1. EC2 instance: Network bandwidth(Mbps)
2. EBS-optimized: EC2 instance option (On/Off)
3. Workload: Block size, read/write ratio, serialization
4. Queue depth: The number of outstanding I/Os
5. RAID: Stripe volumes to maximize performance
6. Pre-warm: Eliminates first touch penalty
1. Amazon EC2 instance• Compute optimized– C3
• Memory optimized- R3
• General Purpose – M3
Amazon
EBS
Amazon
EC2
Select Amazon EC2 instance that has the right network, RAM, and
CPU resources for your applications.
2. Amazon EBS-optimized
b
Use EBS-optimized instances for consistent EBS
performance
3. Workload
EBS SSD-backed volumes measure I/O size up to 256KB
EBS SSD-backed delivers same performance for read and write
NEW
4,000 IOPS
PIOPS volume
4,000 IOPS
128MB/s throughput
You can achieve 4,000 IOPS
when driving smaller I/O
You can achieve up to 128MB/s
when driving larger I/O
4,000 IOPS
PIOPS volume
4,000 IOPS
128MB/s throughput
4,000 X 4KB = 16MB/s
4,000 X 32KB =128MB/s
8,000 x 8KB = 64MB/s
4,000 x 8KB = 32MB/s
2,000 x 128KB=256MB/s
1,000 x 128KB =128MB/s
Block (I/O) size determines whether your application is IOPS bound or
throughput bound
4. Queue depth
An I/O
Amazon
EBSAfter it’s gone, it’s gone
Amazon
EC2
Queue depth is the pending I/O in flight
41
I/O Latency
0.075
35.1
0
5
10
15
20
25
30
35
40
1 4 8 12 16 20 24 28 32
La
ten
cy T
P9
0 (
ms
)
Queue Depth
Random READ latency across various QDs
Latency (TP90)
Read latency linearly increases with increase in queue depth
0.075
35.1
2.09
1,865
4,152
3,851
-
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
0
5
10
15
20
25
30
35
1 4 8 12 16 20 24 28 32
La
ten
cy
TP
90
( m
s)
Queue Depth
16 KB random READ IOPS, Latency across various QDs
Latency (TP90) Avg Read IOPS
IOP
S
Queue depth of one has the lowest latency, but has the lowest IOPS
0.075
35.1
2.09
1,865
4,152
3,851
-
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
0
5
10
15
20
25
30
35
1 4 8 12 16 20 24 28 32
La
ten
cy
TP
90
( m
s)
Queue Depth
16 KB random READ IOPS, Latency across various QDs
Latency (TP90) Avg Read IOPS
IOP
S
Queue depth between 4-8 has the optimal IOPS and latency performance
0.075
35.1
2.09
1,865
4,152
3,851
-
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
0
5
10
15
20
25
30
35
1 4 8 12 16 20 24 28 32
La
ten
cy
TP
90
( m
s)
Queue Depth
16 KB random READ IOPS, Latency across various QDs
Latency (TP90) Avg Read IOPS
IOP
S
Higher queue depths have negative impact on IOPS and latency
0.08
7.71
845
4,152
0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
0
1
2
3
4
5
6
7
8
9
10
1 4 8 12 16 20 24 28 32
La
ten
cy
TP
90
( m
s)
Queue Depth
16 KB random WRITE IOPS, Latency across various QDs
Latency (TP90) AvgIOPS
IOP
S
Write latency, queue depth and IOPS interaction is similar to
that of read latency
Optimal queue depth to achieve lower latency and highest IOPS is
typically between 4-8; ~1 QD per 500 IOPS
Amazon EBS-optimized offers consistent latency experience
5. RAID• Increase performance or capacity or both
• Stripe multiple volumes to achieve 10’s of TB and up-to 48,000 IOPS per Instance
• Don’t mix volume types
• Typically RAID 0 or LVM stripe
• Avoid RAID for redundancy Amazon
EBS
Amazon
EC2
Maximum performance per instance
12×400 GB PIOPS, pre-warmed, RAID 0 LVM, Stripe size 128 KB, attached to CR1 instance
Use stripe size of 128KB or 256KB
6. Pre-warming
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-prewarm.html
Use large block size to speed up your pre-warming.
ex: sudo dd if=/dev/xvdf of=/dev/xvdf conv=notrunc bs=1M
Final tips
• Ext4 or XFS (understand journal impact!)
• Amazon EBS encryption has no visible IOPS
performance impact for many Amazon EC2
instances
• Alignment can matter, check tools use 4k
Workload/
Software
Typical block
size
Random/
Seq?
Max EBS @ 500
Mb/s
Max EBS @
1 Gb/s
Max EBS @ 10Gb/s
instances
Oracle DB Configurable:2 KB –16
KB
Default 8 KB
random ~7800 IOPS ~15,600 IOPS ~96,000 IOPS
Microsoft SQL
Server
8 KB w/ 64 KB
extents
random ~7800 IOPS ~15,600 IOPS ~80,000 IOPS
MySQL 16 KB random ~4000 IOPS ~7,800 IOPS ~48,000 IOPS
PostgreSQL 8 KB random ~7,800 IOPS ~15,600 IOPS ~96,000 IOPS
MongoDB 4 KB serialized ~15,600 IOPS ~31,000 IOPS ~96,000 IOPS
Cassandra 4 KB random ~15,600 IOPS ~31,000 IOPS ~96,000 IOPS
GlusterFS 128 KB sequential ~500 IOPS ~1000 IOPS ~6,000 IOPS
EBS-optimized
Oh, YEAH!!
Amazon
EC2
A “boatload” of I/O
Right sized EBS
1. EC2 instance: Network transfer rate (Mbps)
2. EBS-optimized: EC2 instance option (On/Off)
3. Workload: Block size, read/write ratio, serialization
4. Queue depth: The number of outstanding I/Os (#)
5. RAID: Stripe volumes to maximize performance
6. Pre-warm: Eliminates first touch penalty
Performance (per volume)
GP2 PIOPS GP2 PIOPS
Max Size 1 TB 1 TB 16 TB 16 TB
Max IOPS 3,000 4,000 10,000 20,000
Max Throughput 128 Mbps 128 Mbps 160 MBps 320 MBps
PREVIOUSLYW/ LARGER &
FASTER VOLUMES
Amazon EBS larger and faster volume types
General Purpose (SSD) Provisioned IOPS (SSD) Magnetic
Recommend use casesBoot volumes Small to med DBs Dev and test
I/O intensive Large DBs
Cold storage
Storage media SSD-backed SSD-backed Magnetic-backed
Volume size 1GB- 16TB 4GB- 16TB 1GB- 1TB
Max IOPS per volume 10,000 IOPS 20,000 IOPS ~100 IOPS
burst < 1TB to 3000 IOPS baseline baseline
Read and write peak throughput 160 MB/s 320 MB/s ~50-90 MBps
Max IOPS per node (16k) 48,000 48,000 48,000
Peak throughput node 800 MB/s 800 MB/s 800MB/s
Latency (random read) 1-2ms 1-2 ms 20-40 ms
API name gp2 io1 standard
Price* $.10/GB-Month$.125/GB-Month
$.065/provisioned IOPS$.05/GB-Month
$.05/ 1M I/O
64
66
• 2-tiered key hierarchy using envelope
encryption
• Unique data key encrypt customer data
• AWS KMS master keys encrypt data keys
• Benefits of envelope encryption:• Limits risk of a compromised data key
• Better performance for encrypting large data
• Easier to manage a small number of master
keys than millions of data keys
Master Key(s)
Data Key 1
S3 object EBS
volume
Redshift
cluster
Data Key 2 Data Key 3 Data Key 4
Custom
application
AWS KMS
Select the right volume for your workload
Select the right instance for your workload
Take snapshots
Use encryption if you need it
Please give us your feedback on this session.
Complete session evaluations and earn re:Invent swag.
http://bit.ly/awsevals
Related Documents
