AWS Cost Saving Tip 10: Make right choice between PIOPS vs Std EBS volumes for saving costs

Tip 1: When using EBS PIOPS Volumes saves cost ?
Since OLTP databases are generally IO intensive with random read/write pattern and demand consistent IOPS they are better suited for EBS PIOPS volumes than Standard EBS volumes. At an instant look of the PIOPS pricing table, it may sound expensive compared to Standard EBS volumes, but when you are applying it in right use cases actually you can save better costs than standard EBS volumes.
Let us explore some scenarios on how PIOPS saves cost than Standard EBS Volumes:

Imagine you are running a Database on Amazon EC2 instance with following requirements.

Scenario 1:
Load: Medium load -Read/Write
Type : OLTP
DB : MySQL
IOPS: 1,000
Volume Size : 500 GB
EC2 instance : m1.xlarge
EBS Optimized - Yes

EBS Type	Volume Size	IOPS	IOPS Cost	Volume Price	Overall (USD)
PIOPS	500	1000	100	62.5	~162.5
Standard	500	Best effort	259	50	~309
PIOPS = 500 X0.125 + 1000 X 0.1 = ~162.5 Standard = 500 X 0.1 + (86400 X 30 X 1000 /1000000) X 0.1 = ~309  Standard volumes are assembled in EBS Striping (RAID 0) to get 1000 IOPS in Best effort basis

Savings : You can achieve ~47% cost savings if EBS PIOPS can be used instead of Standard EBS Volumes for OLTP scenario for medium loads


Scenario 2:
Load: Heavy load -Read/Write
Type : OLTP
DB : MySQL
IOPS: 10,000
Volume Size : 1 TB 
EC2 instance : m2.4xlarge
EBS Optimized - Yes

EBS Type	Volume Size	IOPS	IOPS Cost	Volume Price	Overall
PIOPS	1 TB	10000	1000	125	~1125
Standard	10 TB	Best effort	2592	1000	~3592
PIOPS = 1 TB X0.125 + 10000 X 0.1 = ~1125 Standard = (10 X 1TB) X 0.1 + (86400 X 30 X 10000 /1000000) X 0.1 = ~3592 Standard volumes are assembled in EBS Striping (RAID 0 = 1 TB X 10) to get 10000 IOPS in Best effort basis ( not guaranteed)

Savings : You can achieve ~68% cost savings if EBS PIOPS can be used instead of Standard EBS Volumes for OLTP scenario for Heavy loads 

It is clear from the above analysis that if you have OTLP workloads with consistent IOPS requirements then depending upon the Volume Size and number of IOPS required, your savings also proportionately increases if you choose PIOPS volumes over Standard volumes.

Tip 2: When using EBS standard Volumes saves cost ?
When you are designing an online application, it is common that not all the tiers demand consistent IOPS. Example: Your web and application layers usually do not demand consistent IOPS.  Unless you are locally writing lots of data files in web/app EC2 (which is not a advisable practice) you do not need PIOPS volumes for these tiers. For such tiers standard EBS volumes are recommended. Standard volumes are well suited for use as boot volumes for web/app tiers, where the burst I/O capability provides fast instance start-up times (+1 when used in autoscaling). In addition std volumes are better performers for sequential operations. ex: Workloads like OLAP databases, Reporting stores etc which are more sequential, bursty and follows batch-oriented pattern are a better fit for standard volumes.
Let us explore some scenarios on where to use standard volumes and avoid PIOPS overuse and save costs.
Scenario 3:
Load: Low IOPS, boot volumes , bursty workloads
Tier : Web/App 
IOPS: 30-100
Volume Size : 30 GB
EC2 instance : m1.large
EBS Optimized - No

EBS Type	Volume Size	IOPS	IOPS Cost	Volume Price	Overall
PIOPS	30 GB	100	10	3.75	~13.75
Standard	30 GB	30-100	7.7	3	~10.7
PIOPS = 30 GB X0.125 + 100 X 0.1 = ~13.75 Standard = 30 GB X 0.1 + (86400 X 30 X 30 /1000000) X 0.1 = ~10.7

Imagine you 10+ EC2 serving your Web/App Tier then multiply this cost :

13.75 X 10 = ~137.5 USD

10.7 X 10 = ~ 107 USD

Savings : You can achieve ~22% cost savings if EBS volumes can be used instead of PIOPS Volumes for boot/sequential scenarios. 

It is evident that from the above analysis that wisely choosing right EBS volume type saves money. All layers of the application stack will not demand heavy and consistent IO in a larger web scale application; Secondly EBS PIOPS is costlier than standard EBS volumes. Taking both the points into consideration, I usually recommended to mix-match your tiers with standard EBS/PIOPS volumes and other storage options to get better overall cost efficiency and performance in Amazon Cloud.

Other Tips

Cost Saving Tip 1: Amazon SQS Long Polling and Batch requests
Cost Saving Tip 2: How right search technology choice saves cost in AWS ?
Cost Saving Tip 3: Using Amazon CloudFront Price Class to minimize costs
Cost Saving Tip 4 : Right Sizing Amazon ElastiCache Cluster
Cost Saving Tip 5: How Amazon Auto Scaling can save costs ?
Cost Saving Tip 6: Amazon Auto Scaling Termination policy and savings
Cost Saving Tip 7: Use Amazon S3 Object Expiration
Cost Saving Tip 8: Use Amazon S3 Reduced Redundancy Storage  (new)
Cost Saving Tip 9: Have efficient EBS Snapshots Retention strategy in place (new)
Cost Saving Top 10: Make right choice between PIOPS vs Std EBS volumes and save costs (new)

Part 10: Performance Tuning Amazon Elastic Block Store - IO Block Size

The IOPS rate you get usually depends on the I/O size of your applications’ reads and writes. It is very important to know the I/O size your application operates and accordingly expect the performance from EBS Volumes. Provisioned IOPS volumes process your applications’ reads and writes in I/O sizes of 16KB blocks or less. Every increase in I/O size above 16KB will linearly increase the resources you need to achieve the same IOPS rate. For example, if you have provisioned a volume with 2000 IOPS, that means that it can handle 2000 16KB writes per second, or 1000 32KB writes per second, or 500 64KB writes per second, and so on. You can use Amazon CloudWatch to monitor your EBS Volumes throughput and I/O size for a while and decide what best suits for your case. 

Since PIOPS is especially designed for Database context, let us explore how it pans out in Amazon RDS. 

Following are some of the important points to note on PIOPS + RDS context.

P1) The theoretical maximum IOPS rate is also a function of database I/O page size and available channel bandwidth. Each page read or write constitutes one I/O operation. Database operations that read or write more than a single page will use multiple I/O operations for each database operation. For example, for a page size of 16 KB, if a read operation requires 32 KB, you would need 2000 IOPS to achieve 1000 read operations per second.

P2) If you provision an IOPS rate that is higher than the maximum or that is higher than your realized IOPS rate, you may still benefit from reduced latency and improvements in overall throughput.

Applicability of the following points and IOPS rates are based on the m2.4xlarge instance class with full duplex (1000 Mbit/second) and a workload that is perfectly balanced between reads and writes.

·         MySQL uses a page size of 16 KB and can do 12,500 IOPS. On a DB instance with a full duplex I/O channel bandwidth of 1000 megabits per second (Mbps), the maximum IOPS for page I/O is about 6,250 IOPS in each direction for 16 KB I/O. This is for a workload consisting of 50% reads and 50% writes and could reach 12,500 IOPS with 16 KB I/O.

·         Oracle uses a page size of 8KB and can do 25,000 IOPS. On a DB instance with a full duplex I/O channel bandwidth of 1000 megabits per second (Mbps), the maximum IOPS for page I/O is about 12,500 IOPS in each direction for 8 KB I/O. This is for a workload consisting of 50% reads and 50% writes and could reach 25,000 IOPS with 8 KB I/O.

·         SQL Server uses a page size of 8KB and can do 10,000 IOPS. On a DB instance with a full duplex I/O channel bandwidth of 1000 megabits per second (Mbps), the maximum IOPS for page I/O is about 5,000 IOPS in each direction for 8 KB I/O. This is for a workload consisting of 50% reads and 50% writes and could reach 10,000 IOPS with 8 KB I/O.

Note : Please refer the AWS Documentation for latest PIOPS 

EBS Article Series (continued..)

Part 1: Understanding Amazon Elastic Block Store

Part 2: Understanding Standard EBS Volumes

Part 3: Understanding EBS PIOPS Volumes

Part 4: Understanding EBS-Optimized Instances

Part 5: Understanding Latency in EBS

Part 7: 10% of your provisioned IOPS 99.9% of the time

Part 8: Performance Tuning - Pre Warming the EBS volume

Part 9: Performance Tuning - EBS Striping

Part 10: Performance Tuning - IO Block Size

Part 11: Understanding Amazon EBS Snapshots

Part 12: Securing Amazon EBS volumes - EBS Encryption using SecureCloud 

Part 13: Amazon EBS Security Best practices and tips

Part 9: Performance Tuning Amazon Elastic Block Store - EBS Striping

Software RAID systems takes number of EBS Volumes and present them as one volume to the user. A RAID 0 (known as a striped volume) splits data evenly across two or more volumes (striped) without parity information. Example: The chunks of data from the same file are distributed across the disks that form the RAID array as illustrated in the below diagram (coming soon)

By writing data in small chunks across several EBS volumes, the performance of those volumes can be aggregated. Since I/O is distributed across the volumes in a stripe manner, if you add a volume in the array, you get the straight addition of throughput as well (most times). For example, where a single PIOPS volume is restricted to its individual I/O performance and disk RPM’s, a write to an array made up of several volumes combines the I/O of all those PIOPS Volumes. So, a stripe made up of six PIOPS volumes with a throughput of 1800 IOPS each would together have a total I/O performance of up to 10800 IOPS. You can create EBS stripes on both Standard and PIOPS EBS Volumes.  Note: Performance of the stripe is limited to the worst performing volume in the set. Usually EBS Striping is chosen in cases where I/O performance is more important than fault tolerance (such as a heavily used database, Search engine servers) and where data replication is already set up separately.

To create a six-volume stripe set on Linux, use a command like the following.

$ sudo mdadm --create /dev/md0 --level=0 --chunk=64 --raid-devices=6 /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk

Why 64 KB RAID chunk size?

The RAID chunk size should suit the I/O characteristics of the data you are working with. Important factor here is the size of the average I/O request you're going to place on the RAID Volume; as general rule; if you want big I/O requests, you should opt for smaller RAID chunk sizes, and if I/O will be small, you should go for larger chunks sizes.

Let me illustrate the chunk sizing with following scenarios:

Scenario 1:  Let us imagine your application works with lots of video or large image files, which means, you will need to spread data across individual volumes as much as possible. For this scenario use smaller RAID chunk sizes (for example, Block size 512 bytes-> 8 KB) because you want to take data from one volume while the others seek the next chunks to be read.

Scenario 2: Let us imagine you are running a database in which the amount of data read on each operation is small (4KB ->16KB). Here you want a single I/O to deal with one volume (drive) with one seek rather than the action is split between more than one volume and multiple seeks. So, for use cases such as databases and exchange servers, you should go for a bigger RAID chunk size say 64 KB or larger.



EBS Article Series (continued..)

Part 1: Understanding Amazon Elastic Block Store
Part 2: Understanding Standard EBS Volumes
Part 3: Understanding EBS PIOPS Volumes
Part 4: Understanding EBS-Optimized Instances
Part 5: Understanding Latency in EBS
Part 7: 10% of your provisioned IOPS 99.9% of the time
Part 8: Performance Tuning - Pre Warming the EBS volume
Part 9: Performance Tuning - EBS Striping
Part 10: Performance Tuning - IO Block Size
Part 11: Understanding Amazon EBS Snapshots
Part 12: Securing Amazon EBS volumes - EBS Encryption using SecureCloud 
Part 13: Amazon EBS Security Best practices and tips

Part 8: Performance Tuning Amazon Elastic Block Store - Pre Warming the EBS volume

New EBS Volumes always experience a “First Use Penalty” in AWS; means there will be a performance hit on the first write to an unused block on the EBS Volume and will perform slower than subsequent writes. During this time you can experience spike in volume metrics like service times, I/O latency and then subsequently leading to normalization. It is observed that 5 to 50 percent reduction in IOPS when you first access the data on a volume. Performance is restored after the data is accessed once. Therefore, it is recommended that you read or write to all the blocks on your volume before you use it.

$ dd if=/dev/md0 of=/dev/null

In Linux, the above command reads from all blocks on a volume and pre-warms the volume. On Windows, formatting the new EBS volume pre-warms it.

“dd” is not verbose by default so download and use the following script from the github, it shows the status of the pre warming of EBS Volumes, URL:  https://gist.github.com/muhqu/3293988

#!/bin/bash if [ -z "$1" ];   echo "Usage: sudo $0 /dev/sdh1"   exit 1; fi  dd if=$1 of=/dev/null & pid=$! while true; do   ps -p$pid --no-heading || break;   echo "-- $(date) ------------------";   kill -USR1 $pid;   sleep 60s; done echo "-- $(date) ------------------"; echo "DONE \o/"

EBS Article Series (continued..)

Part 1: Understanding Amazon Elastic Block Store
Part 2: Understanding Standard EBS Volumes
Part 3: Understanding EBS PIOPS Volumes
Part 4: Understanding EBS-Optimized Instances
Part 5: Understanding Latency in EBS
Part 7: 10% of your provisioned IOPS 99.9% of the time
Part 8: Performance Tuning - Pre Warming the EBS volume
Part 9: Performance Tuning - EBS Striping
Part 10: Performance Tuning - IO Block Size
Part 11: Understanding Amazon EBS Snapshots
Part 12: Securing Amazon EBS volumes - EBS Encryption using SecureCloud 
Part 13: Amazon EBS Security Best practices and tips

Part 7: Understanding Amazon Elastic Block Store Performance: 10% of your provisioned IOPS 99.9% of the time

Understanding the 10% statement of PIOPS

“Provisioned IOPS volumes are designed to deliver within 10% of your provisioned IOPS 99.9% of the time”. In simpler terms, it means for a volume provisioned with 2000 IOPS, your volume should deliver at least 1800 IOPS 99.9% of the time. (This is measured based on one minute intervals). When you design EBS Striping (RAID 0) with 6 disks (example 1 TB each with 2000 IOPS) you need to factor 6 X 1800 = 10800 IOPS as the bottom line and not 12000 IOPS.


EBS Article Series (continued..)

Part 1: Understanding Amazon Elastic Block Store
Part 2: Understanding Standard EBS Volumes
Part 3: Understanding EBS PIOPS Volumes
Part 4: Understanding EBS-Optimized Instances
Part 5: Understanding Latency in EBS
Part 7: 10% of your provisioned IOPS 99.9% of the time
Part 8: Performance Tuning - Pre Warming the EBS volume
Part 9: Performance Tuning - EBS Striping
Part 10: Performance Tuning - IO Block Size
Part 11: Understanding Amazon EBS Snapshots
Part 12: Securing Amazon EBS volumes - EBS Encryption using SecureCloud 
Part 13: Amazon EBS Security Best practices and tips

Part 5: Understanding Amazon Elastic Block Store Performance: Latency

Understanding the latencies

In general, it is impossible for a storage system to sustain the same peak IOPS number when presented with different I/O types and latency requirements.  Only IOPS numbers alone are meaningless without considering additional metrics such as latency, read/write % and I/O block size etc.

Latency is a measure of how long it takes for a single I/O request to happen from the apps perspective. High Latency and variance is not preferred when it comes to database applications. Database makes variety of requests and ideally IO latency needs to be under 10ms and write especially under 5ms. Example: In Oracle DB with for heavy writes, the redo log under 1ms is preferred. On the other hand, applications doing sequential, throughput-driven I/O (like backup or archival) typically don’t need high IOPS, but rather need high MB/s.

PIOPS Volumes delivers consistent level of latency and performance than standard EBS Volumes on random reads and random read/writes.  In the benchmark conducted by parse, PIOPS showed latency between 0-0.6 seconds and Standard EBS volumes showed 0.0 to 2.5 seconds. Refer URL:

http://blog.parse.com/2012/09/17/parse-databases-upgraded-to-amazon-provisioned-iops/

On the other hand, for random writes you should expect to see roughly similar latencies between Standard and Provisioned IOPS volumes, whereas on sequential writes and reads you can observe Standard EBS performing better than PIOPS sometimes. 


EBS Article Series (continued..)

Part 1: Understanding Amazon Elastic Block Store
Part 2: Understanding Standard EBS Volumes
Part 3: Understanding EBS PIOPS Volumes
Part 4: Understanding EBS-Optimized Instances
Part 5: Understanding Latency in EBS
Part 7: 10% of your provisioned IOPS 99.9% of the time
Part 8: Performance Tuning - Pre Warming the EBS volume
Part 9: Performance Tuning - EBS Striping
Part 10: Performance Tuning - IO Block Size
Part 11: Understanding Amazon EBS Snapshots
Part 12: Securing Amazon EBS volumes - EBS Encryption using SecureCloud 
Part 13: Amazon EBS Security Best practices and tips