Showing posts with label Cost Savings. Show all posts
Showing posts with label Cost Savings. Show all posts

Sunday, January 31, 2016

CloudWatch + Lambda Case 4: Control launch of Specific “C” type EC2 instances post office hours to save costs

We have a customer who has predictable load volatility between 9 am to 6 pm and uses specific large EC2 instances during office hours for analysis, they use “c4.8xlarge” for that purpose. Their IT wanted to control launch of such large instance class post office hours and during nights to control costs, currently there is no way to restrict or control this action using Amazon IAM. In short we cannot create complex IAM policy with conditions that user A belonging to group A cannot launch instance type C every day between X and Y.
Some stop gap followed is to have a job running which removes the policy from an IAM user when certain time conditions are met. So basically what we would do is, to have a job that calls an API that removes the policy which restricts an IAM user or group from launching instances. This will make the IAM policy management complex and tough to assess/govern drifts between versions.
After the introduction of the CloudWatch events our Cloud operations started controlling it with lambda functions. Whenever a Instance type is launched it will trigger a lambda function, the function will filter whether it is a specific “C” type and check for the current time, if the time falls after office hours, it will terminate the EC2 instance launched immediately.
As a first step, we will be creating a rule in Amazon CloudWatch Events dashboard. We have chosen AWS API Call as an Event to be processed by an AWSCloudTrail Lambda function as a target.



The next step would be configuring rule details with Rule definition



Finally we will review the Rules Summary



Amazon Lambda Function Code Snippet (Python)
import boto3

def lambda_handler(event, context):
    #print ("Received event: " + json.dumps(event, indent=2))
    #print ("************************************************")
 
    ec2_client = boto3.client("ec2")
 
 
    print "Event Region :", event['region']
 
    event_time = event['detail']['eventTime']
    print "Event Time :", event_time
 
    time = event_time.split('T')
    t = time[1]
    t = t.split(':')
    hour = t[0]
 
    instance_type = event['detail']['requestParameters']['instanceType']
    print "Instance Type:", instance_type
 
    instance_id = event['detail']['responseElements']['instancesSet']['items'][0]['instanceId']
    print "Instance Id:",instance_id
 
    if( instance_type.startswith( 't' ) and hour > 18 or hour < 8 ):
        print ec2_client.terminate_instances( InstanceIds = [ instance_id ] )

GitHub Gist URL:  https://github.com/cloud-automaton/automaton/blob/master/aws/events/TerminateAWSEC2.py

This post was co authored with Priya and Ramprasad of 8KMiles

Thursday, January 23, 2014

AWS Cost Saving Tip 18: Using AWS Tagging with Graffiti Monkey to Saves costs

Tagging AWS assets and resources is a simple and an important feature that helps IT teams streamline their process of managing the environments. We can categorize AWS resources in variety of ways using the AWS Tagging feature. For example: 

  • Environment based Tags: Environment based tags can be used by customers who have Larger AWS footprints. These tags will help you to distinguish between production, development, and staging infrastructure deployed on AWS
  • Application based Tags: Used to describe the set of disparate resources (or clusters) that work together to deliver a product or service to a customer. Can be applied on Small to Big AWS Production environments to identify the resources used
  • Role based Tags– Used to describe the function of a particular node (web server, database server, load balancer, etc.). Owner – Used to identify the individual who is responsible for the instance. Both these cases are usable for small AWS foot prints.

How it helps ?
Tags can be very useful in understanding our AWS Billing. When you run large foot prints on AWS, constantly we need to monitor and manage our resource usage. Any IT operations team which manages AWS resources, constantly need to work on the following questions to optimize and manage their operations efficiently. Which AWS resources are used ? Which resources are lying idle (volumes, snapshots) in detached state ? Which Tiers or environments are consuming more billing than others? Which owner is exceeding the AWS cost allocated ? In short by starting to tag the AWS resources we will know how those assets are translating to monthly cost and take optimization decisions. Tagging is an simple but an essential feature that helps us to reduce and keep costs in control by improving our management efficiency in AWS.

AWS Tags in Detail:
Each tag consists of a key and a value, both of which the user can define. Currently AWS does not provide facility to TAG all AWS resources. Some of them can be Tagged only using API actions or the Command Line. The following Screen Shot shows all AWS resources and the restrictions that apply to them. Resources with tagging restrictions of None can be tagged with API actions, the CLI, and the console.
The following diagram illustrates how tagging works. In this example, i am assigning two tags to each of my EC2 instances, one called Owner and another called Stack. Each of the tags also has an associated value.
Owner = 8KMiles , Stack = Development, Stack = Production as illustrated in the below diagram



You can assign tags using the AWS Management Console, the Amazon EC2 command line interface (CLI), and the Amazon EC2 API.You can define tags from the EC2 console by selecting the relevant instance and selecting the “Tags” option.  Alternatively, you can select the instance, click “Actions” and select “Add/Edit Tags.” Below Screenshot shows how tags can be assigned using AWS Management Console

We have assigned Name, Owner , Environment and Contact for the 8KMiles Production instance using the AWS Tagging feature. The above is not the usual EC2 Managed Service naming standard , but just an simple illustration of the tag use case.
AWS Tagging Constraints and the Cost Leakage:
Currently AWS Tagging has some limitations as well. For example each tag associated with a single AWS resource, applies only to that particular resource and it is not propagated to the dependent resources attached to it. Example: Imagine if you have an EC2 instance, most of the times this EC2 instance is attached with dependent resources like EBS volumes, Snapshots etc. Usually there will be 1-2 EBS Volumes on normal scenarios and up to 8 EBS Volumes or more on RAID Scenarios. Number of snapshots are more in number and usually becomes unmanageable. Imagine how this problem will be propagated if you are running a Large AWS foot print with thousands of EC2 instances. 
You have to find each EBS volumes associated with a particular EC2 and Tag it separately.Managing this is easier said than done and usually slightest mis- management of these resources will lead to cost leakage. What if i can use a tool to automate tagging for dependent AWS resources. It will surely reduce the manual labor associated and help us remove all the dependent resources when we are removing the EC2. This avoids leakages and reduces cost of Cloud Operations in very dynamic AWS environments.

Welcome to Graffiti Monkey. 
The Graffiti Monkey goes around tagging things automatically. By looking at the tags of an EC2 instance, it copies those tags to the EBS Volumes that are attached to it, and then copies those tags to the EBS Snapshots as well. This small automation reduces hours of Manual labor on large AWS footprints, helps us efficiently manage the cloud infrastructure and reduce the cost leakages.

Step 1: Installation  of Graffiti Monkey 

Easiest way to install Graffiti Monkey using the usual PyPI channels.Please refer the below screenshot
link for installing pip.http://www.pip-installer.org/en/latest/installing.html

Step 2: Using Graffiti Monkey

It will identify the tags associated with EC2 instance and applies them to dependent resources like EBS volumes and snapshots saving hours of manual labor.

This Post is co authored with Nitheesh 8Kmiles


Tuesday, October 22, 2013

AWS Cost Saving Tip 17: Add Amazon RDS Read Replica's on need basis and save costs

Many online applications are usually Read intensive in nature. Though the usage of Cache, NoSQL, CDN are becoming part of the core architecture in many such applications, still most of them are heavily dependent upon Traditional RDBMS (like MySQL, Oracle) for serving their requests.
Architects usually design their DB tier with Master + Read Slaves model to cater the read intensive needs of the application. Writes and Immediate Write + Reads are directed to Master and all other Read queries are balanced between Read Slaves. Translating this model to AWS infrastructure means, using RDS MySQL Master + Multiple Read Replica Slaves.

To know more about Load balancing Amazon RDS Read Replica's Click here




Scenario: Weekly load volatility impacts: 
Online Ticketing and Travel companies are usually prone to weekly fluctuations in their load pattern. Imagine during peak load they need five M2.4XLarge RDS MySQL Read Replica's to serve their read intensive site.

As an Infra Architect you have two options in the Elastic AWS world: 
Option 1: Run your DB tier with peak capacity all days of the week. ( May be suitable for Traditional infra environments, but a bad idea in AWS infra)
Option 2: Elastically increase the number of RDS Read Replica instances during the peak period and decrease the number of instances once the peak period is over. (AWS is built for leveraging such cases)

Number of EC2 Instances
Number of days in a week
EC2 Instance Type
Weekly($)
Monthly($)
5
7
M2.4XLarge
1860
7442



1860
7442
2
4
M2.4XLarge
425
1701
5
3
M2.4XLarge
797
3189



1222
4890

Cost for Option 1: Running all 5 of them for a Week costs : $1860
Cost for Option 2: Running with 2 RR normally for 4 days a week and elastically increasing to Five M2.4Xlarge during the peak days(i.e. three days a week) will cost : $1222

Savings: From the above analysis you can observe that elastically increasing/Decreasing Amazon RDS RR based on your load you can achieve ~30 % cost savings in a week for the mentioned use case.

To know more about "How Amazon Auto Scaling can save costs on your Web/App Tier ? Click here "

Scenario: Impact during Campaigns 
Imagine your online business demands big campaigns once every quarter. The campaign usually runs for 2 weeks and traffic increases 4-5X during these times. During normal period you run 1 RDS Read replica and during campaigns you need 5 RDS Read replicas. 

If the campaign is planned once a year during holiday season, companies lease hardware, but if the campaign is every quarter or more frequent it becomes irrelevant to lease-build-maintain peak capacity.


Number of EC2 Instances
Number of Months
EC2 Instance Type

Monthly($)
5
10
M2.4XLarge
All times
98877




98877
1
10
M2.4XLarge
Other times
16479
5
2*
M2.4XLarge
During campaigns
16479




32959
Instance Hours per day: 744
*Campaign weeks are aggregated to months
US-Oregon region - $2.215 (M2.4Xlarge RDS RR pricing as on posting)

Cost for Option 1: Running all 5 of them for a year costs :  98877 USD
Cost for Option 2: Running with 1 RR on regular weeks and elastically increasing to Five M2.4Xlarge during the peak weeks will cost : 32959 USD a year

Savings: From the above analysis you can observe that elastically increasing/Decreasing Amazon RDS RR you can achieve ~55 % cost savings in a year.

For in depth analysis on Amazon RDS Second tier Read Replica's Click here 

Sunday, October 6, 2013

AWS Cost Saving Tip 16: Periodically remove your unwanted AWS resources and save costs

Thanks "@HighScalability.com"  for referencing this post "Stuff The Internet Says On Scalability For October 11th, 2013"

Following are some of the AWS assets that needs to be periodically reviewed for under utilization and should be removed as part of your IT process to avoid cost leakage in AWS. 

Remove unwanted AMI's: Periodically audit for unwanted AMI's in your dev/test/production environment and remove them. Tagging the AMI's with proper identifiers can help you during the cleansing process.

EBS snapshots are incremental in nature and only the changes are moved to them. The unwanted snapshots should be periodically identified (Tagging snapshots will help you) and deleted as part of your IT Infrastructure operations. EBS snapshots are priced $0.095 per GB-month of data stored and you can save few hundred $$$ depending upon the volume of unwanted snapshots stored.To know more about EBS Snapshots

EBS Volumes: In development and testing environments of large enterprises we can often find lots of EBS volumes provisioned and lying unattached or unused. EBS Volumes are priced at $0.10 per GB-month of provisioned storage. Usually 10-30 GB are the most used volume ranges for the dev/test. Imagine a enterprise having few hundred EC2's for their Dev/Test infrastructure and 10-25% of their storage are unused, Periodically identifying , tagging, consolidating and cleansing them will save few hundred to thousands of $$$ for the enterprises.To know more about EBS Volumes

S3 Buckets and Objects: Unused objects in S3 buckets have to be frequently deleted. Objects with a limited life period can be configured to expire dynamically using S3 object expiration policies. Objects that are rarely used can be archived to Amazon Glacier from S3. This exercise is an important candidate for Media and online companies using AWS. To know more about other S3 Cost saving tips 1 | 2.

Elastic IP is a scarce resource in AWS cloud, in event the EIP's attached to your account are not associated with any running EC2 instance, you will be charged $0.005 per hour on a prorata basis.
EIP's can usually be leaked in: 

  • Dev/Test environments, where the developer disassociates the EIP and keeps it unused.
  • In production, where sometimes EIP are assigned to Auto Scaled EC2 instances using scripts and they are not properly released back after EC2 instances are terminated by AWS.

Though if the EIP is kept unused can lead to around only ~$3.5 leakage per month, if no proper tracking procedures are in place many such EIP's can be residing in your account unused and lead to more such leakages.

ELB: Unused and Unwanted Elastic Load Balancers should be removed periodically to avoid cost leakage. A single ELB can cost around ~216 USD if kept unused a year. Usually ELB leakage is prone to happen in Dev/Test environments compared to production. To know more about Amazon ELB

Monitoring : Detailed monitoring option of CloudWatch is generally not required for Development EC2 instances. On the other hand, they are required on Load Testing and Production environments for monitoring the health. Custom Metrics are also good candidates for Production and Load Test environments. Constant review of the infrastructure in AWS has to be done to identify whether such resources(over provisioning) are associated with Dev/Test env and related cost leakages can be avoided.Detailed monitoring costs $3.50 per instance per month, provided at 1-minute frequency and Custom Metrics costs $0.50 per metric per month.

Glacier archives : Amazon Glacier is designed for use cases where data is retained for months, years, or decades. But there are times where you may find some data life has expired and they can be removed. Deleting data from Amazon Glacier is free if the archive being deleted has been stored for three months or longer. Please ensure it is part of you IT process. To know more about other Cost savings in Glacier 1 | 2 | 3

Other small stuff :

  • AWS charges $1.00 per month for inactive data pipelines. They should be periodically reviewed for leakages.  



Note:
  • There are times where an AWS asset can be created in a non frequently used AWS Region mistakenly by a developer and it can be lying idle for months. Have an automated or manual IT process to periodically cleanse the above leakages points in "ALL" AWS Regions and not just your most frequently used regions.
  • Many times on large AWS deployments/Dev/Test environments, it becomes difficult to track who created and who is currently using the AWS resources. AMI's, EBS snapshots, Volumes and S3 buckets should be tagged for proper identification. Identifiable names should be kept for other assets (wherever applicable) , so that it helps your IT team during the cleansing process. 
  • Employ automation using scripts/programs or use governance tools ( which i will be discussing later) to monitor under utilization of resources and there by remove them.
Other Cost Saving Tips

Cost Saving Tip 1: Amazon SQS Long Polling and Batch requests
Cost Saving Tip 2: How right search technology choice saves cost in AWS ?
Cost Saving Tip 3: Using Amazon CloudFront Price Class to minimize costs
Cost Saving Tip 4 : Right Sizing Amazon ElastiCache Cluster
Cost Saving Tip 5: How Amazon Auto Scaling can save costs ?
Cost Saving Tip 6: Amazon Auto Scaling Termination policy and savings
Cost Saving Tip 7: Use Amazon S3 Object Expiration
Cost Saving Tip 8: Use Amazon S3 Reduced Redundancy Storage  
Cost Saving Tip 9: Have efficient EBS Snapshots Retention strategy in place 
Cost Saving Tip 10: Make right choice between PIOPS vs Std EBS volumes and save costs 
Cost Saving Tip 11: How elastic thinking saves cost in Amazon EMR Clusters ? 
Cost Saving Tip 12: Add Spot Instances with Amazon EMR 
Cost Saving Tip 13: Use Amazon Glacier for archive data and save costs (new)
Cost Saving Tip 14: Plan your deletion in Amazon Glacier and avoid cost leakage (new)

Saturday, September 21, 2013

AWS Cost Saving Tip 15: Plan your Amazon Glacier Retrieval strategy and avoid cost leakage


Tip 1: Use Multi-part upload and range retrieval:
You store data in Amazon Glacier as an archive. An archive is any object, such as a photo, video, or document that you store and usually represents a single file or several files. It is a base unit of storage in Amazon Glacier. You can upload an archive in a single request. For large archives, Amazon Glacier provides a multipart upload API that enables you to upload an archive in parts and enables range retrieval in future.
In reality many times, you may need to retrieve a small selection of those files from a archive, in which case you could retrieve only the ranges of the archive that contained the required files.You may also choose to perform range retrievals in order to reduce or eliminate your retrieval fees. If you exceed your free retrieval allowance, you pay a retrieval fee that is based on your peak retrieval rate. Spreading out a retrieval of an archive in smaller parts could therefore allow you reduce your retrieval fees, by reducing your peak retrieval rate.

Tip 2: Spread your retrievals and avoid cost leakage:
Let’s assume you are storing 75 TB of data in Amazon Glacier and you would like to retrieve 140 GB. The amount you pay is determined by how fast you retrieve the data. 
Option 1: Storing 75 TB and retrieving 140 GB in 4 hours costs $21.60.  
Option 2: Storing 75 TB and retrieving 140 GB in 8 hours costs $10.80 (half the cost of option 1)
Option 3: Storing 75 TB and retrieving 140 GB evenly over 28 hours costs NIL because you would no longer exceed your daily free retrieval allowance and would therefore not be charged a Retrieval Fee. Imagine how much cost is leaked if option 1 is followed frequently instead option 3 where the billing is lower or free most times. This can be avoided by better understanding of Amazon Glacier, planning your retrieval in advance with proper spread.

In Amazon Glacier, the suggested practice to lower costs is to plan your retrieval strategy in advance by spreading out the retrievals over longer periods of time. This will reduce or eliminate the retrieval fees and lowers your billing.
Use the aggressive retrieval mechanisms only on use cases where timing is important, Example : You have an old archived news footage in Amazon Glacier, and suddenly based on the current event this old new footage needs to be retrieved, related and delivered. In this case you can use peak retrieval rate and other times you can spread your retrieval strategy.

To explore Log Analysis and Archive with Amazon S3 and Glacier. Refer this detailed article series.

Other Tips

Cost Saving Tip 1: Amazon SQS Long Polling and Batch requests
Cost Saving Tip 2: How right search technology choice saves cost in AWS ?
Cost Saving Tip 3: Using Amazon CloudFront Price Class to minimize costs
Cost Saving Tip 4 : Right Sizing Amazon ElastiCache Cluster
Cost Saving Tip 5: How Amazon Auto Scaling can save costs ?
Cost Saving Tip 6: Amazon Auto Scaling Termination policy and savings
Cost Saving Tip 7: Use Amazon S3 Object Expiration
Cost Saving Tip 8: Use Amazon S3 Reduced Redundancy Storage  
Cost Saving Tip 9: Have efficient EBS Snapshots Retention strategy in place 
Cost Saving Tip 10: Make right choice between PIOPS vs Std EBS volumes and save costs 
Cost Saving Tip 11: How elastic thinking saves cost in Amazon EMR Clusters ? 
Cost Saving Tip 12: Add Spot Instances with Amazon EMR 
Cost Saving Tip 13: Use Amazon Glacier for archive data and save costs (new)
Cost Saving Tip 14: Plan your deletion in Amazon Glacier and avoid cost leakage (new)

AWS Cost Saving Tip 14: Plan your deletion in Amazon Glacier and avoid cost leakage

Amazon Glacier is designed for use cases where data is retained for months, years, or decades. Deleting data from Amazon Glacier is free if the archive being deleted has been stored for three months or longer. If an archive is deleted within three months of being uploaded, you will be charged an early deletion fee.

Example : In the US East Region, you would be charged a prorated early deletion fee of $0.03 per gigabyte deleted within three months. So if you deleted 1 gigabyte of data 1 month after uploading it, you would be charged a $0.02 early deletion fee. If, instead you deleted 1 gigabyte after 2 months, you would be charged a $0.01 early deletion fee.

Avoid Cost leakage:
Imagine your IT process does not have this important point considered and assume they delete ~10 TB within 3 months every quarter from Amazon Glacier Archives, they are unnecessarily leaking ~2000 USD cost to AWS every quarter. Instead plan your deletion strategy well in advance build your IT process around this Cost constraint and delete items only after 3 months and avoid this unnecessary leakage.
(Note: Storing rarely accessed archives in Amazon Glacier and deleting early is still cheap compared to storing 10 TB in Amazon S3 and paying ~8K USD bill per month.)

To explore Log Analysis and Archive with Amazon S3 and Glacier. Refer this detailed article series.

Other Tips

Cost Saving Tip 1: Amazon SQS Long Polling and Batch requests
Cost Saving Tip 2: How right search technology choice saves cost in AWS ?
Cost Saving Tip 3: Using Amazon CloudFront Price Class to minimize costs
Cost Saving Tip 4 : Right Sizing Amazon ElastiCache Cluster
Cost Saving Tip 5: How Amazon Auto Scaling can save costs ?
Cost Saving Tip 6: Amazon Auto Scaling Termination policy and savings
Cost Saving Tip 7: Use Amazon S3 Object Expiration
Cost Saving Tip 8: Use Amazon S3 Reduced Redundancy Storage  
Cost Saving Tip 9: Have efficient EBS Snapshots Retention strategy in place 
Cost Saving Tip 10: Make right choice between PIOPS vs Std EBS volumes and save costs 
Cost Saving Tip 11: How elastic thinking saves cost in Amazon EMR Clusters ? 
Cost Saving Tip 12: Add Spot Instances with Amazon EMR 
Cost Saving Tip 13: Use Amazon Glacier for archive data and save costs (new)
Cost Saving Tip 14: Plan your deletion in Amazon Glacier and avoid cost leakage (new)

Need Consulting help ?

Name

Email *

Message *

DISCLAIMER
All posts, comments, views expressed in this blog are my own and does not represent the positions or views of my past, present or future employers. The intention of this blog is to share my experience and views. Content is subject to change without any notice. While I would do my best to quote the original author or copyright owners wherever I reference them, if you find any of the content / images violating copyright, please let me know and I will act upon it immediately. Lastly, I encourage you to share the content of this blog in general with other online communities for non-commercial and educational purposes.

Followers