Thursday, January 23, 2014

AWS Cost Saving Tip 18: Using AWS Tagging with Graffiti Monkey to Saves costs

Tagging AWS assets and resources is a simple and an important feature that helps IT teams streamline their process of managing the environments. We can categorize AWS resources in variety of ways using the AWS Tagging feature. For example: 

  • Environment based Tags: Environment based tags can be used by customers who have Larger AWS footprints. These tags will help you to distinguish between production, development, and staging infrastructure deployed on AWS
  • Application based Tags: Used to describe the set of disparate resources (or clusters) that work together to deliver a product or service to a customer. Can be applied on Small to Big AWS Production environments to identify the resources used
  • Role based Tags– Used to describe the function of a particular node (web server, database server, load balancer, etc.). Owner – Used to identify the individual who is responsible for the instance. Both these cases are usable for small AWS foot prints.

How it helps ?
Tags can be very useful in understanding our AWS Billing. When you run large foot prints on AWS, constantly we need to monitor and manage our resource usage. Any IT operations team which manages AWS resources, constantly need to work on the following questions to optimize and manage their operations efficiently. Which AWS resources are used ? Which resources are lying idle (volumes, snapshots) in detached state ? Which Tiers or environments are consuming more billing than others? Which owner is exceeding the AWS cost allocated ? In short by starting to tag the AWS resources we will know how those assets are translating to monthly cost and take optimization decisions. Tagging is an simple but an essential feature that helps us to reduce and keep costs in control by improving our management efficiency in AWS.

AWS Tags in Detail:
Each tag consists of a key and a value, both of which the user can define. Currently AWS does not provide facility to TAG all AWS resources. Some of them can be Tagged only using API actions or the Command Line. The following Screen Shot shows all AWS resources and the restrictions that apply to them. Resources with tagging restrictions of None can be tagged with API actions, the CLI, and the console.
The following diagram illustrates how tagging works. In this example, i am assigning two tags to each of my EC2 instances, one called Owner and another called Stack. Each of the tags also has an associated value.
Owner = 8KMiles , Stack = Development, Stack = Production as illustrated in the below diagram

You can assign tags using the AWS Management Console, the Amazon EC2 command line interface (CLI), and the Amazon EC2 API.You can define tags from the EC2 console by selecting the relevant instance and selecting the “Tags” option.  Alternatively, you can select the instance, click “Actions” and select “Add/Edit Tags.” Below Screenshot shows how tags can be assigned using AWS Management Console

We have assigned Name, Owner , Environment and Contact for the 8KMiles Production instance using the AWS Tagging feature. The above is not the usual EC2 Managed Service naming standard , but just an simple illustration of the tag use case.
AWS Tagging Constraints and the Cost Leakage:
Currently AWS Tagging has some limitations as well. For example each tag associated with a single AWS resource, applies only to that particular resource and it is not propagated to the dependent resources attached to it. Example: Imagine if you have an EC2 instance, most of the times this EC2 instance is attached with dependent resources like EBS volumes, Snapshots etc. Usually there will be 1-2 EBS Volumes on normal scenarios and up to 8 EBS Volumes or more on RAID Scenarios. Number of snapshots are more in number and usually becomes unmanageable. Imagine how this problem will be propagated if you are running a Large AWS foot print with thousands of EC2 instances. 
You have to find each EBS volumes associated with a particular EC2 and Tag it separately.Managing this is easier said than done and usually slightest mis- management of these resources will lead to cost leakage. What if i can use a tool to automate tagging for dependent AWS resources. It will surely reduce the manual labor associated and help us remove all the dependent resources when we are removing the EC2. This avoids leakages and reduces cost of Cloud Operations in very dynamic AWS environments.

Welcome to Graffiti Monkey. 
The Graffiti Monkey goes around tagging things automatically. By looking at the tags of an EC2 instance, it copies those tags to the EBS Volumes that are attached to it, and then copies those tags to the EBS Snapshots as well. This small automation reduces hours of Manual labor on large AWS footprints, helps us efficiently manage the cloud infrastructure and reduce the cost leakages.

Step 1: Installation  of Graffiti Monkey 

Easiest way to install Graffiti Monkey using the usual PyPI channels.Please refer the below screenshot
link for installing pip.

Step 2: Using Graffiti Monkey

It will identify the tags associated with EC2 instance and applies them to dependent resources like EBS volumes and snapshots saving hours of manual labor.

This Post is co authored with Nitheesh 8Kmiles

No comments:

Need Consulting help ?


Email *

Message *

All posts, comments, views expressed in this blog are my own and does not represent the positions or views of my past, present or future employers. The intention of this blog is to share my experience and views. Content is subject to change without any notice. While I would do my best to quote the original author or copyright owners wherever I reference them, if you find any of the content / images violating copyright, please let me know and I will act upon it immediately. Lastly, I encourage you to share the content of this blog in general with other online communities for non-commercial and educational purposes.