In any Cloud IT operations, gaining visibility into user activities is important for achieving compliance and meeting the regulatory standards of the enterprise. For years in AWS, we used to solve this with a combination of tools for log collection, alerts and privileged Identity access management. At times, it becomes cumbersome to install and manage these tools and overall operation cost effectively on Amazon Cloud. Like any other company pioneering managed services in AWS infrastructure, Our Cloud operations was in touch with Amazon Cloud product team for years giving input in this area. As an outcome of this, we see a new product line called as AWS Cloud Trail launched last year.
In this article we are going to explore topics like :
AWS Cloud Trail provides a record of AWS API calls made on your AWS account. AWS CloudTrail can be used for Resource Life Cycle Tracking, Operational Troubleshooting, Compliance Aid, and Security Analytics on AWS cloud infrastructure by integrating with the analytics tools like Splunk, Loggly, Sumo logic and etc.
In this article we are going to explore topics like :
- How AWS CloudTrail can be integrated with Splunk App for AWS ?
- How can we use it to operate our managed services efficiently ( with sample use cases answering following questions)
- How do i get to know when security groups or Network details are changed ?
- How do i know when EC2 instances are launched, terminated etc ?
- How do i know what activities are carried on the Amazon Elastic IP's ?
- What activities and changes happened in the IAM policies over the period ?
- What are the Rare errors related to AWS account activities that can be tracked ?
How AWS CloudTrail + Splunk Integration works ?
Currently CloudTrail logs API calls on the following AWS services and
in future you can expect more services to be launched. The current list is as follows :
·
Elastic Compute Cloud (EC2)
·
Elastic Block Store (EBS)
·
Virtual Private Cloud (VPC)
·
Relational Database Service (RDS)
·
Identity and Access Management (IAM)
·
Security Token Service (STS)
·
Redshift
AWS CloudTrail captures AWS API calls made by or on behalf of an AWS
account. Once the API calls are logged on files, it delivers these log files to
an Amazon S3 bucket that you specify. Each call detail is logged in JSON
format. Usually the JSON file is zipped and delivered to the bucket. SNS
notification can be enabled to operations team when the log file is delivered.
Sample format of the log file nomenclature is shown below:
https://s3-us-west-2.amazonaws.com/<bucket_name>/<prefix_name>/AWSLogs/<AccountID>/CloudTrail/
us-west-2/2013/10/25/<AccountID>_CloudTrail_us-west-2_2014-11-25T12:25Z_bPlKAoJtDqSWWlTT.json.gz
How to activate AWS CloudTrail ?
By default CloudTrail is not enabled in your AWS account. To activate
follow the below steps
1. Open the CloudTrail console in the AWS Management console
2. Click Get Started
3. On the Turn on CloudTrail page, you will be presented with the
option to create a new Amazon S3 bucket or using an existing Amazon S3 bucket. In
this article, I have selected the option to create new S3 bucket and provided
the Bucket name as cloudtraillogs-splunk as illustrated below.
Select
Yes or No to “Include global services”? IAM and AWS STS are the global services so if
you want to log activities of those services as well, please select “Yes”. It
is recommend to Select them because you can review the actions of the IAM.
SNS notification
for every log files delivery? If you
want delivery, please enter the SNS topic name. The entered topic will be
created in the SNS service and you must subscribe to it. After Saving, Click
Subscribe. Once it is done, the service is activated and the log files are generated.
5. You can also edit the CloudTrail configuration and Turn off CloudTrail using the console. The console is illustrated in the below screen Shot.
What is Splunk App for AWS ?
Splunk App for AWS is designed to take the data from the AWS
CloudTrail and provide the results using built-in dashboards. It is provided as
a plug-in. Operations / Managed Services team can view this app and monitor and review some activities efficiently.
What is interesting with Splunk App for AWS?
- It gives important insights into security-related activity such as unauthorized access attempts, simultaneous logins from geographically disparate locations and frequent changes to access control privileges. You can ensure security and compliance with continuous monitoring and a full audit trail of user activity using the Splunk App.
- It provides option to configure email alerts based on the schedule
- It provides easy option to monitor events, generate reports and export the reports and Search for events.
Steps to Integrate Splunk with AWS Cloud Trail :
Stage 1: Installing Splunk on AWS:
In order to use the Splunk App for AWS for this article, we are using the
Enterprise Splunk AMI for illustrating the use cases.
1. Launch an EC2 Linux instance and based on the choice of Operating
system download the Enterprise Splunk from the link
http://www.splunk.com/download?r=header
2. Install the Splunk Enterprise
3. Export the following environmental values and then start the Splunk
service.
# export
SPLUNK_HOME=/opt/splunk
# export PATH=$SPLUNK_HOME/bin:$PATH$
$ splunk start
Commands to Restart and Check Status are below.
$ splunk restart
$ splunk status
4. On successful start the Spunk will be listening on the port 8000.
Access the Splunk using the link http://<EC2 Public DNS>:8000. The below screen
will be presented
Stage 2: Installing and Configuring Splunk App for AWS:
Splunk App for AWS fetches the data from the Amazon S3 bucket and provides
the report in the Splunk GUI. Following are the steps to integrate Splunk app plugin with AWS Cloud Trail
1. Download the Splunk App for AWS from the link http://apps.splunk.com/app/1274/
2. Log into Splunk Enterprise and select the Manage Apps from the App menu
3. Click Install app from the file button.
4. On the Upload a file screen and Browse to locate the downloaded [app].tar.gz file, click Open and Upload
5. Click Restart Splunk, confirm to restart.
6. After Splunk restarted, login again, you can find the below screen.
2. Log into Splunk Enterprise and select the Manage Apps from the App menu
3. Click Install app from the file button.
4. On the Upload a file screen and Browse to locate the downloaded [app].tar.gz file, click Open and Upload
5. Click Restart Splunk, confirm to restart.
6. After Splunk restarted, login again, you can find the below screen.
7.From Settings in the top right corner of the page Select Data input,
In that you can see an option to add AWS CloudTrail Log settings.
8. Click Add new and you will be asked for the following mandatory details.
8. Click Add new and you will be asked for the following mandatory details.
- AWS Access key
- AWS Secret key
- AWS SQS name
- AWS SQS region
In order to use Splunk App, CloudTrail logging should be configured to send SNS topic and that topic should point to the Amazon SQS queue. You can see cloudtrail-spunkalert SNS topic is subscribed to the SQS queue Cloud Trail’s endpoint.
SQS queue should be created manually with the proper permissions to
process messages. We have created a queue named CloudTrail in SQS. Below screenshot details the CloudTrail SQS queue:
The above step completes the configuration.
What we can monitor and review using CloudTrail + Splunk ?
The Splunk App for AWS has built in dashboard with lot of options to review Network Actions, IAM Actions, EC2 options etc. Now lets go through some of the common review activities the IT operations/ Managed Services team can perform using the AWS CloudTrail and Splunk Integration:
What we can monitor and review using CloudTrail + Splunk ?
The Splunk App for AWS has built in dashboard with lot of options to review Network Actions, IAM Actions, EC2 options etc. Now lets go through some of the common review activities the IT operations/ Managed Services team can perform using the AWS CloudTrail and Splunk Integration:
Case 1: Reviewing AWS Security Group related activities using AWS CloudTrail and Splunk:
Case 2: Reviewing Amazon EC2 related activities using AWS CloudTrail and Splunk:
From the above Screen shots , we can infer that a new Security group was created and authorized. The Audit details pertaining to Security group + additional details like IAM User, AWS Account, Source IP address and Time stamp of this activity is captured by AWS CloudTrail and presented in Splunk. The Operations/ Managed Services team can constantly monitor for such changes in the infrastructure and take actions accordingly.
Case 3: Rare errors pertaining the AWS account can be reviewed using AWS CloudTrail and Splunk:
Case 4: Reviewing AWS IAM related activities using AWS CloudTrail and Splunk:
AWS IAM Activities like CreateUser, DeleteUser, DeleteLoginProfile etc are tracked by AWS CloudTrail as logs. These activities are presented to the IT Ops/MS team on the IAM actions module of Splunk. The diagrams representing the same is listed below:
Case 5: Reviewing Amazon ElasticIP related activities using AWS CloudTrail and Splunk:
Case 6: View all the events occurred on your AWS Account as Graph
Case 7: Query your AWS CloudTrail logs for viewing activities and patterns
Please talk to us to understand more about our Managed Services offerings and how we can help your cloud infrastructure. Contact
This article was co authored with Senthil.
