FAQ : Varnish Cache on Amazon EC2 (AWS)

1)Can Varnish Web accelerator run on AWS ?
Yes , Varnish Web Accelerator can be configured and run on AWS . Please refer this URL for configuration 


2)Is Varnish Cache a  Web Service provided by AWS ?
No , it is not a web service . We need to Install Varnish Cache on Amazon EC2.


3)What Varnish compatible Linux distributions are available in Amazon Machine Images (AMI) ?
Red Hat , Ubuntu , Debian and FreeBSD compatible AMI's are available for Varnish


4)Should i go for S3 Backed AMI's or EBS backed AMI's for Varnish ?

• S3 backed AMI's have faster IO but slower launch times . Information stored in Ephemeral disks will be lost during EC2 failure or reboot , if they are not properly backed up in S3.
• EBS backed AMI's have faster launch times and information is persisted across EC2 failures and launches. This AMI is useful if you want to preload or cache warm your Varnish from varnish logs.
• If Varnish is going to run inside VPC make sure compatible AMI distributions are available.

5)How to choose the right Instance Type in Amazon for Varnish ?

• Varnish web accelerator is a Cache and it ideally needs lots of memory, so a 64-bit OS is ideal choice to start with
• Amazon T1.micro is also 64-bit OS , but with 700+ MB of Memory. Since memory might not be enough to store enough pages in cache a larger 64bit is ideal
• M1.Large (64 bit , 7GB RAM) is an ideal candidate to start with , if your Cache requirements can be met within 7GB RAM . We have seen this capacity being used by many customers in HA mode.
• High Memory EC2 Instances are better choices than High CPU EC2 Instances for Varnish Cache
• If your Cache need is more than 7GB RAM you can choose other High memory instances from Amazon EC2
• If you need to cache TB's of page data , we suggest to distribute/shard your cache in Varnish using Reverse Proxy ( HAProxy or NginX) as entry point

6)How do we monitor Varnish in AWS Cloud ?
We recommend our customers to monitor Varnish in the following combinations :

• Using Amazon Cloudwatch + SNS alerts technique we can monitor the Varnish EC2 instance CPU and Network.
• Using Custom Cloudwatch metrics we can monitor Custom varnish Parameters and alert the Ops
• Using Munin we can monitor the VarnishStat and infer quite bit of information on Varnish EC2

7)What are the best practices in Backing up Varnish EC2 ?
Since Varnish web accelerator is used as cache and it performs well when most of the pages are served from memory and stored on disk, we should not complicate the backup process too much. Keeping simplicity in mind i suggest the following easy backup schemes in AWS for Varnish EC2:

• Bundling Varnish EC2 AMI to S3 or Periodic EBS Snapshots is a recommended approach for most use cases 
• Frequent EBS Snaphots or Syncing varnishlog to S3 is recommended for cases where Varnish Pre-loading or Cache Warming is required
• Automation of above approaches is ideal for large scale systems

8)What directors are supported by Varnish on AWS  ?
Most popular Varnish Directors are Round Robin and Random directors. Amazon EC2 does not restrict any direction algorithms of Varnish to back end servers .  So whatever directors Varnish supports it should work in AWS cloud too. 


9)How do we design High Availability for Varnish EC2 in AWS ?

• Simple use cases of Varnish does not demand Varnish running in HA mode. When Varnish is down , Route53 will try to pass the request to Varnish EC2 and error is delivered to the end user till Varnish gets back to running state. 
• Use cases like online newspapers , portals ,  online classifieds , Magazines etc where Varnish EC2 failure can trigger huge performance impact , Cache stampede or heavy load to the back end demand Varnish to run on HA mode. For such use cases we recommend running minimum 2 Varnish EC2 on 2 Availability Zones of AWS. This avoids single point of failure at Varnish EC2 as well as Single Availability Zone level. Varnish HA Implementation architectures are discussed here.

10)Will Varnish work inside Amazon VPC? 
Yes , Varnish works inside Amazon VPC as well , place it in public subnet. if public Varnish AMI's are not available for your AWS region , you can easily build one for the community.

11)Do i need an Elastic IP for Varnish EC2 ?
It depends upon implementation architectures :

• If you place Varnish behind Amazon Route53 , you will have to attach ElasticIP to the Varnish EC2
• If you place Varnish behind Amazon ELB , you need not attach ElasticIP to Varnish EC2
• If you place Varnish behind Reverse proxy, attaching ElasticIP is optional. Most use cases does not demand this need behind Reverse proxy.

12)Can i use Amazon AutoScaling to scale out my Varnish Layer ?

Yes , we can use Amazon AutoScaling to scale out Varnish layer based on CPU or some custom metric. But , still now i had not found a use case for recommending the same. Auto Scaling is suitable for scenarios where entire infrastructure needs to react to load volatility. Varnish in HA or Distributed Varnish in HA mode can usually handle this scenario. If needed we prefer controlled varnish scale out using custom programs rather than Amazon Auto Scaling . We feel an improperly warmed Varnish Cache can pass heavy load to back end servers and affect the overall website performance in Scale out scenario , hence we need to properly add warmed varnish EC2 instance in controlled manner into the acceleration layer.

Article Under progress , Some more answers are on the way 

Does Varnish Compete or Complement AWS CloudFront CDN ?
When do we need Distributed  Varnish Setup in AWS ?
How do we achieve Distributed Varnish Setup in AWS ?
Any Varnish Latency benchmarks in AWS ?