Performance Tuning & Best Practices of Varnish on Amazon EC2

Varnish Cache is by default tuned for high performance on the majority of  production use cases. Following tuning points and best practices are suggested based on our experience in AWS.


64 bit Instance Type for Varnish :
Varnish needs lots of memory to perform better. If you have lots of cache items and less memory , tuning will not help much . So better run your Varnish on 64 bit instances with large memory. High Memory EC2 Instance types are very good candidates for Varnish (Note : Though t1.micro is 64 bit OS , it has only 780 MB of memory so it is not ideal for Varnish in AWS) 


Tuning the Hit rates :
Tuning Cache Hit rates is one of the most important parameters in Varnish. Online newspapers , magazines , classifieds are some of the applications which needs extra care in this parameter.if this parameter is not tuned properly , it will lead to Stampede of your back end heavily. 
Allocate one thread pool for every CPU core of Amazon EC2 instance. Example if a m1.large Varnish EC2 instance has 2 CPU core , then configure :
thread_pools 2
Thread pool minimum can start with 100 and maximum between 1000-2500 . 
Thread pool max is the maximum number of worker threads in each pool.
Configuring Thread pool max is usually based on the concurrent hits that you might expect Varnish.Do not set this higher than you have to, since excess worker threads soak up RAM and CPU and generally just get in the way of getting work done.
thread_pool_min 100
thread_pool_max 2500
Decrease the Thread pool delay to 3 seconds from 2 seconds , so that Varnish process can start up quickly without pile up.
thread_pool_add_delay 3

Tuning the Timeouts :
cli_timeout is a  Timeout for the childs replies to CLI requests from the master. The default is 10 seconds.If your varnish is heavily hit, child might not answer the management thread in a timely fashion, which in turn will kill it off. To avoid that, set cli_timeout to ~20 seconds. This is recommended only for Heavy traffic Varnish sites.

Configuring malloc :

If the cache content can be accommodated in memory , then recommended option is to use -smalloc. Example , if you are using m1.large instance type (7GB RAM) for Varnish , then allocate 5.5-6 GB Cache for Varnish,Since the Varnish cache size you specify is for the cache alone, and does not include session workspace.


Cache hitting disks :
If you cannot accommodate your cache Contents in EC2 instance memory, then you have following options :
Option 1)Increase the EC2 instance type to next High Memory type. Additional cost has to be bared for using High memory EC2 instance type.
Option 2)Use -sfile when the cache starts hitting your disk. -sfile performs far better than -smalloc in this scenario.(Note:filesystem mounted with noatime is preferred sometimes). Use Ephemeral disks in Amazon EC2, it is faster than EBS. Cost is lesser than option 1 and 3.
Option 3) Scale out + distribute your Varnish behind Load Balancer in AWS. This option is recommended only for advanced applications. Cost is more.

Do not tune parameters :
Do not set “tw_reuse” to 1 (sysctl). 
Do not modify lru_interval and sess_timeout.


Related Articles
Varnish Implementation Architectures in AWS 
FAQ - Varnish on Amazon EC2 
Configuring Varnish on AWS
Designing Web/App's with Varnish Page Cache in AWS