Configuring Varnish Page Cache in Amazon EC2 (AWS)

Installing and Configuring Varnish Server

Install the varnish server and make corresponding configuration changes.

COMMAND

§  Install dependants for Varnish server installation

§  Download and Install Varnish server

§  START Varnish server and check for Varnish server run status

§  Make required configurations in default.vcl file

yum install automake

yum install autoconf

yum install libtool

yum install ncurses-devel

yum install groff

yum install pcre-devel

yum install pkgconfig

wget repo.varnish-cache.org/source/varnish-3.0.2.tar.gz

tar -zxvf varnish-3.0.2.tar.gz

cd varnish-3.0.2

sh autogen.sh

sh configure

make

make check

make install

Now Varnish server is installed, to start varnish server we need to know the IP  of this server. To get the IP of the server execute command “ifconfig” and find the IP as shown in the image below,

Once you have received the IP of the instance now build the command for Varnish server launch

varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,4G -T 127.0.0.1:2000 -a 10.64.43.79:80 -p thread_pool_min=500 -p thread_pool_max=4000 -p thread_pool_add_delay=2

Note: Make sure you provide the correct IP address while varnish server launch

Varnish Launch Command Explanation:

Varnishd –> Command

-f –> file location of VCL file

-s –> Backend storage specification. By default the storage is “file” we change it to “malloc” so that the information will be stored in memory

-T –> Telnet listen address and port. Host is set to localhost and port is some random port e.g. 2000

-a –> HTTP listen address and port. Host is this server’s IP and listening port is 80

-p –> Set parameter for service launch. 

There are few essential performance based parameters to be set while launch they are as follows

§  thread_pool_min is the minimum number of threads for each thread pool

§  thread_pool_max is the maximum total number of threads

§  thread_pool_add_delay – Reducing the add_delay lets you create threads faster which is essential - specially at startup - to avoid filling up the queue and dropping requests

Now we should edit the default.vcl ( located at : /usr/local/etc/varnish/default.vcl ) and uncomment the backend default section.

This backend default section is the section where we will configure the host and port of the server, where varnish will send the request; if the requested content is not available in varnish cache.

Here the host is set as localhost i.e. 127.0.0.1 because, we have both web server and Varnish server running in the same Server Instance. If you wish to run the varnish in one server and the web server in another separate server, then you can do so but, make sure you provide the IP of the server instance where the web server is installed in the Varnish server’s default.vcl file’s “backend default” section accordingly.

Check Firewall settings of the Server Instance

By default in Red Hat server it has Firewall settings enabled which will block our setup from responding when requested from browser i.e. only port 22 will be open since our requests come via port 80 it will not be allowed. Hence, to allow access we block Red Hat’s Firewall settings to proceed further. Also we will configure in such a way that this firewall block will be effective even on server restarts.

Instead of blocking we can add configurations in iptables so as to allow port 80 also, but this is totally not required since our setup is in AWS. The security group settings of AWS are much secured and there is no need for another security level. Also, this additional security step will cause more latency in response so it is better to block Red Hat’s Firewall settings.

COMMAND’s for executing the above mentioned are as follows,

iptables  -L

ð  Checks for Red Hat Firewall status

/etc/init.d/iptables stop

ð  Stop Red Hat firewall

chkconfig iptables off

ð   Will make sure that the service won't start when server boots up. This removes the service from the start up list

Configure boot startup for Varnish Server

For the Varnish server to start automatically when the server reboots, we need to update the “rc.local” file in the server instance.

The commands for the same is as follows,

vim /etc/rc.local

Update these lines below in the file

/usr/local/sbin/varnishd  -f /usr/local/etc/varnish/default.vcl -s malloc,4G -T 127.0.0.1:2000 -a 10.64.43.79:80 -p thread_pool_min=500 -p thread_pool_max=4000 -p thread_pool_add_delay=2

save the changes made

Related Articles


FAQ - Varnish on Amazon EC2 
Varnish Implementation Architectures in AWS
Configuring Varnish on AWS
Designing Web/App's with Varnish Page Cache in AWS

Performance tuning & Varnish Best practices on AWS