Building a cache server using Varnish and NGINX for high availability websites

In this tutorial I intend to show you how to reverse proxy a website with no SSL using NGINX, cache it into Varnish and delivery the cached content with SSL on the 443 port. You can use this structure to solve issues like broken SSL on websites that you have no control and no access (which is my case).

What is Varnish?

Varnish is powerful and flexible open-source HTTP accelerator and also a reverse proxy. It caches the content from a specified IP/domain inside the RAM memory and serve it on it’s own port, 6081 (you can change it, if needed). It speeds up your content delivery. You can also use Varnish to keep your website up even when the main server is down.

What is Nginx and why use it with Varnish?

NGINX is a high performance webserver, a mail proxy server, and a generic TCP/UDP proxy server. You can use it to host high performance websites and applications, to proxy webservers and to do reverse proxy.

You can use Varnish to put content inside the 80 port and to reverse proxy, but NGINX gives you more options than Varnish. If you want to cache a website that is under SSL, Varnish won’t do that by itself. You’ll need another software to make the request on 443 port under SSL. NGINX do that. It brings more dynamism into your cache server.


Requisites to this tutorial

  • A running server with Debian 9 and SSH enabled;
  • A running website without SSL certificate;

Note: This tutorial may not works on other Debian versions.

So, let’s begin \o/

Step 0 – Basic server settings

I’m assuming that you’re using a fresh Debian 9 installation. If not, you can jump to step 1.

The first thing we always should do in a fresh Linux install is updating and upgrading our packages. We also must to set the server’s timezone.

$ sudo apt-get update && sudo apt-get upgrade
$ sudo ln  -sf /usr/share/zoneinfo/America/Sao_Paulo /etc/localtime

Change “/America/Sao_Paulo” with your location timezone.

Step 1 – Install NGINX and configure the proxy

After upgrading your packages, you can install NGINX.

$ sudo apt-get install nginx

Now, we must to configure the nginx.conf file located at /etc/nginx/nginx.conf . Use your favorite text editor to do that.

$ sudo vi /etc/nginx/nginx.conf

Delete everything inside the file and paste the text bellow. Don’t forget to change the values with your matching data.

events {}
http {
    server {
    server_name proxy.yourdomain.com;
    access_log /var/log/nginx/proxy.yourdomain.com.log;
    error_log  /var/log/nginx/proxy.yourdomain.com.log;

    location / {
        proxy_pass http://35.200.200.200:80;}
        proxy_set_header Host yourdomain.com;

    listen 8080;
    server_name yourdomain.com;

}
}

In short, we’re telling to NGINX to reverse proxy all the exposed content with the header yourdomain.com located at 35.200.200.200:80 and put the content at your server’s 8080 port.

Save, exit and reload the NGINX configuration.

$ sudo nginx -s reload

If you want to test the proxy, change the 8080 port to 80 and request http://35.200.200.200 on your web browser (change to your ip). Remind to change it back to 8080 port before going on.

Step 2 – Install Varnish and configure

To install and run Varnish on Debian, simply use:

$ sudo apt-get install varnish
$ sudo service varnish start

By default, varnish reads the content from localhost on 8080 port, which is the same port that we’re proxying the content on NGINX. So, it won’t be necessary editing the default.vcl file located at /etc/varnish/

You can edit the default.vcl file to configure your cache memory, availability and other stuffs.

Step 3 – Configure NGINX to listen the 80 port

Now we need to put our 8080 port content on 80 port.
Open your nginx.conf located on /etc/nginx/, go to the last line (before the last brace) and paste this:

    server {
    server_name yourdomain.com;
    access_log /var/log/nginx/yourdomain.com.log;
    error_log  /var/log/nginx/yourdomain.com.log;

    location / {
        proxy_pass http://127.0.0.1:6081;}
        proxy_set_header Host yourdomain.com;

    listen 80;
    server_name yourdomain.com;

}
}

Reming to change the values with your data.

Save, exit and reload your NGINX configuration.

You can check your ports usage using this:

$ sudo netstat -plnt

If everything is ok, you should see something like the screenshot bellow.

Usage of my server ports

To test it, open a browser and type: http://your.server.ip
If your configuration is right, you’ll be able to see your cached content on the page.

Now you can point your domain to the cache server.

Step 4 – Optional – Get SSL credentials using Certbot

To this step, you’ll need to point your domain to your cache server using your domain DNS.

Ok. We got a working cache server responding on 80 port and a reverse proxy pointing to the website server, but, what if someone tries to access it under HTTPS? It will not work. We need to add some SSL credentials.

Let’s create our certificate and install it on our server. To do this, we will use Certbot to get a free SSL certificate from Let’s Encrypt.

Certbot has a NGINX module that is really helpful. Get it using:

$ sudo apt-get install certbot
$ sudo apt install python-certbot-nginx

Now create your certificate and add it into your NGINX configuration:

$ sudo certbot --nginx

Enter your e-mail and agree the license. It will ask if you want to subscribe on their mailing list. Then, it will show all your NGINX configured name servers. Select the one equivalent to yourdomain.com and chose to redirect HTTP to HTTPS.

And it’s done. Open your browser and type https://yourdomain.com

If it doesn’t works, try cleaning all the caches of your web browser and try again.

Now you have your website responding from a cache server and running under SSL encryption, all from outside. Amazing, isn’t? =)

Any suggestions? Leave a comment

One Comment

Leave a Reply

Your email address will not be published. Required fields are marked *