My situation is a little bit complicated:

  • I have a powerful server completely under firewall (no inbound connection from outside)
  • I want to run several websites (mostly blogs)
  • I want to support SSL

At the beginning, DigitalOcean is the best choice. I will have my own server, host unlimited websites, have full control and DigitalOcean is blazingly fast. I selected the smallest plan with 20G SSD and 512MB RAM. It would be more than enough for my blogs. I installed my own LAMP stack, get my own SSL certificate from StartSSL (you should get your own, too. It’s free!)

Everything is fine until after several week. My server crashed every few hours. There are lots of requests coming for wp-comment-post.php, xmlrpc.php and wp-login.php. Unfortunately I can’t disabled them. Apache’s mod_security and mod_qos couldn’t help much. I have to write a temporary cron script to restart apache2 daemon whenever server load bigger than 20.

Doesn’t improved much. My server still crash. There come nginx. Not work.

Then CloudFlare. The same

Until I decided to use my dedicated server to handle requests. Then it works!, not perfectly but we will be there later.

In short, my configuration is like this:

INTERNET < -> CLOUDFLARE < -> NGINX (DIGITALOCEAN) < -> APACHE (MY DEDICATED SERVER)

There are several technical challenges that need to be solved:

  1. How can I forward requests to my dedicated server (completely under firewall)
  2. How can my end point (apache on my dedicated server) recognize IP from visitors correctly (since there are several layers in between?

The solution for my first challenge is actually very simple: SSH Tunnel. There is one catch: Each website in my dedicated server will have to use its own port. And here is why:

  1. Assume I have 2 websites, example.com and codepie.org. I assigned port 81 to example.com and 82 to codepie.org
  2. Assume my dedicated server has IP 1.1.1.1, my DigitalOcean server has IP 2.2.2.2
  3. In my dedicated server, I fire up a SSH connection with the following parameters
    ssh -g username@2.2.2.2 -R 8081:localhost:81 -R 8082:localhost:82
  4. The command above will create a 2-way tunnel (using -g), an open socket in my DigitalOcean server that will forward all requests to my dedicated server on port 81 and 82
  5. Now you can imagine how I forward request from my DigitalOcean server to my dedicated server: using nginx reverse proxy. My configuration looks like this:
    server {
      listen 80 default_server;
      server_name codepie.org www.codepie.org;
      root /dev/null;
    
      access_log  /var/log/nginx/codepie.org_access.log;
      error_log   /var/log/nginx/codepie.org_error.log;
      
      location / {
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-Ssl     on;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto   $scheme;
        proxy_set_header    X-Frame-Options     SAMEORIGIN;
        
        proxy_pass http://localhost:8082/;
      }
  6. That’s it. All requests to codepie.org will be served by my dedicated server at port 82 (or DigitalOcean server port 8082)

Now come the second challenges. This need to be solved at each layer:

  1. CloudFlare. Actually there is nothing can be configured at CloudFlare side. All we need is to add the following lines into /etc/nginx/nginx.conf
http {
    ##
    # CloudFlare
    ##
    set_real_ip_from 199.27.128.0/21;
    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 188.114.96.0/20;
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 104.16.0.0/12;
    real_ip_header   CF-Connecting-IP;
}

These IPs are nginx cache server of CloudFlare. You can find more information here: https://www.cloudflare.com/ips

  1. nginx on DigitalOcean

We need to forward IP information to apache, using this configuration (at host level)

  location / {
    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-Ssl     on;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;
    proxy_set_header    X-Frame-Options     SAMEORIGIN;
    
    proxy_pass http://localhost:8082/;
  1. apache2 on my dedicated server

By using mod_rpaf, apache2 will be able to catch up the correct real IP address

That should be it! If you have any question or suggestion, let me know in the comment section.

My perfect setup (hint: CloudFlare, DigitalOcean, StartSSL, nginx, apache and private servers)
Tagged on:                     

6 thoughts on “My perfect setup (hint: CloudFlare, DigitalOcean, StartSSL, nginx, apache and private servers)

  • August 13, 2014 at 4:17 PM
    Permalink

    Test comment 12345654321

    Reply
    • August 13, 2014 at 4:31 PM
      Permalink

      Another reply with another uniqueID 333222111

      Reply
      • August 13, 2014 at 4:32 PM
        Permalink

        Another one 123123123

        Reply
  • October 24, 2014 at 4:28 AM
    Permalink

    Why not use Nginx to serve your blogs directly, why use Nginx and then Apache ?

    Reply
    • October 24, 2014 at 5:56 PM
      Permalink

      The reason is actually very simple: The droplet at DigitalOcean is very weak and can’t handle > 10 concurrent php-fastcgi connections, so I just use this droplet as nginx proxy, and forward almost all requests to my private server. The apache instance on DigitalOcean is just a backup, in case there is something wrong with my private server.

      Reply
  • October 9, 2015 at 3:01 PM
    Permalink

    Hello, I am having a problem. I have tried your guide, and a number of other guides for creating a self signed certificate.

    No matter what I do, I just can’t get any https to work at all.

    I have continued to get:

    SSL connection error

    ERR_SSL_PROTOCOL_ERROR

    I don’t know what this means. I have checked that the certificate and key are in the write directory.

    This is my nginx config:

    # carsonevans.ca settings
    #

    # Catch http://carsonevans.ca, and http://www.carsonevans.ca
    #
    server {
    listen 80;
    listen [::]:80;
    server_name http://www.carsonevans.ca carsonevans.ca;

    # Redirect to https://www.carsonevans.ca
    return 301 https://www.carsonevans.ca$request_uri;
    }

    # Catch http://www.carsonevans.ca
    #
    server {
    listen 443;
    listen [::]:443;
    server_name http://www.carsonevans.ca;

    # Redirect to http://carsonevans.ca
    return 301 https://carsonevans.ca$request_uri;
    }

    # Catch https://carsonevans.ca
    #
    server {
    listen 443;
    listen [::]:443;
    server_name carsonevans.ca;

    root /usr/share/nginx/carsonevans.ca;
    index index.html index.htm;

    ssl on;
    ssl_certificate /etc/nginx/ssl/server.crt;
    ssl_certificate_key /etc/nginx/ssl/server.key;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    location / {
    try_files $uri $uri/ =404;
    }
    }

    Reply

Leave a Reply