can design 2.7

How to setup an Ubuntu 7.10 server with nginx, php, mysql, ruby, radiant, mongrel (mongrel_cluster), monit and a simple firewall.

Table of Contents

Introduction

A very quick run through of how to setup an Ubuntu 7.10 server with nginx, php, mysql, ruby, radiant, monit and a simple firewall.

ps. if this is a “fresh” slice don’t forget to set a new root password.

passwd

If you're on a unix (or unix like) system you can edit your Host file to point example.com to your IP address (or domain name) so that the test links work properly. For example, on OS X and Ubuntu you'd add this line to the end of /etc/hosts:

10.0.1.2 example.com

Since we'll need it later on go ahead and install the build tools.

aptitude -y install build-essential

Reference Files

Available here.

nginx Setup

aptitude -y install nginx

That’s it. You can now see the default Welcome to nginx! page.

PHP Setup

aptitude -y install php5-common php5-cgi

First, we need to enable nginx as a front-end for our php-fastcgi backend.

Thanks to cleaner416 who wrote in to point me to this article which details how to setup php-cgi and get access to a .pid file for more graceful control of the process. Namely, being able to have monit keep an eye on it as well (check the monit section for more information). Since this is a clearly superior technique I've updated this section to reflect the previously mentioned article's instructions.

/etc/nginx/sites-available/default
ref: defaultPHP
==================================

server {
    listen 80;
    client_max_body_size 50M;
    server_name localhost;
    root /var/www/nginx-default;
    access_log  /var/log/nginx/localhost.access.log;
    location / {
      index  index.html index.php;
      expires  max;
      break;
    }
    location ~ \.php$ {
      fastcgi_pass   127.0.0.1:9000;
      fastcgi_index  index.php;
      fastcgi_param  SCRIPT_FILENAME  /var/www/nginx-default$fastcgi_script_name;
      include        /etc/nginx/fastcgi.conf;
      expires  max;
    }
    error_page  500 502 503 504  /50x.html;
    location =  /500.html { root  /var/www/nginx-default; }
}

Now we need to create the php fastcgi configuration file.

/etc/nginx/fastcgi.conf
ref: fastcgi.conf
=========================

fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;

In order to control our php-fastcgi processes we'll want to use an init script.

/etc/init.d/php-fastcgi
ref: php-fastcgi
=======================

#!/bin/sh
### BEGIN INIT INFO
# Provides:          php-fastcgi
# Required-Start:    $all
# Required-Stop:     $all
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start and stop php-cgi in external FASTCGI mode
# Description:       Start and stop php-cgi in external FASTCGI mode
### END INIT INFO
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="php-cgi in external FASTCGI mode"
NAME=php-fastcgi
DAEMON=/usr/bin/php-cgi
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
[ -x "$DAEMON" ] || exit 0
[ -r /etc/default/$NAME ] && . /etc/default/$NAME
. /lib/lsb/init-functions
if [ "$START" != "yes" -a "$1" != "stop" ]; then
  log_warning_msg "To enable $NAME, edit /etc/default/$NAME and set START=yes"
  exit 0
fi
export PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS
DAEMON_ARGS="-q -b $FCGI_HOST:$FCGI_PORT"
do_start()
{
  start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null || return 1
  start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --background --make-pidfile --chuid $EXEC_AS_USER --startas $DAEMON -- $DAEMON_ARGS || return 2
}

do_stop()
{
  start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE > /dev/null # --name $DAEMON
  RETVAL="$?"
  [ "$RETVAL" = 2 ] && return 2
  start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
  [ "$?" = 2 ] && return 2
  rm -f $PIDFILE
  return "$RETVAL"
}

case "$1" in
  start)
    [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
    do_start
    case "$?" in
      0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
      2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
    esac
    ;;
  stop)
    [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
    do_stop
    case "$?" in
      0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
      2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
    esac
    ;;
  restart|force-reload)
    log_daemon_msg "Restarting $DESC" "$NAME"
    do_stop
    case "$?" in
      0|1)
        do_start
        case "$?" in
          0) log_end_msg 0 ;;
          1) log_end_msg 1 ;; # Old process is still running
          *) log_end_msg 1 ;; # Failed to start
        esac
        ;;
      *)
      # Failed to stop
      log_end_msg 1
      ;;
    esac
    ;;
  *)
  echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
  exit 3
  ;;
esac
:

Make it executable.

chmod +x /etc/init.d/php-fastcgi

Add it as a start-up script.

update-rc.d php-fastcgi defaults

Create the php-fastcgi configuration file.

/etc/default/php-fastcgi
ref: fastcgi_init.conf
========================

START=yes
EXEC_AS_USER=www-data
FCGI_HOST=localhost
FCGI_PORT=9000
PHP_FCGI_CHILDREN=1
PHP_FCGI_MAX_REQUESTS=1000</code></pre>

Don't forget that the php.ini file you are using is for development mode only. Before going into production mode remember to switch to a proper php.ini file.

mv /etc/php5/cgi/php.ini /etc/php5/cgi/php.ini-development
cp /usr/share/doc/php5-cgi/examples/php.ini-recommended /etc/php5/cgi/php.ini

Finally, start up the php-fastcgi processes.

/etc/init.d/php-fastcgi start

Create a test file.

/var/www/nginx-default/info.php
===============================

<?php phpinfo(); ?>

Reload nginx.

/etc/init.d/nginx force-reload

Point your browser to your info.php file and you'll see the regular PHPInfo screen.

MySQL Setup

aptitude -y install libmysqlclient15-dev libmysql-ruby mysql-server php5-mysql

Enter a strong mysql root user password when prompted.

Ruby Setup

aptitude -y install ruby-full rubygems
gem update --system

If you receive a "Could not find rubygems-update in any repository" error just re-run the command.

gem update --system

Now install the necessary gems.

gem install mongrel_cluster mysql radiant

If you receive a GemRunner error of some kind you just need to make a change to your gem script.

/usr/bin/gem
============

...
require 'rubygems'
require 'rubygems/gem_runner' # this line fixes common GemRunner errors
Gem.manage_gems
...

Then try to install the gems again.

gem install mongrel_cluster mysql rails radiant

Since (as of this writing) the current version of Radiant relies on the older Rails 1.2.6 as opposed to the Rails 2 that we installed in the previous step, we also need to install the older Rails. When Radiant supports Rails 2 you can safely omit this step.

gem install rails -v 1.2.6

Radiant Setup

Create a new Radiant site.

radiant --database mysql /var/www/nginx-default

Configure the Radiant database.

/var/www/nginx-default/config/database.yml
ref: database.yml
==========================================

production:
  adapter: mysql
  database: nginxDefault_live
  username: radiant
  password: supersecretpassword
  host: localhost</code></pre>

Create the Radiant database.

mysql -u root -p
================

create database nginxDefault_live;
grant all on nginxDefault_live.* to radiant@'%' identified by 'supersecretpassword';
flush privileges;
exit;

Run the Radiant rake task.

cd /var/www/nginx-default
rake production db:bootstrap

Enter y when prompted about overwriting, since you have an empty database there’s nothing to overwrite. Then your name, username and password when prompted (if you just press enter your site will be setup with the username admin an the password radiant).

Create a mongrel_cluster config file.

/var/www/nginx-default/config/mongrel_cluster.yml
ref: mongrel_cluster.yml
=================================================

---
environment: production
port: 3000
address: 127.0.0.1
pid_file: log/mongrel.pid
cwd: /var/www/nginx-default/
num_procs: 1
servers: 1

Enable nginx as a frontend for your mongrel_cluster.

/etc/nginx/sites-available/default
ref: defaultROR
==================================

server {
  listen 80;
  client_max_body_size 50M;
  server_name example.com dev.example.com www.example.com;
  root /var/www/nginx-default/public;
  access_log  /var/log/nginx/example.com.access.log  main;
  if (-f $document_root/system/maintenance.html) {
    rewrite  ^(.*)$  /system/maintenance.html last;
    break;
  }
  location ~ \.php$ {
    add_header Content-Type "text/html; charset=utf-8";
    fastcgi_pass   127.0.0.1:9000;
    fastcgi_index  index.php;
    fastcgi_param  SCRIPT_FILENAME  /var/www/nginx-default/public$fastcgi_script_name;
    fastcgi_param  QUERY_STRING     $query_string;
    fastcgi_param  REQUEST_METHOD   $request_method;
    fastcgi_param  CONTENT_TYPE     $content_type;
    fastcgi_param  CONTENT_LENGTH   $content_length;
    expires  max;
  }
  location / {
    proxy_set_header  X-Real-IP  $remote_addr;
    proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect false;
    proxy_max_temp_file_size 0;
    if (-f $request_filename) {
      expires max;
      break;
    }
    if (-f $request_filename/index.html) {
      rewrite (.*) $1/index.html break;
    }
    if (-f $request_filename.html) {
      rewrite (.*) $1.html break;
    }
    if (!-f $request_filename) {
      proxy_pass http://mongrel_cluster;
      break;
    }
  }
  error_page   500 502 503 504  /50x.html;
  location = /500.html {
    root   /var/www/nginx-default/public;
  }
}

Create our upstream server for nginx to connect to.

/etc/nginx/mongrel_cluster
ref: mongrel_cluster
==========================

upstream mongrel_cluster {
server 127.0.0.1:3000;
}

Edit the nginx configuration.

/etc/nginx/nginx.conf
ref: nginx.conf
=====================

worker_processes 1;
events {
  worker_connections 1024;
}
http {
  server_names_hash_bucket_size 128;
  charset  utf-8;
  include /etc/nginx/mime.types;
  default_type application/octet-stream;
  log_format main '$remote_addr - $remote_user [$time_local] '
                  '"$request" $status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';
  access_log /var/log/nginx/access.log main;
  error_log /var/log/nginx/error.log debug;
  sendfile on;
  tcp_nodelay off;
  gzip on;
  gzip_http_version 1.0;
  gzip_comp_level 9;
  gzip_proxied any;
  gzip_types application/javascript application/x-javascript text/css text/html text/javascript text/plain text/xml;
  include /etc/nginx/mongrel_cluster;
  include /etc/nginx/sites-enabled/*;
}

Start the mongrel_cluster.

cd /var/www/nginx-default
mongrel_rails cluster::start

Reload nginx.

/etc/init.d/nginx force-reload

Depending on the setup option you chose during the rake, you can now see either a [homepage or a login screen][nginx].

Make sure php is still working.

mv /var/www/nginx-default/info.php /var/www/nginx-default/public/

Now your info.php will still be accessible at info.php

If you chose the Styled Blog option during the rake task you might notice that your site is missing all of it’s pages and it’s stylesheet. Try re-running the rake task (overwrite your database) and removing the crummy cache.* files Radiant 0.6.4 leaves around.

cd /var/www/nginx-default
rake production db:bootstrap
rm cache.*
mongrel_rails cluster::restart

Then login to the admin interface and clear the page cache and try refreshing in your browser (you may need to clear your browser cache also).

Monit Setup

Unfortunately the monit in the Ubuntu repositories is version 4.8.1 which lacks SMTP authentication. So we have to build the latest version ourselves. However, I still like to use some of the Ubuntu package files.

aptitude -y install monit

Now we'll build the latest version.

aptitude -y install bison flex libssl-dev
cd /usr/local/src
wget http://www.tildeslash.com/monit/dist/monit-4.10.1.tar.gz
tar xzvf monit-4.10.1.tar.gz
cd monit-4.10.1
./configure
make
make install

Edit the monit init script.

/etc/init.d/monit
=================

...
PATH=/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/bin/monit
...

Enable monit to startup.

/etc/default/monit
==================

...
startup=1
...

Edit monit control file.

/etc/monit/monitrc
ref: monitrc
==================

set daemon  120
set logfile /var/log/monit.log
set mailserver mail.example.com port 587 username "monit@emample.com" password "supersecretpassword"
set eventqueue
basedir /var/monit
slots 100
set alert monit@example.com
set alert admin@example.com on { nonexist, timeout, resource, icmp, connection }
set httpd port 2812 and
    use address localhost
    allow localhost
    allow admin:monit
include /etc/monit.d/*

Setup monit monitoring.

mkdir /etc/monit.d

/etc/monit.d/system
ref: monitSystem
===================

check system example # change example to your hostname
    if loadavg (1min) > 4 then alert
    if loadavg (5min) > 2 then alert
    if memory usage > 75% then alert
    if cpu usage (user) > 70% then alert
    if cpu usage (system) > 30% then alert
    if cpu usage (wait) > 20% then alert

/etc/monit.d/mongrel_cluster
ref: monitMongrel
============================

check process example.com with pidfile /var/www/nginx-default/log/mongrel.3000.pid
  group mongrel_cluster
    start program = "/usr/bin/mongrel_rails cluster::start -C /var/www/nginx-default/config/mongrel_cluster.yml --clean --only 3000"
    stop program  = "/usr/bin/mongrel_rails cluster::stop -C /var/www/nginx-default/config/mongrel_cluster.yml --only 3000"
    if totalmem is greater than 55.0 MB for 2 cycles then restart
    if cpu is greater than 80% for 2 cycles then alert
    if cpu is greater than 40% for 4 cycles then restart
    if 5 restarts within 5 cycles then timeout
    if failed port 3000 protocol http with timeout 30 seconds then restart

/etc/monit.d/nginx
ref: monitNginx
==================

check process nginx with pidfile /var/run/nginx.pid
  group nginx
    start program = "/etc/init.d/nginx start"
    stop program  = "/etc/init.d/nginx stop"
    if failed host 127.0.0.1 port 80 protocol http then restart
    if 5 restarts within 5 cycles then timeout
    if cpu is greater than 25% for 2 cycles then alert
    if cpu is greater than 10% for 5 cycles then restart
    if totalmem is greater than 20.0 MB for 5 cycles then restart
    if children is greater than 10 then restart

/etc/monit.d/ssh
ref: monitSSH
================

check process sshd with pidfile /var/run/sshd.pid
  group ssh
    start program = "/etc/init.d/ssh start"
    stop program = "/etc/init.d/ssh stop"
    if failed port 22 protocol ssh then restart
    if 5 restarts within 5 cycles then timeout

/etc/monit.d/mysql
ref: monitMySQL
==================

check process mysql with pidfile /var/run/mysqld/mysqld.pid
  group mysql_server
  start program = "/etc/init.d/mysql start"
  stop program = "/etc/init.d/mysql stop"
  if failed host 127.0.0.1 port 3306 protocol mysql then restart
  if 5 restarts within 5 cycles then timeout

/etc/monit.d/php-fastcgi
ref: monitPHPFastcgi
========================

check process php-fastcgi with pidfile /var/run/php-fastcgi.pid
  group php-fastcgi
  start program = "/etc/init.d/php-fastcgi start"
  stop program  = "/etc/init.d/php-fastcgi stop"
  if totalmem is greater than 20.0 MB for 2 cycles then restart
  if cpu is greater than 80% for 2 cycles then alert
  if cpu is greater than 40% for 4 cycles then restart
  if 5 restarts within 5 cycles then timeout
  if failed host 127.0.0.1 port 9000 type TCP with timeout 30 seconds then restart</code></pre>

Make a link to make things simply work.

ln /etc/monit/monitrc /usr/local/etc/monitrc

Start monit.

/etc/init.d/monit start

Check it.

monit status

You should see version 4.10+ and some information about your monit monitored processes.

Test it.

killall mongrel_rails nginx php-cgi

Wait a few minutes, if all went well the mongrel_cluster, php-fastcgi and nginx processes will be brought back online automatically and you'll get a some emails letting you know about it.

If anyone has a nice way to use monit to monitor your php-cgi processes I'd really like to hear about it.

cleaner416 wrote in to say, I used these instructions for php-fastcgi with great success: http://blog.codefront.net/2007/06/11/nginx-php-and-a-php-fastcgi-daemon-init-script/. Then I was able to use the following conf to monit the php instances: (the mem and cpu limits could probably be tweaked, depending on the slice)

Another update from cleaner416, ...the last line of this conf needed to be tweaked in my case: (wrong port, and don't try with http protocol)

I haven't had time to check this out myself, but thought it should be passed on as quickly as possible. Hopefully, I'll have a chance to test it out and update this page accordingly within the next day.

Firewall Setup

Create a test setup.

/etc/iptables.test.rules
ref: iptables
========================

*filter

-A INPUT -i lo -j ACCEPT
-A INPUT -i ! lo -d 127.0.0.0/8 -j REJECT
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A OUTPUT -j ACCEPT
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7
-A INPUT -j REJECT
-A FORWARD -j REJECT

COMMIT

Use it.

iptables-restore < /etc/iptables.test.rules
iptables-save > /etc/iptables.up.rules

Make sure it gets applied when the system boots.

/etc/network/interfaces
=======================

...
iface lo inet loopback
pre-up iptables-restore < /etc/iptables.up.rules
...

Reboot

Make sure all services restart automatically and your firewall is turned on.

shutdown -r now

Updates & Upgrades

aptitude -y update
aptitude -y safe-upgrade
aptitude -y full-upgrade

Automating Deployment with Capistrano

Read about how to do all of this with Capistrano tasks

Notes

Of course, a generic run through how to setup a basic system will have it’s share of omissions and inaccuracies; when you find them please “let me know”:co. If you find this how-to helpful and you plan to implement a similar system for a production site there are a few things to remember:

Credits

2008/01/23