How to deploy an Ubuntu 7.10 server with nginx, mysql, ruby, radiant, mongrel (mongrel_cluster), monit and a simple firewall using Capistrano. { john muhl }

Introduction
Building your own Radiant distribution
Pre Capistrano tasks
Capistrano setup
The stack:setup task
The stack:config task
Modifying the deploy task
Good luck
More tasks
Patching Radiant
Adding extensions

Introduction

If you’ve never deployed Radiant it’s strongly recommended that you, at least, read through my other entry so you have some idea of what’s in store. It’d be even better if you went through everything in that entry and came back here with a running server and a blank one. Amazon charges very little by the hour for their Elastic Compute Cloud service if you need something you can beat on and not have to worry about breaking. The only real problem with using EC2 for any kind of trial and error stuff is the waiting around between breaking something and getting back to a blank state. Since I don’t want to be waiting around all the time I decided to do everything in a locally virtualized environment; going from broken back to blank takes about 5 seconds. Of course it takes about an hour to get your virtual machine installed and configured but in the long run it’s well worth the effort. You also want to take the time to get it as close to your real server as possible.

I’m assuming you’ll be deploying to a single server, it’s running Ubuntu 7.10 Server, and on that server you have, a root user account, ssh access and a sources list capable of getting the necessary software. Minimally, your /etc/apt/sources.list needs the following:

deb http://us.archive.ubuntu.com/ubuntu/ gutsy main restricted universe
deb-src http://us.archive.ubuntu.com/ubuntu/ gutsy main restricted universe

deb http://us.archive.ubuntu.com/ubuntu/ gutsy-updates main restricted universe
deb-src http://us.archive.ubuntu.com/ubuntu/ gutsy-updates main restricted universe

deb http://security.ubuntu.com/ubuntu gutsy-security main restricted universe
deb-src http://security.ubuntu.com/ubuntu gutsy-security main restricted universe

The Radiant I’ll be deploying is the empty template, no pages, no styles, no extensions. If you’re building your own Radiant distribution feel free to include whatever extras you want. Keep in mind that including extras will probably complicate some part of getting the basic setup working and part of the point of using Capistrano is to make deploying updates trivially easy. So you may want to hold off including those extras until you get the basic setup running.

Building your own Radiant distribution

I’ve made my Radiant distribution files publicly available, if you like you’re free to use those and skip this section.

At very least you’ll need subversion installed and available in your PATH, optionally you may want to instal piston through rubygems.

svn co -r 848 http://svn.radiantcms.org/radiant/trunk/radiant radiant-0.6.6
cp radiant-0.6.6/config/database.mysql.yml radiant-0.6.6/config/database.yml
svnadmin create svn
svn mkdir file:///svn/trunk --message="create trunk" 
radiant-0.6.6/script/generate instance -d mysql ./radiant
svn co file:///svn/trunk radiant
mkdir radiant/cache
mv radiant/config/database.yml radiant/config/database.yml.sample
cd radiant
svn add *
svn commit --message="Initial import of a Radiant instance" 
svn propset svn:ignore "database.yml" config
svn propset svn:ignore "*.log" log
svn propset svn:ignore "*" cache
svn commit --message="ignore stuff we don't want in the repository" 
svn up
gem install piston
piston import -r 848 http://svn.radiantcms.org/radiant/trunk/radiant vendor/radiant
touch vendor/radiant/REVISION_848
svn add vendor/radiant/REVISION_848
svn commit --message="Freeze Radiant to r848 aka 0.6.6"

Pre Capistrano tasks

If you skipped the previous section and want to use my files, you’ll need a local checkout of them where you can edit as necessary.

svn co http://svn.caenim.com/tags/066

First, we need to install MySQL on the server.

ssh root@host
aptitude -y install mysql-server

In order to save myself needing to bootstrap the database on a new server I’ve dumped an empty database and placed it online. You can use it like so:

wget http://svn.caenim.com/tags/710-config/radiant-066.sql
mysqladmin -u root -p create radiant_live
mysql -u root -p radiant_live < radiant-066.sql

Now you need to setup a Radiant user for database access.

mysql -u root -p
=============================
  grant all on radiant_live.* to radiant@'%' identified by 'supersecretpassword';
  drop user '';
  flush privileges;
  exit;
=============================

Next we need a deploy user.

adduser --ingroup www-data deploy
visudo
=============================
  root    ALL=(ALL) ALL
  deploy  ALL=(ALL) ALL #--let deploy sudo
=============================

Clean up and leave.

rm radiant-066.sql
exit

Setup your database.

cp config/database.yml.sample config/database.yml
nano config/database.yml
=============================
  production:
    adapter: mysql
    database: radiant_live
    username: radiant
    password: supersecretpassword
    host: localhost
=============================

Capistrano setup

gem install capistrano
cd radiant
capify .
nano config/deploy.rb
============================= 
  set :application, "Radiant CMS" 
  set :repository,  "http://svn.caenim.com/tags/066" #--use your address
  set :user, "deploy" #--use root for stack:tasks
  set :deploy_to, "/home/deploy/radiant" 
  set :port, "22" 
  set :deploy_via, "export" #--use copy for file:/// address
  role :app, "example.com" 
  role :web, "example.com" 
  role :db,  "example.com", :primary => true
=============================

Now you should be able to get an error free setup.

cap deploy:setup

If you don’t get any errors you’re ready to start writing tasks.

The `stack:setup` task

nano config/deploy.rb
=============================
  ...
  role :db,  "example.com", :primary => true

  namespace(:stack) do
    desc "Installs server software. This calls 'aptitude' to download and install a base set of server applications and their dependencies. Should be run with 'set :user, “root”'." 
    task :setup, :roles => :web do
      run "aptitude -y update" 
      run "aptitude -y full-upgrade" 
      run "aptitude -y install bison build-essential flex libmysqlclient15-dev libmysql-ruby libssl-dev monit mysql-client ruby-full libzlib-ruby nginx subversion subversion-tools" 
      run "cd /usr/local/src && wget http://rubyforge.org/frs/download.php/35283/rubygems-1.1.1.tgz && tar xzvf rubygems-1.1.1.tgz && cd rubygems-1.1.1 && ruby setup.rb && ln -s /usr/bin/gem1.8 /usr/bin/gem" 
    end
    task :after_setup, :roles => :web do
      run "cd /usr/local/src && wget http://www.tildeslash.com/monit/dist/monit-4.10.1.tar.gz && tar xzvf monit-4.10.1.tar.gz && cd monit-4.10.1 && ./configure && make && make install && ln /etc/monit/monitrc /usr/local/etc/monitrc" 
      run "gem install mongrel_cluster mysql radiant" 
      run "aptitude -y update" 
      run "aptitude -y full-upgrade" 
    end
  end
=============================

The namespace(:stack) line puts your command in the stack namespace, so instead of deploy you prefix the task you want to run with stack. Before you test it change the user to root.

nano config/deploy.rb
=============================
  set :user, "root" #--use root for stack:tasks
============================= 

cap stack:setup

You should see all the stuff you’d expect to see had you run the commands directly on the server. As should be obvious, all we’re doing is is putting a group of shell commands inside a container Capistrano understands as a single task. The after_setup task is detected and run by Capistrano every time you run cap stack:setup, all the magic of associating before and after tasks with the proper parent task is handled by Capistrano.

The `stack:config` task

While this task isn’t necessarily more complex than the setup one it may require a bit more effort to get right; configuring a server is a fairly involved task and there are lots of places for things to go wrong. Again I’ve made my configuration files available online. If you’ve been through the previous entry and have a running Radiant install that’s a great place to get your configuration files from.

If your handling your database.yml like I am then you’ll need to upload it to the server before you run the configure task.

scp config/database.yml deploy@host:database.yml

Of course, you’ll want to point at your own configuration files (or if you think mine are good enough at least thoroughly inspect them, they work for me but that’s no sort of assurance they’ll work you).

nano config/deploy.rb
=============================
  ...
  namespace(:stack) do
  ...
    desc "Configures server software. This uses pre-made configuration files hosted on the internet to configure the server software. You will need to run 'cap stack:setup' before this task. Should be run with 'set :user, “root”'." 
    task :configure, :roles => :web do
      run "wget http://svn.caenim.com/tags/710-config/nginx.conf && wget http://svn.caenim.com/tags/710-config/mongrels.conf && wget http://svn.caenim.com/tags/710-config/assets.conf && wget http://svn.caenim.com/tags/710-config/radiant.conf && wget http://svn.caenim.com/tags/710-config/monitrc && wget http://svn.caenim.com/tags/710-config/mongrels && wget http://svn.caenim.com/tags/710-config/mysql && wget http://svn.caenim.com/tags/710-config/nginx && wget http://svn.caenim.com/tags/710-config/ssh && wget http://svn.caenim.com/tags/710-config/system && wget http://svn.caenim.com/tags/710-config/my.cnf && wget http://svn.caenim.com/tags/710-config/monit && wget http://svn.caenim.com/tags/710-config/firewall.sh" 
      run "mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf-old && mv nginx.conf /etc/nginx/nginx.conf" 
      run "mv mongrels.conf /etc/nginx/mongrels.conf" 
      run "mv assets.conf /etc/nginx/assets.conf" 
      run "mv radiant.conf /etc/nginx/sites-available/radiant.conf && ln -s /etc/nginx/sites-available/radiant.conf /etc/nginx/sites-enabled/radiant.conf && rm /etc/nginx/sites-enabled/default" 
      run "mv /etc/monit/monitrc /etc/monit/monitrc-old && mv monitrc /etc/monit/monitrc" 
      run "mkdir /etc/monit.d && mv mongrels /etc/monit.d/mongrels && mv mysql /etc/monit.d/mysql && mv nginx /etc/monit.d/nginx && mv ssh /etc/monit.d/ssh && mv system /etc/monit.d/system" 
      run "mv /etc/mysql/my.cnf /etc/mysql/my.cnf-old && mv my.cnf /etc/mysql/my.cnf" 
      run "sed -e 's/startup=0/startup=1/g' -i /etc/default/monit" 
      run "update-rc.d -f monit remove && mv /etc/init.d/monit /etc/init.d/monit-old && mv monit /etc/init.d/monit && chmod +x /etc/init.d/monit && update-rc.d monit defaults" 
      run "chmod +x firewall.sh && /bin/bash firewall.sh" 
    end
    task :after_configure, :roles => :web do
      run "mv /home/deploy/database.yml #{deploy_to}/shared/system/database.yml" 
      run "/etc/init.d/nginx stop" 
      run "/etc/init.d/mysql stop" 
      run "/etc/init.d/monit restart" 
      run "monit reload" 
    end
==============================

You could just as easily pack all that into a single shell script, download and execute that. In fact, that’s how I started but quickly realized how painful that was going to be to debug when it didn’t work so I decided to break nearly everything its own run command.

Once you think you’ve got everything in place go ahead and test it.

cap stack:configure

The server should now show a 404 error message.

Modifying the `deploy` task

Now you’ve got the server running and configured it’s time to get your application deployed. Before you run cap deploy:cold let’s override and extend some of Capistrano’s built-in tasks.

nano config/deploy.rb
=============================
  namespace(:stack) do
  ...
  end

  namespace(:deploy) do
    task :start, :roles => :app do
      start_mongrels
    end
    task :stop, :roles => :app do
      stop_mongrels
    end
    task :restart, :roles => :app do
      restart_mongrels
    end
  end
=============================

If you don’t modify the built-in start, stop, restart tasks (in the deploy namespace) Capistrano will assume you’re using fast-cgi and you’ll get errors at the end of cap deploy. Now you need to define the start|stop|restart_mongrels tasks.

nano config/deploy.rb
=============================
  namespace(:deploy) do
  ...
  end

  desc "Start Mongrel cluster." 
  task :start_mongrels, :roles => :app do
    run "/usr/bin/mongrel_rails cluster::start -C #{deploy_to}/current/config/mongrel_cluster.yml --clean" 
  end
  desc "Stop Mongrel cluster." 
  task :stop_mongrels, :roles => :app do
    run "/usr/bin/mongrel_rails cluster::stop -C #{deploy_to}/current/config/mongrel_cluster.yml --clean" 
  end
  desc "Restart Mongrel cluster." 
  task :restart_mongrels, :roles => :app do
    run "/usr/bin/mongrel_rails cluster::restart -C #{deploy_to}/current/config/mongrel_cluster.yml --clean" 
  end
=============================

You don’t need to define the _mongrels tasks inside the deploy namespace, Capistrano can find them. If you’re going to handle your database.yml file like I am you’ll need to eventually link to your current deployment; I do this by extending the built in symlink task. I chose after_symlink because it runs after the new code has been linked to the current directory and before the mongrels are (re)started.

nano config/deploy.rb
=============================
  task :after_symlink, :roles => :app do
    run "ln #{deploy_to}/shared/system/database.yml #{deploy_to}/current/config/database.yml" 
  end
=============================

You may also find, like I did, that occasionally you get strange permissions on the production.log, to remedy this I just extended the built-in after_start|restart tasks.

nano config/deploy.rb
=============================
  task :after_start, :roles => :app do
    run "chmod 666 #{deploy_to}/shared/log/production.log" 
  end

  task :after_restart, :roles => :app do
    run "chmod 666 #{deploy_to}/shared/log/production.log" 
  end
=============================

Lastly, don’t forget to create your mongrel configuration.

nano config/mongrel_cluster.yml
=============================
  ---
  environment: production
  address: 127.0.0.1
  port: 3000
  cwd: /home/deploy/radiant/current
  docroot: public
  pid_file: log/mongrel.pid
  num_processors: 1024
  servers: 2
=============================

And add it all to your subversion repository.

svn add Capfile config/mongrel_cluster.yml config/deploy.rb
svn commit --message="Add Capistrano files and mongrel_cluster.yml"

Good luck

cap deploy:cold

If everything went according to the plan then your 404 page should now be a login screen. If not you should be close at least.

More tasks

If you’re Radiant application is running you’re probably thinking “how else can I use this?” If you’re application isn’t running you’re probably wondering “how do I get this working?” Here’s a task everybody can find useful.

nano config/deploy.rb
=============================
  namespace(:monitor) do
    desc "Watch the logs. Uses sudo for access to mysql logs." 
    task :logs, :roles => :web do
      sudo "tail -f /var/log/nginx/*.log /home/deploy/radiant/shared/log/*.log /var/log/mysql/mysql-slow.log /var/log/mysql.log" 
    end
  end
=============================

cap monitor:logs

While we’re in there we can add a few more useful tasks to the monitor namespace.

nano config/deploy.rb
=============================
  namespace(:monitor) do
    desc "Check status" 
    task :status, :roles => :web do
      sudo "monit status" 
    end

    desc "Check memory" 
    task :ram, :roles => :web do
      run "free -m" 
    end
  end
=============================

Why not add some more process controls?

nano config/deploy.rb
=============================
  namespace(:control) do
    desc "Start nginx" 
    task :start_nginx, :roles => :web do
      sudo "/etc/init.d/nginx start" 
    end
    desc "Stop nginx" 
    task :stop_nginx, :roles => :web do
      sudo "/etc/init.d/nginx stop" 
    end
    desc "Restart nginx" 
    task :restart_nginx, :roles => :web do
      sudo "/etc/init.d/nginx restart" 
    end
    desc "Reload nginx" 
    task :start_nginx, :roles => :web do
      sudo "/etc/init.d/nginx force-reload" 
    end

    desc "Start monit" 
    task :start_monit, :roles => :web do
      sudo "/etc/init.d/monit start" 
    end
    desc "Stop monit" 
    task :stop_monit, :roles => :web do
      sudo "/etc/init.d/monit stop" 
    end
    desc "Restart monit" 
    task :restart_monit, :roles => :web do
      sudo "/etc/init.d/monit restart" 
    end

    desc "Start mysql" 
    task :start_mysql, :roles => :web do
      sudo "/etc/init.d/mysql start" 
    end
    desc "Stop mysql" 
    task :stop_mysql, :roles => :web do
      sudo "/etc/init.d/mysql stop" 
    end
    desc "Restart mysql" 
    task :restart_mysql, :roles => :web do
      sudo "/etc/init.d/mysql restart" 
    end
  end
=============================

Patching Radiant

One of the things I was really hoping would make it into 0.6.56 was patch 626 which brings Radiant’s response caching to nginx. I’m guessing it didn’t make it in due to lack of interest, nginx is after all a minority web server. Whatever the reason we can take care of it ourselves. You can copy/paste from the patch page or grab the patch file. Once you’ve applied the patch to vendor/radiant/app/models/response_cache.rb you need to tell Radiant to use your new proc object.

nano config/environment.rb
=============================
  ...
  # Response Caching Defaults
  ...

  # Response Caching for nginx
  ResponseCache.defaults[:sendfile_proc] = lambda { |request, response, cache_dir, file|
    filename = file.slice(File.expand_path(cache_dir).length, file.length)
    response.headers.merge!('X-Accel-Redirect' => "/cache#{filename}.data")
  }
=============================

Add it to your repository.

svn commit --message="Add some more capistrano tasks and patch Radiant for nginx response caching"

Deploy it.

cap deploy

Adding extensions

If you’re patched Radiant deployed and restarted and everything is ok you can start adding extensions and static assets, checking them into your subversion repository and cap deploy‘ing them. Let’s go through adding the Shards extension.

svn up
piston import -r 848 http://svn.radiantcms.org/radiant/trunk/extensions/shards vendor/extensions/shards
touch vendor/extensions/shards/REVISION_848

nano config/environment.rb
=============================
  # config.extensions = [ :all ]
  config.extensinos = [ :shards, :all ]
=============================

svn add vendor/extensions/shards/REVISION_848
svn commit --message="Add Shards extension"

nano config/deploy.rb
=============================
  namespace(:deploy) do
    desc "Install Shards extension." 
    task :shards, :roles => :app do
      symlink
      run "rake production --rakefile radiant/current/Rakefile radiant:extensions:shards:migrate radiant" 
    end
    task :before_shards, :roles => :app do
      update_code
    end
    task :after_shards, :roles => :app do
      restart
    end
  end
=============================

You could, presumably, put the before|after tasks into the parent task, but again I prefer to break things up as much as possible to make debugging a little easier. As far as I can tell there is not a “standard” way to uninstall Radiant extensions, so you’re on you own if you need rollback tasks for extensions but I’d imagine you need to at least add a database dump to the before_shards task.

Criticism and corrections are welcome. I’ve only just begun using Capistrano, if there is some obvious power it has that I’ve missed let me know.

can design 2.7

How to deploy an Ubuntu 7.10 server with nginx, mysql, ruby, radiant, mongrel (mongrel_cluster), monit and a simple firewall using Capistrano.

Table of contents

Introduction

Building your own Radiant distribution

Pre Capistrano tasks

Capistrano setup

The `stack:setup` task

The `stack:config` task

Modifying the `deploy` task

Good luck

More tasks

Patching Radiant

Adding extensions

can design 2.7

How to deploy an Ubuntu 7.10 server with nginx, mysql, ruby, radiant, mongrel (mongrel_cluster), monit and a simple firewall using Capistrano.

Table of contents

Introduction

Building your own Radiant distribution

Pre Capistrano tasks

Capistrano setup

The stack:setup task

The stack:config task

Modifying the deploy task

Good luck

More tasks

Patching Radiant

Adding extensions

The `stack:setup` task

The `stack:config` task

Modifying the `deploy` task