Deprecated! This page is deprecated. We recommend using Knife to manage Cloud Computing based servers.

Goal

Deploy a three node infrastructure for hosting a Rails based web application using Opscode's public AMIs.

1. Operations Master, providing centralized control over the infrastructure.
2. Web application server, running the Rails application.
3. Database server, running MySQL to store the application data.

Requirements

• Amazon Web Services account, keys, certificates.
• AWS EC2 API commandline tools.
• Security group 'rails_infra'.
• Keypair 'rails_infra'.
• Ubuntu 9.04
• Chef 0.6.2
• New! Chef 0.7.0, see below.

Assumptions

While we will cover setting up the EC2 environment in detail at a later time, to get started the EC2 API tools need to be in the user PATH and execute without specifying the keys and certificates for every command.

Getting the Images

We have two images available. Both were built from the Alestic Ubuntu 9.04 public AMI.

First is an 'operations master' node, which will be the Chef Server, OpenLDAP master, central Rsyslog server, and monitoring/trending server.

Second is a 'client' node, which can be either an web application server, or a database node. Or anything else a user wants to add to their environment!

• Ops Master AMI: ami-6d20c604 - Chef 0.6.2
• Ops Master AMI: ami-756b8d1c - Chef 0.7.0 (preliminary release!)
• Client AMI: ami-9d23c5f4

The AMIs will be updated with new features and improvements, the latest will always be posted here, and we'll use some generic names for AMIs and instances in the examples below.

Chef Repository

We are using a new Chef Repository, which can be found here. If the repo is private, don't worry, it is already checked out on the operations master AMI, in /root/rails_infra_repo. This repository started as a bare chef-repo, with the addition of site-cookbooks to accomplish the goals listed above. The repo on the operations master also includes the Opscode Cookbooks.

All that said, let's get rolling!

Bootstrap Operations Master Node

Start up a new instance using the ops master AMI using the ec2-run-instances command.

Note the instance number, and check on its progress with ec2-describe-instances (ec2din).

When it is ready, ssh as root, using the key from rails_infra keypair.

Now we'll handle a few setup tasks and the master system will be ready to cook up some piping hot infrastructure.

For the Chef 0.7.0 AMI, we'll lock down the Chef server so only some_user can login with their 'myopenid'. Change this to reflect your OpenID.

We'll stop the Chef Client while we're working on getting it up and running.

Next, let's get the fully qualified domain name (fqdn) of the ops master all lower case (due to an issue with mixed case internally in Chef).

Go into the rails_infra_repo directory.

Note at this time the repo is on the 'chef070' branch if you're using the AMI for Chef 0.7.0.

Configure the Rakefile for the local environment (company name, locality, etc) by editing config/rake.rb.

Generate self signed certificates for use by OpenLDAP and copy the certificate to the site-cookbooks.

Since the Chef Server webui uses OpenID and we have a publicly available server, we want to have the authorized OpenID providers and identifiers locked down. To do this, edit site-cookbooks/chef/templates/default/server.rb.erb.

On Chef 0.7.0, this should match the values used above in /etc/chef/server.rb.

Modify the providers list to allow the external provider sites (myopenid.com, blogger.com, etc) and add the URL for each specific identifer that should be authorized. Note that when signing in, this must match exactly, so if 'some_user.myopenid.com' is entered, 'http://some_user.myopenid.com' will fail to login on the webui.

We will pass a JSON attributes file to Chef to tell it how to configure this node. Edit config/ops_master.json as applicable. Replace 'chef.example.com' and 'ldap.example.com' with the value from $my_fqdn. The openldap: rootpw should be a password hash that openldap supports, for example SSHA, generated from slappasswd -s.

Deprecated configuration! The ops_master.json was the way to go on Chef 0.6.2. With Chef 0.7.0 we use Roles, so read on...

We use Chef 0.7.0's Roles feature here to manage default_attributes for the 'base' server role and the 'ops_master' server role, and we're using the Ruby DSL for Roles. First, edit the 'base' role.

Next edit the 'ops_master' role.

Once the JSON is saved, we'll install the repository on the Chef Server, using rake.

We like to use OpenLDAP for single-sign on. We provide a default LDIF file that should be editted at this point for the local environment. There's a sample user (USERID) and group (sysadmin). Feel free to modify as necessary, replacing the following as indicated:

• USERID - username of the user, eg "jsmith"
• FIRSTNAME - First name of the user, eg "Joseph"
• LASTNAME - Last name of the user, eg "Smith"
• PASSWORD_HASH - Password hash that OpenLDAP supports, see OpenLDAP FAQ for more information.

Now run the Chef Client. Note that we're using the contents of a validation token to automatically authorize the client, and passing the JSON file we just editted.

If the client times out trying to connect, restart Apache2. This doesn't happen every time.

When the client runs, it may fail on configuring the OpenLDAP server (its finicky). Remove /etc/ldap/slapd.conf and /etc/ldap/slapd.d and rerun the client.

It should finish okay. The slapd process won't be started because we'll now use slapadd to enter the LDIF file.

If there's any parse errors, fix those and rerun. Once the directory is loaded from LDIF, start slapd or rerun chef-client. If there's an object class error, the /etc/ldap/slapd.d directory contains stale config from the package installation; remove it and slapd.conf as above and rerun chef-client.

The operations master is ready! The user created in LDAP will be available, though due to the default SSH settings, will need an SSH public key added. Assuming '$ldap_user',

Obviously this is a great place where Chef can help, but out of scope for this particular document; we'll talk about it more in [Managing OpenLDAP Users with Chef].

Bootstrap Client Nodes

We will create two host types, "database" and "application". They will use the same steps, but different JSON data files. When we're done, the web site will be available on the public hostname of the application node, running RadiantCMS.

The operations master fully qualified domain name in lowercase is required to ensure the clients are set up correctly using the right server. For example purposes, we'll use domu-12-31-xx-xx-xx-xx.compute-1.internal and we'll use a perl command to substitute the hostname where we need it.

Next, take a look at the JSON file as applicable on the application node (application.json) and database node (database.json), and edit as required. Once modifications are complete, run the chef-client.

In testing, the validation token wasn't working for me, so the client needs to be run, then the node validated on the Server webui, and then the client run again.

The database needs to be available before the application can connect to it of course.

Once the application server is up, the database needs to be bootstrapped using the rake task available from RadiantCMS. We'll do this as the 'railsdev' user.

Select defaults, unless the values should be changed per environment.

Now navigate to the public hostname of the application node, and the RadiantCMS site will be available!

At this point, we have built a running Rails application infrastructure in the Amazon cloud. Feel free to modify the site-cookbooks for additional applications and services.

For additional help, stop by IRC, or post on the Mailing Lists.