This page describes how to launch API-driven "Cloud" servers with Knife and automatically run Chef on them. Currently supported services (version 0.9.6):

• Amazon EC2
• Rackspace Cloud
• Slicehost
• Terremark vCloud

These are supported via sub-commands in knife that use the Ruby library fog. During the "bootstrap" process that runs Chef, knife will SSH to the systems, and uses the net-ssh-multi library. Finally, knife uses colorized output via highline. You will need to install the fog, net-ssh-multi and highline gems / libraries on your local management workstation where you use knife.

Once complete, read on for how to continue setting up Knife with your "Cloud" credentials.

Cloud Credentials

Add the Cloud Credentials to your knife.rb. They are specifically named for each of the service providers Knife supports as keys under the knife configuration option.

Add the appropriate lines for your preferred Cloud Computing service to your knife.rb.

Cloud Sub-commands

The sub-commands for cloud management in knife follow the pattern:

Where service is ec2, rackspace, slicehost, or terremark. Each of the services has a server create sub-command that uses their API to launch a new instance of the specified flavor or image with a run list. For example, if I want to launch a new EC2 instance with the webserver role:

See the Knife documentation for more information on the syntax for the subcommands and further options.

Instance Bootstrap

The server creation sub-command bootstrap the instance. The current implementation is opinionated and is focused on Ubuntu servers, but we are working to support other platforms and methods.

When the instance is launched, knife determines whether it is available, and once it is, ssh's to the instance using the specified username (or your local user if none specified) and does the following.

1. Installs Ruby and packages to support installing RubyGems with native extensions.
2. Installs RubyGems from source.
3. Installs Chef from RubyGems.
4. Creates the /etc/chef directory.
5. Writes your validation certificate to /etc/chef/validation.pem.
6. Writes an /etc/chef/client.rb config file.
7. Writes a JSON file, /etc/chef/first-boot.json with the roles and recipes specified as the run list.
8. Executes chef-client with the first-boot.json, connecting to the server in the client.rb.

Once the Chef run is complete, the new instance will be registered with the Chef Server and be saved as a node object. This process does not set up the Chef Client service, you'll need to add a recipe in the run list that does this.

You can see the config file that will be copied by creating one with:

The client.rb and validation.pem will be copied to the specified directory, in this case client-config.

Bootstrap Subcommand

Currently, the bootstrap subcommand is used on EC2 instances. This isn't cloud-provider specific, and can be used on any kind of system, like a bare metal server or a VMware machine.

The bootstrap sub-command is not used by the rackspace, terremark or slicehost server creation yet. There's an open ticket for this, CHEF-1445

SSH Configuration

The bootstrap process uses SSH, so you'll need to configure your system to SSH in a particular way. The ticket CHEF-1386 will make this easier, by allowing you to specify a private key that may be different than your default SSH key.

The easiest way to make sure that SSH uses your Cloud service's SSH key is to add it to your SSH key agent. For example, if you have saved the private key for EC2 to ~/.ssh/ec2_ssh_key:

On Mac OS X Snow Leopard, ssh-agent is already running by default when you use Terminal.

You also need to specify the current user to SSH as. On Canonical's Ubuntu AMIs, this is ubuntu. If you're using an image that uses a different user, specify it with -x in the server create command.

The bootstrap currently SSH's to the IP of the launched instance, so this can be problematic for configuring with ~/.ssh/config. There's an open ticket to resolve this, CHEF-1444.