Skip to end of metadata
Go to start of metadata


There are two resources for deployment, the Deploy Resource - with extensive attributes available to specify use, and the SCM Resources - which can be used within the Deploy Resource, or independently.

There are many approaches and strategies that can be undertaken to use these resources for managing and automating deployment within your infrastructure. The discussion below goes over some of those approaches and provides some examples of use.

SCM Resources

Source Control Management (SCM) Resources, which may be used within the Deploy resource, or independently.

Actions

"sync" is the default action.

Action Description Default
sync Update the source to the specified revision, or get a new clone/checkout  
checkout Clone or checkout the source. Does nothing if a checkout is available  
export Export the source, excluding or removing any version control artifacts  
force_export (Subversion only) Force an export of the source overwriting the existing copy if it exists, excluding or removing any version control artifacts  

Attributes

Attribute Description Default Value
destination Name attribute Path to clone/checkout/export the source to.  
repository URI of the repository  
revision revision to checkout. can be symbolic, like "HEAD" or an SCM specific revision id HEAD
reference (Git only) alias for revision  
user System user to own the checked out code nil (user chef runs as)
group System group to own the checked out code nil(user chef runs as)
svn_username (Subversion only) Username for Subversion operations  
svn_password (Subversion only) Password for Subversion operations  
svn_arguments (Subversion only) Extra arguments passed to the subversion command  
svn_info_args (Subversion only) svn_arguments are not used when chef calls "svn info". If you need arguments to be passed to "svn info", put them in svn_info_args  
depth (Git only) Number of past revisions to include in Git shallow clone nil (full clone)
enable_submodules (Git only) performs a submodule init and submodule update false
remote (Git only) remote repository to use for syncing an existing clone origin
additional_remotes (Git only) Array of additional remotes to add to the repositories configuration
ssh_wrapper (Git only) path to a wrapper script for running SSH with git. GIT_SSH environment variable is set to this.  



Doing Deployment from Your Git Repo?

Don't hit a dead end when you try to pull the code from your private git repo! If you do, one of the following should help:

1. Have a passphrase-less key.
You can strip your current key of the passphrase with:

Run that from /srv/app_name, type your passphrase where 'YOURPASSPHRASE' appears, and the ssh-key will no longer require a passphrase.

or

2. Add the deployment private key to a Data Bag.

The key needs to be a string with the newlines converted to "\n"'s

Providers

Provider Shortcut Resource Default For Platforms
Chef::Provider::Git git  
Chef::Provider::Subversion subversion  

SCM Examples

Did you need the absolute latest version of CouchDB?

grab_couchdb_from_svn

Prefer the git mirror?

grab_couchdb_from_git_mirror

Want to use different branches depending on the environment of the node?

using_different_branches_in_git

In the above example, the branch_name variable is being set to "staging" or "master" depending on the environment of the node. Once that is determined, it uses the branch_name variable to set the revision for the repository.

Keep in mind that if you use the command "git status" after running this recipe it will return the branch name as "deploy" regardless, as this is a default value. You should be able to see it checking out the correct branch when you run chef-client with debugging on:

Deploy Resource

A resource to help you manage and control your deployments. Please note the number of attributes available for specifying behavior, and review the Discussion for strategies and examples of use.

Actions

Action Description Default
deploy Deploy the application yes
force_deploy For the revision deploy strategy, this removes any existing release of the same code version and re-deploys in its place  
rollback Rollback the application to the previous release  

Attributes

Attribute Description Default Value
deploy_to The "meta root" for your application.  
repository URI of the repository  
repo alias for repository  
revision revision to checkout. can be symbolic, like "HEAD" or an SCM specific revision id HEAD
branch alias for revision  
user System user to run the deploy as nil (user chef runs as)
group System group to run the deploy as nil (group chef runs as)  
svn_username (Subversion only) Username for Subversion operations  
svn_password (Subversion only) Password for Subversion operations  
svn_arguments (Subversion only) Extra arguments passed to the subversion command  
shallow_clone (Git only) boolean, true sets clone depth to 5 nil (full clone)
enable_submodules (Git only) performs a submodule init and submodule update false
remote (Git only) remote repository to use for syncing an existing clone origin
ssh_wrapper (Git only) path to a wrapper script for running SSH with git. GIT_SSH environment variable is set to this.  
git_ssh_wrapper alias for ssh_wrapper  
scm_provider SCM Provider to use. Chef::Provider::Git
repository_cache Name of the subdirectory where the pristine copy of your app's source is kept cached-copy
environment A hash of the form {"ENV_VARIABLE"=>"VALUE"}  
purge_before_symlink An array of paths, relative to app root, to be removed from a checkout before symlinking %w{log tmp/pids public/system}
create_dirs_before_symlink Directories to create before symlinking. Runs after purge_before_symlink %w{tmp public config}
symlinks A hash that maps files in the shared directory to their paths in the current release {"system" => "public/system", "pids" => "tmp/pids", "log" => "log"}
symlink_before_migrate A hash that maps files in the shared directory into the current release. Runs before migration {"config/database.yml" => "config/database.yml"}
migrate Should the migration command be executed? (true or false) false
migration_command A string containing a shell command to execute to run the migration  
restart_command A code block to evaluate or a string containing a shell command nil
rollback_on_error Boolean value that controls whether to rollback on error. When true, the deploy resource will rollback to the previously deployed release if an error occurs when deploying the new release. (Available in Chef 0.10.6+) false
before_migrate A block or path to a file containing chef code to run before migrating deploy/before_migrate.rb
before_symlink A block or path to a file containing chef code to run before symlinking deploy/before_symlink.rb
before_restart A block or path to a file containing chef code to run before restarting deploy/before_restart.rb
after_restart A block or path to a file containing chef code to run after restarting deploy/after_restart.rb

Providers

Provider Shortcut Resource Default For Platforms
Chef::Provider::Deploy::TimstampedDeploy timestamped_deploy all
Chef::Provider::Deploy::Revision deploy_revision, deploy_branch  

Discussion

The Deploy Strategies

The deploy resource is ported from Capistrano and works similarly.

In your deploy directory, you will need to create a subdirectory named shared, where your configuration and temporary files will be kept. For a rails application, you will typically have config, log, pids, and system directories within the shared directory to keep the files stored there independent of the code in your source repository.

In addition to these, the deploy process will create subdirectories named releases and current in your deploy directory. The releases directory holds up to the five most recently deployed versions of your application, and the current directory will be a link to the currently released version.

Deployment happens in four phases: checkout, migrate, symlink, and restart.

  1. When the deploy is run, Chef uses the SCM resource to get the specified revision, placing the clone or checkout in a subdirectory of the deploy directory named cached-copy.
    • A copy of your application is then placed in a subdirectory under the releases directory.
  2. Next, if a migration is to be run, Chef symlinks the database configuration file in to the checkout, by default config/database.yml, into the checkout and runs the migration command.
    • For a rails application, migration_command is usually set to rake db:migrate
  3. After migration, the symlink proceeds by removing directories for shared and temporary files from the checkout, by default log, tmp/pids, and public/system.
    • After this step, any needed directories, by default tmp, public, and config, are created if they don't yet exist.
    • The symlink step completes by symlinking shared directories into the current release, public/system, tmp/pids, and log by default, and then symlinking the release directory to current.
  4. Following the symlink step, the application is restarted according to the restart command set in the recipe.

Timestamped Deploy

The two available deploy providers mainly differ in how they name the directories that contain your releases. Although the difference might seem minor, it has a large effect on how the deployment behaves.

The "timestamped" deploy strategy is the default. It names release directories with a timestamp of the form "YYYYMMDDHHMMSS". For example, your release might be located at the path /my/deploy/dir/releases/20040815162342.

The deploy provider determines whether or not to deploy your code based on the existence of the release dir it is attempting to deploy to.

  • Because the timestamp will be different for every chef run, the timestamped deploy provider is not idempotent.
  • If you choose to use this deployment strategy, you will need to manually manage the action setting on the resource to prevent unintended continuous deployment.

Deploy Revision

Although the "revision" deploy strategy is not the default for compatibility reasons, it is the recommended strategy.

With this strategy, the release subdirectory will be named after the revision id used by your SCM. For git users, this will be the familiar SHA checksum; For subversion users, it will be the integer revision number. Note that even if you supply a branch name, tag, or other name instead of the revision id, Chef will lookup the revision id from your SCM and name this directory with the revision id.

As noted above, the deploy providers determine whether or not to deploy based on whether or not the releases directory it is trying to deploy to exists. This means that deploy_revision will be completely idempotent if you specify an exact revision to deploy. You could also use this in more interesting ways, for example, if you have a "deploy" or "production" branch in git, you can set the provider to track this branch. This will cause your deployment to get updated the next time chef runs after pushing to that branch.

The deploy_revision resource results in deployed components under the destination location being owned by the user who runs the application. If this is an issue for your particular workflow, there are three approaches you can undertake:

  • you'll need to incorporate changing permissions to their desired end state within your recipe, or
  • add a before_restart block to fix up the permissions, or
  • have an unprivileged user (e.g., "opscode") as the owner of the deploy directory, and another unprivileged user (e.g., "opscodeapp") run it. This is the approach we'd recommend.

The drawback to the revision deploy strategy is that if the deploy fails for any reason, and you want to re-deploy with the same code, you'll need to manually set the action to :force_deploy. Forcing a deploy will remove the old release directory and then proceed with the deployment as usual. Be forewarned that this can cause downtime if you force a deployment over the current release.

The deployed revisions are stored in (file_cache_path)/revision-deploys/(deploy_path)

Interacting with Your Infrastructure During Deployment

In between each step of the deployment process, you can run arbitrary ruby or chef code using callbacks. The available callbacks are:

  • before_migrate
  • before_symlink
  • before_restart
  • after_restart
  • restart_command

All of the above callbacks support embedded recipes given in a block, but assume a shell command (instead of a deploy hook filename) when given a string.

Each of these callbacks can be used in one of three ways:

  • You can pass a block:
    callback_blocks
  • You can specify a file. Files are searched relative to the current release. The code in the file is evaluated exactly as if you had put that code in a block.
    callback_files
  • If you don't provide a block or file, Chef will look for a file in the current checkout named deploy/callback_name.rb, such as deploy/before_migrate.rb. If it exists, it will be evaluated as above.

Within your callbacks, there are several ways to get access to information about the deployment.

  • As shown in the example above, you can use release_path to get the path to the current release.
  • You can access the deploy resource itself using the new_resource method, allowing you to access the environment variables you've set.
  • If you've relied on access to an @configuration variable using the original chef-deploy gem, you can continue to use that as well, however, new_resource is the preferred way.

These are all available only at the "top level" within callbacks, so you need to assign any values you intend to use to local variables as shown in the callback_blocks example above.

Custom Application Layouts

By default, the deploy resource expects your application to be structured like a rails app, but you can customize the layout for any kind of application.

  • symlink_before_migrate
    symlink_before_migrate is a hash mapping files in the shared directory to files in the current release. The primary use is to link required configuration files into place so they're available during the migration step. The default is
    {"config/database.yml" => "config/database.yml"}
    Note that if you try to do "symlink_before_migrate {}", the "{}" will be interpreted as a block rather than an empty hash. Instead, use "symlink_before_migrate({})" or "symlink_before_migrate nil".
  • purge_before_symlink
    purge_before_symlink is an array of directories which will be removed before the symlink step. This removes directories before the production copy from the shared directory is symlinked in. Defaults to
    ["log", "tmp/pids", "public/system"]
  • create_dirs_before_symlink
    This list is used to create directories that need to exist in the release before the symlink step. To illustrate in the context of a rails app, you might use .gitignore to keep all tempfiles out of your repository and not have the tmp/ directory in your checked out code, but the symlink step will create a symlink from shared/pids to RELEASE/tmp/pids. Since this would fail if the RELEASE/tmp directory did not exist, the directory must be created for the deploy to proceed. Note that it is not necessary to create directories that will be symlinked in the symlink step, such as log/, as these don't rely on the existence of other directories in the release. Directories that already exist won't be overwritten. Defaults:
    ["tmp", "public", "config"]
  • symlinks
    symlinks is a hash mapping files and directories from their locations in the shared dir to locations in the release dir, used in the main symlinking step. Defaults to
    {"system" => "public/system", "pids" => "tmp/pids", "log" => "log"}

How to Nuke and Start Over

Chef keeps track of the order in which you've deployed each revision of your application in a cache file. If you delete the deployed code from a box in order to force a redeployment, you need to delete this cache file as well. Under the default configuration, this cache file is located in /var/chef/cache/revision-deploys/APPNAME/

Deploy Examples

Basic attribute usage

basic_usage

Any recipes using the git-deploy gem can continue using the same API

backwards_compatible_API

The layout of the application matches a Rails app by default, but you can customize it to fit your needs

customizing_the_application_layout

When deploying non-rails code and you don't want any symlinks to the shared folder, use parentheses or Hash.new to avoid an ambiguity in Ruby

How to Deploy Without Symlinking to Shared

Using resources from within your callbacks as blocks or within callback files distributed with your application's source code

using_embedded_recipes_for_callbacks

Open Issues

  • Only One Level of Submodules
    If you have submodules that themselves depend on submodules, the second level of submodules won't be initialized/updated. See the comments below.
  • Directories are Assumed to Exist (CHEF-630)
    You'll need to have most of the directories needed by deploy already created before the deploy runs. This issue is addressed with version 0.10.6+.





Resources


Yum Package Resource



Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Oct 02, 2009

    would be great if enable_submodules works recursively from the start ... (like http://speakmy.name/2008/10/capistrano-and-nested-submodules-in-git/)

    1. Oct 02, 2009

      I believe our code still has this same enable_submodules problem; thanks Edmund I'll look into it!

  2. Dec 21, 2009

    It's not clear how to use the alternate providers. I stumbled around before I figured out I had to say:

    As a side note: Doing this made me realize why I never liked capistrano: Why should my app know where it's source code lives? I shouldn't need to change my app if I move my source code. Also, having the app start/stop itself doesn't make sense because there are so many servers in ruby (mongrel, thin, passenger, unicorn, etc.)  Why should I update my app when I change server technology?

  3. Jul 23, 2010

    I'm getting the release directory sym-linked below the current/ directory, i.e. something like

    /srv/myapp/current/20100723075228 -> /srv/myapp/releases/20100723075228

    instead of

    /srv/myapp/current -> /srv/myapp/releases/20100723075228

    I think here is why http://github.com/opscode/chef/commit/b86c5b063f6544b3767abec03ce0509aaa8c372b#L2R150 but can somebody check that?

  4. Aug 07, 2010

    There also seems to be a bug with caching & failed deploys. Chef will write to $CACHE/revision-deploys at the start of a deploy. If the deploy fails, the next chef-client run will not even try to deploy b/c it sees the latest deploy already in this cache. Perhaps chef should wait to write to the cache until after the deploy block exits successfully?

  5. Aug 27, 2010

    How do you change the deployment strategy? Above it says: "The 'timestamped' deploy strategy is the default" but I can't figure out how to change the default!

    1. Aug 27, 2010

      I figured it out--instead of:

      deploy "xxx" do
      ...
      end

      you need to do

      deploy_revision "xxx" do
      ...
      end

  6. Sep 10, 2010

    "For a rails application, you will typically have config, log, pids, and system directories within the shared directory"

    I think 'config' should be removed from this sentence. When I deployed a Rails app, the config dir didn't get symlinked. It's not capistrano-y to do so, anyway.

    EDIT: the individual files seem to get symlinked. Oops!

    1. Feb 14, 2011

      Yep. Capistrano was written before config management was popular in the rails community and one of the design goals was to allow you to set your production db password without having to check it in to source control, which it accomplishes by reusing the same database.yml and symlinking it in to place. Chef matches this behavior, though it's easy enough to grab your db password from a data bag and render your database.yml from a template.

  7. Mar 03, 2011

    Any ideas on what to do with SSH keys and Github? copying the key to Github for each server is rather arduous. Maybe this isn't the correct way to do it?

    1. May 19, 2011

      Will:

      I am new to this, so if my answers seems overly simple please say so. That said, I simply put the private key in template/default and included this in the recipe that configures the machine that my app lives on:

      Bar.ruby

      So I am not sure why this would be "arduous". I manage about 20 servers this way.

      1. Oct 13, 2011

        Since it's a private key, I'd suggest an encrypted data bag instead of storing it semi-openly in your chef-repo.

    2. Jan 29, 2012

      One way seems to be via ssh_wrapper, although it's a bit messy. Basically you create a ssh private/public key pair that's used for deployment only, and enter the public key part in the "Admin" section the github repo. Then, to use the private key during deployment itself, you write it to a file, the create an ssh_wrapper shell script that uses the key. You can see this being done in the application::php recipe:

      https://github.com/opscode/cookbooks/blob/master/application/recipes/php.rb

      (Search for deploy_key.)

  8. Apr 27, 2011

    I was surprised by the deploy_revision resource changing ownership of everything under destination to be owned by user.

    This doesn't seem to be what I want, for a Rails application. I'd like shared/logs to be writeable by the user running my rails app, but the majority of the code should not be writeable by that same user. If my permissions are rw-r-----, then changing ownership to a single-user makes it difficult to meet this goal.

    For now I've added a before_restart block to fix up the permissions.

  9. Nov 22, 2011

    Is it possible to have multiple callbacks?
    I'm using the default application recipe in my setup, but I'm intentionally not triggering the bundle command, as I have some tasks I need to perform first.

    Application recipe automatically does a deploy_revision and before_migrate block, does this prevent me from doing one of my own in myApp recipe?