Opscode
Home     Introduction to Chef     Cookbooks     Blog     GitHub     Tickets 

Deploy Resource

SCM Resources


SCM Resources are used by the deploy resource but can be used independently as well.

Actions

There's no default action in 0.7.x. "sync" is intended to be the default in 0.8.x.

Action Description Default
sync Update the source to the specified revision, or get a new clone/checkout  
checkout Clone or checkout the source. Does nothing if a checkout is available  
export Export the source, excluding or removing any version control artifacts  

Attributes

Attribute Description Default Value
destination Name attribute Path to clone/checkout/export the source to.  
repository URI of the repository  
revision revision to checkout. can be symbolic, like "HEAD" or an SCM specific revision id HEAD
reference (Git only) alias for revision  
user System user to own the checked out code nil (user chef runs as)
group System group to own the checked out code nil(user chef runs as)
svn_username (Subversion only) Username for Subversion operations  
svn_password (Subversion only) Password for Subversion operations  
svn_arguments (Subversion only) Extra arguments passed to the subversion command  
depth (Git only) Number of past revisions to include in Git shallow clone nil (full clone)
enable_submodules (Git only) performs a submodule init and submodule update false
remote (Git only) remote repository to use for syncing an existing clone origin
ssh_wrapper (Git only) path to a wrapper script for running SSH with git. GIT_SSH environment variable is set to this.  

Providers

Provider Shortcut Resource Default For Platforms
Chef::Provider::Git git  
Chef::Provider::Subversion subversion  

Examples

Did you need the absolute latest version of CouchDB?

grab_couchdb_from_svn

Prefer the git mirror?

grab_couchdb_from_git_mirror

Deploy Resource


Actions

Action Description Default
deploy Deploy the application yes
force_deploy For the revision deploy strategy, this removes any existing release of the same code version and re-deploys in its place  
rollback Rollback the application to the previous release  

Attributes

Attribute Description Default Value
deploy_to The "meta root" for your application.  
repository URI of the repository  
repo alias for repository  
revision revision to checkout. can be symbolic, like "HEAD" or an SCM specific revision id HEAD
branch alias for revision  
user System user to run the deploy as nil (user chef runs as)
group System group to run the deploy as nil (group chef runs as)  
svn_username (Subversion only) Username for Subversion operations  
svn_password (Subversion only) Password for Subversion operations  
svn_arguments (Subversion only) Extra arguments passed to the subversion command  
shallow_clone (Git only) boolean, true sets clone depth to 5 nil (full clone)
enable_submodules (Git only) performs a submodule init and submodule update false
remote (Git only) remote repository to use for syncing an existing clone origin
ssh_wrapper (Git only) path to a wrapper script for running SSH with git. GIT_SSH environment variable is set to this.  
git_ssh_wrapper alias for ssh_wrapper  
scm_provider SCM Provider to use. Chef::Provider::Git
repository_cache Name of the subdirectory where the pristine copy of your app's source is kept cached-copy
environment A hash of the form {"ENV_VARIABLE"=>"VALUE"}  
purge_before_symlink An array of paths, relative to app root, to be removed from a checkout before symlinking %w{log tmp/pids public/system}
create_dirs_before_symlink Directories to create before symlinking. Runs after purge_before_symlink %w{tmp public config}
symlinks A hash that maps files in the shared directory to their paths in the current release {"system" => "public/system", "pids" => "tmp/pids", "log" => "log"}
symlink_before_migrate A hash that maps files in the shared directory into the current release. Runs before migration {"config/database.yml" => "config/database.yml"}
migrate Should the migration command be executed? (true or false) false
migration_command A string containing a shell command to execute to run the migration  
restart_command A code block to evaluate or a string containing a shell command nil
before_migrate A block or path to a file containing chef code to run before migrating deploy/before_migrate.rb
before_symlink A block or path to a file containing chef code to run before symlinking deploy/before_symlink.rb
before_restart A block or path to a file containing chef code to run before restarting deploy/before_restart.rb
after_restart A block or path to a file containing chef code to run after restarting deploy/after_restart.rb

Providers

Provider Shortcut Resource Default For Platforms
Chef::Provider::Deploy::TimstampedDeploy timestamped_deploy all
Chef::Provider::Deploy::DeployRevision deploy_revision, deploy_branch  

Discussion

The Deploy Strategies

The deploy resource is ported from Capistrano and works similarly. In your deploy directory, you will need to create a subdirectory named shared, where your configuration and temporary files will be kept. For a rails application, you will typically have config, log, pids, and system directories within the shared directory to keep the files stored there independent of the code in your source repository. In addition to these, the deploy process will create subdirectories named releases and current in your deploy directory. The releases directory holds up to the five most recently deployed versions of your application, and the current directory will be a link to the currently released version.

Deployment happens in four phases, checkout, migrate, symlink, and restart. When the deploy is run, Chef uses the SCM resource to get the specified revision, placing the clone or checkout in a subdirectory of the deploy directory named cached-copy. A copy of your application is then placed in a subdirectory under the releases directory. More on this below.

Next, if a migration is to be run, Chef symlinks the database configuration file in to the checkout, by default config/database.yml, into the checkout and runs the migration command. For a rails application, migration_command is usually set to rake db:migrate

After migration, the symlink proceeds by removing directories for shared and temporary files from the checkout, by default log, tmp/pids, and public/system. After this step, any needed directories, by default tmp, public, and config, are created if they don't yet exist. The symlink step completes by symlinking shared directories into the current release, public/system, tmp/pids, and log by default, and then symlinking the release directory to current.

Following the symlink step, the application is restarted according to the restart command set in the recipe.

Timestamped Deploy

The two available deploy providers mainly differ in how they name the directories that contain your releases. Although the difference might seem minor, it has a large effect on how the deployment behaves.

The "timestamped" deploy strategy is the default. It names release directories with a timestamp of the form "YYYYMMDDHHMMSS". For example, your release might be located at the path /my/deploy/dir/releases/20040815162342.

The deploy provider determines whether or not to deploy your code based on the existence of the release dir it is attempting to deploy to. Because the timestamp will be different for every chef run, the timestamped deploy provider is not idempotent. If you choose to use this deployment strategy, you will to manually manage the action setting on the resource to prevent unintended continuous deployment.

Deploy Revision

Although the "revision" deploy strategy is not the default for compatibility reasons, it is the recommended strategy. With this strategy, the release subdirectory will be named after the revision id used by your SCM. For git users, this will be the familiar SHA checksum; For subversion users, it will be the integer revision number. Note that even if you supply a branch name, tag, or other name instead of the revision id, Chef will lookup the revision id from your SCM and name this directory with the revision id.

As noted above, the deploy providers determine whether or not to deploy based on whether or not the releases directory it is trying to deploy to exists. This means that deploy_revision will be completely idempotent if you specify an exact revision to deploy. You could also use this in more interesting ways, for example, if you have a "deploy" or "production" branch in git, you can set the provider to track this branch. This will cause your deployment to get updated the next time chef runs after pushing to that branch.

The drawback to the revision deploy strategy is that if the deploy fails for any reason, and you want to re-deploy with the same code, you'll need to manually set the action to :force_deploy. Forcing a deploy will remove the old release directory and then proceed with the deployment as usual. Be forewarned that this can cause downtime if you force a deployment over the current release.

The deployed revisions are stored in (file_cache_path)/revision-deploys/(deploy_path)

Interacting with Your Infrastructure During Deployment

In between each step of the deployment process, you can run arbitrary ruby or chef code using callbacks. The available callbacks are:

  • before_migrate
  • before_symlink
  • before_restart
  • after_restart
  • restart_command supports embedded recipes given in a block, but assumes a shell command (instead of a deploy hook filename) when given a string.

Each of these callbacks can be used in one of three ways:

  • You can pass a block:
    callback_blocks
  • You can specify a file. Files are searched relative to the current release. The code in the file is evaluated exactly as if you had put that code in a block.
    callback_files
  • If you don't provide a block or file, Chef will look for a file in the current checkout named deploy/callback_name.rb, such as deploy/before_migrate.rb. If it exists, it will be evaluated as above.

Within your callbacks, there are several ways to get access to information about the deployment. As shown in the example above, you can use release_path to get the path to the current release. You can access the deploy resource itself using the new_resource method, allowing you to access the environment variables you've set. If you've relied on access to an @configuration variable using the original chef-deploy gem, you can continue to use that as well, however, new_resource is the preferred way. These are all available only at the "top level" within callbacks, so you need to assign any values you intend to use to local variables as shown in the callback_blocks example above.

Custom Application Layouts

By default, the deploy resource expects your application to be structured like a rails app, but you can customize the layout for any kind of application.

  • symlink_before_migrate
    symlink_before_migrate is a hash mapping files in the shared directory to files in the current release. The primary use is to link required configuration files into place so they're available during the migration step. The default is
    {"config/database.yml" => "config/database.yml"}
  • purge_before_symlink
    purge_before_symlink is an array of directories which will be removed before the symlink step. This removes directories before the production copy from the shared directory is symlinked in. Defaults to
    ["log", "tmp/pids", "public/system"]
  • create_dirs_before_symlink
    This list is used to create directories that need to exist in the release before the symlink step. To illustrate in the context of a rails app, you might use .gitignore to keep all tempfiles out of your repository and not have the tmp/ directory in your checked out code, but the symlink step will create a symlink from shared/pids to RELEASE/tmp/pids. Since this would fail if the RELEASE/tmp directory did not exist, the directory must be created for the deploy to proceed. Note that it is not necessary to create directories that will be symlinked in the symlink step, such as log/, as these don't rely on the existence of other directories in the release. Directories that already exist won't be overwritten. Defaults:
    ["tmp", "public", "config"]
  • symlinks
    symlinks is a hash mapping files and directories from their locations in the shared dir to locations in the release dir, used in the main symlinking step. Defaults to
    {"system" => "public/system", "pids" => "tmp/pids", "log" => "log"}

Examples

Basic attribute usage is shown here:

basic_usage

Any recipes using the git-deploy gem can continue using the same API:

backwards_compatible_API

The layout of the application matches a Rails app by default, but you can customize it to fit your needs:

customizing_the_application_layout

Using resources from within your callbacks as blocks or within callback files distributed with your application's source code:

using_embedded_recipes_for_callbacks

Open Issues

  • Only One Level of Submodules
    If you have submodules that themselves depend on submodules, the second level of submodules won't be initialized/updated. See the comments below.
  • Directories are Assumed to Exist (CHEF-630)
    You'll need to have most of the directories needed by deploy already created before the deploy runs.
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Oct 02, 2009

    Edmund Haselwanter says:

    would be great if enable_submodules works recursively from the start ... (like h...

    would be great if enable_submodules works recursively from the start ... (like http://speakmy.name/2008/10/capistrano-and-nested-submodules-in-git/)

    1. Oct 02, 2009

      AJ Christensen says:

      I believe our code still has this same enable_submodules problem; thanks Edmund ...

      I believe our code still has this same enable_submodules problem; thanks Edmund I'll look into it!

  2. Dec 21, 2009

    Dan DeMaggio says:

    It's not clear how to use the alternate providers. I stumbled around before I f...

    It's not clear how to use the alternate providers. I stumbled around before I figured out I had to say:

    As a side note: Doing this made me realize why I never liked capistrano: Why should my app know where it's source code lives? I shouldn't need to change my app if I move my source code. Also, having the app start/stop itself doesn't make sense because there are so many servers in ruby (mongrel, thin, passenger, unicorn, etc.)  Why should I update my app when I change server technology?

  3. Jul 23

    Evgeniy Dolzhenko says:

    I'm getting the release directory sym-linked below the current/ directory, i.e. ...

    I'm getting the release directory sym-linked below the current/ directory, i.e. something like

    /srv/myapp/current/20100723075228 -> /srv/myapp/releases/20100723075228

    instead of

    /srv/myapp/current -> /srv/myapp/releases/20100723075228

    I think here is why http://github.com/opscode/chef/commit/b86c5b063f6544b3767abec03ce0509aaa8c372b#L2R150 but can somebody check that?

  4. Aug 07

    Dhruv Bansal says:

    There also seems to be a bug with caching & failed deploys. Chef will write...

    There also seems to be a bug with caching & failed deploys. Chef will write to $CACHE/revision-deploys at the start of a deploy. If the deploy fails, the next chef-client run will not even try to deploy b/c it sees the latest deploy already in this cache. Perhaps chef should wait to write to the cache until after the deploy block exits successfully?

  5. Aug 27

    Michael Stillwell says:

    How do you change the deployment strategy? Above it says: "The 'timestamped' de...

    How do you change the deployment strategy? Above it says: "The 'timestamped' deploy strategy is the default" but I can't figure out how to change the default!

    1. Aug 27

      Michael Stillwell says:

      I figured it out--instead of: deploy "xxx" do ... end you need to do deploy...

      I figured it out--instead of:

      deploy "xxx" do
      ...
      end

      you need to do

      deploy_revision "xxx" do
      ...
      end


Copyright © 2009 Opscode, Inc. All Rights Reserved.