Skip to end of metadata
Go to start of metadata



Spiceweasel is a command-line tool for batch loading Chef infrastructure.

It provides a simple syntax in either JSON or YAML for describing and deploying infrastructure in order with the Chef command-line tool Knife.

Please note, Spiceweasel is not an "official Chef tool", but it is developed by Opscode team member Matt Ray.


Requirements

  • Spiceweasel currently depends on knife to run commands for it. Infrastructure files must either end in .json or .yml to be processed.
  • It is written with Chef 0.9.12 and the 0.10.x series and supports cookbooks, environments, roles, data bags and nodes.
  • Also, as Spiceweasel is not an "official Chef tool", it is not supported for Hosted Chef customers.

Installation

The source code is available on github, but Spiceweasel is distributed and installable as a gem.

Syntax

The syntax for the spiceweasel file may be either JSON or YAML format of Chef primitives describing what is to be instantiated. Below are 2 examples describing the same infrastructure.



OpenStack Swift, Chef and SpiceWeasel

Our friends at Voxel put together two great blog posts on the use of OpenStack, Chef, and Spiceweasel to automate the deployment, configuration and management of their infrastructure.

Read Part One, and Part Two, and love the weasel.


Spiceweasel 1.0

For more information, check out the blog post detailing Spiceweasel 1.0 put together by Opscode Team Member Matt Ray.
YAML

Given the `example.yml`:

Spiceweasel generates the following knife commands:

Cookbooks

The `cookbooks` section of the manifest currently supports knife cookbook upload FOO where `FOO` is the name of the cookbook in the `cookbooks` directory. The default behavior is to download the cookbook as a tarball, untar it and remove the tarball. The --siteinstall option will allow for use of knife cookbook site install with the cookbook and the creation of a vendor branch if git is the underlying version control. If a version is passed, it is validated against the existing cookbook `metadata.rb` and it must match the `metadata.rb` string exactly. You may pass any additional arguments if necessary.

Assuming the apt cookbook is not present, the YAML snippet

produces the knife commands

Environments

The `environments` section of the manifest currently supports knife environment from file FOO where `FOO` is the name of the environment file ending in `.rb` or `.json` in the `environments` directory. Validation is done to ensure the filename matches the environment and that any cookbooks referenced are listed in the manifest.

The YAML snippet

produces the knife commands

Roles

The `roles` section of the manifest currently supports knife role from file FOO where `FOO` is the name of the role file ending in `.rb` or `.json` in the `roles` directory. Validation is done to ensure the filename matches the role name and that any cookbooks or roles referenced are listed in the manifest.

The YAML snippet

produces the knife commands

Data Bags

The `data bags` section of the manifest currently creates the data bags listed with knife data bag create FOO where `FOO` is the name of the data bag. Individual items may be added to the data bag as part of a JSON or YAML sequence, the assumption is made that they `.json` files and in the proper `data_bags/FOO` directory. You may also pass a wildcard as an entry to load all matching data bags (ie. `*`). Encrypted Data Bags are supported by listing `secret filename` as the first item (where `filename` is the secret key to be used). Validation is done to ensure the JSON is properly formatted, the id matches and any secret keys are in the correct locations.

Assuming the presence of `dataA.json` and `dataB.json` in the `data_bags/data` directory, the YAML snippet

produces the knife commands

Nodes

The `nodes` section of the manifest bootstraps a node for each entry where the entry is a hostname or provider and count. A shortcut syntax for bulk-creating nodes with various providers where the line starts with the provider and ends with the number of nodes to be provisioned. Windows nodes need to specify either `windows_winrm` or `windows_ssh` depending on the protocol used, followed by the name of the node(s). Each node requires 2 items after it in a sequence. You may also use the --parallel flag from the command line, allowing provider commands to run simultaneously for faster deployment.

The first item after the node is the run_list and the second are the CLI options used. The run_list may be space or comma-delimited. Validation is performed on the run_list components to ensure that only cookbooks and roles listed in the manifest are used. Validation on the options ensures that any Environments referenced are also listed. You may specify multiple nodes to have the same configuration by listing them separated by a space.

The YAML snippet

produces the knife commands

Usage

To run a spiceweasel file, run the following from you Chef repository directory:

or

This will generate the knife commands to build the described infrastructure. Infrastructure files must end in either `.json` or `.yml`.

-c --knifeconfig
Specify a `knife.rb` configuration file to use with the knife commands.

--debug
This provides verbose debugging messages.

-d --delete
The delete command will generate the knife commands to delete the infrastructure described in the file. This includes each cookbook, role, data bag, environment and node listed. Node deletion will specify individual nodes, attempt to pass the list of nodes to the cloud provider for deletion, and finish with `knife node bulk delete`. If you are mixing individual nodes with cloud provider nodes it is possible that nodes may be missed from cloud provider deletion and you should double-check (ie. knife ec2 server list).

--dryrun
This is the default action, printing the knife commands to be run without executing them.

-h --help
Print the currently-supported usage options for spiceweasel.

--novalidation
Disable validation ensuring existence of the cookbooks, environments, roles, data bags and nodes in infrastructure file.

--parallel
Use the GNU `parallel` command to execute `knife VENDOR server create` commands that may be run simultaneously.

-r --rebuild
The rebuild command will generate the knife commands to delete and recreate the infrastructure described in the file. This includes each cookbook, role, data bag, environment and node listed.

--siteinstall
Use the 'install' command with 'knife cookbook site' instead of the default 'download'.

-v --version
Print the version of spiceweasel currently installed.







Shef


Documentation



Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Jun 04, 2011

    I think you should not embed the knife options as a single line in the node object, but instead use a hash where the keys are the full text names of the options as a child of nodes. You should also have the run_list as one of the options and have the actual roles and recipes as an array under the run_list key:

    Example node

    You could also have a special key for passing knife options as you are doing now as a fall back/compatability option.

    If you are going to have the mechanism to specify the quantity of the nodes to generate (which is not passed to the knife command, but it is used to determine how many times to run the command), it should be an explicit key/option.

    It should also take the node-name and append a number to the node-name that is the index of the loop.

    I would suggest adding another parameter that is also not passed to knife but is used as an offset for the number appended to the node-name. So if you add nodes of the same basename you can explicitly specify the starting number of the next batch of nodes of the same basename.

    You can also make an option to append the environment to the name or some other string to further namespace node-names if need be.

    1. Jun 06, 2011

      I'm not sure I want to introduce additional complexity to the yaml and json files for Spiceweasel. By keeping the syntax restricted and simple, it's easier to share and understand the files. By adding a hash of all the available options, the format of the files will greatly change, introducing a lot more verbosity to the file. I prefer having the list of CLI options because in my opinion it encourages looking those up and becoming more familiar with knife, rather than introducing another layer of abstraction.

      Of course Spiceweasel is free to fork, so perhaps if you implement it and make it an additional option we can see how that looks, but I don't believe I want to add that myself.