What are Data Bags?

Data bags provide an arbitrary stores of globally available JSON data.

Data Bags are not directly associated with Node or Role attributes. When using Data Bags with a chef server, they are stored on the server and indexed for searching. When using Data Bags with chef-solo, data bags are stored in a directory hierarchy on the machine running chef-solo.

Recipes can load data bags directly or search a data bag for specific values similar to attributes in node indexes.

Store Data Bags In Source Control

If you clone the Chef Repository from Opscode, there is a directory called data_bags. You can create a directory for each bag, and put the JSON files for each item in the bag's directory. This will provide for more easily managing data bags in source control systems and in understanding and use. Access to your data bag structure can then be done through the knife command to load from file.

Data Bag From File

A individual data bag can refer to either a sub-directory of the chef-repo/data_bags directory or a .json file in that directory or one of the sub-directories. From the earlier example of admins:

Either create the new data bag item as a JSON file in the admins directory, or if the item already exists on the Chef Server, write to a file.

Or:

Here is an example of what a complete data bag directory structure could look like:

The directory structure is used by knife as a convenience so you don't have to type the full path when using the data bag from file sub-command.

The data bag itself provides a container of related items. For example, admins and db_users in your output would be separate bags. The "standard_packages" and "global_shell_settings" are things that might be under a "common" bag or whatever makes sense to you for grouping.

Each item in the bag gets its own JSON file. The only structural requirement of items is that they have an id.

Then when using the knife command to load the data bag item from the JSON file, you don't have to specify the path, just the name of the bag and the filename of the item. Knife's data bag from file automatically knows how to find the directory and the file under data_bags.

The command:

Loads the file:

(Note: the bag "BAG_NAME" must exist already.)

If you are not in the root directory for chef, you can also specify the path with this command:

Managing Data Bags With Knife

Data bags can be managed with the knife command line client. See Managing Data Bags With Knife to accomplish this.

Using Data Bags in Recipes

Data bags can be loaded by name using the DSL in a recipe, or can be accessed via the search indexes. When using a single data bag item, use the direct method whenever possible, as it avoids the search index entirely and so has lower overhead. On the other hand, if you plan to loop over all of the items in a data bag, using the search interface will cause Chef to bulk load the items, resulting in lower overhead.

Loading a Data Bag Directly

The recipe DSL provides access to data bags via the data_bag(BAG) method and access to data_bag_items via the data_bag_item(BAG, ITEM) method.

The data_bag method returns an array of the keys of the items that belong to the data bag. For the admins data bag shown in the above examples, we'd use the data_bag method like this:

To get the contents of the 'charlie' data bag item, we use the data_bag_item method. Note that each time you use data_bag_item in a recipe, Chef makes an API call to the server to load the item.

A More Complete Example

If we want to create a user on our systems for each admin in our admins data bag, we might do something like this:

Loading Data Bags through the Search Interface

In some situations you may not know which data bag items you want information from in advance, or you may want to load all of the items in the data bag. In these cases you can use the search index to find the data you want. When searching for data bag items, you provide the name of the data bag to search and the query string as arguments to the search(BAG, QUERY_STRING) method. Using the admins data bag, you can search for items based on any value, for example:

Note that even though the search is returning the data bag items as Chef::DataBagItem objects, you can use these objects just like they were a Hash:

Here's the same "create a user for each admin" recipe as shown above, modified to load the admins using the search interface:

Creating and Editing Data Bags within a Recipe

Danger! If two clients are simultaneously attempting to update a data bag, the data written last wins. This can lead to data loss if the user is not careful about multiple clients writing to data bags concurrently. Altering data bags from the node when using the Open Source chef-server requires giving the node's API client admin privileges. In most cases, this is not advisable. Please use this methods in this section with extreme care.

Data bags can be created within recipes using a variety of methods. The following examples describe how to create and edit data bag and data bag items within recipes by directly using the methods provided by the Chef::DataBag and Chef::DataBagItem objects.

Data Bags and Environments

Data Bags are global and can be accessed by nodes in any environment. For details on strategies that you can use to provide per-environment values in data bags, see Data Bags and Environments on the Environments page.

Using Data Bags with Chef Solo

Chef 0.10.4+ Only Since 0.10.4 Data bags are available for use in Chef Solo as well as in the Chef Server.

Data bags can also be used by chef-solo. Chef solo loads data bags from a directory structure on the local file system. The location of the data bag directory is configurable with the data_bag_path configuration option in solo.rb. Each subdirectory within data_bag_path corresponds to a data bag. JSON files within each data bag directory corresponds to a data bag item. This is the same convention used by knife and described in the Data Bags from File section.

Since search is not available in recipes run with chef-solo, you must use the data_bag() and data_bag_item() functions to access data bags and items within a recipe.

Search using Chef Solo A community member, Edelight, created a chef-solo search implementation for data bags. This functionality is provided via a cookbook that can be found on GitHub: https://github.com/edelight/chef-solo-search

Cookbooks

Environments