401 Unauthorized (using node's API client)

Note: This error is very similar to the error in the next section. If you see a line that states c:/chef/client.pem is not present, follow the directions in the next section for 401 Unauthorized (using validator API client) instead. NODENAME below could be any name, but will never be ORGANIZATION-validator like the error in the next section.

Error on Hosted Chef:

Error on Open Source Chef:

Fix:
This error can typically happen for 1 of 3 reasons:

• 1. Your client.pem file is incorrect.
  This can be fixed by deleting client.pem in /etc/chef, deleting the client and node with knife, and re-running chef-client:

  Saving your Node Data The following directions involve deleting the data associated with this node. If the node attributes or run_list would be hard to recreate, you may want to save them:

on management station

on affected node:

When chef-client runs, it will register the API client and generate the correct key.

• 2. You are trying to authenticate with a node_name that is different from the one you used on your first chef-client run.
  This can happen for a number of reasons. For example, if your client.rb file does not specify your node name and you have recently changed the systems hostname. Running chef client with debug logging will allow you to see the node name the client is trying to authenticate with:

  You can fix this by explicitly setting the node name in the client.rb file or with chef-client's -N option to match the name originally used to register. Alternatively, you can re-register using the method described above.

• 3. If you are using Hosted Chef, your node name matches another users username on the platform.
  This issue is documented here: http://tickets.opscode.com/browse/CHEF-2240. If you have unsuccessfully attempted the solutions above, click the Help link from http://manage.opscode.com and request an analyst to verify this is the issue. To rename the node without changing the hostname, follow these steps:
  • Capture you node data by either
    or by
    and copy/paste.
  • Delete the node, client and client.pem on the node.
    • On the node to be renamed:
  • Run the chef client on the node and specify the new node name:
  • Replace your node data
  • Run the chef-client again (and in all future instances) as
    or add
    to the /etc/chef/client.rb file.

To rename your node by changing the host name, delete the node,client and client.pem as described above, change the hostname on the host and run the chef-client as normal.

401 Unauthorized (using validator API client)

Error on Hosted Chef:

Error on Open Source:

Fix:
If you get this error, you most likely need to recreate the validation key. If you lose this file, or perhaps you end up with an empty CouchDB database and no longer have this client in the database, you could get a 401 Unauthorized error when trying to use it.

In Hosted Chef, you can recreate this key by going to http://manage.opscode.com and selecting 'organizations' in the upper right side of the screen. You can then select 'Regenerate validation key' next to the organization you need the key for. You will then want to replace this on the client. Be aware that any other nodes you have will no longer work until you distribute the new validation.pem to them.

Get an "Organization not found." error? If you see this error when trying to recreate the key, it's possible that the client itself was deleted. You will need to create the validator key again and fix it's permissions by using the following directions. In these directions, ORGANIZATION should be replaced by your organizations short name. 1. Click on the 'clients' tab at http://manage.opscode.com 2. Click on the 'create' sub-tab 3. Enter "ORGANIZATION-validator" as the name, substituting your organization's name, and click on the 'create' button 4. Click on the 'Download API Key' link to get a copy of the new ORGANIZATION-validator.pem. Save a copy of this private key in a save place, this will be your organization's new validator key. 5. Now we will need to fix the permissions on the validator client. Click on the 'list' sub-tab. 6. Click on the 'edit permissions' sub-tab 7. Check the 'create' and 'list' checkboxes for the ORGANIZATION-validator client and click on the 'update permissions' button You will then want to copy this key to your nodes, and you can continue following the steps below.

If you've also lost your client_key for knife, you can also regenerate this at http://manage.opscode.com by selecting your username in the upper right and then 'get private key'.

Copy these keys over to your node. Your node should be able to register now, as long as there is not still a client by the same name. If there is, you will need to delete that client first. Note that on Opscode Hosted Chef, you currently will need to delete the node as well, because the default permissions only allow the client that created the node to modify it.

On a Chef Server, you can recreate the key by following these steps. First you'll want to remove your validation key on the server, which is typically stored at /etc/chef/validation.pem. Afterwards you can restart the chef-server to create a new key pair on both the disk and in the database:

The same process works with the webui key pair, which knife uses as the default “admin” key to create initial knife clients:

If you’ve also lost your client key for your knife client, you will need to create another one. Use a new client name unless you’re sure that the server does not still contain a registration for the previous client. After creating the new client, you can delete the old one from the server using ‘knife client delete MY_OLD_CLIENT’ by replacing MY_OLD_CLIENT with the name of the former client.

Copy these keys over to your node. Your node should be able to register now, as long as there is not still a client by the same name. If there is, you will need to delete that client first.

You should now be able to run the chef-client command without errors.

401 Unauthorized (Failed to Authenticate as "NODENAME")

Error on Hosted Chef:

There is an open Hosted Chef bug that user names and client names cannot be the same.

Creating a new client with an id that clashes with an id in the User namespace results in a successful client create.

Trying to use this client, however, results in failed authentication because the client tries to authenticate in the user namespace.

The system currently does not differentiate between the User and Client namespaces during authentication, and this needs to happen.

Future activities to implement name spacing will address this, but pending this bug being fixed, if your proposed NODENAME is already an existing User name - the authentication will fail for this reason. To validate that this is the cause, search the Opscode Community Site Users to see if there is an existing user with the same name as what you attempted to name the node. Changing the NODENAME to being unique will address this error.

If your node name is unique, and there is no overlapping user name, please provide support with the debug output of the chef-client run and we'll determine and address the cause.

401 Unauthorized (Please synchronize the clock)

Error on Hosted Chef:

Error on Open Source:

Fix:
Your system clock has drifted from the actual time by more than 15 minutes. This can be fixed by syncing your clock with an NTP server. You'll need to rerun chef-client after synching NTP.

403 Forbidden

Error on Hosted Chef:

Fix:
On Hosted Chef, this indicates an issue with the permissions that are setup at http://manage.opscode.com. You can get more information on the specific permission to check by re-running chef-client with -l debug at the end of the command to get the debug logs. Once it is done running, you'll need to scroll up in the logs until you find the last HTTP request. There are two different types of permissions issues, object specific and global permissions.

This is an example of a 403 error on a specific object:

What we are specifically looking for is the type of http request, in this case PUT, and where it tried to PUT to, in this case a specific node/object. Because this points to a specific node instead of the nodes group, this will be an issue with the object permissions. If this pointed to the nodes group, this would be an issue with the global permissions on nodes instead. These errors do not just apply to nodes, you may see these errors for any section of the management console such as Roles, Cookbooks or Environments.

Creating Clients with Knife Are you seeing the 403 error when doing a PUT request to /clients/NODENAME whenever you start up a new node? If you are creating the client with knife first, remember to copy the key it provides to /etc/chef/client.pem or chef-client will get a 403 error when trying to create it.

To fix the object permissions, you'll want to follow these steps:
1. Go to http://manage.opscode.com and click on nodes in this case, or wherever you see the request failing
2. For object specific permissions, click on the specific object that showed up in the error, in this case NODENAME
3. Click on the permissions sub-tab. Which permission it needs, depends on the type of request that failed:
GET - Under the client section, make sure it has the READ permission checked next to the NODENAME
PUT - Under the client section, make sure it has the UPDATE permission checked next to the NODENAME
DELETE - Under the client section, make sure it has the DELETE permission checked next to the NODENAME
4. Check the checkboxes needed and click on 'update permissions'.

If this is an issue with the global permissions the error will look a bit different. This is an example of a 403 error on the global permissions:

The difference is that here it is referring to the entire nodes group, instead of a specific object/node.

To fix the global permissions you'd want to follow these steps:
1. Go to http://manage.opscode.com and click on nodes in this case, or wherever you see the request failing
2. Click on the permissions sub-tab. Which permission it needs, depends on which request that failed:
GET - Under the group section, make sure it has the LIST permission checked next to the clients group
POST - Under the group section, make sure it has the CREATE permission checked next to the clients group
3. Check the checkboxes needed and click on 'update permissions'.

More information on the API requests and associated errors can also be found on the Server API page.

Commit or stash your changes before importing cookbooks

This isn't really an error, but can be confusing to new users. When you try to install a cookbook with changes that have not been committed to git you will get this error:

Fix:
You can solve this by committing your changes. For example, this would commit all new changes with the message "updates".

Alternatively, if you do not want to commit the changes, you can save the changes for later (reverting the branch to the state of the latest commit) by running

Afterwards you can re-enter the command to install the cookbook.

No such file or directory - /etc/chef/validation.pem

Error:

Fix:
Make sure your validation.pem (for Chef server Users) or ORGANIZATION-validator.pem (for Hosted Chef Users) is downloaded. Edit your client.rb to point to the location of your validator pem.

Can not find config file

Error:

Fix:
This error is related to bug CHEF-2317 on Windows, and can also happen if you are not in your chef directory on Linux and Mac. You can work around it by supplying the full path to your client.rb. For example, if your client.rb was in C:\chef you could use this command:

Can't convert Array into String

Error:

Fix:
There are a few things that can cause this error. Check the following to resolve this:

1. Make sure you are running this command from your chef-repo directory which has the .chef file in it. Knife will need direct access to the .chef file, which contains it's config.

2. If you are using the --distro switch, confirm the bootstrap file exists in one of the directories it checks:

• bootstrap directory in the installed Chef Knife library (Location varies depending how you installed Chef).
• bootstrap directory in the $PWD/.chef, e.g. in ~/chef-repo/.chef.
• bootstrap directory in the users $HOME/.chef.

In the example above, the bootstrap file could be located at ~/chef-repo/.chef/bootstrap/ubuntu.erb. More information on this is on the Knife Bootstrap and the Custom Knife Bootstrap Script pages.

3. Check the knife.rb file for any relative paths and try using the full absolute path instead.

Directory not found error when uploading cookbooks

Error:

Fix:

The vendored runit is supposed to come with its own .chef directory, but in this case, it was not available, causing the ENOENT.

Windows Specific

These errors below are common on Windows systems and may not apply to Linux/Mac.

Knife cookbook site install

Error:

Fix:
Knife cookbook site install currently doesn't work due to bug CHEF-2250. You can work around this by using knife cookbook site download and then untaring the file:

Cannot find a version for node

Error:

Fix:
This is fixed in Ohai versions 0.6.8 and above. You can update to it with:

Alternatively, you can downgrade your Ruby to 1.8.7 to get around this error.

No such file to load – ruby-wmi

Error:

Fix:
Install the Ruby Gems needed for Windows:

Chef Server Specific

These errors are specific to using the Open Source Chef Server, and are not something you will see when using Hosted Chef.

Starting rabbitmq-server: FAILED

This error only occurs in Ubuntu versions 11.04 and below when installing chef-server. It is a rabbitmq-server issue that is related to this Rabbitmq bug: https://bugs.launchpad.net/ubuntu/+source/rabbitmq-server/+bug/653405

Error:

Fix:
This error can occur when the hostname is not properly resolving, or when the machine has not been rebooted after changing the hostname. You can test this by running the command ping $HOSTNAME if it responds with unknown host or the old hostname, follow these directions to get Chef Server installed:

1. Edit /etc/hostname and /etc/hosts so the hostname in /etc/hostname resolves to 127.0.1.1 or 127.0.0.1 in /etc/hosts.
2. Reboot the machine.
3. Remove rabbitmq-server, chef, and chef-server:

4. Remove this rabbitmq file so it gets properly installed next time:

5. Re-install Chef and Chef Server: