|
The Chef Indexer is comprised of a RabbitMQ message queue, chef-solr-indexer, which pulls messages from the queue and formats them, and chef-solr, a thin wrapper around Solr. |
Table of Contents |
RabbitMQ
RabbitMQ provides a queuing service which stores requests for updates to the [search index]. This allows the API server to handle load spikes while maintaining an even load on Solr.
We recommend you read RabbitMQ's administration guide before deploying it into production.
Configuring Chef's RabbitMQ-Specific Settings
| Setting | Default Value | Description |
|---|---|---|
| amqp_host | '0.0.0.0' | The IP address or resolvable hostname of the server running RabbitMQ |
| amqp_port | '5672' | The port on which RabbitMQ listens for requests |
| amqp_user | 'chef' | The username to use when connecting to RabbitMQ |
| amqp_pass | 'testing' | The password to use when connecting to RabbitMQ |
| amqp_vhost | '/chef' | The RabbitMQ vhost to use |
| amqp_consumer_id | nil | A tag to append to the queue name. The implications of this setting are discussed in the redundancy and scaling section below |
More information:
- RabbitMQ User Management
- RabbitMQ Access Control and VHosts
- Getting status information about a running RabbitMQ Note: be sure to include -p /chef (or -p /VHOST_NAME if you are using a custom vhost) when running rabbitmqctl to query for information about the chef vhost.
Configuring RabbitMQ for Chef
Chef requires you to configure a user and vhost in RabbitMQ. Normally, this configuration will be handled for you by either post-install scripts in Chef packages, or in the rabbitmq_chef recipe included in the bootstrap install. You only need to run these commands if you are doing a fully manual installation or have destroyed your RabbitMQ configuration somehow.
To configure RabbitMQ for Chef using the default vhost "/chef", user "chef", and password "testing":
To verify that your configuration is correct:
Chef Solr Indexer
Chef Solr
Redundancy and Scaling Options with Chef Indexer
The point of setting the queue name is to switch between messages going from 1 producer to N consumers or messages going from 1 producer to (1 of N) consumers. When multiple consumers share the same queue name, you'll have the 1->(1 of N) behavior; with multiple consumers with distinct queue names, you'll get the 1-> N behavior.
The use cases these address in Chef are:
- With 1 to (1 of N) behavior, you can have multiple chef-solr-indexer processes reading data from the queue, munging the data and then posting the data to the same SOLR instance. You would want to do this if: you want redundancy in chef-solr-indexer and/or the data munging that chef-solr-indexer does becomes a bottleneck. To configure this behavior, configure each instance of chef-solr-indexer to use the same amqp_consumer_id setting.
- With 1 to N behavior, you have multiple chef-solr-indexer processes each passing the data to unique SOLR instances. You want to do this if: you want redundancy in chef-solr, and you prefer to achieve it this way instead of replicating at the SOLR level. To configure this behavior, configure each instance of chef-solr-indexer to use a different amqp_consumer_id setting, or set amqp_consumer_id to nil.