|
Cluster Orchestration:
Convener: Nathaniel Eliot, Peter Norton, Carl Perry
Participants:
Kyle Bader
Kurt Yoder
Sean P. Kane
Summary of discussions:
- Nathaniel Eliot: Overview of what cluster_chef provides on top of chef
- Discussion on tools for orchestrating large clusters
- cluster_chef & rundeck
- dominodes: databag-based locking mechanism (security issue: nodes require write on all databags)
- chef-solo + push mechanism
- crowbar: has its own internal orchestration
- ruote: ruby orchestration library
- nanite: messaging bus
- rightscale: has a tool ("remote recipe"?) for invoking single recipes on a target node
- Monitoring tools
- zabbix - monitoring
- sensu - monitoring framework from Sonia, using Rabbit MQ
- nagios - perpetual monitoring standard
- PeterN: use case of launching a cluster, use case for orchestration
- Standard web app, discussing options for orchestration. Rundeck will be used with cluster_chef
- Standard stack of rabbitmq, web apps, and nginx needing to be completed in order and asap
- Learned: rightscale provides a platform that offers orchestration. They are adding chef support (maybe done already? I'm not clear on this)
- Also there is a non-server-based discovery protocol that can be used for orchestration called dominodes (I'm not sure how this works - in the end I see things like https://github.com/websterclay/chef-dominodes which uses a databag as the lock? I guess dominodes facilitates intra-node communication).
- Desired use cases:
- Spin lock (wait_for_foo "foo_name" so that only one node is ever doing "foo_name" in a cluster)
- event coordination (wait_until_foo "foo_happened" so that when a resource becomes available, nodes that need it can use it)
- Remote execution (run_then_activate "foo_nodes" where after a node runs an action, either it directly, or through a chef server, can invoke a resource or a chef run on the nodes that comprise "foo_nodes").
- Takeaway: chef needs to provide orchestration syntax. There are at least 3 use cases. The use cases may be orthogonal, but some API decisions need to be made and these need to be approached. It would be preferred if the chef server could do this instead of just cooperating with an outside infrastructure (in my opinion)
Other stack tools mentioned:
- elastic search
- hadoop
- hbase
- goliath
- unicorn
- apistack
- mongo
- rails
- noah - zookeeper - doozer (doozerd?)
- chef plugin for Run Deck
- mcollective
What will we do now? What needs to happen next?
- Desired feature: ability to run a single recipe (+ dependencies?) on a node, without fouling up run-list or node data
|
|