You should consider using SSH-based Configuration Management
At the first mention of configuration management (CM), everybody and their dog (and links on their respective blog posts!) will direct you to one of what I take the liberty of calling ‘The CM Triumvariate’: Puppet, Chef and CfEngine 3. But I reckon the interwebs is presenting a skewed view and hides from you some real, useful alternatives.
“Alternatives?! With a gazillion resource types, cloud integration, and promise theory readily available, what else could you you possibly want to manage your thousand server infrastructure with?”
If I may reply with a question, can a developer use any of them, say, to provision a bunch of remote hosts directly from his developer machine without installing any additional software?
My sentiments exactly.
I don’t think it’s too much to ask. An ideal of configuration management setup is to go from 0 to 60 with one button press.
I’ve recently run into the problem of wanting to write simple, reproducible configurations for a very limited number of boxes. I’m talking tools that could help provision QA, stage or production environment of fewer than, say, three boxes each, each with well defined roles within your cluster. It turns out that Puppet or Chef just doesn’t seem to be the right tool for the job when you want to provision some machine B from machine A.
Don’t get me wrong, I am not dismissive of either of these tools (unlike the title of this post). I think they certainly have their rightful place in every sysadmin’s / developer’s toolbox. I am saying that there are more befitting tools that help us better in this particular use case.
Let me explain. Let’s say we want only to configure one remote host without explicitly having to install any extra software on it.
It doesn’t take long before you realise that Puppet becomes virtually unusable whenever there is no separate puppetmaster host. Granted, you can bodge it by copying manifests and then running puppet-apply remotely, but really, let’s not go there.
Chef is in a better situation, since it has no fewer than four usable solutions (namely Chef Solo, Spatula, Soloist and LittleChef) that con Chef into thinking that there is a local Chef repository running on the single remote machine you’re trying to provision. Chef is then happy to do the rest of the work of retrieve cookbooks from said local repository. Having said that, using any of these will require a remote installation of Chef on your remote machine along with all of its dependencies, which includes a complete Ruby runtime. Yuck.
My understanding of CfEngine is limited, having had no experience using it, but as far as I know, it’s a similar story, requiring an agent application to run on the target host.
Not very KISS at all. What’s more, I doubt that any respectable sysadmin is going to say that developing Puppet or Chef scripts is easy. Having to wrap your head around their respective abstractions and collection of cheesy metaphors (you know what I’m talking about) is quite a high barrier to entry.
In a sense, what I sought was a simple, lightweight solution that was a bit less painful than to attempt writing (idempotent) bash scripts, which CM set out to eliminate in the first place.
Alternatives: CM through SSH with Sprinkle and Ansible
There are actually a couple of tools I’d like to mention in this post that both sort of somewhere in between performing server provisioning tasks manually through SSH and full-on client-server configuration management.
If you’re thinking of Fabric and Capistrano, I would say that they are helpful tools for defining tasks, but lack the declarativeness, idempotency and clarity of execution order making them unsuitable for CM. For instance, I cringe a little to find that introducing conditionals in Capistrano, for example, requires its own GitHub project.
So actually there’s this thing called Sprinkle. I recently used it to maintain a configuration of my home server. The main thing going for it is its simplicity and flexibility. It mightn’t be as feature-rich, but you can still write declarative, idempotent scripts in its Ruby DSL a bit like with Puppet and Chef, as long as the developer provides sensible declarations. It provides a simple dependency-based system for deciding how to reach the desired system state.
It comes with a number of useful ‘installers’ and ‘verifiers’ which can be used to build up ‘package’ definitions. It’s sad that the author called these packages instead of something more generic like ‘resource’, since actually you can use packages to describe virtually any aspect of the target machine’s state.
An installer is essentially to execute a command to ensure the ‘package’ (e.g. a UNIX user) manifests itself in the target system (e.g. by creating the user if he doesn’t exist).
A verifier is a set of commands that if executed successfully, witnesses that the ‘package’ is indeed present (e.g. by grepping through /etc/passwd). ‘Policies’ allow you to manage groups of resource requirements according to nodes’ roles.
The real differentiator of Sprinkle is the way the developer can Sprinkle to use Capistrano to push commands to target hosts for remote execution (in parallel) over SSH all from the comfort of your developer machine.
Since the commands executed remotely are just shell commands, it meets our requirement of requiring no extra software to be installed on the remote hosts. All you need is sshd.
This mode of working also makes the script more amenable to iterative development, and could save you from having to develop the script on a VM. Just run the script against the actual target environment if the circumstances allow. Applying the configuration to your local machine is as simple as changing one line.
In fact, you could almost treat Sprinkle as a lightweight execution framework for bash commands that installs things in dependency order. If you want to discover details about the running system you could use Facter, just as you could do so with Chef or Puppet. Just grab a quick glance at the GitHub page for the README. Actually, I put together a set of slides about Sprinkle for your viewing pleasure.
Ansible If you’re more of a Pythonista you may enjoy using Ansible. From the guys who made Cobbler, Ansible shares many benefits that Sprinkle has. The biggest difference is that your idempotent configuration specification is defined in YAML instead of Ruby. It’s intended to be fed into the ansible Python executable, which spits out SSH commands in a mostly idempotent way.
I urge you to go forth and find out more about these SSH-based configuration management tools if you find yourself in a similar hole. You’ll thank me for it!