ISconf

Puppet was heavily influenced by two tools, but almost more importantly, it was heavily influenced by what I couldn’t do with those tools.  I attended LISA in 2001, where I saw Steve Traugott give his talk on automating infrastructures.  I was impressed enough that I convinced my then-company to bring him in for a week of consulting.  During that week, he convinced me to give a tool he’d written, ISconf, a try.

ISconf is an insanely simple tool — it’s just a way to apply a set of make stanzas in the same order every time.  Steve’s theory was that the only way to get consistency from your computers was to do exactly the same thing on them every time.  So, he wrote up everything he did as a make stanza, and then built ISconf as a way of executing them in a consistent order.

Or at least, that was the theory.  When I got his code for ISconf, it was, em, not terribly usable.  It was written in shell and perl, with shell calling perl calling shell calling shell calling perl then exploding.  I had already bought into the basic theory, but I couldn’t see how to his code (which he was calling ISconf 2), so I rewrote it with some pretty significantly different functionality but following, as closely as possible, his basic theory. Naturally, I called this ISconf 3. On the ISconf web site, you can read more of this history, along with Steve’s relatively bitter version of how this portion of the history goes. (Incidentally, I wrote the majority of the content on that site.)

I won’t go into the detail of how it worked, except that all of the actual work happened with make stanzas.  This meant that, really, ISconf was a huge collection of very small shell scripts. This seems straightforward, until you start thinking about things like package installation — the command to install one package is almost the same as that to install a different package, which meant that a lot of my make stanzas had a lot of duplication.

To make my life easier, I started abusing the hell out of make, switching to gmake, and starting to make simple rules using ‘%’ to avoid duplication:

pkg/%:
...

I can’t even remember how this worked, thankfully; I just know you can extract what the ‘%’ matches and use it to choose the package name when installing a package.

This worked great for packages, since in most cases (and all the cases I was dealing with), you can easily specify a package by name.  It didn’t work so well for things like cron jobs, though — you’ve got between 1 and 6 fields, and the last field can have spaces and all manner of crap.  So, I started maintaining a separate file of all of these cron jobs, written in a perl Data::Dumper format, with each cron job having a unique name.  Then my make stanza would just refer to the arbitrary name I’d chosen.

This at least made ISconf more usable, but there were still a lot of problems.  You can read my writeup from around then in my LISA paper, but there were some serious problems with using ISconf.  First and foremost, it wasn’t a real solution — it was just a way to organize billions of little shell scripts, and as most anyone knows by now, I’m not so fond of shell scripts.  Also, the whole theory was bogus — there’s just no way you can guarantee consistency, not through order or any other mechanism.  Your dev servers and test servers can never be exactly the same, so stupidly replaying the same scripts in order also can’t guarantee sameness.  As a ridiculous example, I’ve seen printers that don’t work if their hostnames have an odd number of characters, and there’s just no way you can test this kind of case.

Beyond the theoretical problems, though, there were even worse practical issues.  You were supposed to never touch any code once it’d been deployed to a machine, because otherwise you couldn’t guarantee consistent behaviour, but this is just silly, because it means that you can never refactor code or change how you manage a given aspect of your system.  Imagine a solution that required immediate calcification of anything you ever did — once you deploy it, it’s deployed like that forever, even if it’s stupid.  No reason to learn; even worse, you get punished for learning, because you just add a new system to maintain every time you learn anything.

So, I wanted the ability to refactor my solutions, and ISconf couldn’t deliver that, so I started looking for something else that could.  Check back for part 2 to see what I did next.