Anyway, so I’m once again wrestling with data lifetime in Puppet.  This is one of those problems that always seems licked but then crops up again.

See, there’s plenty of data in a Puppet transaction whose lifetime should only be that transaction:  File stats, user name to gid mappings, and all kinds of information about the current state of the machine.  We need to collect this information, and we don’t want to collect it more than once a transaction (e.g., we need a file’s uid, gid, and mode, but we can get them all from a single Stat instance); but we also don’t want the data lying around for the next transaction.

So Puppet has support for a ‘flush’ method throughout most of the RAL:  By default, the transaction calls ‘flush’ on each resource, and the resource calls ‘flush’ on its provider.  This makes it easy to get rid of data that should be cleared up after the transaction.

Kind of.  See, the *real* reason for the ‘flush’ method is actually to flush changes to disk; e.g., the provider might have multiple attributes changed, and then you call ‘flush’ on it to make all of those changes at once.  It’s just that it’s also a convenient place to clean up data because, well, it’s the only place to do so.  So some time in the last few months or years or decades, my brain decided it does both things, but it seemed to hide this conclusion from me until yesterday.

But yesterday I was trying to fix all of the broken tests resulting from my file serving refactoring, and I was finding that, not surprisingly, I kept having these cached stat instances lying around — but *only* if the file hadn’t changed.  E.g., consider this code:

assert_events([], resource)
File.unlink(file)
assert_events([:file_created], resource)

Ignore, please, whether this is a good idea or what; the point is that the first line runs a transaction that results in a cached stat but no changes; and because there are no changes, there’s nothing to flush to disk; and because there’s nothing to flush to disk, ‘flush’ isn’t called; and because ‘flush’ isn’t called, the ‘stat’ is lying around still.  Which means the next transaction uses the cached stat, but of course, reality has changed in the meantime.

So, I need something that will make it easy for my data to match the lifetimes I want.  I made this Cacher Module that purportedly solves this problem for me, but noooo, it solves a different data-lifetime problem:  I have a lot of initialization code that generally only runs once but ends up running many times during testing, so I needed a clean way to remove the initialized data after every test.  So, this module has a single, global boolean that defines whether a given chunk of data is expired or still valid.  That’s all fine and dandy for one-time configuration data, but it doesn’t work for transactions.

So, now I’m trying to enhance that module to support either the global expiration marker or a per-instance marker, so that I can have all of the resources use a timestamp in their catalog to determine whether their cached data is expired.

One of the hard problems here is that you don’t want to find yourself maintaining a global list of anything anywhere.  My first design for the Cacher module involved it keeping a reference to all the cached data, which would have made it easy to clear the cache but would have had lots of code accessing tons of global data; stupid.  Instead, everyone keeps references to their own data, and the only global data is the expiration marker.  This works great for global things.

But resources and catalogs aren’t global.  Even worse, I’m (reasonably) using an internal class to check expiration.  So, that internal class needs to have access to the catalog’s timestamp in order to figure out if its cached data is still valid.  This is automatically ugly — an instance you shouldn’t even need to know exists now needs access to a big collection of resources.  Yuck.  Without this, the internal class doesn’t have a reference to anything except the data it’s managing, but there’s just no escaping it needing access to that timestamp somehow, so you either give it a reference, or you pass it one every time you check for a value.

The *stupid* thing is that this isn’t the problem I want to solve.  I just want the old fileserving behaviour but now with gooey RESTfulness.  That, plus the fact that I’ve apparently decided that I can no longer use spackle and sheetrock mud during development means that I have to fix this even though it wouldn’t normally be on my critical path.  Yuck.