No, I’m actually not dead, as much as it might appear from my lack of updates. Fortunately there are other means to verify my vitality.
Anyway, the goal here is to help you understand how the imminent release of Puppet 0.25 affects you. And by “you” here I am, of course, making some assumptions – if you don’t use or care about Puppet, um, not so much. But if you use Puppet, or, even better, if you care about how well Puppet solves your problems, then this post is for you. I hope.
I’ve no short-term memory and don’t feel like planning this blog post out, so I’m going to just do what I can to ad-lib the reasons that Puppet’s new-found RESTfulness kicks ass. And stuff.
Smaller and Faster
First and foremost, 0.25 does not have many what you would call features. It’s mostly a refactoring release. This refactoring in general results in a faster system that takes less memory. Experiments have borne this out: People are seeing 10-40x speedups in fileserving, at least 30% less ram consumed on client and server, and much faster run times on the client.
So, look not for features but for benefits.
What is REST?
REST is more of a principle than a standard, but the things that matter in it for Puppet are:
- All data transferred is treated as a stand-alone ‘resource’ (and here ‘resource’ means something different than a normal Puppet resource). For instance, Puppet 0.25 treates catalogs, file content, file metadata, certificates, certificate requests, fact collections, and probably a bit more as resources.
- All resources have a unique URI. E.g., a given host’s catalog can be found at ‘http://puppet/$environment/catalog/$hostname’.
- Standard HTTP infrastructure is used as the ‘api’ – HTTP ‘put’, ‘get’, and ‘delete’ are the primary operations, and HTTP content-type handling is used for serialization negotiation.
Of course, none of these are really helpful to you, whomever you are. What’s helpful is how we’ve made these basic ideas useful in Puppet. So here are some of the things you get:
Faster and Smaller
The big problem with our current network protocol (XMLRPC, which is essentially XML over HTTPS) is that everything has to be considered XML, which means that anything that isn’t actually XML (which is, um, everything in Puppet) has to be encoded and escaped so it can act like XML. For file serving, this means that every file has a minimum of three copies in memory – plaintext, encoded, and escaped. This is time- and memory-expensive.
Oops I Did It Again
Sorry, couldn’t resist. One of the benefits of switching to REST is that we’re breaking compatibility, so we can fix some design problems we (i.e., I) made early-on. The biggest benefits have been around fileserving and the catalog.
In pre-0.25 releases, recursive file-serving involves a minimum of one call per directory and one call per file. This is obviously slow and bad. 0.25 fixes this so it’s one call for an entire directory structure, with possibly one call for each file that actually needs to be transferred. The combination of this plus the lack of encoding and escaping has resulted in most people seeing a 10x speedup in recursive fileserving.
For catalogs, well, pre-0.25 releases don’t really talk about them. They transfer this serialized recursive tree structure, rather than the whole catalog (which is really mostly a graph). This matters a bit in 0.25, because it’s allowed us to remove some extraneous stupidity that resulted from the limitations of the tree structure (relationships no longer propagate to all contained resources – see the ticket for more information), but it’s going to be a bigger deal going forward as we can start relying on the complete graph being passed around.
Both of these refactors could have been done with XLMRPC, but they would have required breaking compatibility, which is always a difficult step to take, so doing it all at once makes the most sense.
Easier Integration
The big deal in our use of REST is really an internal system that enables REST – this is the all-powerful Indirector. It’s essentially a plugin system that makes it easy to add new sources or destinations for data we care about. For example, when we wanted to queue the storage of catalogs to a database (rather than forcing the client to wait until it’s done), we had to develop the queueing infrastructure, of course, but integrating it with Puppet was less than 100 lines of code that knew how to accept catalogs and send them to the queue.
This ease of integration opens up all kinds of possibilities, like moving our CA to be database-backed, but the main thing is that if you come up with some cool integration, it’s similarly easy for you — you’re not dependent on us.
We’re still missing one piece – a user-configurable routing system to help you configure which plugin is used – but we’ve got that mostly done and it’ll be in the release after 0.25.
Note that this integration was often possible in earlier releases, but it was much harder because you had to know a lot more about every individual class you were integrating with. Because we use the same methods and the same interface and the same location pattern for everything, you don’t have to know as much.
Enforced Good Citizenship
Without the Indirector at the heart of everything, and an assumption that other people are going to be integrating with our code, we are forced to build more decoupled, portable components. This isn’t much, but it does always enforce a kind of good citizenship, which is something.
Pingback: BotchagalupeMarks for June 3rd - 23:49 | IT Management and Cloud Blog
I just check out Puppet 0.25 It is rally amazing, all your hard works with it is really worth it. Congratulations
Is it free to download? Where can I download it?
Cheers,
Dave
I've been using Puppet 0.25 as well for catalogs, file content, file metadata, certificates, certificate requests, and fact collections. Very useful and efficient to use.
Puppet is the best. I am using this & it is giving me good results.
Puppet 0.25 It is rally amazing, all your hard works with it is really worth it. Congratulations
well with all of these add-ons. I hope good work keeps coming to it.
I am using this & it is giving me good results.
well with all of these add-ons. I hope good work keeps coming to it.
I am using it & totally satisfied. It is helping me a lot in my work.
Excellent article , you cleared all my doubts!
I sometimes feel like that with my posts too
. Those are impressive improves with 0.25.How long did it take you to make the improvements?Keep them coming though.
That was a really cool article bout Puppet. Appreciate the good work!
this was a very interesting post, you have said it very well, thanks for explaining it to us.
That was a real good article on Puppet..An eye opener I would say.. Appreciate the good work..Thanks..
Good job on Puppet article. But this post is not for every category.
Yea, we appreciate your hard work.
Thanks
I haven't tried REST. I will put a stronger emphasis on it though. Great post you have here!
I entered your blog from google search. You really have done a good effort. Thanks for the wonderful blog.
regards
charcoal grill
wow, this was really informational, thanks a lot for sharing this guide, I'll be looking forward on more info's from you..
OK – I'll be the first of these posters to admit – I'm not really sure what Puppet is. I'm in IT and this is all foreign to me.
Good summary. Looking forward to hearing more
this is so good how you explained this. i will have to try this.
First and foremost, 0.25 does not have many what you would call features. It’s mostly a refactoring release. This refactoring in general results in a faster system that takes less memory. Experiments have borne this out: People are seeing 10-40x speedups in fileserving, at least 30% less ram consumed on client and server, and much faster run times on the client.
I’ve no short-term memory and don’t feel like planning this blog post out, so I’m going to just do what I can to ad-lib the reasons that Puppet’s new-found RESTfulness kicks ass. And stuff.
CHEERS
Hey, thank you for this post, it's really a great read, I will share it.
That Sounds interesting, I agree with you.Please keep at your good work, I would come back often.*
Thanks for sharing this valuable information on this article
modern furniture
Thanks for the really interesting post.. Appreciate it..
It's interesting indeed, no doubts.
One of the benefits of switching to REST is that we’re breaking compatibility, so we can fix some design problems we (i.e., I) made early-on.
The is lot of benefits using REST. So one must use it on daily basis to get maximum outcome. This I can guarantee on.
I completely agree with you on this point and I sincerely hope too see more great articles like this on your blog. I just love to read it all the time when you write something new.
I’ve no short-term memory and don’t feel like planning this blog post out, so I’m going to just do what I can to ad-lib the reasons that Puppet’s new-found RESTfulness kicks ass. And stuff.
nice article dude.. hope it will be useful for many… thanks for sharing it!!!
Thank you for this great article
Great as usual!
Another good one, thanks for the share.
That was a really interesting post… Appreciate the effort..
That Sounds interesting, I agree with you.Please keep at your good work, I would come back often.*
City scraps Web site plan; will seek new bids”The City Council was set to vote in March to approve the $704,000 site design contract with Cignex.
voip phone service provider
what is the definition of being alive? is it only breathing??? no, its a happy life and a happy ending.. .
best credit protection
did people make puppets by observing people or did God make people by observing puppets???
Cheap Package Holiday
Similar to QuickSilver, there is a system-wide keystroke that invokes the persistently hovering (modal) to-do entry form.
Very quickly, one can enter to-dos from any app, tag it, and add it to an in-bin. Not sure if the tagging is necessary, but that’s for a different post.
REST is an awesome feature, it is really a cut the BS tool, direct to the point!
Thank you for the great information, I like the information here.
The big deal in our use of REST is really an internal system that enables REST