Managing Your Puppet World with r10k
Puppet is a great tool for managing your infrastructure, most people start of with a few modules to manage various core services. However as the number of modules grows and therefore the complexity better systems are required to keep things running smoothly.
I’m going to give an overview of how we have structured things. This wont be an extensive technical how-to, more tips and pointers on how we have things setup which will hopefully help out others.
- Version all the things
- Config Examples
- Work-Flow & Tips
Version all the things
One of the core tenants of DevOps is to version everything. If you can’t track and trace whats changing in your infrastructure you’re storing up a world of pain as things grow in complexity.
Git makes branching really easy, some would say too easy. By using branching and tagging effectively you can still make large changes to your modules and preserve a stable production state (which can be destroyed and rebuilt identically).
We used to operate with a single monolithic git repo which contained all our puppet configuration, but once we moved to multiple environments things started getting a bit out of hand. So start by making sure all your modules are in their own repo and versioned. We went through pulling each module out one at a time and versioning them as we went (using semver of course). These were then moved to our local GitLab.
Now you have all your modules source controlled in their own repo, but how do you tie them all together? Enter r10k.
Environment Management with r10k
To manage our environments we use r10k. This builds on librarian-puppet making dynamic environment much easier. r10k will create an environment for each git branch it finds in your r10k repo, you can also selectively build/update environments if needed. The Puppetfile in each branch is then used to build the modules.
The Data (Hiera)
Finally we need to source control our module data in Hiera. This can be either in YAML or JSON, we’ve increasingly found JSON better for our needs. We have a single branch for all our data as we split things based on custom facts, but you can now also have hiera data per environment which is something we are looking at moving to.
Our r10k structure looks like this:
Work-Flow & Tips
So how does this work in practice?
You can also then update a single module with the following:
You can also quite easily automate these changes using CI when things are committed to your r10k repo.
Difference Between Environments
One of the great advantages to this is visibility. Difference between testing and staging:
Another useful tip is to have a bleed edge environment which tracks the
master branch of your modules and potentially the
latest versions from the forge. You can then set this to automatically build and you can test against this for possible problems before bumping your versioned environment.
The flexibility you gain from r10k’s dynamic environment has certainly help increase the speed and safety of our puppet changes, hopefully the above will help you to as well.