Automating an Automation Server
Job DSL or scripted/declarative pipelines are becoming the standard when it comes to defining jobs in Jenkins. Now we need a similar solution to maintain Jenkins itself.
Jenkins can be installed through native system packages, Docker, or run standalone by any machine with a Java Runtime Environment (JRE) installed. But configuring Jenkins is a tedious process, no matter how you choose to run it, and the more plugins you use the more tedious it gets. Given that there are more than 1400 plugins available it’s easy to reach an unmanageable state.
Both Jenkins and its plugins require manual configuration, by default available only via UI. The Jenkins Configuration as Code plugin (JCasC) allows us to manage such configuration via human readable configuration files.
Configuration as code is not a new concept and Jenkins already provides a way to automate its deployment e.g. by introducing support for init scripts written in Groovy. The problem is that it requires you to know Groovy and understand the inner workings of Jenkins and its plugins. It also mixes scripts with configuration and doesn’t honor one of our important principles of separating the “what” from the “how.”
One specific example is JenkinsAsCodeReference developed at Praqma. I worked with a customer who wanted to introduce this approach and I loved it from the very beginning. I didn’t have much experience with Jenkins at the time, so having a way to preserve old configuration, without the need of saving non-readable xml or taking screenshot of existing setup for documentation, was a big relief. And it allowed me to experiment and implement without fear.
JenkinsAsCodeReference is now being used by a number of companies, but it has issues. It forces you to use Docker and it also handles deployment of Jenkins. Redeployment is time-consuming and the actual configuration is mixed with Groovy scripts. This started to get annoying so we decided to come up with a new, better solution that would provide the same features (and much more) but would be easier to maintain, faster, and could be used in various (not only Docker) scenarios.
By we I mean me and other Praqma developers, later joined by CloudBees engineers, and by better solution I mean a plugin.
While preparing the prototype of the plugin we found existing ones. CloudBees engineers and regular Jenkins users were trying to do exactly the same thing. We saw this as a great opportunity for collaboration, so we got in touch with CloudBees and found that our visions, goals and motivations were very much aligned.
And now we’re working together on making this the Configuration as Code solution for Jenkins.
The goal of the plugin is to do for Jenkins configuration what Job DSL once did for jobs. Move it to code! We also want to learn from the pains that were involved in making Job DSL. That is, for the plugin to succeed, it has to support as many plugins as possible out of the box, without needing to write glue code for each plugin. Let’s remind ourselves that there is (much) more than 1000 of them. Not ALL of them require some sort of global configuration, but the number that need to be supported by the Jenkins Configuration as Code plugin is still quite high.
It is a challenge and we already know we won’t be able to support all of the plugins. Some glue code and a number of Pull Requests to make the plugins support our requirements will be needed, but we believe it’s worth it.
Jenkins Configuration as Code does not help you deploy Jenkins - you will still need to do that your favorite way using containers, Kubernetes, native system package, Ansible or the like, but as soon as you’re about to start Jenkins, JCasC will be there to save you. There’ll be no more manual editing of the Jenkins global configuration under Manage Jenkins after deployment.
Once you have prepared your configuration file Jenkins can be configured automatically when it starts. You can restore your Jenkins easily - within seconds. You can also reuse the same file to configure multiple Jenkins instances, or you can spin up a local instance to test your changes before you make them part of the production environment.
If you decide to keep your configuration file in some VCS e.g. Git repository - which I believe you should - you can keep track of changes in the Jenkins global configuration. This makes it possible to quickly restore a previous configuration if the new one proves to be unsatisfying. It also makes it easy to compare and trace changes when something is broken.
Jenkins Configuration as Code is the last missing piece in the puzzle for you to completely manage your Jenkins instance as code. You can now have your infrastructure as code.
One of the many challenges we set ourselves was to mimic Jenkins UI in a configuration file as much as possible. We didn’t want to have to scroll through long documentation pages to find out how to configure the Mailer plugin or Artifactory Server details. And, of course, we didn’t want to implement separate solutions for each plugin either. So, if you have some Jenkins management experience, writing such a file will come naturally to you. But don’t worry if it doesn’t, we’re not the enemies of documentation, we still allow it to generate itself! Furthermore, it will be generated based on your actual Jenkins setup and you can access it directly on your own server.
The configuration files for the global Jenkins configuration should be human readable. We don’t want it to be language specific, it can’t require specific knowledge, and it should support comments. These are the reasons we chose YAML. It is easy to write and because of available schema it is also possible to get support within your IDE and verify your configuration without running Jenkins instance.
And, most importantly, once you introduce the change in your configuration file you’ll be able to reload it without restarting Jenkins. No more time consuming redeployment!
I mentioned the involvement of CloudBees engineers in the plugin, but there’s a little more to that story. We’ve presented the concept at a number of meet-ups and CloudBees made some noise about it during FOSDEM at the beginning of February. The reaction has been great - we’re are already getting external contributors. The Pull Requests and feature ideas are coming!
You can get involved too - the more PRs the better. Feel free to create issues with features or bugs. We don’t just want to solve our problems, we want to solve yours too!
https://github.com/jenkinsci/configuration-as-code-plugin is the place to go now!
Jenkins Configuration as Code is finally here. In this blog I’ll cover how to convert an existing Jenkins instance to JCasC and also how to start from scratch. Let’s get it up and running!
How to spin it up from scratch or migrate an existing instance
What do we need from a CI/CD system? How should we decide which one to use? In this blog we ask ourselves what a modern CI/CD system should look like and compare two commonly used build systems: Jenkins pipelines and Concourse CI.
An in-depth comparison of two CI/CD servers
Do you have a tendency to use the backlog as an eternal placeholder? If so, you probably have a lot of clutter that’s creating a lot of frustrations for your end-users. In this post we’ll show you how to clean up your Jira issues and reduce the backlog with some basic JQL queries.
Tips to improve project management in the Atlassian suite
How to test Kubernetes artifacts like Helm charts and YAML manifests in your CI pipelines with a low-overhead, on-demand Kubernetes cluster deployed with KIND - Kubernetes in Docker.
Low overhead, on-demand Kubernetes clusters deployed on CI Workers Nodes with KIND
Had enough of sluggish polling? With instant Artifactory event triggers you can give responsiveness in Jenkins a real boost. Here’s an easy way to set it up.
A super easy configuration guide
With the arrival of microservices code is becoming disposable. Does this mean that we no longer need maintainable code? Is it the end of refactoring?
Still relevant or increasingly redundant?
In software development tight coupling is one of our biggest enemies. On the function level it makes our application hard to change and fragile. Unfortunately, tight coupling is like the entropy of software development, so we have always have to be working to reduce it.
How to safely introduce modular architecture to legacy software
I am an Atlassian certified trainer and over the years I have been spending much time with clients and their Jiras. In this blogpost, I have collected some small tips and tricks that will make your Jira usage better.
Jira Software is a powerful tool deployed in so many organizations, yet in day to day usage people are missing out on improvements, big and small.
In this post, I’ll take a closer look at the version of Jenkins X using Tekton, to give you an idea of how the general development, build, test, deploy flow looks like with Jenkins X. How does it feel to ship your code to production using a product coming from the Jenkins community that has very little Jenkins in it?
A crash course in Jenkins X and how to test it out on a local Kubernetes cluster
Hear about upcoming events in Scandinavia, latest tech blogs, and training in the field of Continuous Delivery and DevOps