A new card game to design Continuous Delivery pipelines
What testing steps should you include in your Continuous Delivery pipeline? Don’t just string together existing manual processes - use simple, collaborative tools to design something better!
Creating a Continuous Delivery (CD) pipeline is a key development in an organization’s transformation to DevOps. A CD Pipeline covers all the activities needed to transform a code change made by a developer into updated software bringing value to users. Steps in the pipeline include building a new version of the software as well as testing and deploying it. Exactly what kinds of build, test and deployment steps will depend on many factors and there is no ‘one perfect pipeline’ which will suit all situations.
I created the card game ‘Pipeline’ as a quick and fun way to explore alternatives for a CD pipeline without actually building anything. The goal of the game is to design a pipeline for a given scenario and optimize the deployment lead time. You work in a small group and get to discuss what steps are needed and which order you want to do them in. You will run into design tradeoff decisions and may discover people have different risk tolerances. If you play the game a second time with a different scenario, or compare notes with another group, you can learn more about how different scenarios drive different decisions.
Playing the game should help you to build a real pipeline for the real software system you are working on. Building a CD pipeline needs specialist knowledge of particular tools and could be many weeks of work. Playing this game should help you to avoid some costly misunderstandings. It’s also a fun way to engage a small group for an hour or two while you think through the issues you’re facing.
To play the game each group begins with a sketch of a guiding scenario. For example, imagine you work for a startup aiming to displace an established business, like a social media website or an online bank. The reason for the scenario is that it makes the discussions more concrete and productive. You can discuss some specific risks you need to mitigate through testing. With that in mind it’s easier to reason about how long to spend on each testing step, and also assess the potential impact of excluding a step entirely.
Knowing the deployment lead time of the competitor in the scenario helps guide the group to discussions about tradeoffs and where quality comes from. Can we leave out this slow testing step? Can we mitigate that risk another way?
The ‘pipeline step’ cards form the majority of the cards in the deck. These are the ones you lay out on a table to define the structure of your pipeline. You can choose which steps are important and which can be optional or omitted entirely. You can put several steps in parallel to reduce overall lead time. You can choose to perform a step twice, both before and after building the deployable component for example, or to do some steps before the beginning of the pipeline. The deck includes a few blank cards so that you can invent completely new steps. Just put a new sticky note on one with a text explaining what the new step involves.
You have a lot of freedom about how to design your pipeline, but in reality there are a few rules about how you must order the steps. For example, you can’t deploy to production before you’ve built a release candidate. The “Game Rule” cards help you to keep track of these rules. One of the cards reminds you to add sticky notes to each step in the pipeline with an expected execution time. You should estimate this - either how long you think it will take, or a timebox for how long it is allowed to take. If it’s a manual step this estimate should include the time spent waiting for the person to be available.
It might take a group of 4-6 people an hour or so to design a pipeline for their scenario and estimate the overall deployment lead time. When this is done (or the time for this part of the exercise is used up!) you should bring out the ‘review’ card. Each group can discuss what the implications of having this pipeline would be for their startup business. How does your lead time compare with your competitor’s? You might discuss how deployment lead time relates to time to market, or whether you would be able to do Lean Product Development, or A/B testing.
Some of the fictitious competitors have rather short deployment lead times. That’s quite deliberate on my part. I’d like people to think about how the competitor is achieving that. What must their pipeline look like? What testing must they be doing differently? You may think they are taking too many risks, and they will be easy to out-compete. Or perhaps they have found a better way, and you’d better imitate them.
Some of the fictitious competitors have much longer deployment lead times. I’d like people to think about whether that is really necessary. Perhaps they have included heavyweight change processes that take time without providing any benefits.
In any case, it’s interesting to think about, and might help you make more informed choices when you come to design a pipeline for your production system. Playing this game could alert you to some of the real life pitfalls that often arise.
If you want to play this game with your team I hope it will be straightforward for you to organize. The instructions are on the box. Allocate something between 1-2 hours for the exercise, split people into groups of about 3-6 people, with one deck of cards per group. You will also need a block of small sticky notes and pens for each group.
Your job as facilitator is to make sure everyone understands the point of the exercise, follows the pipeline game rules, and correctly calculates their deployment lead time. You should keep time and answer questions as they come up.
One thing a group might do is to decide on a microservices architecture, or front and back ends that are deployed separately. They might want to design a different pipeline for each service or each deployable component. That’s harder to do with only one deck of cards. For the purposes of this exercise, ask them to pick only one pipeline to design to start with. If they complete it quickly, you could give them another deck of cards to design another one later on.
As facilitator you may want to be quite active in the ‘review’ part of the game. Ask leading questions about the pipeline design to get people to explain their reasoning. If there are several groups you’ll want to have them look at one another’s work. This will help people to understand that different scenarios might call for different pipeline design choices. Everyone should come away with a better understanding of how to design a CD Pipeline and the hard choices that sometimes have to be made. Remind them that what they are learning in this game could help you all to avoid costly mistakes when designing a CD pipeline for real.
Throughout, try to keep these goals in mind:
The first idea for this game came from a fun exercise at a workshop at the European Testing Conference in February 2018, led by Lisa Crispin and Abby Bangser. They divided us into groups, then handed each a pile of cards showing names of steps you could include in a pipeline. For example, “automated test: unit test”, “manual test: exploratory test”, or “deploy step: deploy to performance testing environment”. We had to collaborate to lay out the cards on a table in the form of a pipeline, and estimate the overall lead time.
I really enjoyed the exercise. There is something about having cards to move about and wave at people that encourages discussion and group engagement. Some cards provoked lively controversy - I remember discussing whether to include a manual code review step before merging to master, with someone who was quite convinced it was essential.
I liked the exercise so much I decided to adapt it for a workshop I lead about Testing in Continuous Delivery. As I did this exercise with different groups I tweaked it a little each time. I also did it once with Llewellyn Falco and incorporated some of his suggestions too.
As the game gradually grew more sophisticated; I realized it would be worth getting some custom cards printed, so I started working with Phable on a layout and design. My colleagues at Eficode Praqma did some play-testing of an early version, and contributed some useful ideas. I’d like to thank everyone who played earlier versions of this game with me along the way. It’s a better game because of your feedback!
The second version of the cards came about six months after the first print run, following many successful CD workshops. Some of the ‘scenario’ cards have a lead time measured in weeks, and I wanted to make it possible to model those pipelines too. I added a few cards with more traditional release activities like a ‘Go/No Go meeting’ and ‘Decide Release Version number’. I also took on board some feedback from Steve Smith that ‘Constraint’ cards were poorly named - it reminds too much of ‘Theory of Constraints’, which has nothing to do with this game. I now call them ‘Game Rule’ cards, which I hope you’ll agree is more accurate.
I’d like to thank everyone who played earlier versions of this game with me along the way. It’s a better game because of your feedback!
If you’re interested in finding out more about DevOps, Continuous Delivery and metrics like deployment lead time, we at Eficode Praqma have lots of experience to share. Catch us at a conference or meetup - we have lots going on as you can see over at code-conf.com. For an in-depth report on the state of DevOps and the science of what works, I highly recommend the book ‘Accelerate’ by Nicole Forsgren, et al. You might also be interested in other blog posts by Praqma.
Let’s talk about cakes and baking. It’s similar to making software. No really! How quickly you can bake a fantastic cake depends on three main factors. Turns out, it’s the same three for software.
Three factors affecting your agility
In this blog I will show you how to create snapshots of Persistent volumes in Kubernetes clusters and restore them again by only talking to the api server. This can be useful for either backups or when scaling stateful applications that need “startup data”.
Sneak peak at CSI Volume snapshotting Alpha feature
When I read Fowler’s new ‘Refactoring’ book I felt sure the example from the first chapter would make a good Code Kata. However, he didn’t include the code for the test cases. I can fix that!
Writing tests for ‘Theatrical Players’
Nicole Forsgren and the Accelerate DORA team has just released the newest iteration of the State of DevOps report. The report investigates what practices make us better at delivering valuable software to our users as measured by business outcomes. Read on for our analysis of the report, and how it can be best put to use.
The latest drivers of software delivery performance
A major challenge of software development is that our work is by and large invisible. This makes our folklore essential in business matters. Some of our commonly used arguments and visualizations are digital urban legends rather than solid foundations for informed decisions. Here, we’ll go through a few examples and some measures to address our misconceptions.
How the stories we tell influence our decisions
When you embark on your cloud native journey there will be important choices to make about cloud providers, continuous deployment, environments’ setup and separation. This guide will help you make the right choices by sharing lessons learnt from running cloud native apps in production.
Kubernetes has become the de facto container orchestration platform. When we help clients of different sizes and domains start their cloud native journeys in Kubernetes, we assist them in making sound decisions and technology choices. There is no one-size-fits-all solution when it comes to choosing cloud providers, CI tools, continuous deployment pipelines etc., so it is important to make the right decisions at the start. Failing to do so can be very costly in terms of lost time and money.
How to make the right technical choices on your cloud native journey
Learn how Docker and Kubernetes work and the key benefits they bring. Using real demos, I show how Docker is a great packaging and distribution technology, and how Kubernetes provides a powerful runtime for containerized applications.
Watch this introduction to Docker and Kubernetes at the Trondheim Developer Conference (TDC)
In the world of Agile and DevOps we use many figures, charts and diagrams to argue and reason about our world and how we prioritize and make choices. However, at all levels of the organization, we misuse and misinterpret figures. It’s time to be explicit, measure the right things and act on them. Watch this talk from DevOpsDays Zurich in May 2019.
Watch this talk from DevOpsDays Zurich
Hear about upcoming events in Scandinavia, latest tech blogs, and training in the field of Continuous Delivery and DevOps