DevOps Metrics Canvas

Running a DevOps Metrics Workshop

Step 1: Understanding your outcome (5-10 mins)

The first section of the workshop is for the facilitator to present to the team the outcome that will form the core of this workshop.  This can then be written in the “Outcome” box.

Hint: make sure you are focussing on an outcome and not on outputs (see below) and make sure you spend some time ensuring everyone has a clear understanding of the outcome, this will make the remainder of the workshop much clearer.

Step 2: Measuring the outcome (15-20 mins)

The second section of the canvas to be completed are the measures for the outcome (bottom right).  This box contains the data points or proxies for the outcome we have established.

The team now works to identify measures that ensure they are achieving the outcome.  These should be written on post-its and stuck in the outcome/data box.  Once the team has identified all of the options they should be prioritised and ensure the selected measures (3-7) contain counter-measures (see below).

Hint: a valid data/proxy must be something that you can measure but it doesn’t need to be easy to measure!  Just because something is difficult to measure doesn’t make it the wrong thing to measure.  Validate your proxy by ensuring that the following makes sense – If we see an increase/decrease in [data/proxy] we might infer that we are achieving our outcome.  (Because outcomes are typically complex one proxy in isolation is unlikely to prove it is being achieved).

You should look to have approx 5 data proxies that have been agreed by the team, prioritised and confirmed for balance when you finish this section.
but just because something is hard to measure.

Step 3: Inputs (20-30 mins)

In this section of the workshop the team identify their ‘Inputs’.  Inputs are the actions/activities that the team believe will help achieve the outcome as measured by the values determined in step 2.  The team should write down as many ideas as they can within the allotted time before grouping and ranking them to end up with approx 5 activities.  It sometimes helps to think of these are the levers that we can pull to help us achieve our outcome.

Hint: validate that your input is valid by asking yourself the following – “We believe that doing [action] will drive/impact/ correlate to our [outcome].  It is common for people to try and put input measures in this box not the actions.  In this case ask the following: “what would you do to move that number?”.

You should look to have approx 5 data actions that have been agreed by the team and prioritised.

Step 4: Input Data/Proxies (20-30 mins)

For each of the actions prioritised in Step 3, the team now have to ask themselves ‘how will we measure that we have performed that activity’.  This should result in a list of measures aligned to your actions.

Hint: This sometimes works well by dividing the team into smaller groups each working on one activity.  This is especially useful if you are running short of time.

You should look to have at least two measures for each action by the time you finish this section.

Step 5: Concluding the session (10 mins)

To wrapping up the session you should summarise the canvas and agree on next steps with your team.  The canvas can be summarised as follows:

As a team we are committed to [outcome].  We will know we have achieved this outcome by monitoring the following measures [outcome data/proxies].  To achieve our outcome we believe focus on [actions] will deliver the outcome.  We will track these actions by monitoring [input data/proxies].

The Metrics Canvas is Copyright DevOps Research and Assessment LLC.