One of the things I love about SF novels is that they are always full of fascinating ideas that sometimes spark resonances in my own thinking (in this case, on DevOps).
“There’s a value in chaos theory, a number called the Lyapunov exponent, which tells you how to predict a chaotic systems boundary – it’s knowledge horizon, if you like. Hyperion’s Lyapunov exponent is just forty day – we can’t predict this moon’s motion beyond the next forty days. That’s the maximum limit of our foreknowledge! If my life depended on this moon’s motion, I would still not be able to say a word about its state beyond forty days”
My first thought when I read this was “Yep, sound like every IT project I’ve ever worked on… pretty much turns to chaos after about 40 days…”.
Now, the mathematics of the Lyapunov Exponent are way beyond my long-forgotten basics of high school calculus so I can’t comment on whether the author’s usage of the exponent is valid but the concept – that there is a limit on our foreknowledge beyond some sort of chaos event horizon – really stuck with me.
So what’s this all got to do with DevOps?
Well, “classical” project management takes a rather linear, deterministic view of the world – that we can accurately define a set of initial conditions that are complete (the mythical “requirements spec”) and that we can define a series of steps (the “waterfall process”) that will result in us achieving a desired end state (a working system that matches the specification AND the customer’s intent).
But what happens if software development is “non-linear”, chaotic and subject to “the butterfly effect”?
If “small changes to the initial conditions” (i.e. the spec is a tiny bit wrong, or more likely the customer changes their mind) can have a major impact on the end state (not to mention any of the other factors that might turn a project plan to “chaos” in the common usage of the term) does any given software project have a “Lyapunov Exponent” that limits our ability to predict the future?
I’d argue that it might… and this is intuitively what DevOps seeks to address in the “2nd Way of DevOps”
The Second Way is about creating the right to left feedback loops. The goal of almost any process improvement initiative is to shorten and amplify feedback loops so necessary corrections can be continually made.
By shortening the feedback cycle we, consciously or unconsciously, are seeking to bring back our project cycles within our “Lyapunov Exponent”. If we can carry out our planning cycles (Sprints in an Agile development world) within that “limit of our foreknowledge” we create a system where we can correct and tweak the “initial conditions” of our next cycle to seek to keep our project “on track”.
What I also find interesting is the idea that different projects have different “Lyapunov Exponents” depending on their degree of “non-linearity” and sensitivity to initial conditions etc.
Perhaps this is why different projects can have success (or failure) with different sprint durations.
Just because a one, two or four week sprints work for project X doesn’t mean that’s the best duration for Project Y because it has a different exponent. Part of our feedback cycle also involves finding the optimal value for our “knowledge horizon” – not so far out we descend into “chaos” but not so short we spend more time re-planning than we might need to otherwise do.
Sprint Duration < “Lyapunov Exponent” = DevOps Success?
Sadly I don’t think that there is any theoretical way to determine a project (or organisation’s) hypothetical “Lyapunov Exponent” when it comes to project planning… it has to be empirical based on trial and error, but I’d be curious to know if, even anecdotally, that for a given organisation or type of project the likelihood of a “successful project” falls off a cliff beyond some planning horizon. If there was some way to determine the “Lyapunov Expontent” for a project and optimise an organisation’s feedback cycle (or sprint duration) below that limit then I think that would be of significant benefit.
p.s. yes, I know I’ve played fairly fast and loose with the mathematical usage of the word “chaos” compared to its vernacular usage. I apologise to any mathematicians reading this blog but I hope you’ll forgive me in the spirit of exploring an interesting concept!