S02E02 First principles of business agility
Getting things done in organizations, why digital transformations fail, agility vs agileᵀᴹ, cargo cults, sociotechnical systems thinking, explore & exploit through optionality and flow
In season 2, I alternate between long-form essays and curated tidbits on organization design. This is an essay.
Last season, I looked at organizations through the lens of complexity theory, far removed from the day-to-day business reality. In season two, I go down a level. Instead of theorizing about evolution writ large, I’ll write about the stripes of the tiger and the shell of the turtle. Like animals, companies co-evolve with their environment, and some are better at this game than others.
In corporate evolution, rapid changes in technology are usually the forcing function. Digital transformation is the boardroom term for the evolutionary process of getting toward the desired state of business agility - the holy grail of being able to adapt to any market evolution.
In today’s article, I want to tackle the following questions:
Why do so many digital transformations fail?
What are the principles that underlie true business agility?
Why do digital transformations fail?
Companies are usually aware that the playing field is changing and that they need to adapt in order to survive. The trickier question is how to internalize that ability. The seemingly safe path is to hire a reputable consulting firm to come up with solutions that aren’t too much of a stretch compared to existing practices. For example:
Erect a skunkworks circus tent and launch a digital studio that is unencumbered by the bureaucracy of the rest of the organization — this allegedly worked for Lockheed, but you are more likely to subsidize rival factions. The IT department down in the basement won’t be fond of the hipster studio.
Adopt the latest trending architecture (eg MACH) — not featured in the consultant’s slides: the social and technical challenges hidden underneath this recommendation
Implement a scaled agile framework — no one ever got fired for recommending SAFe, but I don’t know of aany successful cases either.
When companies hire management consultants, it is because they need to get something complex done (and things get complex quickly in enterprises). To persuade others to adopt a thing, enterprise execs usually need more than logic; they need evidence that can withstand scrutiny and criticism when someone inevitably questions the decision.
Enterprise companies turn to the McKinsey’s of this world to rationalize and clarify their decisions. They’re buying a seal of approval, not original thinking. There is nothing inherently wrong with this approach, but it does lead to mimetic behavior.
Take agile software development. Over the past decade, enterprises have been flocking to agile as an organizational fountain of youth. If that sounds too good to be true, it is. Any moment now, companies will start to realize that their expensive agile software development frameworkᵀᴹ did not translate to actual business agility.
The failure of digital transformation programs begins with outdated mindsets at the board and C-suite levels. Instead of addressing this behavioral change from first principles, many companies attempt to purchase a packaged solution.
Agility is not something you can buy off the shelf, but there are plenty of agileᵀᴹ certified consultants who are willing to take your money and claim otherwise. This usually results in cargo-cult agile - the superficial adoption of agile practices without a deep understanding of the underlying principles.
I’m not sure if I’m still allowed to define a cargo cult in 2023, so I have asked an LLM to do it for me:
Imagine you live on a remote island in the Pacific Ocean. One day, you see strange metal birds flying over your head. They land on a nearby airstrip that was built by foreign soldiers during a war. Out of these birds come men in uniforms who bring food, medicine, tools and other goods. You are amazed by these gifts and want to receive more of them.
You observe how the soldiers behave: they wear headphones, wave flags, march in formation and salute each other. You think that these actions must be some kind of magic that summons the metal birds and their cargo. You decide to copy them: you carve wooden headphones, make flags out of leaves, march around and salute each other.
You hope that by doing this, you will attract more metal birds and more cargo. But nothing happens. The war ends and the soldiers leave. The metal birds stop coming. You are confused and disappointed.
You have just become part of a cargo cult.
Halfbaked transformation initiatives lead to frustration and disillusionment with the agile philosophy, further entrenching traditional development practices. In a failed attempt to one-click-install agileᵀᴹ, the enterprise ends up even further removed from its stated goal of business agility.
First principles beat imitation
Modern digital leaders understand that there are no turnkey solutions when it comes to organization design and business strategy. Instead of imitating patterns, they look beyond the boundaries of the current best practice.
A world without best practices is a scary vision for traditional managers, but there are ways to deal with variation and non-determinism.
True leaders articulate the principles that underlie their views on management and update their priors with new information. I believe that this Bayesian approach is a managerial superpower. I will come back on probabilistic methods throughout the season but here is a short primer:
What are the first principles of business agility?
Optionality is one way to use Bayesian reasoning to make better decisions in volatile and uncertain environments. Nassim Nicholas Taleb popularized this concept in his book Antifragile. Taleb defines optionality as "the property of asymmetric upside (preferably unlimited) with limited downside":
Taleb arrived at optionality from the perspective of financial options trading. In the world of capital allocation, the concept seems to come naturally. Venture capitalists or private equity firms face uncertainty about the future performance of investments, and they leverage optionality to hedge their bets:
Develop models to find the companies with low downside risk but high upside potential.
Holding cash as a reserve to take advantage of opportunities as they arise.
Diversifying across sectors, geographies, and stages of development.
In today’s markets, no matter how much analysis we perform, we rarely know with certainty what the best course of action is. In any complex adaptive system, open-ended options are an excellent alternative to knowledge and prediction. To leverage the concept of optionality, we need to maximize the options at our disposal. Options give us the right - but not the obligation - to take an action.
Companies are not that different from capital allocators. They too have finite resources to allocate and strategic bets to place. Great leaders manage their company as a portfolio of real options, essentially creating a probabilistic framework for decision-making that allows them to adjust their strategies and actions based on new information.
How does optionality enable business agility?
Leaders who understand the underlying principles of optionality, non-determinism, and complexity will have a much stronger grasp on the implementation of lean management and agile software development.
For first principle thinkers, lean and agile are not a matter of imitation. They are a matter of minimizing waste and maximizing value. Truly agile companies invest only in what is necessary while building in the flexibility to pivot if market conditions or customer needs change.
They appreciate that the lowest common denominator implementation of agileᵀᴹ (JIRA, sprints, backlogs, story points,…) does not automatically unlock the ability to pivot or jump on an opportunity when one presents itself.
Instead, modern leaders leverage their understanding of the underlying principles toward true business agility.
Let’s make this tangible.
Aligned autonomy
Product teams can only adapt to changing market circumstances when they have the autonomy to do so. However, their hands are often tied in an enterprise context:
In hiring and rewarding people, a product team is constrained by an HR department.
In selecting tools and partners, they are constrained by a procurement department.
In deploying software, they are constrained by the enterprise architecture.
Autonomy is not a goal in itself. The above constraints are only a problem when the different parts of the organization are not aligned. If the product team is optimizing for agility (“we need a time & materials agreement with partner X”) but the procurement team is optimizing for control (“we want fixed scope and budget”), the company is wrestling with itself.
Top-down or bottom-up transformation?
This is a false dilemma.
In the procurement vs product team example, leadership has a role to play in setting the direction but the lack of alignment may just as well be identified from an operational point of view.
If the boots on the ground are permanently at odds with the generals, you should fix that problem before attempting a digital transformation. Let’s assume a minimally functional enterprise.
Leadership must have an excellent grasp of complexity theory and its downstream ideas of Bayesian reasoning, optionality, agility… There are countless techniques in those toolboxes but they don’t come prepackaged. Every context is unique, and leaders will have to do the hard work of figuring out what makes sense for their company.
At the same time, there is plenty to tackle on the operational level:
Take behavior-driven development (BDD) and test-driven development (TDD), for example. These practices create the option to change the software by telling us when we have broken it. The earlier we catch errors, the easier it is to change course.
The same can be said of the practice of ensemble programming. The best way to learn as a team is to actually collaborate and get the most out of each other’s unique set of skills, perspectives, and experiences. Teams that work together like a jazz band benefit from immediate feedback, faster flow, increased focus, and better software quality.
Companies are socio-technical systems
When we are talking about alignment and collaboration, these are social challenges. At the end of season one, I introduced the idea of socio-technical systems thinking, an approach that recognizes the interaction between people and technology in our workplaces.
The socio-technical approach to architecture is a good example. It emphasizes modularity as an important design principle for agility. Sociotechnical architects strive to align architectural boundaries with team topologies, domain knowledge, and business value. Good architecture is about more than producing UML diagrams in Archimate.
Limit your downside
Optionality (and downstream ideas like lean and agile) is not just about capturing the upside. They can also help in managing risk. To do that, teams and companies want to avoid decisions with a fixed upside but an unlimited downside. For example:
Accruing technical debt. On the upside, you get to release slightly sooner, but on the downside, you are jeopardizing the ability of the product to evolve in the long term.
Postponing infosec investments. On the upside, you get to cut some costs or avoid those pesky ISO27001 audits. On the downside, you might find yourself locked out of your data and being held hostage by ransomware attackers.
The best options are asymmetrical in the other direction. They have a fixed cost but an unbounded upside. John Cutler has a great visualization that looks at optionality from a capacity planning perspective:
It is also a good introduction to the logical follow-up questions. Companies have to explore and exploit. At what point do we sink our teeth into something? And how do we balance optionality and focus? I’ll come back to these questions in my next essay.