The problem with Agile

The Problem with Agile

Nov 19, 2019

GUEST POST: By tech guru, Andrew Wilkinson – Andrew looks at the phenomenon that is… Agile

The problem with Agile is that it’s brilliantly branded. This has fueled uninformed appeal and encouraged its application too universally. So much so that it risks being wrongly discredited. That would be a shame as it can be a powerful tool if applied appropriately.

Agile just sounds great – who doesn’t want to be agile?

Agile creates images of being nimble, faster, smarter, and innovative. If you embrace Agile, you are or want those things. If you don’t embrace Agile, what does that say about you or your organisation? It’s no wonder corporations are embracing it in droves.

Would it be as popular if it was called something else… for example Innovation and New Execution Processes for Technology (INEPT)? I suspect not.

Before we can consider its correct (or incorrect) application, we need to consider what it is and what it’s designed for.

A (really) brief history

Agile was formalized in a manifesto by 17 software developers in 2001.^[1] It united the many new lightweight software development methods that had emerged prior to this date (e.g. Scrum, Rapid Application Development, Extreme Programming etc). It also has its roots in, or shares commonality with, manufacturing and quality control concepts such as Kanban, just in time, Lean, Kaizen, production lines and continuous improvement. Some of these concepts date back to W Edwards Deming and even Henry Ford. So, it’s an amalgamation and evolution of concepts, some of which have been around for 100 years or more.

What is Agile?

Agile comprises a set of principles, practices and ceremonies wrapped up in an adaptable process. Put simply, it is a way of working that empowers teams to self-direct and enables feedback-induced changes in scope throughout development. It avoids hardening of requirements at the start – when the team knows the least about the efficacy of the proposed solution.

Agile is ideally suited for developing solutions to complex problems / opportunities where there is a high degree of uncertainty or lack of understanding associated with the problem / opportunity itself or how to best address it. It mitigates the risk that uncertainty represents by enabling continuous improvement (learning) using feedback cycles. Each feedback cycles starts with applying current understanding to arrive at a proposed solution, delivering that solution, gaining feedback (learning) which increases our understanding. This process is repeated until a problem is understood, a satisfactory solution is found, or the team abandons the work.

It’s a tool and not a universal panacea to cure all orgaisational ills

Oxford dictionary defines a tool as “a thing that helps you do your job or to achieve something”^[2]. In this light, Agile is a tool that is designed to get a desired outcome within a certain context. Like any tool, Agile has a specific application and requires skill to use well. My beef is that people and organisations are applying it universally when it shouldn’t be. For someone who’s favourite tool is a hammer, everything looks like a nail.

Unwinding Agile is also damaging on culture as it empowers teams to select what they work on and how they go about it. As John Culkin said, “We shape our tools and, thereafter, our tools shape us.”^[3] Agile is a great example of this – it becomes entwined with cultural values of an organization and its individuals. Ever tried to tell an Agile team what they should do? Retreating from Agile risks sacrificing cultural values, leaving an organization worse off than when it started its Agile journey.

Agile risks getting a bad name. Both from ill-conceived implementations and any unwinding. All it will take is an organization misusing it, creating internal havoc, spending a lot of time and money, ending up unwinding it and damaging culture and morale – “…we tried it, but it didn’t work”.

When and how to go Agile?

So, when, where, and how do we apply it or any other tool to get the job done most effectively? There are two key questions to ask when considering applying Agile to address a problem or opportunity:

Question 1. How well understood is the problem, opportunity, solution and execution?

How understood is the problem or opportunity you’re trying to address? How certain are you that the proposed solution will have the intended affect? How clear are you in the execution on the proposed solution within your technical and business context?

I don’t mean that you understand it enough to write a powerpoint strategy document about it, I mean deep and intimate understanding. This requires self-honesty about what you really know, wat are hypotheses and what you want to be true (e.g. if X was true, we could create a really innovative product customers would love). This usually only comes from having been actively working on solutions like this in the space. Usually repeatably.

The less you understand the problem, opportunity or solution, the more valuable early end-user feedback is in refining our understanding. Successive rounds of testing and feedback loops allows a team to narrow in on the actual problem and a suitable solution which increases customer value or the likelihood of success. Technology companies like realestate.com.au generally do this well. In this context, Agile programs are often focused on delivery of value to customers as early as possible.

Conversely, the more understood the problem, solution and execution, the less you need feedback to refine your understanding. In this case, Agile if not usually the best tool. Property Developers tend to use traditional project management methods as the problem or opportunity and execution varies little and the fastest path to the end result is desired.

What is the penalty for getting it wrong?

What is the likelihood of getting it wrong? What is the impact of getting it wrong? How costly is it to correct a failed solution? In summary, what is the consequence of failure? The lower consequence of failure, the more you don’t have to over think it.

The greater the consequences of failure, the more important our understanding of the problem / opportunity, its solutions and the executions are. For NASA’s Apollo moon program in the 1960’s, a failure could lead to loss of life. Space exploration is dangerous, and lives have been and will be lost. For NASA, the right way to proceed should facilitate early learning to aid in refining their design to accomplish their mission, shorten development timelines and reduce risk to life. I would argue that NASA used a form of Agile for this program.

If the consequence for failure is low and the time, effort or cost to pivot to a different solution is low, you can choose to try your best idea first and see if it works. If it fails, try the second and so forth.

A framework for considering when to apply Agile (and other tools)

The combinations of these two factors (understanding and consequence of failure) determine the selection of the most appropriate tool and specifically, when Agile should be and should not be applied (see Figure 1).

Figure 1: The application of Agile given consequence for failure vs understanding

Quadrant A. High Consequence of Failure & Low Understanding (Extreme Risk)

Tools: Staging. Agile (front load risk reduction).

This is what NASA’s Apollo moon program faced. It had never been done before (very low understanding) and the consequences of failure were very high (both political and in human life). They chunked the problem into parts, conducted experiments and explored and perfected solutions one part at a time. They used what they learned from one part to inform the solution to the next. They didn’t call it Agile back then, but it was a form of it.

Quadrant B. Low Consequence of Failure & Low Understanding (Med Risk)

Tools: Agile (front load accelerated value delivery).

This situation is faced by product squads in tech companies every day. The impact of failure may be revenue falling below expectation or launching a new feature that is not used by customers. While they are wasteful and revenue is always important, these impacts are not usually life threatening. The consequence of failure is therefore relatively low. Customers can also react in ways we can’t predict and conducting research on never before seen products or features is unreliable. Consequently, our understanding of opportunity, solution and execution is low. Agile applies well here. Successive rounds of feedback facilitate learning, improve the proposed solution and accelerate delivery of value.

Quadrant C. Low Consequence of Failure & High Understanding (Low Risk)

Tools: Lean, Kanban, Just in Time (JIT)

This situation is faced by customer service teams on a daily basis. Each individual interaction usually doesn’t risk the company and processes are well understood (e.g. ticketing, triaging, prioritizing, resourcing, resolution and escalation). The feedback cycles of full blown Agile are not needed here. Lean, Kanban and/or just in time are more suited. These techniques are often grouped within the broader or colloquial “Agile” definition but for clarity, I have called them out separately. For customer service, Lean is an efficient way of allowing the team to prioritise and process work parcels effectively and transparently. At its simplest, it involves a visible card wall which shows each parcel of work, the progress and status of the work and resourcing. At regular (usually daily) stand-up team gatherings at the wall, teams can respond to changing demands using available resources to maximise customer value. Issues and system improvements can be identified, and blockages escalated.

Quadrant C. High Consequence of Failure & High Understanding (High Risk)

Tools: Staging. Waterfall. Project Management.

This is what migrators of data centres faced before Cloud computing. The consequence of failure can be high (loss of valuable data and risk to reputation). The migration process varies little and for seasoned practitioners, the execution is routine and well known. For the most part, property development is similar; the same steps and processes which are repeated often by seasoned practitioners. A traditional waterfall method is ideally suited. Feedback (and therefore Agile) is not needed and often less efficient.

To apply this to your own project, team or organisation, simply use this framework. Consider where you sit on the matrix being honest with yourselves.

In conclusion

Agile is a great tool. It’s just branded in a way that appeals to enterprises that are attracted by the connotations with little understanding of what it was designed to do, the value it can drive and therefore the implications for culture and damage is poorly applied.

References: