joelkuiper.eu

The Agile cargo cult

Introduction

Last week I attended, for the first time, the annual XP Days. While fun, I could not shake my cynicism towards Agile. I could rant about their goofy ways, such as grown guys in bunny suits being shot with nerf-guns for the sake of “kill all projects”. Or the blatant misunderstanding and misuse of neuroscience, similar to the way New-Age people give credibility to their cures with quantum mechanics (simply stating something is science does, in fact, not make it science). But maybe I’m just too much of a downer to appreciate those things (“That’s like, just your opinion man”).

Instead this piece will focus on the inherit problems of Agile if embraced unconditionally (Drinking the Kool-Aid). Since there are plenty of problematic things which I feel are often unaddressed.

Cargo cults

During the second world war the indigenous inhabitants of Melanesia were confronted with modern products from military cargo drops. The act of production was held in extremely high regard in their cultures, and they found the products so magnificent that they could not comprehend their existence. They explained the products as gifts from their ancestors, believing that if they would appease the ancestors enough they would bestow them with wealth beyond measure. The rituals for appeasement took the form of mimicking the conditions from the first cargo drops, including military parades and most iconically the replication of an entire airfield.

Obviously building an airfield with wooden headphones and other fake apparatus seems silly to us. But sometimes we are guilty of similar behavior. Richard Feynman popularized the term Cargo Cult science, indicating activities that might look like scientific work, but are far from. Often the metaphorical use of Cargo Cult indicates having rituals or activities that probably have no influence on the actual outcome, and perhaps were once only coincidental to success.

I’d like to propose that sometimes Agile is, in fact, Cargo Cult. My reason for believing this is twofold. First, I do not believe that some of the rituals of Agile such as Test Driven Development, daily stand-ups or planning poker have in itself any influence on actual (product) success. Maybe some are necessary but certainly not sufficient for proper development. Secondly I think that the Agile methodology can be actively harmful to a successful development cycle.

Coincidental

The Agile manifesto is clear:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

But many of these core principles are often expressed within standardized frameworks such as Scrum. In these frameworks the rituals become the goals, instead means to the principles they want to uphold and tend to be done without understanding why they are done.

For example, I’ve often seen daily standups not being used for the benefit of development, but instead being abused as a method for the product owner to exert force on team members.

Likewise planning poker is a tool for gaining better understanding of the complexities involved in a story. But often it is used simply as a bean counting tool for project managers.

Furthermore, embracing Test Driven Development “because it is Agile” while completely ignoring the costs would be very non-Agile; the process and tools are imposed over the individuals and interactions.

If rituals are done without understanding why they should be done, they become Cargo Cult. I’ve seen this often: the team claims Agile but many problems are not tackled in the appropriate way.

Case in point: teams often feel that the product owner becomes a boss, while many of the Agile principles focus exclusively on mutual trust and communication. Thus if an idea is clearly stupid “it’s what the boss wants” is not a valid answer.

And if such an impasse arises transperancy and consensus are the tools for resolving the conflict, not (ab)using planning poker to collectively overestimate the complexity or push the story back on the list, which I’ve witnessed first hand.

Agile resolves around principles which should be understood properly. Implementing just the rituals would be rather ironic.

Harmful

Now that I’ve covered why some of the Agile rituals can be Cargo Cult if not understood properly I will outline why Agile principles themself can be, especially if embraced unconditionally.

Agile principles are focused on iteration and adaptation, as opposed to planning and rigidity. The belief is that many things are not very solid to begin with, so imposing rigidity would be like trying to build the pyramids on quick sand. This is true … up to a point.

In many ways the Agile process resembles evolution: adapting to local changes by making small iterations. And we know this works well in nature, mistakes get punished and positive things get rewarded. But it is not very elegant. Evolution has often been compared to a drunkards walk on a mountainous landscape, surely the drunkard will reach the top (or bottom) at one point, but someone with a map would’ve done a far better job.

If we are all Übermensch then we know what is right and what is wrong instinctively. But we are not, and thinking about what you, the software or the company needs to do in order to be successful is often non-iterative, it takes a long term strategy or vision (the map). But in Agile vision and strategy have the tendency to become dirty words. It rewards short term gain more than long term pay-off. In some ways, especially if done with inexperienced people (which we all are at some point), Agile invites De kantjes er vanaf lopen (Dutch for doing just barely or not enough for a passible job).

But I would argue that strategy and vision are needed desperately when trying to build an elegant piece of software or empire. Without them you risk getting stuck in a local optimum instead of a global one. Elegance is non-iterative. Planning is not a dirty word, and sometimes it means paying a huge cost for pay-offs much later.

Conclusion

Don’t drink the Kool-aid, it is cyanide. Planning the long game is not a sin. Understand the reasons for the rituals, and make reasonable choices (as Kant would’ve put it: wisdom is required for morality).

Perhaps I just love Anarchy too much, if done with the right people that is.