Agile Development and Project Design

Agile development has found itself under a fair bit of criticism in recent years, from both managers and developers alike: managers tend to complain about Agile’s apparent lack of accountability, structure, and ability to deliver on schedule or budget; while developers tend to complain about pretty much the same things (just from a different angle). In truth, it is hard to blame anyone for becoming disenchanted, given how badly Agile tends to be practised in the real world: the chaos spawned from leaping to the keyboard at the merest whiff of a specification change; the seemingly wilful disregard for quality assurance (“We’re doing Agile - we don’t have time to test - just make sure you write it without any bugs”); and the almost pathological fear of committing to (literally) anything. Why should anyone have faith in Agile, especially if it gives you an in-built excuse to never think ahead? Regardless of what some would have you believe, a business of any size needs a plan. A business without a plan is little more than an expensive hobby, and the same goes for any significant software project.

Here is a quick thought experiment: suppose you had an idea for the next big “killer app”, one that you felt was certain to make you the next Gates, Ellison or Zuckerberg; what would you do? You would probably pool your life savings, remortgage the family home, maybe even take out a loan on top of that just to fund a business venture into which you were ready to pour your heart and soul. Wouldn’t you want to know how that money was going to be spent? Wouldn’t you expect the team you hired to have a well researched plan of action? Wouldn’t you expect them to have mechanisms in place to deal with the constantly shifting needs of the market place? Of course you would. So why treat your company’s money any differently? Or worse, your customer’s money?

As I have mentioned in previous posts, the real victim in all this is the good name of Agile, and the way in which many practitioners drag its reputation through the mud every time they dive into a project with complete disregard for what it actually says.

Planning? Architecture? Research and requirement gathering? An Agilista craves not these things.

I would challenge anyone to demonstrate where in the Manifesto or the Principles is says that an Agile project cannot have a plan, an architecture, a proper phase of research and discovery, or a deep understanding of business requirements (note: I said “business requirements”, not “software requirements”). In fact, I would go as far to say that every one of those things is actually a strong prerequisite for an Agile project.

No system, no matter how elegant or robust, can be considered well-designed unless it can be properly built. So, much in the same way a software architect would design a system, the project to build that system must also be designed. Only then can you provide a practicable estimate of its eventual cost (resources), schedule (time), and functionality (results).

However, it is crucial to understand that any project is only as valid as the information available at the time; so when requirements change, as they inevitably do, so must your project design in order to accommodate those changes. Think of it like charting the course of a ship across the Atlantic: constantly monitoring and replotting the trajectory of travel as the surrounding elements jostle and jolt the vessel in all directions. Just because you cannot control the elements does not mean that they should control you. Remember: “responding to change over following a plan” does not mean “instead of following a plan”.

The progenitor of Agile, the Toyota Production System, explicitly states that stability and well-understood working practices form the very centre of its foundation. From the middle of the 20th century, and within a generation, Toyota went from being a third-rate Japanese car producer to the most successful and efficient manufacturer of automobiles on the planet.

By making (what we would now recognise as) Agile Principles part of their lifeblood, they didn’t just do Agile, they became Agile. At any given point, they knew exactly what their plan of action was, how long it would take, and how much it would cost. The instant a requirement changed Toyota were perfectly placed to harness that change, replot their course, recalculate their schedule, and redetermine their budget. The difference was that they could do this 12 months in advance of the original deadline instead of 12 hours, giving them plenty of time to work out (a) whether the change was feasible, (b) whether it was worth doing, and (c) how they could make it happen.

Agile is not about building software more quickly; Agile is about ensuring that building good software takes as long as it must take, but no more (even in the face of changing requirements). While change is inevitable, no-one ever said that it has to be free - all change comes with a cost, be that in time, money or both (and to pretend otherwise is just plain dishonest). However, through technical excellence, good design, and a deep understanding of the business requirements, it becomes possible to minimise the cost of change while sustainably harnessing it for competitive advantage. By doing this in the framework of a well-designed project, not only can you bring real Agility to your planning process, but to the whole of your development lifecycle.