Agile Software Development – A Paradigm Change
I’ve recently realised how difficult it is to clearly communicate the concepts of agile software development. As a starting point, I think it is useful to point people to the Agile manifesto (http://agilemanifesto.org) which lists the high level shared principles of agile methodologies. However, I have often found myself discussing something with a senior member of a software development team, under the assumption that we are working from similar shared principles, only to find from a question or request that my assumption is blatantly wrong. For example, in a discussion about continuous-integration or regression testing, they may ask a question or give a request such as:
Therefore, I realise I need to be better at explaining the paradigm change from which everything else flows. This is my first attempt.
In a nutshell, agile development is about being nimble: being able to change direction, quickly and at low cost, when new requirements are discovered or an unknown is resolved. When working on a project with a lot of unknowns and when the cost of change is low, it is far more effective to take small steps and change direction frequently.
The original concept of change in a software project can be illustrated in the following graph:
Exponential rise in cost for fixing a defect later in a project.
This exponential curve is the increased cost of change the later that a defect or a requirement is discovered. This was quite true when software projects were developed in Cobol: the requirement that we can represent dates after the year 2000 was incredibly expensive to modify. If only they had realised that during requirements analysis! Hence the solution was to try to avoid any late discovered requirements or defects by spending significantly more time capturing requirements up front and doing software design.
This curve doesn’t have to be this steep though and can be flattened significantly. Good software engineers are now well practiced with object-oriented principles such as encapsulation, abstraction, and decoupling. Applying these principles is simplified with a modern object-oriented programming language. With a large suite of automated regression tests developed in conjunction with the software, it is easy to determine if any change has an unintended impact. Therefore, if software is developed using these principles and with a good coverage of tests, the cost of a change made late in the project is not that much more than a change early in the project. Imagine if the old Cobol programs had a class called ‘Date’ with a method called ‘Compare()’, and they had been developed with a suite of automated regression tests. Then I suspect that instead of years of effort, the defect could have been handled with an afternoon’s work.
This is the enabling factor of agile development. It is normal for a software project to contain a large number of unknowns, such as requirements that even the customer doesn’t fully appreciate, capabilities and limitations of the technology employed, and the significant changes to technology over time. Trying to resolve these unknowns with up front analysis has diminishing returns. So if the cost of change is low, it is far better to take small steps, correcting direction frequently when new requirements are discovered or unknowns are resolved.
So in my opinion, that is the big mind-shift that people need to make to fully grasp agile software development. Unlike building another harbour bridge, where there only a few unknowns and the cost of change late in the project is large, it is best to avoid big upfront planning efforts and develop easy to modify software in small incremental steps.
