On the surface, the world of agile software development is bright, since it is now mainstream. But the reality is troubling, because much of what is done is faux-agile, disregarding agile’s values and principles, writes programmer Martin Fowler. The three main challenges we should focus on are: fighting the Agile Industrial Complex and its habit of imposing process upon teams, raising the importance of technical excellence, and organizing our teams around products (rather than projects), he added.

The State of Agile Software in 2018
https://martinfowler.com/articles/agile-aus-2018.html

Many of today’s best software companies tout agile development as a way to release software early and often, as longer-term planning can be more heavily impacted by project unknowns.

Yet why do IT projects using agile still consistently hit delays and exceed budgets?

Our development team was able to triple productivity by looking outward for inspiration, and found that the project planning methods commonly used in non-IT engineering projects have the key to solving this problem. It’s called reference class forecasting.

As a consequence of better estimates and improved productivity, team morale and confidence markedly improved.

Oxford researchers Alexander Budzier and Bent Flyvbjerg published research in 2013 in which they found that agile methods appear to improve project delivery times. Yet, in the more than 4,000 IT projects they surveyed, the average schedule overrun was +37 percent , and the average cost overrun was +107 percent. Outliers that significantly exceed schedule or budget can tremendously impact business.

One problem is that the human brain’s forecasting capabilities are limited, leaving us prone to cognitive biases that lead to systematic errors of judgement. This bias tends to be overly optimistic rather than pessimistic regarding the amount of work required for projects. The use of “story points” does little more than foster overconfidence by suggesting an objective quantifiability which, in reality, is merely an illusion.

different strategies have been developed within the agile development community to address this problem. Most prominent perhaps is the #NoEstimates movement

As your database of comparable cases grows, your predictive accuracy for building new product features will improve.