Complexity & Empiricism

Before Scrum & Agile became widespread, traditional approaches to software development utilised a predictive method. This was based on the belief/assumption that the work could be effectively planned in advance and change could be controlled. This is not the case with complex work, and most software development work falls into the complex domain.

As a result, plans and budgets frequently overrun, quality may decline as pressure increases and as a result, expectations are often not met. The consequence is typically a loss of trust, money and damaged relationships. Products fail and organisations may fail with them.

Predictive approaches do not help when we are doing complex work. Work where outputs, outcomes and challenges cannot be well predicted and fully understood ahead of time. In these environments, we cannot standardise and rely on best practices. Each challenge we face is in some way unique and requires its own new and somewhat unique solution.

Scrum is designed to help us carry out such complex work. It is based around empiricism and the scientific method. The 3 pillars of this empirical approach are:

  • Transparency
  • Inspection
  • Adaptation

Scrum directs us to develop iteratively and incrementally combined with an empirical approach to help us deliver products of the highest possible value.

We learn as we move forward and re-plan and adapt as we discover new insights. We accept that ahead of time, it is not possible to know everything about what our customers want, what technical challenges we will encounter and how people will work together. As a result, any plans we make are based on incomplete data and so likely to be less, rather than more reliable. Plans will need to change and improve over time as more becomes known.

As more becomes known about our product domain, our ability to make accurate predictions may increase. Knowledge comes from experience and the only way to get experience is to build the product, deliver it to customers and inspect the results. Scrum creates an environment where these activities occur by design.

“Scrum is a lightweight framework that helps people, teams and organizations generate value through adaptive solutions for complex problems.” – Scrum Guide 2020