Complexity & Empiricism
Before Scrum & Agile became widespread, traditional approaches to software development utilised a predictive method. This was based on the belief/assumption that the work could be effectively planned and change could be controlled. This is not the case with complex work; most software development work falls into the complex domain.
Complexity refers to software development projects being often unpredictable and involving multiple interdependent variables, making it difficult to plan and execute linearly or sequentially. In complex environments, the best approach is to embrace uncertainty and adapt to changing circumstances as they arise. This contrasts traditional “waterfall” predictive project management approaches, which assume that requirements can be fully defined upfront and that the project can be planned accurately and executed linearly. When using this approach in complex environments, plans and budgets frequently overrun, quality may decline as pressure increases, and as a result, expectations are often not met. The consequence is typically a loss of trust and money and damaged relationships. Products fail, and organisations may fail with them.
Predictive approaches do not offer the best chance of success when doing complex work. Work where outputs, outcomes and challenges cannot be well predicted and fully understood beforehand. In these environments, we cannot standardise and rely on best practices. Each challenge we face is unique and requires its own new and unique solution.
Empiricism refers to the idea that knowledge comes from experience and observation rather than theory or speculation. Agile teams use an empirical process control approach to manage work. This involves raising transparency on the actual state of the environment and progress and then regularly inspecting and adapting based on feedback and observations. It uses data and metrics to guide decision-making and measure progress wherever possible.
Scrum utilises this empirical approach to complex work to help maximise value delivery. It directs us to develop our product iteratively and incrementally using small, self-managing, cross-functional teams. We learn as we move forward and re-plan and adapt as we discover new insights. We accept that in advance, it is not possible to know everything about what our customers want, what technical challenges we will encounter, and how people will work together. As a result, any plans we make are based on incomplete data and are likely less reliable. Plans will need to change and improve over time as more becomes known.
As more about our product domain becomes known, our ability to make accurate predictions may increase. Knowledge comes from experience; the only way to get experience is to build the product, deliver it to customers, and inspect the results. Scrum creates an environment where these activities occur by design.
“Scrum is a lightweight framework that helps people, teams and organizations generate value through adaptive solutions for complex problems.”