Complexity-aware PM&E

Complex projects require complexity-aware M&E approaches to shine lights on blind spots. (c) Can Stock Photo / alphaspirit

Developing a Theory of Change (ToC) and accompanying impact pathways (or logic models) can assist by supporting diverse stakeholders to work together and plan for outcomes across different time horizons and scale.  However, this implies a move to contribution rather than attribution. It requires us to more overtly acknowledge the role and inputs of partners and other actors both in achieving outcomes, and in providing evidence for those outcomes. At the same time, there is a growing recognition that the relation between cause and effect in these complex situations can be unpredictable, and unexpected (both good and bad) results often happen.  In such complex contexts, an accompanying logic model can guide basic monitoring (is what is happening what was intended?) and evaluation (have the actions delivered what was expected?). However, there is a corresponding need to support R4D approaches by complexity-aware M&E approaches that can aid in tracking a fuller range of outcomes, causal factors, and pathways of contribution. The papers and articles below shed more light on theory, approaches and tools that underpin complexity-aware monitoring and evaluation.

Discussion Note: Complexity Aware Monitoring
This Note is a starting point for USAID staff to integrate complements ADS 201 and outlines general principles and promising approaches for monitoring complex aspects of USAID development assistance. Complexity-aware monitoring is a type of complementary monitoring that is useful when results are difficult to predict due to dynamic contexts or unclear cause-and-effect relationships. This builds on 2016 USAID discussion note by Heather Britt outlines general principles and promising approaches for monitoring complex aspects of development initiatives. It discusses five particular approaches to complexity-aware monitoring for USAID projects and strategies: sentinel indicators, stakeholder feedback, process monitoring of impacts, most significant change, and outcome harvesting.

Twinning “Practices of Change” With “Theory of Change”: Room for Emergence in Advocacy Evaluation
This 2017 paper by Bodille Arensman, Cornelie van Waegeningh, and Margit van Wessel argues that ToC’s focus on cause–effect logic and intended outcomes does not do justice to the recursive nature of complex interventions such as advocacy. They propose putting “practices of change” at the center by emphasizing human interactions, using the analytical lenses of strategies as practice and recursiveness. This provides room for emergent outcomes and implies a different use of ToC.

Performance monitoring’s three blind spots
This 2013 slideshare powerpoint by Ricardo Wilson-Grau reminds us that linear approaches to program evolution miss three key areas – unintended results,  alternative causes and multiple pathways.

Contribution analysis: A new approach to evaluation in international development
This article by Fiona Kotvojs and Bradley Shrimpton examines AusAID’s shift to contribution analysis and a system of outcome-based monitoring and evaluating. It looks at the method of contribution analysis, its implementation in the Fiji Education Sector Program, an assessment of its use, and the challenges faced in the application of contribution analysis.

Complex adaptive systems: a different way of thinking about health care systems
This 2004 short paper by Beverly Sibthorpe, and colleagues  provides a brief synopsis of literature relevant to CAS and health care systems. It reminds us of the need for M&E systems to take account of this.

Appreciating the recursive and emergent nature of  CAS

Strategy-as-Practice: Taking Social Practices Seriously
This 2012 paper by Eero Vaara and Richard Withington reviews research in Strategy-as-Practice (SAP) and suggests directions for its development. They point to the power of this perspective via in its ability to explain how strategy-making is enabled and constrained by prevailing organizational and societal practices. They go on to outline five directions for the further development of the practice perspective: placing agency in a web of practices, recognizing the macro-institutional nature of practices, focusing attention on emergence in strategy-making, exploring how the material matters, and promoting critical analysis.

Using programme theory to evaluate complicated and complex aspects of interventions
This 2008 paper by Patricia Rogers proposes ways to use programme theory for evaluating aspects of complex programmes.  It highlights how complex programme theory may be used to represent recursive causality (with reinforcing loops), disproportionate relationships (where at critical levels, a small change can make a big difference – a ‘tipping point’), and emergent outcomes.