Complexity-aware monitoring, evaluation and learning
Developing a Theory of Change (ToC) and using impact pathways (or logic models) can help facilitate collaboration and planning among diverse stakeholders to achieve outcomes over different timeframes and scales. However, especially in complex settings, this requires a shift from focusing on attribution to recognising contribution, which means explicitly acknowledging the roles and inputs of partners and other actors in achieving and demonstrating outcomes. It is increasingly recognised that the relationship between cause and effect in complex situations can be unpredictable, resulting in unexpected outcomes, both positive and negative. In such contexts, a logic model can serve as a guide for basic monitoring (ensuring that actions align with intentions) and evaluation (assessing whether the expected results have been achieved).
To address the complexities and uncertainties inherent in these situations, it is important to support various complex programmes, such as those in agriculture, public health, natural resource management, and energy, with monitoring, evaluation, and learning (MEL) strategies that are sensitive to complexity. Complexity-aware monitoring and evaluation (CAME) techniques can be used to identify unexpected results, different causes for outcomes, and various ways to achieve impact, recognising that this work emphasises contribution rather than direct cause and effect. By incorporating continuous learning as part of the management, stakeholders can adapt and respond to emerging insights and challenges, enhancing the overall effectiveness of their initiatives. This approach acknowledges the complexity of various programmes and the multifaceted nature of change.
The papers and articles below provide further insights into the theories, approaches, and tools that underpin complexity-aware monitoring, evaluation, and learning (CAME/CAMEL).
Complexity-Aware Monitoring and Evaluation
This 2021 paper by Tilman Hertz, Eva Brattander, and Loretta Rose explores the elements of a complexity-aware monitoring and evaluation (M&E) system for international development programmes focused on sustainable development. Recognising that social and ecological systems are integrated and complex, the authors argue that traditional M&E approaches, which rely on rigid theories of change, are inadequate for capturing the dynamic and unpredictable nature of these systems. Instead, they advocate for a real-time, iterative M&E approach that continuously redefines narratives for change, embracing uncertainty and fostering flexibility, experimentation, and innovation. The paper highlights the importance of involving donors in developing a shared understanding of systems and longer timeframes, shifting from tracking pre-determined plans to reviewing emergent trajectories. By focusing on understanding and learning from unsuccessful experiments, this approach aims to build greater transformational capacity and address the inherent complexities of sustainable development.
Introducing CAMEL: Counterpart’s Complexity-Aware Monitoring, Evaluation, and Learning Framework
Counterpart’s Complexity-Aware Monitoring, Evaluation, and Learning (CAMEL) framework integrates complexity-aware monitoring (C-AM) with Collaboration, Learning, and Adaptability (CLA) to address the challenges of implementing development programs in complex environments. This approach, exemplified in the 2019 USAID-funded Participatory Responsive Governance – Principal Activity (PRG-PA) project in Niger, uses sentinel indicators to track significant changes within the system. By organizing quarterly pause-and-reflect sessions the framework ensures adaptive management in response to emerging insights. The CAMEL approach, informed by Ralph Stacey’s Agreement and Certainty Matrix and Michael Bamberger’s Complexity Checklist, emphasizes the importance of flexibility, continuous learning, and stakeholder engagement to navigate the unpredictability of complex systems.
Discussion Note: Complexity Aware Monitoring
This Note outlines general principles and promising approaches for monitoring complex aspects of USAID development assistance. Complexity-aware monitoring is a type of complementary monitoring that is useful when results are difficult to predict due to dynamic contexts or unclear cause-and-effect relationships. This builds on 2016 USAID discussion note by Heather Britt outlines general principles and promising approaches for monitoring complex aspects of development initiatives. It discusses five particular approaches to complexity-aware monitoring for USAID projects and strategies: sentinel indicators, stakeholder feedback, process monitoring of impacts, most significant change, and outcome harvesting.
Twinning “Practices of Change” With “Theory of Change”: Room for Emergence in Advocacy Evaluation
This 2017 paper by Bodille Arensman, Cornelie van Waegeningh, and Margit van Wessel argues that ToC’s focus on cause–effect logic and intended outcomes does not do justice to the recursive nature of complex interventions such as advocacy. They propose putting “practices of change” at the center by emphasizing human interactions, using the analytical lenses of strategies as practice and recursiveness. This provides room for emergent outcomes and implies a different use of ToC.
Performance monitoring’s three blind spots
This 2013 slideshare powerpoint by Ricardo Wilson-Grau reminds us that linear approaches to program evolution miss three key areas – unintended results, alternative causes and multiple pathways.
Contribution analysis: A new approach to evaluation in international development
This article by Fiona Kotvojs and Bradley Shrimpton examines AusAID’s shift to contribution analysis and a system of outcome-based monitoring and evaluating. It looks at the method of contribution analysis, its implementation in the Fiji Education Sector Program, an assessment of its use, and the challenges faced in the application of contribution analysis.
Complex adaptive systems: a different way of thinking about health care systems
This 2004 short paper by Beverly Sibthorpe, and colleagues provides a brief synopsis of literature relevant to complex adaptive systems (CAS) and health care systems. It reminds us of the need for M&E systems to take account of this.
Appreciating the recursive and emergent nature of Complex Adaptive Systems
Strategy-as-Practice: Taking Social Practices Seriously
This 2012 paper by Eero Vaara and Richard Withington reviews research in Strategy-as-Practice (SAP) and suggests directions for its development. They point to the power of this perspective via in its ability to explain how strategy-making is enabled and constrained by prevailing organizational and societal practices. They go on to outline five directions for the further development of the practice perspective: placing agency in a web of practices, recognizing the macro-institutional nature of practices, focusing attention on emergence in strategy-making, exploring how the material matters, and promoting critical analysis.
Using programme theory to evaluate complicated and complex aspects of interventions
This 2008 paper by Patricia Rogers proposes ways to use programme theory for evaluating aspects of complex programmes. It highlights how complex programme theory may be used to represent recursive causality (with reinforcing loops), disproportionate relationships (where at critical levels, a small change can make a big difference – a ‘tipping point’), and emergent outcomes.
