Key evaluation questions (KEQs)

This post explores how evaluations benefits from being focused on a small set of key questions. These are often referred to as key evaluation questions (KEQs). They should be seen as high level questions that assess progress towards the main specified outcomes, and will be answered by combining data from several sources and methods. 

Evaluations provide an opportunity for your (or your clients’s) intervention’s overall progress to be considered, including focused consideration of specific aspects of the initiative. A well-developed theory of change (TOC) and accompanying logic models provide an outline that helps to develop measures of success that traces the intervention’s development and impact over time. These measures, in turn, need to be focused with appropriate KEQs that are driven by funders, project participants and other key stakeholders.

The five criteria to evaluate interventions (relevance, effectiveness, efficiency, impact, and sustainability outlined in the OEDC/DAC evaluation guidelines provide a good starting framework for a range of initiatives in development areas (health, natural resource management, community resilience, etc.) . Evaluation questions also to be considered in a complex intervention such as this should address context, reasons for adaption and emergence of activities and outcomes, different perspectives and inter-relationships that impact project success, sustainability and transferability.

A useful starting set of key evaluation questions to guide initial analysis are:

• Is the research delivering on outputs and outcomes as planned? (efficiency and effectiveness)
• Have applied activities and their delivery methods been effective? Are there aspects that could have been done differently? (process effectiveness)
• Is the wider project story being told? What range of outcomes (intended and unintended) has the research project contributed to – taking account of each of social, economic, environmental and cultural considerations (impact)
• How has the project influenced the stakeholder community, and what capacities has it built? (impact)
• Is the project being delivered on budget? What aspects of the participatory elements of the project could be done differently next time to cut costs while still delivering achievements? (efficiency)
• Is the project impacting positively on key groups and issues that have been identified as important in project design? (impact)
• Is there evidence that the initiative is likely to grow – scaling up and out – beyond the project life? (sustainability)
• To what extent did the initiative deliver against the needs of key stakeholders? Were the size, scale and approach taken for each need appropriate? (impact & efficiency)

These questions need to be clarified by key project stakeholders. Some may be amended, others dropped, and new questions can be included. Developing these questions also provides an opportunity to revise the underlying theory of change and any accompanying logic or outcome models. In this way KEQs can be seen to help intervention planning and evaluation.

A number of other Learning for Sustainability pages provide additional information on this topic. The page on selecting evaluation questions and types directly builds on this topic and includes links to additional external resources on KEQs. A number of other related M&E themes are introduced through the introductory page – Planning, monitoring & evaluation – closing the loop.

Will Allen( Principal, Will Allen & Associates )

An independent systems scientist, action research practitioner and evaluator, with 30 years of experience in sustainable development and natural resource management. He is particularly interested in the development of planning, monitoring and evaluation tools that are outcome focused, and contribute towards efforts that foster social learning, sustainable development and adaptive management.