Using rubrics to plan and assess complex tasks and behaviors

A rubric is an easily applicable form of assessment. They are most commonly used in education, and offer a process for defining and describing the important components of work being assessed. They can help us plan and assess complex tasks (e.g. essays or projects) or behaviors (e.g. collaboration, team work). Increasingly rubrics are being used to help develop assessments in other areas such as community development and natural resource management.

Developing rubrics involves articulating and clarifying ‘the things that matter’ in a complex task or behaviour.

In a recent report we describe how rubrics can be used to assess complex tasks and behaviours such as engagement and partnerships. More resources on rubrics can be found from the main Learning for Sustainability rubrics page – Rubrics as a learning and assessment tool for project planning and evaluation. Rubrics are both an instructional tool and a performance assessment tool. They act as a guide to help practitioners clarify and understand both the objectives required to complete any particular initiative and the qualities required for achieving high standards in those objectives.

Although the format of a rubric can vary, they all have two key components:

  • A list of criteria – or what counts in an activity or task
  • Graduations of quality – to provide an evaluative range or scale.

Developing rubrics helps clarify the expectations that people have for different aspects of task or behavior performance by providing detailed descriptions of collectively agreed upon expectations. Well designed rubrics used for assessment increase the reliability and validity and ensure that the information gathered can be used to help people assess their management efforts, and improve them. It is different than a simple checklist since it also describes the gradations of quality (levels) for each dimension of the performance to be evaluated. It is important to involve program participants in developing rubrics and helping define and agree on the criteria and assessment. This broad involvement increase the likelihood that different evaluation efforts can provide comparable ratings. As a result, the assessments based on these rubrics will be more effective and efficient.

Involving people in developing rubrics involves a number of steps.

  • Defining the task to be rated. This can include consideration of both outputs (things completed) and processes (level of participation, required behaviors, etc.).
  • Defining criteria to be assessed. These should represent the component elements that are required for successful achievement of the task to be rated. The different parts of the task need to be set out simply and completely. This can often be started by asking participants to brainstorm what they might expect to see where/when the task is done very well … and very poorly.
  • Developing scales which describe how well any given task or process has been performed. This usually involves selecting 3-5 levels. Scales can use different language such as:
    –   Advanced, intermediate, fair, poor
    –   Exemplary, proficient, marginal, unacceptable
    –   Well-developed, developing, under-developed

Co-developing rubrics in this way helps groups and teams clarify and negotiate the expectations that members people may have for different aspects of project performance. It is important to involve key stakeholders in developing the final versions of rubrics that may be used to assess their activities.By working together members can develop shared descriptions of collectively agreed-upon measures for key areas of performance.  By involving stakeholders in helping define and agree on the criteria and assessment scales – as something they feel is achievable and within the limits of normal operations – the assessment is more likely to be used and acted upon by those involved.

Rubric assessments are not just about ticking boxes, their strength is that they support evidence-based performance reflection.

Equally, it is not just a tick-box exercise. Assessments should be evidence-based, providing an ongoing process of learning and adaptive management that continues throughout the life of the initiative. Facilitated learning debriefs and After Action Reviews support a reflective process that can be used as a basis for supporting this adaptive management approach. In this manner, the rubric development and assessement process itself can contribute towards adapting key project tasks and behaviours  in a way that grows and strengthens  performance over time.

More resources on rubrics can be found from the main Learning for Sustainability rubrics page –Rubrics – as a learning and assessment tool for project planning and evaluation. More material can also  be found from the indicators page. Other related PM&E resources can be found from the Theory of Change and Logic Modelling pages.

More information on my own use of rubrics can be found through recent research papers. One 2016 paper looks at the use of rubrics to support collaboration in an integrated research programme – Bridging disciplines, knowledge systems and cultures in pest management. Another recent book chapter  (2018) looks at the Use of Rubrics to Improve Integration and Engagement Between Biosecurity Agencies and Their Key Partners and Stakeholders: A Surveillance Example.

[Note: This post is an update of an earlier 2016 version with the original title – Using rubrics to assess complex tasks and behaviors.]

Share

An independent systems scientist, action research practitioner and evaluator, with 30 years of experience in sustainable development and natural resource management. He is particularly interested in the development of planning, monitoring and evaluation tools that are outcome focused, and contribute towards efforts that foster social learning, sustainable development and adaptive management.

Leave a Reply

Your email address will not be published.

*

Rubrics – as a learning and assessment tool for project planning and evaluation

Rubrics offer a process for defining and describing the important components of work being assessed
A rubric is an easily applicable tool for supporting learning, assessment and evaluation and professional development. They offer a process for defining and describing the important components of complex tasks and behaviours. They can then help us assess these tasks (e.g. essays or projects) or behaviors (e.g. collaboration, team work), and make evaluative judgments about about the quality of performance and effectiveness. Rubrics are most commonly used in education, and increasingly they are being used to help develop instruction and evaluation/assessments in other sectors such as community development and natural resource management. The term ‘rubrics’ is used most commonly in education and evaluation circles – but the similar use of multiple scaled criteria in this way has long underpinned the monitoring and evaluation of complex environmental and physical systems.

A brief introduction to rubrics can be found in the recent LfS posting – Using rubrics to assess complex tasks and behaviors. The first set of links below provide recent examples and learnings from their emerging use in the evaluation and strategic planning sectors (in areas such as community development and natural resource management). The second set of links points to material recognising the long use of rubrics for simultaneously supporting learning, assessment and professional development in the education sector.

Emerging uses of rubrics in project planning, implementation and evaluation


The use of rubrics to improve risk communication and engagement between biosecurity agencies and their key partners and stakeholders
This 2018 open-access chapter by Will Allen and colleagues outlines an action research approach to the participatory development of rubrics as a design and assessment approach to improve surveillance systems in a biosecurity setting.  They show how rubrics can provide a way for reaching a shared understanding of what matters, and how to assess that in terms of what can be confidently regarded as good practice—and equally what can be agreed on as emerging practice.


Evaluation building blocks – a guide
In this 2018 e-book, the Kinnect Group highlight the place of evaluative reasoning in evaluation. This guide sets out a step-based approach to evaluation, drawing on the work of many evaluation theorists, as well as their own practice-based body of knowledge. The approach is designed to be collaborative, practical and user-oriented. The guide is particularly helpful in reminding practitioners of the key questions to ask at each phase of the work.


Understanding the components of evaluative rubrics and how to combine them
In this 2018 e-book, Judy Oakden explores the different ways evaluative rubrics can be constructed from three basic components: key aspects of performance; levels of performance; and importance of each aspect of performance. She outlines some alternative ways she has combined the components in her own practice, and discusses the benefits and challenges of each approach.


To rubrics or not to rubrics? An experience using rubrics for monitoring, evaluating and learning in a complex project
In this Practice Note, Samantha Stone-Jovicich shares her experience using an evaluation and monitoring approach called ‘rubrics’ to assess a complex and dynamic project’s progress towards achieving its objectives. Rubrics are a method for aggregating qualitative performance data for reporting and learning purposes.


 Bridging disciplines, knowledge systems and cultures in pest management
This 2013 paper by Will Allen, Shaun Ogilvie, Helen Blackie and colleagues describes how their research team with a range of disciplinary and stakeholder expertise was able to use rubrics in an action research based approach to critically reflect on their engagement practice and identify lessons around how to collaborate more effectively. They discuss the implications of these experiences for other researchers and managers seeking to improve engagement and collaboration in integrated science, management and policy initiatives.


Evaluative rubrics: a method for surfacing values and improving the credibility of evaluation
This practice-based 2013 paper by Julian King and colleagues unpacks the learnings of a group of evaluators who have used rubrics to help make judgments about performance, quality, and effectiveness. They have found that while evaluative rubrics look beguilingly simple they are hard to do well. However, when done well, the use of this tool can substantially increase the use and credibility of evaluation.


Evaluation rubrics: how to ensure transparent and clear assessment that respects diverse lines of evidence.
This 2013 BetterEvaluation report by Judy Oakden provides a practical example of how rubrics can be used to ensure a transparent process for articulating the aspects of performance that are important. Rubrics also help evaluators to identify the data required to make judgements about the performance of the programme so that suitable data can be collected, and identify early any likely information gaps. When using rubrics, reporting can be succinct, but with sufficient detail that users of the evaluation consider the value judgements robust. This process results in evaluation that supports use by users of the evaluation


Selected rubric presentations and briefs:   The rubric revolution. (Jane Davidson, Nan Wehipeihana, Kate McKegg 2011); Rubric methodology basics. (Jane Davidson 2013); Focusing the Evaluative Dance – The Value of Rubrics in Messy Non-Profit Evaluation Contexts. (Kate McKegg 2013);  Rubrics – an assessment tool for the urban biosecurity toolkit (Will Allen, Andrea Grant, Lynsey Earl 2017).


Rubrics: a history of use in the education sector


Rubrics: sharing the rules of the game
This useful 2016 paper by David Balch and Robert Blanck summarise the evolutionary uses of rubrics within the educational environment. They describe holistic, analytic (and developmental), and single-point rubrics. They provide a checklist for measuring the qualities of good rubric. They also review the incorporation over time of different goals, objectives and learning outcomes of rubrics in this setting.


What Are Rubrics and Why Are They Important?
This 2013 chapter by Susan Brookhart describes the different types of rubrics – holistic, analytic, general, or task specific. Rubrics are defined in terms of their two main components: criteria and descriptions of levels of performance. The main point about criteria is that they should be about learning outcomes, not aspects of the task itself. The main point about descriptions of levels of performance is that they should be descriptions, not evaluative statements. The “evaluation” aspect of assessment is accomplished by matching student work with the description, not by making immediate judgments.


Teaching With Rubrics: The Good, the Bad, and the Ugly
This 2010 paper by Heidi Andrade gives a brief overview of the structure and purposes of rubrics; reviews the benefits of using rubrics as both teaching and grading tools; warns against approaches that limit the effectiveness of rubrics; and urges instructors to take simple steps toward ensuring the validity, reliability, and fairness of their rubrics. Tips for using rubrics with undergraduate and graduate students are also included.


Rubrics: Tools for making learning goals and evaluation criteria explicit for both teachers and learners
This 2006 paper by Deborah Allen and Kimberly Tanner highlights the benefit of rubrics as teaching and professional development tools. Used wisely, rubrics not only make the instructor’s standards and resulting grading explicit, but they can give students a clear sense of what the expectations are for a high level of performance on a given assignment, and how they can be met. This use of rubrics can be most important when those undertaking the activity to be assessed are novices with respect to a particular task or type of expression. Finally, they remind us that by their very nature, rubrics encourage reflective practice on the part of both students and teachers.


Using rubrics to promote thinking and learning
This 2000 paper by Heidi Andrade highlights that instructional rubrics help teachers teach as well as evaluate student work. She reminds us that  creating rubrics with your students can be powerfully instructive. Rubrics make assessing student work quick and efficient, and they help teachers justify to parents and others the grades that they assign to students. At their very best, rubrics are also teaching tools that support student learning and the development of sophisticated thinking skills. When used correctly, they serve the purposes of learning as well as of evaluation and accountability. Like portfolios, exhibitions, and other authentic approaches to assessment, rubrics blur the distinction between instruction and assessment.


What’s wrong – and what’s right – with rubrics
In this 1997 paper James Popham emphasises that rubrics have the potential to make enormous contributions to instruction quality – but first we have to correct the flaws that make many rubrics almost worthless. He notes that we have to abandon both overly task-specific and excessively general rubrics, and strive to end up with rubrics that actually and practically help instruction.


Share

An independent systems scientist, action research practitioner and evaluator, with 30 years of experience in sustainable development and natural resource management. He is particularly interested in the development of planning, monitoring and evaluation tools that are outcome focused, and contribute towards efforts that foster social learning, sustainable development and adaptive management.