Assessing students
| .Ask us. | .Tell us . |
 |
 |
Why are rubrics so important?
Rubrics are marking tools which are developed hand-in-hand with assessment tasks; they reflect the intention and purpose of the assessment, and, through the assessment, the learning outcomes of the course. They aim to separate out the standards of student work, and communicate those standards to students and other staff - that is, they bring clarity and transparency to the assessment.
In teaching and learning, rubrics are far more important and powerful tools than generally assumed.
It could be argued that the development of the rubric is more important than the development of the detailed assessment task, because the rubric is critical in ascertaining whether or not the students have achieved the learning outcomes of the course. At its worst, a poor rubric can result in students passing the assessment, and even the course, even though they haven't achieved the outcome that you are looking for.
Rubrics communicate to students (and to other markers) your expectations in the assessment, and what you consider important. The rubric tells students much about the assessment, so you need to make sure that it's telling them what you want them to hear. You should be able to give a well-written rubric to students INSTEAD of the assessment, and they should be able to identify what the assessment is and what is important - in fact, in some cases, it may be possible for the students to negotiate the assessment task with you, based on the rubric.
You can also use good rubrics as teaching resources, to give students a clear indication of any critical gaps or weaknesses in their performance, what they need to do to improve, and where to focus their efforts (feed-forward rather than feedback).
Good rubrics can also go a long way in eliminating awkward conversations on questions such as, 'Why did I get this mark', 'This mark is not fair', etc.
In order to create a really powerful rubric, there are some important steps to consider, which we have outlined below.
However, one other point before you start - try not to start out by using a template. There are literally hundreds of them out there - some good, some terrible - but it is important that your rubric is absolutely aligned to your course outcomes and your assessment outcomes, and using a provided template can skew the process. You need to consider the unique aspects of your assessment and outcomes independently of pre-formatted templates. If you definitely want to use a template, first work through the rubric development process, and then find a template which fits your requirements.
.
The steps ...
1. Stand back. The first step must ALWAYS be to take a good, hard, holistic look at the very foundation of your assessment task - what is the purpose of this assessment?
Always, always, always start by considering the Big Idea of the assessment. What is the one main essential purpose of this assessment - what it is that students MUST do to pass? What learning objectives do you intend the assessment to further or to demonstrate?
Ideally, there should only be one Big Idea for every assessment (which might focus on more than one course objective). Take all the time that you need to think about this, because if it is not reflected as the basis of your rubric, the rubric is not only not effective as a marking tool, but could actually be causing harm.
.
2. Next ... where does your assessment fit with the program and with the course?
Your assessment must fit with the program objectives and with the course objectives. If it doesn't, you may need to change the assessment, or even consider changing the course objectives. It may be that the Big Idea for your assessment is actually a re-statement of one of the course objectives ... nothing wrong with that. You need to consider the program dimension to make sure that the assessment and rubric reflect the appropriate stage in the student's progress.
As the assessment aligns to the program and course, so too should the rubric. If the assessment is aimed at demonstrating achievement of higher order, more complex learning outcomes such as critical thinking and analysis, the rubric must be able to quantify that.
.
3. The big question ... holistic or analytical?
A holistic rubric judges an assessment product as a whole, without judging the various component of the assessment separately. An analytic rubric judges separate elements, components or aspects of the assessment product..
There are strengths and weaknesses with both types of rubric, depending on your situation. For example, a holistic rubric is more valuable in judging whether or not students have achieved the Big Idea - but requires extensive discussion to ensure that multiple markers are all on the same page.
Holistic rubrics are often useful at the more complex levels of learning, especially in creative or hard-to-quantify disciplines. They are also useful for judging achievement in the affective domain - for example, valuing, responding, or being receptive to ideas or material.
Our advice is that if the assessment 'whole' is bigger than the parts, then use a holistic rubric (an essay is a good example of this). If each of the components or aspects are all important, then use an analytical rubric.
However, in developing an analytical rubric, be careful not to over-simplify or over-segregate the assessment elements or components, because this can lead to trivialising and loss of the Big Idea. A fine example of this is dividing up an essay or report task based on the various sections of the document - introduction, body, conclusion, language use, referencing, etc. You may end up with a rubric which comfortably passes students who have no real idea of the Big Idea, and have not actually solved the problem posed in the assessment.
4. The line in the sand ... what is a pass?
The most critical element of the rubric, and the one on which you should spend the most thought, is what we call the Minimum Acceptable Level of Performance. The whole purpose of this is to accurately separate out competent students from incompetent students – the passes from the fails. The rubric MUST be set up so that no student can pass without achieving this minimum level in the critical assessment outcomes, especially higher order and hidden outcomes.
To find this, think about what is the minimum point at which you would be happy with a student's performance in the Big Idea, taking into considering the complexity of the task, the student level and experience, and the outcomes which you are assessing.
Have a very clear picture in your head of what the Minimum Acceptable Level of Performance looks like, for students at this level, in this type of activity, according to these course objectives. This minimum level of performance will become your 'P1 pass point': the defining point for the student to pass the assessment. Receiving a P1 for an assessment should indicate to both student and teaching staff that the student has achieved an acceptable professional level of performance in these assessment outcomes relative to their time in the program. Higher levels of performance will be encouraged and rewarded, but are not essential for professional effectiveness.
You can see an animation of different Minimum Acceptable Levels of Performance (MALOPs) below - just move the arrow on the slider backwards and forwards.
.
5. Good, better, best ... bad. How do you establish other levels of performance?
Using the minimum level of performance as a pivot, decide on the other levels of performance, especially the levels of pass. You can do this by establishing in your mind what you think would classify as an outstanding student response to the task, completely fulfilling all of the assessment outcomes – this becomes your HD. You can then work up from a pass to a HD, deciding on your levels of performance.
Forming a picture of a failure is much harder, because it's virtually impossible to anticipate all the possible ways in which students might not achieve the assessment outcome. You may need to keep this section of the rubric quite general, but make up for this by providing very specific, detailed feedback to the failing students. Some teachers populate the fail sections of the rubric by outlining the 'typical critical errors' which commonly prevent students from achieving the Big Idea.
If you have chosen to develop an analytical rubric, you can now draw out performance criteria which encompass the various assessed aspects of the task. Only create as many as you need to accurately assess the student. You should give each performance criteria a weighting which will the assessment outcomes – again, adjust the weightings to make sure that a student can’t pass without having achieved the critical outcome (even if they did brilliantly in all the other outcomes!).
Finally, before you present the rubric to your students and staff, give it the acid test ... go back and test it to see whether it is possible for a student to pass without achieving the Big Idea.
.
6. Getting everyone on board. Discuss the rubric with students and staff.
We can't emphasise enough how important it is to discuss the rubric in detail with students and with other marking staff, to the point that everyone is absolutely on the same page in their understanding of the language it uses, the Big Ideas it is judging, and the guidance it gives. Remember that different words and terms can mean totally different things to different people. You might understand exactly what the rubric means to you, but others may have a quite different picture in their heads.
Take the time to compare and contrast the various levels of achievement and create a shared understanding. This may take quite a lot of time in discussion and negotiation, but if everybody is crystal clear on what the various levels of performance are, it will pay off in the long run.
.
Rubric scales
The scale will define the various levels of achievement above and below the minimum acceptable level. The most common scale for tertiary assessments is based on grades (distinction through to F2). However, other scales might be useful if you are setting a formative activity, or if your assessment is not suited to the full range of grades. Some options are:
- pass/fail
- qualitative descriptions (Excellent, Very good, Good, Satisfactory, Poor, Very poor)
- marks (either discrete or accumulative)
- percentages
Getting students to participate in writing rubrics and marking using rubrics
At the very least, your assessment rubric should be handed out to students prior to the assessment task, to make them aware of all your expectations related to the assessment task, and to help them to evaluate their own progression and readiness.
However, you can take your use of rubrics to another level by involving the students in their creation. This is a very powerful way to help them to build their understanding of what what different levels of understanding actually look like, and what differences there are between powerful and weak work.
Discussions of which performance criteria they think are important, and what the different performance levels actually mean increases the students' engagement in the assessment process.
Another effective strategy is to get the students to use your assessment rubric prior to the formal assessment, as a self-assessment exercise, or to get students to mark each other using the rubric. In this way, they become more cognisant of assessment processes and procedures, and improve their capacity to assess their own work.
Interesting links and resources
Biggs, J & Tang, C 2007, Teaching for Quality Learning at University, 3rd edn, McGraw Hill, Maidenhead.
David Birbeck's PowerPoint presentation on designing rubrics