Course evaluation
| Site: | learnonline |
| Course: | Teaching and learning in Health Sciences |
| Book: | Course evaluation |
| Printed by: | Guest user |
| Date: | Saturday, 23 November 2024, 9:58 AM |
Evaluating a course
Evaluation needs to be a formative process that influences student learning, not a merely an institutional process done at the end of a course 'to' both academics and students.
Evaluation is not only about students, but needs to consider all stakeholders.
Two big issues with the UniSA formal evaluation tool (MyCourseExperience) are the low response rates, and the problem of how to influence students' perception of the course and the teaching to obtain appropriate recognition.
Extensive 'help' resources such as how to set open and close dates access to the list of additional questions can be found here
.
Action research and continuous evaluation
This type of evaluation involves assessing a course as it is actually running, and reflecting continually on their own actions, with the aim of improving 'on the go', rather than waiting for the next iteration of the course.
John Hattie argued that for improvements to happen in the classroom, it is critical that teachers have the right mind frame to become effective evaluators of their own practice; in fact, effective continual evaluation is one of the few elements in a classroom which does lead to noticeable increase in student learning.
"Effective teachers change what is happening when learning is not occurring... An effective teacher makes calculated interventions, and provides students with multiple opportunities and alternatives to learn, at both surface and deep levels." (Hattie, J 2011, Visible learning for teachers.)
Ellie Chambers & Marshall Gregory have written on the process of 'action research' to evaluate courses and teaching, in their book Evaluating Teaching: Future Trends, in the book Teaching & Learning English Literature. They talk about the ways in which teachers can satisfy themselves about their courses and performance, quite separately from the formal evaluation processes the institution has put in place.
Action research means that teachers investigate and reflect on courses of action to improve various elements of their course and compare the aims and intentions to what actually happened on the ground, by continual monitoring of the student engagement and progress, by peer and student review, and by careful consideration of conflicting feedback.
One view on evaluation"I think that the best approach is to come at evaluation 'sideways'... we use the same approach when we interview young children; you can never ask them directly about the subject or they tell you what they think you want to hear. Rather you ask about something that they relate to that takes you to the subject. If you ask direct questions you get skewed results based on what it is they think you want... it's about power. I think the best way is to ask students about their own learning, and their own perception of their progress. So questions like "How do you think you are going? How have you been challenged? Do you think your understanding of the content is getting better?" offer opportunities for students to signal problems early and you a chance to address them early. David Birbeck |
.
Strategies to help with course evaluation
The following concertina presentation lists some ideas and tips which may help you with evaluation. Just click on the three headings.
Useful links, readings and references
This is a presentation prepared by David Birbeck for a workshop on evaluation.
Course evaluations (1.6 MB)
If you want to publish your evaluation findings these are the two documents you need to read first.
Zabaleta, F. (2007). The use and misuse of student evaluations of teaching. Teaching in Higher education, 12(1), 55–76.
Macdonald, R. (2006). The use of evaluation to improve practice in learning and teaching. Innovations in Education and Teaching International, 43(1), 3–13.
Baldwin, C., Chandran, L., & Gusic, M. (2011). Guidelines for Evaluating the Educational Performance of Medical School Faculty: Priming a National Conversation Teaching and learning in medicine, 23(3), 285–97. doi:10.1080/10401334.2011.586936
The maths behind your numbers
The maths behind the MyCourseExperience numbers works as below:
| Strongly disagree | -100 |
| Disagree | -50 |
| Neutral | 0 |
| Agree | +50 |
| Strongly agree | +100 |
The score you receive is simply the total of the scores divided by the number of people who respond.
One unfortunate thing to recognise is that a 'neutral' result is actually acting as a 'negative', because it adds nothing to your numerator, but increases the denominator by '1' .
If you go to the 'Cube' to look up your data, it's important to remember that if you only refer to one year, you will see a simple 'mean', but if you access across multiple years, a weighted mean is used.
If you are not a maths person (like me) an explanation of what that means (no pun intended) is below.
Download an explanation of the figures in CEI cubes and reports (267 KB)
Evaluation resources from David
Self-assessment / reflection form
I use a reflective task, called “How have you been challenged this week?” I hand out the self-assessment form (downloadable below) and then give the students 15 minutes to fill it in. They then hand it up.
It takes me about 20 minutes to read the completed forms ( for about 30 students). I then feed that back to the group during the next week's session. Note that this will often mean a slightly modified session, because I build in small adjustments based on this feedback.
Most importantly, I make sure that I tell the students, “Thank you; based on your comments, we want to do this”.
Sometimes it takes students a week or two to get into the rhythm of this, but by the end I am getting useful responses.
Critically, if means that if any student has a serious problem, he/she has had 13 opportunities to mention it!
Download the self-assessment/reflection form (24 KB)
Note that it doesn’t matter so much what tool you use for this type of evaluation; what matters is that you make it about the student, and offer the opportunity for them to communicate with you.
Workshop: Dealing with student feedback
In early 2017, we ran a staff workshop on Dealing with student feedback, which many academics found very helpful.
You can see details of the workshop in the 'notes' section of the interactive chart below.
Important note: If you are, or you know of, a staff member who is very worried and needs to talk over things with a professional in counselling or psychology, there is support for you with the Employee Assistance Program. |
Background to the workshop
Mycourseexperience is a tool that is intended to enable students to have a 'voice' in order to express their view about their personal experience of learning at UniSA. It is an important tool and in the end its about making the teaching and the learning better.
There are questions that relate specifically to the course, and other questions that relate specifically to the 'teacher' of the course. Of course, in reality, we know that while there is a difference between 'the course' and 'the teaching of the course', in practice it is difficult to separate these aspects in any meaningful way.
It is not a new phenomena for students to express their views using this tool in inappropriate ways and using inappropriate language. I personally am reminded of a Christmas eve (please don't ask why I was here) where I came across a colleague who was also working and had opened their evaluations. An intended short stay at University turned into a two hours of listening and talking with this colleague. The effect of the students' comments on this person was disturbing, and the effect on me profound.
While it may not be new, there is a perception among many that it seems to be increasing in frequency and the effect on some teaching staff is significant. We all have to be able to accept reasonable criticism of our work and our teaching but there is an appropriate way to make these comments. Some people are arguably less affected by personal comments than other people; but irrespective of the affect, it is unprofessional behaviour for students to make personal comments.
The purpose of this workshop was to explore the issue as sensitively and as sensibly as possible, and to ensure people understand that help exists and you are not on your own.
Download a presentation on Course Evaluation (1.6 MB)
After the workshop, participants were not asked to evaluate the workshop, but rather, were asked to summarise their experiences, and their ideas about a way forward. The results of this are in the chart below, for you to explore and use as you see fit.
.
Interactive chart on dealing with student feedback
You can manipulate the chart below by clicking on the various headings.
Alternatively, if you are working with a small screen, you can open the chart in a new window.