Assessing students

.Ask us.	.Tell us .

Writing multiple choice questions

Multiple choice questions can be an effective and efficient way to assess your students' learning, especially at the lower cognitive levels of recall and application, and even analysis and evaluation. They can be consistent and reliable methods of assessment for certain learning outcomes, and are relatively easy and fast to mark, more so if they are online.

However, if multiple choice questions are to be effective, they need to be carefully considered and planned. Writing multiple choice questions is not an easy task. There are several principles which you need to consider before even deciding on this form of assessment.

.

The principles

1.   Are Multiple Choice questions a suitable assessment for the course learning outcomes?

Firstly, you need to ensure that the questions you intend to ask provide rigorous evidence related to the achievement of the learning outcomes. There are limits to what can be tested using multiple choice questions.

For example, choosing from a limited set of potential answers is not an effective way to test skills of organisation, reflection, communicative abilities or creativity. 

However, you can use MC questions to get students to interpret facts, evaluate situations, explain cause and effect, make inferences, and predict results. A MC problem that requires application of course principles, analysis, or evaluation of alternatives can be designed to tests students’ ability in higher level thinking. Furthermore, MC questions can be designed to incorporate multi-logical thinking, applying more than one facet of knowledge to solve the problem. 

Start by thinking about competence. What would a student need to demonstrate or to convince you that they were competent in relation to the course objective that relates to the MCQ test / exam? 

If your concept of competence cannot be met by a MCQ test / exam, you may need a different form of assessment.

.

2.   Are my MC questions effectively testing the learning?

To produce questions which are effective in assessment the students' understanding, you need to spend some time working on both the stem (the initial problem) and the alternatives (the list of suggested solutions).

In the alternatives, you will have a right answer and a number of distractors (wrong answers).  It is critical that while there is only one right answer, the distractors are plausible options, but are also unambiguously wrong. In fact, if possible, the distractors should be designed to increase the learning for those students who get them wrong.

For example, when designing the distractors for a formative  MC activity, consider the common errors that students make in the type of content you are testing, and design the distractors to follow those errors. Then you can incorporate feedback which corrects the students and shows them how to avoid these common errors. Which brings us to the final principle ... 

.

.3.  How can I incorporate useful feedback?

Properly designed and crafted feedback for online formative MC questions can be one of the more powerful tools in the online toolbox. If the distractors are carefully composed, and the feedback is contextualised for each distractor, you can produce a resource which is along the line of a tutorial, and has as much value.

The trick is to use the distractors to present solutions which may incorporate common errors, and then compose contextualised feedback which is quite specific to that error, shows students how to avoid it, and if possible, has links to further resources covering that content.

The feedback in online MC questions is an invaluable teaching tool, because the students are particularly receptive to this information when they are trying to improve assessment results, and eager to see where they have gone wrong in a testing activity. 

.

Other considerations

In order to optimise the reliability of the MC test, you can focus a number of MC questions on a single learning objective.

Frame questions in a real-life context if possible, so that students are recalling and applying principles, rather than just recalling. 

.

Technical dos and don'ts

There are a number of hints and tips for improving your multiple choice questions, which are given in the following online presentation.

Stream the presentation (about 9 minutes)

Download the presentation for playing offline (7.9 MB; unzip the zip folder, extract all files and play the index.html file)

You can also download a pdf version of these hints and tips (504 KB)

.

Interesting resources and links

Toledo, CA 2006, 'Does your dog bite?: Creating good questions for online discussions', International Journal of teaching and Learning in Higher Education, vol. 18, no. 2, pp.150–154.

Vanderbilt University Writing good multiple choice questions.

Morrison, S & Free, K 2001, 'Writing multiple-choice test items that promote and measure critical thinking', Journal of Nursing Education, vol. 40, pp. 17-24.

Collins, J 2006, Writing Multiple Choice Questions for Continuing Medical Education Activities and Self-Assessment Modules.