Print bookPrint book

Peer Assessment using Moodle Workshop Update

Information and instruction for using the Workshop module.

Site: learnonline
Course: learnonline Help (Staff)
Book: Peer Assessment using Moodle Workshop Update
Printed by: Guest user
Date: Wednesday, 22 November 2017, 5:47 PM

Overview

Moodle’s Workshop module is a peer assessment activity with numerous options. Students receive two marks, one for their own work and another for their peer assessments of other students’ work. Both these marks appear in the grader report. 

Note: At UniSA, this module can only be added as a formative assessment or as an activity. Therefore the assessment method menu in the grader report will need to be set to Activity. select Activity as the assessment method

Workshop allows students to submit their work via an online text tool and/or by uploading files. 

Some of the basics of using the Workshop module:

  • Every student submits their own work. The Workshop module does not support team work (in the sense of one submission per group of students).
  • The submissions are assessed using a structured assessment form defined by the Course Coordinator. Workshop supports several types of assessment forms.
  • Workshop supports peer assessment. Students may be asked to assess a selected set of their peers’ submissions. The module coordinates the collection and distribution of these assessments.
  • To help students understand the assessment form (and the process of peer assessment in general), instructors can provide example submissions and a reference assessment. Students can assess these examples and compare their own assessment with the reference provided by the instructor.

n

Workshop in Moodle 2.0 video

Adding a Workshop

There are numerous settings to complete when adding a workshop activity. They are:

  • General settings
  • Grading settings
  • Submission settings
  • Assessment settings
  • Feedback and conclusion settings
  • Example settings
  • Availability settings
  • Common module, Restricted access and Activity completion settings

For ease of reading, they have been separated into individual pages of this book.

Adding the Workshop activity and general settings

Adding a workshop

  1. Turn editing on
  2. Select Add an activity or resource
  3. Select Workshop from the list of activities
  4. Click on Add

The Add a new Workshop page is displayed.

General settings

General settings

  1. Enter the workshop name.
  2. Add a description (optional).
  3. Decide if you wish to display the description on the front page of your course site.

Grading settings

Grading for the workshop activity covers both the students' own submission and their assessment of other students' submissions. The maximum marks for each of these components can be defined by the Course Coordinator.

Grading settings

  1. Select the grading strategy you wish to use. The options are:
    • Accumulative grading: Comments and a mark are given regarding the aspects of the workshop specified.
    • Comments: Comments are given but no mark can be given to the specified aspects
    • Number of Errors: A yes/no assessment is used and comments are given for specified assertions
    • Rubric: A level assessment is given regarding specified criteria
      NOTE: Assign a mark for each 'Level grade and definition'.  The mark for the first 'Level grade and definition' must be zero for the rubric to grade as expected.
      You can also refer to Rubric instructions: https://lo.unisa.edu.au/mod/book/view.php?id=783159&chapterid=121986    
  2. Enter the maximum mark attainable for submitted work.  Select the assignment activity from the dropdown alongside
  3. Enter the grade the student must gain for their submission to pass.
  4. Enter the maximum mark attainable for assessment of other submissions.  Select the assignment activity from the dropdown alongside.
  5. Enter the grade the student must gain for their feedback to pass.
  6. Set the number of decimal places allowed in the marks.

Note: For more detailed description, visit Workshop grading strategies - Moodle docs (opens in new window) 

Submission settings

Submission settings

  1. Enter submission instructions
  2. Set the maximum number of attachments allowed for submissions.
    Note: Setting this to zero will only allow students to enter text only and not upload any files.
  3. Select allowed file types, separated by commas, for the submission files.
  4. Set the maximum file size. This can be left at the default setting.
  5. Select to allow late submissions. 

Assessment settings

Assessment settings

  1. Enter instructions for the students to use during their assessment of other students submissions.
  2. Decide if students will be allowed to assess their own submission.
    Note: Students are not guaranteed to be assigned their own submissions.

Feedback and conclusion settings

Feedback settings

  1. Decide if overall feedback will be included in the assessment of submissions. The options are; Disabled, Enabled and optional or Enabled and required. If enabled, a text field is displayed at the end of the assessment form.
  2. Set the maximum number of overall feedback attachments.
  3. Enter a comma separated list of allowed feedback file types.
  4. Set the maximum file size for overall feedback attachments. 
  5. Enter the conclusion text that will be displayed at the end of the workshop activity.

Example submissions settings

If enabled, students can practice assessing one or more example submissions and compare their assessment with a reference assessment. This will allow them to refine their assessment techniques before applying them. The mark they receive for the example submission is not counted in their mark for assessment.

Example submission settings

  1. Select the check box if you wish to use example submissions
  2. If using example submissions, decide whether:
  • Assessment of example submission is voluntary
  • Examples must be assessed before submission
  • Examples are available after submission and must be assessed before peer assessment.

Note: if using an example submission, you will need to prepare it during the setup phase

Availability settings 

This section allows you to set the timelines for the submission and assessment phases of the workshop activity. You can also decide if you wish the workshop activity to automatically change from the submission phase to the assessment phase.

Availability settings

  1. Set the opening date and time for the submission phase.
  2. Tick the Enable check box and then set the deadline for submissions.
    Note: this setting is optional. If not set, the activity will remain open and be ongoing.
  3. Tick the check box if you wish the workshop to switch to the assessment phase automatically. If the submissions deadline is specified and this box is checked, the workshop will automatically switch to the assessment phase after the submissions deadline.
    Note: If you enable this feature, it is recommended to set up the scheduled allocation method as well. If the submissions are not allocated, no assessment can be done even if the workshop itself is in the assessment phase.
  4. Tick the Enable check box and then set the open date and time for the assessment phase.
  5. Tick the Enable check box and then set the deadline for the assessment phase.

Common module, Restricted access, Activity completion settings and Tags

These are standard across all Moodle activities and resources. Help for these settings can be accessed via the following links.

Workshop common settings

  1. Change the common module settings if required.
  2. Select if you wish to restrict access to the Assignment.
  3. Select if you wish to track activity completion of the Assignment.
  4. Select if you wish to add tags to the Assignment. 
  5. Select Save and Display to show the Workshop activity.

Whilst the activity has now been added to your site, it will not be ready for your students to use. Please see the remainder of this help resource for instruction on how to complete the setup of the Workshop and how to progress the activity through to completion.

Phases of the Workshop 

The work flow for the Workshop module can be viewed as having five phases. The typical workshop activity can cover days or even weeks. The Course Coordinator simply clicks on the light bulb icon for each phase to switch the activity from one phase to another.

Switching between phases

The typical workshop follows a straight path from Setup to Submission, then Assessment  and Grading/Evaluation, ending with the Closed phase. 

The progress of the activity is visualised in the Workshop planner tool. It displays all Workshop phases and highlights the current one. It also lists all the tasks the user has in the current phase with the information of whether the task is finished or not yet finished or even failed.

The phases are:

  1. Setup
  2. Submission
  3. Assessment
  4. Grading
  5. Closing

Workshop planner tool

(The phase headings are highlighted)

Workshop planner tool

Setup phase

During this phase you can change workshop settings, modify the grading strategy, create/edit assessment forms or prepare example submissions. It is possible to switch to this phase any time you need to change the Workshop setting and prevent users from modifying their work.

When the workshop is in the setup phase, students can only see the description and submission information. They can't submit. Notification is displayed informing the student that the workshop is being set up.

Instructions

The criteria for an assignment are added via an assessment form during the setup phase. Students can view this assessment form in the submission phase and focus on what is important about the task when working on their assignment. In the assessment phase, students will assess their peers’ work based on this assessment form.

Initial view of setup phase in planner

The Setup phase column in the workshop planner indicates the status of the various components required in this phase. In this example, the following are still to be completed:

Edit assessment form

Select edit assessment form

  • Select Edit assessment form link to display the assessment form page. The components on the settings page will vary depending on the grading strategy selected when adding the Workshop.

The instructions below are for the Accumulative grading strategy. The other forms will follow a similar layout.

Assessment form settings

  1. Enter the aspect (criterion) details in the description field.
  2. Enter the maximum mark for the aspect.
  3. Select the weighting.
    Note: Leave the weighting set to 1 for all aspects if they are of equal importance.
  4. Click on Aspect 2 or Aspect 3 to expand and enter details.
  5. Select Save and close.

The Workshop planner page will be displayed.

Add and assess an example submission

Before you can add an example submission, you need to have:

Add example submission

  • On the Workshop planner page, select the Add example submission.

The submissions page will be displayed.

example submission

  1. Enter the title.
  2. Add the submission content.
  3. Upload a file (optional).
  4. Click Save changes.

The page will refresh to display the example with the option to assess to provide a reference assessment.

Assessment of example submission

You can either:

  1. Select Continue to assess the example submission.
  2. Select Cancel to return to the Workshop planner. You will be able to return and complete the assessment later.
  3. Edit the Example.
  4. Delete the example.
  5. Assess the example submission.

n

Assessing the example submission

The assessment screen will display the submission, the Instructions and the Assessment form with the marking criteria.

The Assessment form

Assessment form

  1. Select the mark for the criterion being assessed in aspect 1.
  2. Enter a comment for the criterion being assessed in aspect 1.
    Note: Repeat steps 1 and 2 for all aspects.
  3. Enter any overall feedback (optional).
  4. Click on Save and close.

Submission phase

In the submission phase, students submit their work.

Dates to manage access can be set so that even if the Workshop is in this phase, submitting is restricted to the given time frame only. Submission start date (and time), submission end date (and time) or both can be specified.

explanation of the submission phase in the workshop planner
Information displayed in the submission phase column:
A. The status of Assessment instruction.
B. The status of submission allocations. See Submission allocation for instructions on the various options.
C. The status of submissions.
D. The submission period dates.

Allocating submissions for assessment

Allocation selection

  • Select either Set up scheduled allocation or Allocate submissions to access the allocations page.

allocation options

There are three options for allocating students as reviewers and reviewees. They are:

  • Manual allocation
  • Random allocation
  • Scheduled allocation

Manual Allocation

assigning reviewers and reviewees

For each student:

    1. Select who will review the particular student's submission.
    2. Select whose submission this particular student will review.

The screen will refresh to show the additions and a success message is displayed.

To remove assigned students

removed assigned student

      1. Click on the X next to the student you wish to remove. The screen will refresh to display a confirmation message.
      2. Click Yes, I am sure to complete the removal.

n

Random allocation

Random allocation settings

      1. Select the Random allocation tab.
      2. Select the number of reviews and whether this is per submission or per reviewer, i.e. each reviewer assesses 5 submissions OR each submission has 5 reviewers.
      3. If you already have some students allocated, tick this check box to remove them.
      4. Decide if students can assess submissions without first having submitted their work.
      5. Click Save changes.

Scheduled allocation

The Scheduled allocation method will automatically allocate submissions for the assessment at the end of the submission phase as defined in the workshop setting Availability settings - submissions deadline. Note: Manually switching to the assessment phase will not trigger this process, only the deadline will. If you wish to move to the assessment phase earlier, you will need to use the Random allocation method.

scheduled allocation

      1. Select the Scheduled allocation tab.
      2. Tick the Enable scheduled allocation check box.n
        Current Status -  Note: This section is collapsed by default.
      3. Displays the current scheduled allocation status. n
        Allocation settings - Note: This section is collapsed by default. 
      4. Select the number of reviews and whether this is per submission or per reviewer, i.e. each reviewer assesses 5 submissions OR each submission has 5 reviewers.
      5. If you already have some students allocated, tick this check box to remove them.
      6. Decide if students can assess submissions without first having submitted their work.
      7. Click Save changes.

The screen will refresh and display a confirmation message.

The Current status section will now display the date and time of the allocation execution.

scheduled allocation information

 Note: This section is collapsed by default.

Assessment phase

During this phase students assess submissions allocated to them.

As in the submission phase, access can be controlled by specified date and time. See Availability settings for instructions. 

The assessment phase column
Information displayed in the Assessment phase column.
  1. The date/time that the assessment phase commences.
  2. The date/time that the assessment phase closes and the time remaining.
  3. Whether any time restriction applies.
n

The grades report

Below the planner tool, you will see the Workshop grades report.
Workshop grades report
n
This report displays the following for each student:
  1. Student name.
  2. The students submission.
  3. The mark the student has received.
  4. The mark the student has awarded.
Note: Surnames have been removed from the image above.
n
Columns C and D also allow you to track how students are progressing with the assessment phase.
  1. Initially student names will be displayed in red.
  2. Once a submission has been assessed, the student names corresponding to the submission and the assessment will change to black, and the mark will be displayed.

Grading evaluation phase

The major task during this phase is to evaluate the assessments completed by the students, calculate the final marks for submissions and assessments and provide feedback for authors and reviewers. It is not for you to mark the submissions. Once in this phase, students cannot modify their submissions or their assessments. Course Coordinators can:

  • Manually override the calculated marks.
  • Mark individual submissions (optional).
  • Set selected submissions, e.g. high quality submissions, as "published" so they become available to all Workshop participants in the closed phase.  

Grading phase in the planner tool

Information displayed in the Grading phase column:

  1. Status of submission grade calculations.
  2. Status of assessment grade calculations.
  3. The status of the conclusion information.

Calculate the marks

Grading method and settings

  1. The Grading evaluation method. Note: Currently, the Workshop activity only has the default grading method.
  2. Select how strict the comparison of assessments should be. The options range from very lax to very strict.
  3. Calculate the grade. Re-calculate grades will need to be clicked to generate the initial grades as well.

The screen will refresh and calculated grades will be added to the Workshop grades report table.

Grades table showing calculated grades

Evaluating the submissions and assessments

You can view the submission and peer reviews (and override the final mark), add your assessment of the submission and review/override each peer assessment of the submission. To do so:

Reveiw of submission and assessment

  • click on the submission title

n

The following options are then available:

Viewing the submission and peer assessments 
Assessing the Submission 
Evaluating the Peer Review

n

Viewing the submission and peer assessments 

Evaluation of submission and assessment

A. The student's submission.
B. The Assess button will allow you to assess the submissions. 
C. The Assessment of the submission by peers. 
D. The Assessment settings button will allow you to evaluate the assessment and override the marks awarded.

 

At the end of this page you will also find the feedback section for the author of the submission. 

Submission feedback and publish option

  1. Select the checkbox if you wish to publish this submission. Publishing a submission will make it available to all students as an example paper after the Workshop activity has been closed.
  2. The existing calculated grade for the submission is displayed.
  3. To override the mark awarded for the submission, select the new mark from the drop-down menu.
  4. Enter feedback for the author of the submission (optional).
  5. Select Save and close.

n

Assessing the Submission

Access the submission assessment form.  See point B in second image on this page.

The submission assessment form

A. The submission.
B. Instructions for assessment.
C. The assessment form.

  1. Select the mark 
  2. Enter feedback. Note: Repeat 1 and 2 for all Aspects being assessed.
  3. Give overall feedback (optional).
  4. Select the weighting to apply to your evaluation. Leave as 1 to have your mark calculated with the same influence as others assessing the submission. Increasing the weighting will increase the impact your grade has on the overall submission grade for the student.
  5. Select Save and close.
    n

Evaluating the Peer Review

Access the peer review by clicking on the Assessment settings button. See point D in second image on this page.

The screen will refresh and display:

  • The submission
  • The submission instructions
  • The assessment (mark and comment) for each aspect
  • Assessment Settings (explained below)

Evaluating the assessment component

  1. Select the weighting to apply to this assessment of the submission.
  2. The existing calculated mark is displayed.
  3. If wanting to change the mark awarded by the student, select the new mark from the drop-down menu. 
  4. Enter feedback for the reviewer (the student who assessed the submission).
  5. Select Save and close.

Closing phase

Switching the Workshop into this phase will result in the final calculated marks appearing in Gradebook. Students will also be able to view their marks, their own submission (with feedback from the peer review), their assessments of other students submissions and any submissions you set to publish.

FAQs

Q. How the are these marks calculated?

A. Make sure you read carefully the pages explaining workshop grading and grading strategies (these links go to the external moodle documents site).

x

Q. How can I get the workshop to switch phase automatically?

A. In Administration > Workshop Administration > Edit settings tick the box 'Switch to the next phase after the submissions deadline' in the access control section. The workshop will then automatically switch to the assessment phase. See Availability settings for details.

x

Q. How can I make selected submissions available to all students?

A. When in the Grading evaluation phase, it is possible to publish (i.e. make visible to all participants) selected submissions you might consider are helpful examples. To do this, you will need to tick the Publish submission check box in the Feedback for the Author section when viewing the submission. See Assessing the submission- feedback for the author for instructions.

x

Q. Why can't the students assess submissions?

A. For students to assess submissions the workshop needs to be in the assessment phase, an assessment form must have been created and submissions must have been allocated for assessment.

x

Q. Can I assess a student's submission as well as the already allocated students?

A. Yes. You can do this during the grading evaluation phase. See Assessing the submission for instructions.

x

Q. Can I change a mark awarded by one of the students for a submission?

A. Yes. See Evaluating the Peer Review for instructions.

x

Q. Can I change the total grade for a submission or give overall feedback on the submission?

A.  Yes, See Assessing the submission- feedback for the author for instructions.

Useful resources

Below are links to YouTube videos discussing the Workshop module. 

Note:  These videos are not from the University of South Australia and any screenshots will not be representative of our moodle installation. However, the concepts and information are correct. 

Using the 'Workshop' module for self and peer assessment in Moodle

Using the 'Workshop' module for self and peer assessment in Moodle
by iTeachWithMoodle (time = 18:38)

This is a re-run of the presentation by Frederic Nevers at the MoodleMoot Hong Kong 2012.

Introduction of Moodle's Workshop Module

Introduction of Moodle's Workshop Module
by Ellen Marie Murphy (time = 7:15)

This video explores the Workshop Activity Module in Moodle 2. It demonstrates the various phases in the workshop from both the student and instructor perspectives.

Setting up a Moodle Workshop

Setting up a Moodle Workshop
by Ellen Marie Murphy (time = 14:58)

This video features the Workshop Module in Moodle 2. It demonstrates how to set up a Workshop module.