Paper Types

Background

A Rogo 'paper' performs roughly the same function as an paper-based examination or coursework exercise. 

Summative Assessment

Summative assessments are examinations which contribute to the student's degree.  They are also arguably Rogo's most important application.  Students sit a summative assessment much as they would a traditional paper based examination.  They attend an examination room at a scheduled time and sit the examination on a computer.  At the end of the examination, Rogo marks the papers and they are reviewed and re-marked  by people where appropriate.  Rogo then generates reports detailing the summary of performance.

Summative assessments do not give feedback initially. 

From Rogo 7.2 a summative exam can be set to be 'remote' this allows the exams to be taken outside of exam labs. i.e. open book exams.

Locking papers and questions

Up to thirty minutes before a summative assessment is due to be sat, any user with the correct permission can make changes to it or any of its questions.  After that threshold passes, Rogo locks the paper and all the questions.  The paper cannot be changed and neither can any of the questions. 

Formative Assessment

Formative assessment is a generic term for exercises that allow students to measure their own progress. They also allow students to become familiar with Rogo and its question types.  Answer feedback is very important for formative assessment.

Creating a formative assessment follows the same process as a summative assessment.  They are made available for a much longer period, in years rather than hours and minutes.  Set the Session to N/A (under the Security options)

A formative quiz can be sat at any time and for any number of attempts.

Progress Tests

A progress test is rather like a mid-term exam that does not count towards the student's final degree.They differ from formative assessments in two key ways:

1. They can be only taken once;
2. No feedback is given.

A timed progress test's duration starts from when the student opens the test and concludes when the duration has expired or the student finishes early.  Students can leave and re-enter a progress test as many times as they like up until the expiry date or they click on finish.

Objective Structured Clinical Examination (OSCE) Stations

OSCE Stations are summative assessments completed by assessors, rather like a driving test.  OSCEs are timetabled and tend to be quite short duration, about 5-10 minutes. 

During the OSCE Station, the examiner will score the student on a number of criteria, generally with a 0 or 1 (although some may score with a 0,1,2 scale for partial compliance). 

Killer Questions

KIller questions must be satisfied.  If the student is awarded 0 on a killer  question they will fail the whole assessment.

Offline Papers

Offline papers comprise a set of dummy questions mapped to learning objectives.  Assessors use them to collate the results of paper based examinations which are then entered into Rogo against the skeleton of the paper.  The objective-based feedback in Rogo can then generate  reports for the paper-based examination that resembles standard Rogo reports.  This generates personalized feedback for each student.

Peer Reviews

Most peer reviews work with groups of students.  Each student is asked questions relating to other students in the group.  Rogo provides reports on the averages of the reviews.

Survey

Simply canvasses participants for their views on a number of topics.

Summary of Paper Types

Creating a paper

All Rogo papers reside in modules.  To create a new paper, you need to open the module and then click the New Paper icon (step 1):

Then you select the paper type (step 2), give it  a meaningful name following the preferred format 'Module Code: Month (Session)' (step 3) and move onto the next step (step 4):

You must then select which students can take the paper by selecting the correct module (step 5) and the dates that it will be available from (step 6) and to (step 7).

Setting the Paper Properties

You set the paper properties by opening the paper and clicking Edit Properties in the sidebar.  The Properties screen opens:

Clicking on an item in the sidebar brings up a form allowing various classes of property to be set:

General

Paper Details

• URL: Cannot be changed
• Name:  A simple name for the paper that complies with preferred naming format
• Type: Indicates the paper type.  Cannot be changed unless the type is Progress Test or Formative Self-Assessment 
• Folder:   Sub-folder of module where the paper resides.  Can be changed

Display Options

• Display:  can be Windowed or Full Screen (IE only)
• Background: colour of the paper's background
• Theme: selects a theme colour for the paper
• Calculator:  can be toggled on or off to display the system calculator
• Navigation:  Unidirectional (forward only) or Bidirectional.  The latter is the default:  certain papers might only permit the former depending upon content.
• Foreground:  colour of text
• Labels/Notes:  colour of label text for various attached items
• Audio:  whether the paper supports audio questions

Marking

• Pass mark:  percentage that denotes a pass mark
• Distinction:  percentage that denotes a distinction]

  Locking papers and questions

• Method:
  • No Adjustment: marks are presented as is.
  • Calculate Random Mark:  Rogo works out how a student would perform if  they answered every question randomly.  It then adjusts percentages accordingly.  This is sometimes referred as 'monkey marking'. This option is disabled if negative marking has been used on the paper.
  • Std Set:  standard setting is being used for the paper.

The only users who can access a summative paper outside of the examination time are administrators and staff members.  Students may only access the examination during the scheduled time.

Security

• Session:  controls which academic year the paper applies to.  Set to N/A for formative assessments.
• Password:  puts an extra password on the paper which students need to enter.  This password can be given out during the lab
• Time zone, Duration,Available from/to:  governs availability of the paper.  

From Rogo 7.2

A summative paper can set to be a remote summative:

The minim valid time is also displayed to the admin (highlighted in the image above).

Feedback

Controls what kind of feedback is available.  This varies between formative and summative assessments

Formative assessments make different kinds of feedback available:

• Objectives Based:  Use this for formative assessments to give feedback to students how they have met their assessment targets as mapped to curriculum objectives
• Class totals:  allows external examiners to view this report
• Textual feedback:  The Above dropdown sets a threshold value above which a student will receive a feedback message depending on their score (a 'pat on the back').

Summative assessments allow the following feedback:

• Objective based: as above
• Question based: allows feedback on specific questions.  This is a very dangerous option that has been turned off by default and is controlled by a module-level switch.  As papers can be shared between modules, the switch must be turned on in every parent module for this to work.
• Cohort performance:  allows students to see how they have performed relative to their peers.  Performance is plotted on a box-plot diagram
• Class totals:  allows external examiners to see the class totals report for a cohort.  Examiners can then drill down to individual scripts.

Reviewers

Sets external reviewers and deadlines.

You select internal reviewers and external examiners from a list. Internal reviewers come from all Rogo staff in the current school, external examiners comprise Rogo users with an external examiner role.

Exam Rubric
Rogo displays the rubric at the initial paper information screen after logon. This rubric will contain the usual proscriptions and warnings about acceptable conduct. The invigilator screens will also display the rubric.
The rubric page supports a rich-text editor for special content such as tables and equations.

Prologue
Screen 1 is the first screen the student sees: the prologue appears at the top of this.

Postscript
The student sees this after clicking the Finish button.

Reference Material
Displays approved reference text associated with a module. Has been used for displaying drug names and the suchlike

Changes
A history of all changes to the paper so far, emphasizing the part, old and new values, and who changed it and when it was changed.

Summative Assessment Lifecycle