Summative Locks

Owned by Neill Magill
Oct 25, 2022

The Problem

Many of the reports in Rogo are dynamic. When they are called they use the information from the 'questions' table in its current state. Early in the development of Rogo it was found that changing a question could cause problems when analysing old log information. For example, in 2007/08 an MCQ question had 6 options. Students in the cohort had a spread of answers across all 6 options and the reports functioned correctly. However, it was found through the 'Frequency & Discrimination Report' that not many students selected one of the distractors. It was thus decided that it should be removed the following year. The question was re-used on an exam in 2008/09 with now 5 options. The report worked perfectly for the 2008/09 database but when going back to 2007/08 it through up errors for any student selecting option 6. The reason for this is that Rogo is using the current structure of the question which now has only 5 options.

The Solution

To solve the above problem Rogo now locks all questions used on a summative exam after the exam starts. Locking is carried out by a member of staff going into /paper/details.php after the exam has started (e.g. start time/date has passed). QuestionUtils::lock_question() is the function which actually performs the locking.

Question locking has some distinct advantages:

  • Stops reports going wrong when historical data is changed (see above).
  • Stops questions being changed mid-exam
  • Creates an accurate archive of an exam which can be reviewed for years to come. Useful for inspections by professional bodies.

Locked questions cannot be edited by members of staff (including SysAdmin). The only parts of a question which may be altered are the marks and the correct answer itself. These can be changed and then the 'Limited Save' button used to re-mark the cohort.

Limited Save

The parts of a question than can be saved after a question has been locked are any aspects not critical to the meaning of the question itself. Examples of parts that can be changed include:

  • Marks
  • Correct answer (see Re-Marking below)
  • Bloom's Taxonomy
  • Status
  • Keywords
  • Teams
  • Feedback text

What cannot be changed, because it would alter the actual questions that students saw include:

  • Theme/Heading
  • Notes
  • Scenario
  • Lead-in
  • Options

The array _fields_change in the relevant question type class controls which fields can be saved on a 'Limited Save' operation.

Re-Marking

When the 'Limited Save' button is used a corrector class for the type of question being edited will be used. These corrector classes are held in: /classes/behaviours/corrections The will read in all answers for the question on a particular paper and remark. The relevant log file will then be updates (mark and totalpos).

Limitations

  • Limited save only remarks questions on a particular paper. This is to avoid the situation where correcting a question on one paper could upset the marks on another paper taken in the past.
  • Remarking only uses student answers from the log of the relevant paper type. Formative quizzes and progress test type papers can switch types. The remarking will only occur for questions in the log of the current type.