Marking Principles
- Neill Magill
Rogo marks in two ways: 1) in real-time during an exam as it stores responses and marks to the relevant log tables, and 2) it re-marks when displaying certain reports like 'Class Totals'.
In Real-time
Marking is handled by /include/marking_functions.inc and is called by /papers/start.php during an assessment and /papers/finish.php at the end of an assessment. User responses together with marks are stored in a relevant log table. Storing marks in log is necessary for determining cohort order for frequency/discrimination analysis.
Two types of log table are used log_metadata for general information about a student attempt on a paper and logX which are used to store student answers to individual questions. Different log tables are used for different paper types as follows:
| Log | Purpose |
|---|---|
| log_metadata | Holds as assessment attempt for a student. One record per student, per paper, per attempt. Most papers should have one record per user, Formative Quizzes can have multiple records. |
| log0 | Holds responses for Formative (Quiz) assessments. |
| log1 | Holds responses for Progress Test assessments. |
| log2 | Holds responses for Summative assessments. |
| log3 | Holds responses for Surveys. |
| log4 | Holds responses for OSCE assessments. |
| log5 | Holds responses for Offline papers. |
| log6 | Holds responses for Peer Review assessments. |
| log_late | This holds records submitted after the end time of the exam up to 1 hour later. For these marks to be counted in a students' final score they must first be approved or rejected by a member of staff. If approved the records are copied over to the appropriate normal log (e.g. log2 for a summative exam). |
AJAX Saving
AJAX technology is used on the student screens to ensure that a positive response has been received from the server that answers have been saved. Saving can happen in two ways: 1) when a student navigates to a different screen, and 2) automatically after a certain time period (see `$cfg_autosave_frequency` in `/config/config.inc.php`). The main saving mechanism in /paper/start.php is coded in two ways - normally AJAX will be used but if there is a JavaScript error then the system should revert back to using a standard HTML form submission.
- Screen navigation-save: JavaScript replaces the action of the 'next' and 'previous' buttons and calls `userSubmit` JavaScript function. This in tern will call `ajaxSave` which will POST to `save_screen.php`.
- Automatic-save: `startAutoSave` sets up a new `setTimeout` which will call 'autoSave' function, this in tern will call `ajaxSave`.
If the AJAX either returns `fail` or `error` then it will check `retry()` to work out if it should re-try saving to the server. In `save_user_responses` function in `/include/marking_functions.inc` there is a check on how many row are affected, if zero then a rollback will occur and false returned. If all goes well then `saveSuccess` function is called and the form submitted to change screen. However, because the AJAX has already saved for us the action has been set to `finish.php?id...&dont_record=true`. The `'dont_record=true'` stops `start.php` from re-saving again.
Marking Exceptions
Calculation question
The Calculation question can be set up in a number of different ways to be marked. Two standard ways are to use the internal PHP `eval()` function or to use an external `rserve` service. Formative and progress test papers will mark Calculation questions when the student gets to the end of the assessment. Summative exams containing Calculation questions will only be marked once the current time passes the end time/date set for the paper. Calculation questions in Progressive tests are only marked when the class totals report is run. The Class Totals report will automatically determine if there are any Calculation questions that are unmarked and if so will call `/ajax/reports/mark_all_enhancedcalc.php`. When successful it will reload the current script.
Textbox question
Textbox questions are marked manually by academic staff. After an exam staff can go into the 'Reports' submenu and select 'Primary Mark by Question'. Primary and secondary marking is performed in a kind of cache (textbox_marking table). When primary and/or secondary marking has been completed 'Finalise Marks' will insert final marks into the relevant log table (usually Log2).
Reporting
One of the main reports to use marks is the summary report 'Class Totals'. It is possible to exclude certain poorly performing questions post-exam through the Frequency/Discrimination Analysis report. If a question has been excluded then the 'Class Totals' report needs to do two things: 1) ignore the mark for this question for the user, and 2) ignore the mark for this question when calculating the total number of marks for the paper.
If a question is excluded as a whole (i.e. MCQ or Multiple Response) then the mark from the relevant log table is used (or not if it is excluded). However, if parts of a question can be excluded individually like the scenarios in an Extended Matching question or blanks in a Fill-in-the-Blank question then Rogo will remark the whole of that question based on the user's response.
Negative Marking
Negative marking can be considered a controversial topic. Some schools/modules use it while others, such as the Medical School, have been told by their professional body that they cannot us it. For this reason the ability to use negative marking on a question is controlled at a high level. To activate it requires a user with SysAdmin access to edit the module the question is in and tick 'Negative Marking'.
The system is designed to fail safe so that when a question is on multiple modules all the modules must be set to have negative marking on for this to become an option when editing a question.