Script Concordance Test
- Neill Magill
Description
Allow for the incorporation of SCT questions. These are useful in areas such as Medicine and Veterinary Medicine.
Rationale
This question type assesses reasoning skills, specifically in situations with uncertain scenarios such as those in clinical subjects. It presents initial information and intended course of action or hypothesis. It then shows some further information and asks to what extent this supports the original course of action/hypothesis. This type of question does not have a correct answer so the marks for the different options are calculated from the opinions given by a number of experts.
| Script Concordance Test (SCT) | Multiple Choice (MCQ) |
| Ambiguous scenario and vague data | Well defined scenario and accurate data |
| More than one acceptable answer | Only one correct answer |
| Assesses reasoning | Assesses knowledge |
Fit Criterion
Presentation
- The UI allows staff to edit three important parts of the question: 1) clinical vignette, 2) hypothesis, and 3) new information.
- Dropdown list is included to specify how many experts selected each option.
- The number of experts can be automatically populated from the 'SCT Responses/Reasons' report as a time saving feature.
Marking
Unlike other question types SCT is always marked out of 1. The number of marks awarded for each option is a percentage of the number of experts who selected the option. To calculate this it is first necessary to find the maximum number of experts who have rated an option. The number of marks to award to the student is the number of experts who selected the student option divided by the highest number of experts on an option. Where a student coincides with the maximum number of experts this figure is thus 1.
Feedback
Feedback includes:
- Tick/Cross show what the student has got right/wrong
- Bold options show the most correct option (highest number of experts).
- Student mark out of possible number.
- Textual feedback is available for each option.
- List of learning objectives that the question is linked to.
Limited Save
After a question has locked it should only be possible to change the following aspects:
- General feedback
- Status - only 'Normal' or 'Retired' (not 'Incomplete', 'Experimental' or 'Beta')
- Bloom's Taxonomy
- Keywords
- Teams
Standard Setting
SCT questions cannot be standard set.
Data Format
Correct Answer
A record in the options table is used for each option. The 'correct' field holds the number of experts who selected this option. Zero can be used for options that no expert selected.
Student Answer
The 'user_answer' field of the relevant log table is used to store the student's answer. One represents the first option, two the second and so on like an MCQ question. A zero represents the student not answering the question.