Are Spot Test Multiple Choice Questions Easier to Answer than Short Answer Questions?

Machine marked multiple choice assessments are often preferred over those requiring manual marking, with most medical schools and boards using multiple choice questions (MCQs), usually single best answer, as the main format for written tests. While these often give excellent reliability and validity, disadvantages are that students can guess the answers with no penalty, or can rely on recognition rather than having to produce information. In order to investigate whether this makes tests easier for students, 277 first year medical students were randomly allocated into two groups (A and B). Both groups wrote a formative spot test, delivered by PowerPoint. The questions were largely anatomical, but with a few other basic science questions included. The test was composed of both MCQ and short answer question (SAQ) format questions. Each group answered 40 questions in total, with 20 of each type, but the 20 questions that Group A answered in MCQ format were given to Group B as SAQs and vice versa. A Student t test was carried out to determine if there was a difference between the mean scores achieved for the two types of questions in each format. The mean SAQ score (with no half marks awarded) was 34.3% (±14.8), while that for the MCQs was 54.7% (±15.6). If half marks were included in the scores attained for the SAQs, the mean score was 37.0% (±14.9). There was a statistically significant difference (p<0.01) between the SAQ and MCQ scores, regardless of whether half marks were included. It is therefore suggested that MCQs alone should not be relied upon, at least for spot tests, as scores achieved with MCQs are significantly higher than for SAQs. Free written response type questions should also be included.