Using a diary to quantify learning activities

Background Diaries of actual learning activities can fill the gap between the planned curriculum and students' opinions and outcomes. We report the development and validity of such a method, estimate sources of variation and model sampling strategies to determine efficient ways to obtain information about a curriculum or about individual students. Methods Following development and piloting, the diary was administered to fourth‐ and fifth‐year medical students. Each student was asked to complete a diary on 3 randomly selected days of the academic year. Sources of variance and generalisability were determined using variance components analysis. Validity was explored by comparing activities with what is known about the curriculum, assessment, timetables and the 2 classes of students. Results Response rate was 83% (287/345). Learning activities varied as expected with timing of assessments, and on weekdays compared with weekends. For most activities, 14 days per student would be needed to obtain generalisable information about an individual student. The variation between days is greater than the variation between students, meaning that sampling for information on a curriculum should include all students and all days of the year but the number of diaries per student could be kept low depending on the desired power to detect any differences. Conclusion Such an evaluation method is feasible and can provide reliable and valid information about study activities. Reasons for good compliance are discussed. Sampling strategies should be tailored to the purpose of the study.