Is There a Main Effect? Improving Data Literacy Using Practice Examples and Peer Collaboration

Learning to interpret visual representations of data is an important step towards becoming an informed consumer of research. The current study assesses the effectiveness of two versions of a scaffolded online module in improving students’ ability to identify main effects and interactions in 2 × 2 factorial designs. Across two experiments ( N = 313), we compared exam performance between sections of a lower-division conceptual statistics course that completed the module (in addition to other course activities) to a section that did not complete the module ( n = 91). The first iteration of the module (used in Experiment 1, n = 96) was completed once individually and once with peers and did not enhance students’ individual performance on conceptually-related exam questions. However, performance on the module was low, indicating that students may have needed more support to benefit from this experience. In Experiment 2 ( n = 126), we made three empirically-driven changes to better scaffold student learning: we added a worked example, offered a greater variety of examples, and instructed students to complete the whole activity with peers. Under these conditions, performance increased on related exam questions. We conclude that this freely available module is a promising intervention to strengthen students’ ability to understand factorial designs.