How do students feel and collaborate during programming activities in the productive failure paradigm?

ABSTRACTBackground and Context Productive failure (PF) is a learning paradigm that flips the order of instruction: students work on a problem, then receive a lesson. PF increases learning, but less is known about student emotions and collaboration during PF, particularly in a computer science context.Objective To provide insight on students' emotions and reasoning during PF activities and their relation to performance.Method We conducted a PF study (N = 48) with a programming problem. We used a mixed-methods approach to analyze students' emotions, reasoning, and pretest/posttest performance. Findings Uncertainty occurred frequently compared to confusion, frustration, and positive affect. The constructive contributions made during problem solving correlated positively with posttest scores.Implications Although students failed to produce a correct solution, there were few instances of frustration and a promising amount of constructive reasoning during collaboration. This, coupled with prior work showing that PF improves learning over standard instruction, indicates that PF is a promising approach for computer science education.KEYWORDS: Productive failureemotionsreasoning during collaboration Disclosure statementNo potential conflict of interest was reported by the author(s).Ethics approvalThis study has received clearance by the Carleton University Research Ethics Board B (Clearance #112108).Notes1. A subset of these results was presented in (Savelson & Muldner, Citation2021) as a six page paper – this paper did not include the reasoning analysis.2. Academic Emotions Questionnaire (Pekrun et al., Citation2011) that measures trait emotions, trait-level self-efficacy (Sherer et al., Citation1982), trait-level curiosity (J. A. Litman & Spielberger, Citation2003), and intolerance for academic uncertainty (Freeston et al., Citation1994).3. There were eight instances where frustration co-occurred with confusion or impasse and so both codes were applied to the segment during the coding process described above. When calculating interrater for these, we labelled the utterances with two separate labels, which allowed for agreement to be calculated for each separately; thus, the raters could have agreed on a label of confusion for a segment while disagreeing on whether that segment contained frustration.4. We did not calculate interrater agreement for the segmentation, for the following reasons. The granularity of the segmentation in our analysis was at the utterance level, which means that in some cases, sentences and/or speaker turns were segmented to accommodate the constructs being labelled. Note that in this situation, unique marker locations often do not exist, and minor fluctuations do not impact subsequent coding; moreover, in this situation using two researchers for segmentation can be detrimental, because if they did "not unitize a text exactly the same way, it may become difficult to determine whether their coding is the same" (Campbell et al., Citation2013, p. 302). Thus, the norm is that a single researcher segments the transcripts prior to the coding process (Campbell et al., Citation2013; Lam & Muldner, Citation2017).Additional informationFundingThis work was supported with an NSERC Discovery Grant #1507 and a Masters' SSHRC grant.