Development and Validation of Content Domains for Paramedic Prehospital Performance Assessment: A Focus Group and Delphi Method Approach

Background The chaotic and complex nature of delivering patient care in the prehospital setting complicates the provision of real-time formative feedback to paramedic students. Although the use of simulations is widespread in emergency medical services (EMS) education, a high degree of variability precludes consistent performance assessment in EMS. Objectives: The objective of this study was to define and validate key domains required to evaluate paramedic prehospital performance. Methods: We conducted a two-phase study that combined focus group and Delphi methodology. Participants were purposefully selected to attain diverse panels regarding sex, race, ethnicity, professional roles, levels of education, geographical area, and experience as a paramedic and educator. In Phase I, a panel of 11 subject matter experts (SMEs) were tasked with identifying the essential domains to be evaluated in a paramedic performance assessment. In Phase II, another panel of 11 SMEs and a four-round modified Delphi method with 28 paramedic program directors were used to validate the domains identified in Phase I. Results: The first focus group identified and achieved consensus on five domains: (1) effective communication, (2) scene management, (3) patient assessment, (4) patient management, and (5) professional behavior. These domains were validated by the second focus group. The first round of the Delphi process generated 64 content domains, which were reduced to nine unique content domains via thematic analysis. These nine content domains fit well within the broader domains identified by the focus groups with one specific area, critical thinking and reasoning, being listed in two key areas based on the definitions of the focus group domains of patient assessment and patient management. Conclusion: The content domains identified in this study provide EMS educators a theoretical framework for designing the performance assessment of newly trained professionals in the prehospital setting.