Foreword: The Role of Relevance in Medical Education Research

On behalf of the Research in Medical Education (RIME) Program Planning Committee, welcome to the 57th annual Association of American Medical Colleges (AAMC) RIME program. We hope you will benefit from reading the research papers, reviews, Foreword, and Commentary included in this RIME supplement to Academic Medicine and from participating in the RIME sponsored sessions during Learn Serve Lead 2018. Our sincere thanks to the medical education researchers who submitted manuscripts for consideration, and to the 52 reviewers who participated in the peer review process. Your work is the substance of RIME, and each year we look forward to seeing what new ideas come our way. And thanks to the AAMC staff who provide so much more than administrative support. Their insightful perspective informs and guides us. Each year, as the committee conceptualizes the Foreword to the RIME supplement, we consider both our past few years’ activities and our future direction in terms of work submitted, work accepted, themes of our invited addresses during Learn Serve Lead; and, perhaps a little selfishly, our own desired directions for the future of medical education research. We hope that our Forewords provide useful guidance in medical education research, especially to medical trainees and early-career faculty. The 2016 RIME Foreword outlined the elements of a successful RIME submission,1 while the 2017 Foreword explored the roles of reviewers and authors.2 This year’s Foreword examines the role of “relevance” in medical education research to encourage us all to consider a wider range of research topics with the potential to inform and improve medical education across the continuum and future health care delivery. What Is Relevance? The traditional measures of value for medical education research include the clarity of the research question, the appropriateness of the study design, the rigor of the methodology (quantitative, qualitative, and/or review), the strength of the evidence, and the generalizability/transferability of the findings.3,4 These elements can be evaluated, and many resources5–10 provide a wealth of information to guide design and evaluation of studies. Many reviewer guidelines and discussions of good research also mention “relevance”—a more elusive and contested construct.3,4 Relevance is situated in time and place—it depends on the political, sociocultural, historical, technological, and perhaps even epistemological context. Some argue that education is an applied field in which research is grounded in “problems of practice.” From this perspective, relevance is based on the extent to which research addresses and yields findings that are useful for resolving “problems of practice.”11,12 Of course, deciding what constitutes a useful question or findings depends on who you ask. A dean, a student or resident, a clinician–educator, a researcher, a patient, or a member of an accrediting body will likely have quite different views of relevance and value. In light of these perspectives, how do we as a research community think about “relevance” and define criteria that guide our design and evaluation of research in a productive way? Conceptualizing “Relevance” The 2004 RIME Committee was among the first to explore the construct of “relevance” in conjunction with “quality” of medical education research.4 They characterized relevance as studies that address “the ongoing concerns of the research in the medical education community” and that “hold implications for the enduring concern of improving either practices of medical education and/or medicine.” Frequent RIME contributor Larry Gruppen13 argued as early as 1990 that physicians in training learn best when they see the immediate relevance of content: “Relevance and practicality are central.” Although he was writing in the Bulletin of the Medical Library Association for librarians, the message also applies to those of us who develop that literature. We must not forget to explicitly note the relevance and practicality of our work. When a researcher views a question as important, the researcher often assumes the question will be important to a wider audience. But, as George Gershwin said, “It ain’t necessarily so!” Even if the results of a study are generalizable (or, in the case of qualitative studies, the concepts or theories transferable),14,15 researchers cannot assume that the individuals or organizations they believe will benefit from the findings will see the relevance and be able to make changes. Along with the other elements of good research design, researchers must present a strong argument why and to whom their study is relevant.16–18 A pragmatic approach to improving relevance is to acknowledge multiple types of relevance and to encourage authors to be explicit about ways in which their work is relevant (or useful) to others. For example, a scoping review may not offer specific solutions to a practical problem, but it may bring clarity to a complex topic and identify problems that warrant further research and/or innovative approaches in practice. Such work could be relevant to both researchers and clinician–educators. By contrast, an evaluation of a curricular reform may offer solutions to a problem faced by a single institution, but a stronger case for relevance should indicate the value of the reform and the evaluation findings to broader problems in the field. Such work might be useful to educational leaders or clinician–educators seeking to address similar problems in their own institutions. Or it might be useful to researchers who wish to delve more deeply into questions about why the reform produced certain outcomes. Authors can also use their choice of journals as an opportunity to maximize relevance. Each journal has a different mission and audience. The journal in which work is published helps establish relevance by placing work in context, relating it to other published work, and reaching readers who will likely know how to use or apply the study findings. Often, particularly when exploring issues of social or societal relevance, researchers develop true passion for an issue. While passion can drive the desire to research certain topics, one’s stance as a researcher must remain critical and open to all perspectives and findings. Soapbox rhetoric and linear regression are an awkward combination, so how and where in scholarly writing can researchers make a clear and compelling case for the relevance of their research without appearing biased? Relevance needs to be argued differently depending on the intent of the manuscript. For example, research studies are a different genre from commentaries and editorials and are written (and cited) differently. In commentaries and editorials, authors can argue for relevance using a larger toolbox including both scholarly work and more community-based sources. They must commit to using reputable data, but descriptive and reflective language is also appropriate. They can express themselves and their perspectives (biases?) because their goal is persuasion. In research studies, authors wear a different hat. While it is essential to state the case for relevance, they must be rigorous and systematic to minimize bias so as not to compromise the validity or trustworthiness of the research and acknowledge the perspectives they bring to the study. There are several places in a well-written study where this can be done. This first is in the introduction where authors place their work in context and describe the problem or gap that generated the research question.19 Context may include citations of related studies but may also include editorials or commentaries when identified as such. Citing an editorial or commentary from a well-respected journal, especially one that has made a call for the research being presented, can help build the argument for the relevance of the research. The research problem must be observable and measurable, and, while stated in unbiased language, it may certainly imply a need for change. For example, “many primary care physicians in the United States feel unprepared to provide care to transgender patients.”20,21 There are tremendous social implications behind this statement, but the research finding itself is expressed in unbiased terms. The Discussion section also provides authors opportunities to help readers see the relevance of their work. Once outcomes are stated, authors can compare their findings with outcomes of studies in other contexts or disciplines. This approach not only supports the argument for relevance but also shows that many problems faced in medical education are not unique.22,23 Many methodological approaches offer opportunities for providing evidence of relevance. As medical education research ventures into new content areas, members of the medical education research community can draw on a variety of methodologies to study pressing and emerging problems they know well from their own observations and experiences. For example, in phenomenological approaches, the researcher explores people’s lived experiences of a situation or event (a phenomenon).24 This methodological approach can be a highly effective way of capturing trainee and faculty insights in the classroom and clinical learning environments. Yet another example is epidemiological research which has both medical and behavioral applications and thus can be applied to current issues such as gun violence25 and opioid addition and their importance in medical education.26 West et al1 reminded us in the 2016 RIME Foreword that methodology can be innovative, but must be well referenced so readers understand not only what you did, but why you did it. And your argument for why is an additional opportunity to remind readers of the relevance of your work. Conclusions Medical education research is maturing. The growing community of trainee and faculty researchers benefits from an increasing body of knowledge on which their work can build. Medical education research has addressed many pragmatic questions common to medical educators and provided evidence to support good curricular decision making. It is now time to dive deeper into the more difficult questions—those that deal with the social and psychological impact of medical education on trainees, faculty, and ultimately on patients and society as a whole. While it is true that “relevance” is situational, that in no way reduces its importance. The work of medical education researchers resides at the intersection of the postsecondary education system and the health care system. Both systems have tremendous complexity. As we broaden our scope of work and our methodologies, we must also be innovative in expanding our audience. How might administrators, politicians, and policy makers benefit from our findings? How might academicians in other disciplines become our research partners? How can we make our work more accessible and comprehensible to a variety of stakeholders and audiences? We argue that including clear statements of relevance (both immediate and potential) will increase the likelihood that our work will be found, and outcomes applied. The “so what?” factor that we were taught as novice researchers remains essential. Gratitude We thank several individuals and groups who helped make this year’s RIME program a reality. First, we thank the dedicated AAMC staff—Nesha Brown, Steve McKenzie, and Kate McOwen—who worked tirelessly to make the RIME program and supplement happen. We also thank the 2018 RIME Program Planning Committee: Lara Varpio, Tanya Horsley, Win May, Jeanne Farman, Douglas Larson, Zareen Zaidi, and Yoon Soo Park. It is an honor and privilege to work with such a talented group of individuals who are so devoted to promoting the art and science of medical education. Finally, we thank Dr. David Sklar and the editorial team of Academic Medicine. We sincerely appreciate their continued support of medical education research and the RIME program.