Enhancing Artificial Intelligence Literacy in Nursing Education to Combat Embedded Biases

Artificial intelligence (AI) continues to challenge us as nurses on two fronts. Evaluating and implementing AI applications in the clinical setting presents enormous opportunity for supporting nursing cognitive work. On the educational front, AI is similarly an opportunity for expanding and exploring new ways of teaching and learning. In both cases, we will need to continue to explore the known and emerging ethical facets of its risks and benefits. This article explores the unintended consequences of the growing presence of AI in our professional lives. Mathew D. Byrne, PhD, RN, CNE , Editor, Emerging Technologies Center In health care and health professions education, artificial intelligence (AI) has quickly become a driving force in both clinical and educational settings. The integration of AI is based on its potential to revolutionize care delivery and pedagogies through the application of technologies, including chatbots (e.g., ChatGPT/BingBot/Bard), early warning systems, and other AI tools. These tools can improve diagnostic precision, optimize health care delivery, and transform education, but they also create new challenges. A critical challenge is a potential gap between AI literacy and meaningful understanding in emerging health care professionals. The rapid deployment and uptake of AI tools has outpaced the development of comprehensive educational strategies designed to build AI literacy. This lack of understanding can lead to misusing AI tools, resulting in decreased learning, inaccurate conclusions, reporting, or inappropriate application, which could translate to adverse health outcomes. Therefore, nurse educators must be aptly informed to leverage the benefits of AI while mitigating biases and preventing the exacerbation of existing health disparities. As AI takes a more central role in health care and education, it is critical to be aware of the likelihood of bias within the results. AI carries biases embedded in its algorithms and outputs, which are often reflective of historical and existing social prejudices. AI, particularly large-scale language models, is trained by processing vast data sets sourced primarily from the Internet, which, by and large, mirrors and magnifies existing societal structures and biases. Racism, homophobia, ableism, heteronormativity, sexism, ageism, and others find their way into AI data, creating a system where misinformation and discriminatory practices continue to harm marginalized groups. This is similar to the way traditional medical research has been disproportionately focused on White male populations, often sidelining crucial data from woman and minorities. Consequently, the data processed by AI bears a striking resemblance to this historical skewness. In addition, research conducted with underrepresented groups is often scant or missing; thus, data that AI synthesizes may be based on existing data sets of the only populations that are available, thereby providing false or untested conclusions for the underrepresented. The application of biased information may impact the quality of education nursing students receive and exacerbate existing health disparities. CHALLENGING EMBEDDED BIAS When teaching about AI, nurse educators should inform students about the nature and manifestation of biases in AI. To facilitate understanding, educators may describe the process of how AI generates information. AI uses existing data sources such as information found in books, articles, opinion pieces, and research; extracts these data; and synthesizes data based on patterns and predictions. Therefore, the output is only as good as the input — and at this time, AI struggles with deciphering high-quality evidence and reputable sources. The American Nurses Association (ANA, 2022) cautions that without proper oversight, AI may perpetuate existing societal biases. Because of the possibility that algorithms that profile or prioritize specific populations can propagate discrimination, the ANA recommends that users evaluate the methodology used, including the development and design of the AI and its testing, reliability, and integration. Furthermore, the White House (2023) issued an executive order to establish new standards for AI safety, security, equity, civil rights, and more. This order advises the Department of Health and Human Services to establish a safety program to evaluate and remedy harms or unsafe health care practices involving AI. AI may contribute to transformations in health care and education, but it is clear that if not applied or evaluated properly, it may perpetuate inequity or even cause harm. CONSEQUENCES OF AI BIAS In nursing education, the permeation of biases in AI systems poses significant challenges. As nursing curricula increasingly integrate AI tools for training and simulation, any biases within these systems can skew learning experiences and inadvertently introduce prejudiced thinking. Low AI literacy among nursing students and educators further compounds these issues. Without a robust understanding of AI's limitations and inherent biases, the next generation of nurses might unwittingly rely on or propagate these biases in their practice. This cycle is particularly concerning when considering the broader implications for patient care. Biases in AI, when not recognized or corrected, have the potential to perpetuate and exacerbate existing health disparities, especially for marginalized communities. As such, the intersection of nursing education and AI bias requires urgent attention to ensure that as technology evolves, the foundational principles of care and equity in nursing remain unshaken. Building upon the need for AI literacy in nursing education, it is crucial to deliberately incorporate bias education into the curriculum. One practical approach would be to introduce coursework dedicated to AI and its biases. For instance, students could engage in hands-on activities using simulated AI platforms to diagnose medical conditions. Through this, they would be challenged to recognize when the AI's suggestions are tainted by biases, such as overdiagnosing particular conditions in specific ethnic groups because of skewed data inputs. Case studies can serve as illuminating tools for building AI literacy. Consider a scenario where an AI system, trained primarily on data from older male White patients, misdiagnoses a young African American woman with a heart condition because her symptoms manifest differently. Such a case study would not only highlight the potentially grave consequences of unchecked AI biases in health care but would also underscore the importance of human oversight and critical thinking in the nursing profession. Integrating elements like these into the curriculum would equip nursing students with the tools and knowledge to navigate the complex landscape of AI in health care. Another exercise nurse faculty may choose to conduct is to illustrate the potentially limiting impact of AI for underrepresented populations. Students may be directed to try using different languages in select AI applications. For example, a nursing student might be directed to ask a question of a chatbot in English, Spanish, and Creole. Depending on the application, the answers may be culturally inappropriate or unavailable for the requested language. This exercise can help to emphasize how underrepresented populations may be further disadvantaged, underserved, or wholly overlooked with this technology. Faculty may debrief with students regarding the implications of outputs based on the synthesis of data from different populations and settings that are inappropriately applied or missing altogether. Following the introduction of AI literacy and bias training into nursing education, fostering a culture of critical analysis and open discourse is equally vital. Establishing safe academic environments where students feel comfortable voicing concerns, sharing experiences, and critically examining AI biases is essential. Within these spaces, educators should emphasize the necessity of critically assessing AI outputs. For instance, students can be encouraged to critically assess AI recommendations, questioning the underlying data sets and algorithms and identifying potential biases. Such critical thinking exercises help in distinguishing between genuine AI insights and possible misinterpretations because of biases. Comparing the outputs of AI systems with human discernment can be a valuable exercise. By juxtaposing AI-generated suggestions with human clinical judgments in hypothetical scenarios, students can gain a clearer understanding of where AI might falter and the inherent biases it may carry. These activities, and others, collectively drive home the importance of human oversight, critical thinking, and open dialogue in the rapidly evolving intersection of AI and health care. CONCLUSION The very essence of nursing — a profession steeped in empathy and human understanding — reminds us that though AI can be an effective tool, human judgment remains paramount. It is important to cultivate a generation of nurses adept at harnessing the capabilities of AI yet astutely aware of its inherent biases and potential to perpetuate harm. To safeguard patient care and safety and uphold the integrity of nursing as a profession, schools of nursing must proactively embed AI literacy and bias recognition into their curriculum. This will enhance the quality of care and reify the ethical foundations upon which nursing stands (ANA, 2015).