The impact of artificial intelligence-driven decision support on uncertain antimicrobial prescribing: a randomised, multimethod study

BACKGROUND: Challenges exist when translating artificial intelligence (AI)-driven clinical decision support systems (CDSSs) from research into health-care settings, particularly in infectious diseases, an area in which behaviour, culture, uncertainty, and frequent absence of a ground truth enhance the complexity of medical decision making. We aimed to evaluate clinicians' perceptions of an AI CDSS for intravenous-to-oral antibiotic switching and how the system influences their decision making. METHODS: This randomised, multimethod study enrolled health-care professionals in the UK who were regularly involved in antibiotic prescribing. Participants were recruited through personal networks and the general email list of the British Infection Association. The first part of the study involved a semistructured interview about participants' experience of antibiotic prescribing and their perception of AI. The second part used a custom web app to run a clinical vignette experiment: each of the 12 case vignettes consisted of a patient currently receiving intravenous antibiotics, and participants were asked to decide whether or not the patient was suitable for switching to oral antibiotics. Participants were assigned to receive either standard of care (SOC) information, or SOC alongside our previously developed AI-driven CDSS and its explanations, for each vignette across two groups. We assessed differences in participant choices according to the intervention they were assigned, both for each vignette and overall; evaluated the aggregate effect of the CDSS across all switching decisions; and characterised the decision diversity across participants. In the third part of the study, participants completed the system usability scale (SUS) and technology acceptance model (TAM) questionnaires to enable their opinions of the AI CDSS to be assessed. FINDINGS: 7·73, p=0·0054; logistic regression odds ratio 0·13 [95% CI 0·03-0·50]; p=0·0031). AI explanations were used only 9% of the time when available. Our software and AI CDSS obtained a good SUS score of 72·3 out of 100 (SD 8·79) and, for the TAM questionnaire, scores of 3·6 out of 5 (0·31) for perceived usefulness, 3·8 out of 5 (0·20) for perceived ease of use, and 4·1 out of 5 (0·05) for self-efficacy. INTERPRETATION: This AI CDSS was positively received and has the potential to support antimicrobial prescribing, with the greatest influence on clinicians when it recommended not switching from intravenous to oral treatment. Further prospective research is needed to gather safety and benefit data and to understand behavioural changes as AI CDSSs enter clinical practice. Our research suggests that AI explanations are likely to have a minor role at the point of care, and that AI CDSS adoption and utilisation depends on systems being easy to use and trusted, primarily through clinical evidence. FUNDING: The UK Research and Innovation Centre for Doctoral Training in AI for Healthcare, and the National Institute for Health and Care Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance at Imperial College London.