The Influence of Artificial Intelligence Scribes on Clinician Experience and Efficiency among Pediatric Subspecialists: A Rapid, Randomized Quality Improvement Trial

Artificial intelligence (AI) scribes may reduce the documentation burden and improve clinician experience through generative AI automatically producing provider note sections from recordings of patient-provider encounters.We aimed to examine the impact of AI scribes on clinician experience, clinician efficiency, and business efficiency measures among pediatric subspecialty physicians.We randomized pediatric subspecialty providers with ≥0.5 clinical full-time equivalent and stable electronic health record (EHR) log metrics to use Microsoft/Nuance Digital Ambient eXperience (DAX) Copilot from May 1, 2024, to July 31, 2024 (intervention group) or controls. Using difference-in-differences, we compared quantitative measures of subjective clinician experience using the KLAS Net EHR Experience survey, objective measures of clinician efficiency from EHR logs (e.g., pajama time), and business efficiency measures. At-the-elbow support checked in with intervention providers approximately weekly, and we assessed the sentiment of qualitative comments.Twelve providers were randomized to the intervention and 11 to the control group. One intervention provider stopped using DAX due to ineffectiveness. In the intervention group, DAX was used to populate one or more characters in 53% of visit notes (range across providers: 10.6-98.2%). Nine intervention and eight control providers completed pre- and postsurveys. KLAS Net EHR Experience improved among intervention providers from 52.6 (70th percentile) to 75.2 (99th percentile) but dropped from 37.3 (38th percentile) to 30 (14th percentile) among control providers. Experiencing burnout dropped from 8 (89%) to 5 (56%) among intervention providers but remained stable at 3 (38%) in the control group. There was no significant change to pajama time (-9.4 minutes per scheduled day, 95% CI: -41.2 to +22.4), time in notes per encounter (+0.2 minutes per note, 95% CI: -6.6 to +6.9), or work Relative Value Units (wRVUs) per encounter (-0.03, 95% CI: -0.5 to +0.44). Of 48 qualitative comments, 69% had a positive sentiment, 15% neutral, and 17% negative.Among pediatric subspecialists, AI scribes improved clinician experience and burnout without changing charting time or EHR work outside work hours.