Clean SOAP

Healthcare systems worldwide face a crisis: there are far too few Primary Care Physicians (PCPs), and they are severely overworked. Modern Electronic Health Records (EHR) systems, while solving critical issues with paper-based records, have significantly increased the burden of medical case documentation. An “AI Scribe” is an artificial intelligence system designed to streamline clinical documentation by automatically generating “doctor visit notes,” which all physicians must produce following a patient visit. AI Scribe systems leverage natural language processing (NLP) and machine learning to interpret and document patient encounters in a format called SOAP (Subjective, Objective, Assessment, Plan). However, implementing AI in medical documentation raises multiple risks, including hallucinations, critical detail omissions, misclassifications, narrative quality and organization issues, security and privacy concerns, bias and discrimination, and legal and ethical challenges. Automated testing of AI-generated SOAP Notes is essential but poses its own challenges. Using Large Language Models to test the output of other Large Language Models can automate and scale semantic and contextual SOAP testing. The critical questions are: what can we test successfully, and how do we avoid an infinite regress of validators?