From AI models to AI agents: governance through recursive oversight and revocable access

Recent developments in AI are shifting the main source of risk from the flawed outputs of individual models to the unintended effects of agentic AI systems. Limited but growing evidence of misaligned behavior in swarms of AI agents suggests that such systems can pursue unassigned tasks and act beyond the reach of traditional corporate governance tools, which presuppose agents with interests, fiduciary duties and legal personality. This paper argues that the object of governance should be expanded from the model in isolation to the agentic AI system as a whole, and that oversight should be calibrated to the combined level of capability, autonomy, external access, permissions and scale. It proposes a framework of recursive governance, in which monitoring, evaluation, intervention and authorization form a continuous feedback loop rather than a fixed checklist. Four propositions follow: (P1) governance must become more stringent as capability and operational autonomy combine with external access and scale; (P2) autonomous deployment at a given capability level should not occur unless predefined criteria for evaluation, monitoring, containment and independent verification are met; (P3) some governance functions should be embedded in the technical architecture rather than being left solely to ex post human intervention; (P4) enforcement should rest on conditional and revocable access to the infrastructure that autonomous operation requires, with the kill switch as the last resort of a broader regime, and should be anchored as far as possible to institutions, rules and forms of self-regulation that already exist. To this end, the paper proposes a three-stage enforcement architecture: (i) a lean, independent multilateral agency, composed mainly of experts in advanced AI, certifies that the governance process for AI agents is actually followed; (ii) the certificate is fully incorporated into the ESG framework already operating in the European Union and linked to existing EU legislation on sustainability reporting, sustainable finance and ESG ratings; (iii) the IMF includes AI governance compliance in its periodic surveillance of member countries through the Financial Sector Assessment Program (FSAP). Apart from the new agency, which to our knowledge does not yet exist, enforcement would thus rely on institutions that are already in place. The proposed policies are framed as responses to three market failures: information asymmetry, negative externalities and a collective action problem among competing developers (a safety race to the bottom). The last of these is to be addressed through common safety standards, not through any relaxation of antitrust rules.