How context matters: Human oversight of automated decision-making systems in welfare administration

This article examines the capacity of frontline staff to oversee automated decision-making (ADM) systems, which are increasingly used in digital welfare states to make life-altering decisions. While prior research had focused on cognitive limitations that lead to human over-reliance on ADM, the role of contextual factors that also shape caseworkers’ supervisory activities has neither been conceptualised, nor systematically investigated. To address this gap, we develop an analytical framework for the context-sensitive study of frontline oversight, inspired by the street-level bureaucracy perspective and human–computer interaction studies. We also demonstrate the framework’s relevance through findings from a mixed-methods study of a profiling algorithm used by Public Employment Services in Poland. We identify four types of factors – policy-, organisation-, professionalism-, and technology-related – that effectively shape frontline oversight. Our findings also have practical implications, as the inclusion of humans in the decision-making loop is a central element of regulatory efforts aimed at protecting individuals from algorithmic harms.