Effects of training operators on situation-specific automation reliability

Decision support automation's reliability can vary predictably according to situation, resulting in inappropriate trust in automation, and possible performance decrements. Training might ameliorate these problems. In a realtime simulation, air traffic controllers detected aircraft conflicts, assisted by an automated tool which was very reliable in normal situations, but less so in Free Flight (FF) scenarios where certain aircraft deviated from their flight plans. One group of participants was given a small amount of pre-experiment training, being informed that the automation reliability would suffer in these scenarios. As hypothesized, subjective trust was lower in FF for the trained participants. Overall performance did not differ, but the trained group were more likely to detect both real and perceived conflicts (bias shift). Also, they were more likely in general to unquestioningly accept the automation's judgments (which in general was appropriate), as assessed by a new experimental scale of self-reported use-of-information. Only the non-trained group showed a relationship between subjective trust and unquestioning acceptance of the automation's judgments on the new scale.