What is Human in Judgment? Testing Automation Bias and Algorithm Aversion Among United States Military Academy Cadets

Abstract: Human judgment has always been central to conflict and escalation, but how will a world of AI change the role of humans in war? As militaries increasingly adopt AI-enabled decision-support systems (DSS), including the United States in the war against Iran, concerns about automation bias -- over-reliance on algorithmic recommendations -- and algorithm aversion -- premature distrust of automated outputs -- raise fears that relying on AI too much could increase the risk of error, miscalculation, and accidents. Yet existing evidence on how militaries actually interact with AI remains limited. We test theories about the susceptibility of militaries to automation bias by comparing the results from a survey experiment conducted with 236 cadets at the United States Military Academy at West Point to a demographically similar cross-national public sample. Respondents completed a target identification task and then received advice from either an algorithm or a human analyst and had the opportunity to re-assess their initial identification, allowing direct measurement of automation bias and algorithm aversion. Contrary to prominent concerns, we find that West Point cadets are less prone to cognitive distortion than members of the general public, displaying better calibrated trust in algorithmic decision support systems. While the findings are limited, they suggest that military education and exposure to AI can meaningfully shape how AI influences international politics in matters of war and peace.

• The paper finds that USMA cadets exhibit a significantly lower rate of automation bias (3.9% error rate) compared to the general public (9.0%) in AI-assisted decision tasks.
• The study employs a within-subject design with ten aircraft identification scenarios, comparing outcomes based on human versus AI recommendations.
• The paper indicates that targeted AI training reduces automation bias but leaves algorithm aversion unchanged, suggesting distinct intervention strategies are needed.

Testing Automation Bias and Algorithm Aversion in Military Decision-Making: A Comparative Analysis of USMA Cadets and the General Public

Introduction

This study addresses a critical concern in the application of AI-enabled Decision Support Systems (DSS) in military contexts: the extent of human susceptibility to automation bias and algorithm aversion. The integration of AI in military operations, from logistics to targeting and command and control, has prompted questions about how future officers interact with automated systems in high-consequence environments. The research contrasts cadets at the United States Military Academy (West Point) with a demographically matched general public sample, focusing on actual behavioral responses in a target identification task aided by either an algorithmic or human recommendation.

Methodological Framework and Experimental Design

The experimental paradigm employs a pre-post, within-subject design involving ten aircraft identification scenarios with increasing difficulty. Participants made initial classifications, received advice labeled from either a human analyst or an AI algorithm (with randomized confidence and correctness), and had the opportunity to revise their decisions. This structure allows direct behavioral observation of (1) automation bias—over-reliance on erroneous algorithmic output, and (2) algorithm aversion—distrust or discounting of correct algorithmic recommendations.

The West Point sample consists of 236 cadets enrolled across disciplines, all with substantial exposure to AI concepts as mandated by their training and academic environment. The general public comparator includes young, university-educated respondents from a prior large-scale cross-national survey, ensuring demographic alignment and controlling for confounders such as STEM background and general AI familiarity.

Population Differences: Background, Sentiments, and Pre-Task Calibration

West Point cadets display systematically higher AI background indices across knowledge, practical experience, and familiarity compared to the general public.

Figure 1: AI indicator values clearly show elevated knowledge, experience, and familiarity with AI among West Point cadets relative to the general public.

Trust, sentiment, and perception of AI further delineate the two groups. While both express positive views and willingness to use AI, cadets are distinguished by higher recognition of AI's risks, lower attribution of malice (e.g., “sinisterness”), and heightened critical caution regarding claims of algorithmic superiority.

Figure 2: Net perceptions of AI are more positive but also more critically calibrated among West Point cadets, who acknowledge both potential and limits.

This suggests that the West Point sample starts from a position of justified confidence, being attentive to both the strengths and limitations of AI systems—a key theoretical prerequisite for reducing automation bias.

Behavioral Outcomes: Switching, Bias, and Aversion

Switching rates (revising initial identification post-suggestion) capture responsiveness to DSS recommendations. While the general public switches more frequently and is more sensitive to source framing (AI vs. human), West Point cadets switch less and are primarily modulated by explicit confidence cues, not merely the automation frame.

Figure 3: Switching frequencies are lower among cadets and strongly modulated by recommendation confidence; the general public is more susceptible to generic automation cues.

Quantitative task accuracy analysis demonstrates strong separation in automation bias rates: only 3.9% of West Point cadet trials involved following an incorrect AI recommendation (vs. 9.0% in the public sample, $p<0.001$). However, algorithm aversion—rejecting correct AI advice—remains comparable between populations (44.4% vs. 45.1%).

Figure 4: Cadets exhibit a sharply reduced probability of automation bias, but similar levels of algorithm aversion compared to the public.

Figure 5: Regression-adjusted predicted probabilities show a persistent and substantial reduction in automation bias likelihood among cadets.

This indicates that cadets are not only less prone to over-trusting AI but are also not systematically more dismissive of correct algorithmic output, contradicting hypotheses that military training uniformly boosts either over-reliance or reflexive rejection of AI recommendations.

Theoretical and Practical Implications

The data challenge common arguments in the military AI DSS literature positing that increased automation in high-consequence domains inevitably produces heightened automation bias through cognitive over-delegation and diminished active monitoring. Instead, the sample of USMA cadets, representing future U.S. Army officers with explicit AI education and operational training, shows better calibration and a lower tendency to outsource judgment to algorithms. Crucially, while automation bias can be attenuated by targeted training and exposure, algorithm aversion appears more resistant, potentially requiring distinct interventions—thus, operational policies must treat these phenomena as orthogonal.

From a systems design and policy standpoint, these results imply that effective mitigation of automation bias is achievable with sufficient investment in operator training, scenario-based exposure, and curricula emphasizing both algorithmic strengths and limitations. However, top-down procedures mandating algorithmic deference can still induce risk—bias is a function of organizational doctrine as much as individual cognition. Broader implications pertain to crisis decision-making models and deterrence—reliance on military practitioners’ cognitive calibration, modulated by technical training, may reduce escalation risks when DSS recommendations are imperfect.

Limitations and Prospective Research Directions

The scope is limited to USMA cadets and does not encompass non-commissioned officers, senior leadership, or other branches. The task domain—aircraft identification under time pressure—captures only commission errors (acting on incorrect advice) and not omission (failing to act absent automation cues). Comprehensive future studies should incorporate field operators from diverse military roles, measure longitudinal effects of AI integration, and extend to team-based or adversarial decision contexts.

Conclusion

Direct experimental evidence contradicts the assertion that military AI DSS adoption necessarily increases automation bias among operators. Training and technical familiarity markedly reduce the risk; however, algorithm aversion remains unaffected. The results advocate for differentiated, evidence-based mitigation strategies and highlight the continued necessity of evaluating both human and organizational factors for safe and effective AI integration into military decision-making processes.