Anthropomimetic Uncertainty in AI

• Anthropomimetic uncertainty is a concept where language models communicate uncertainty using human-like verbal cues rather than solely numerical confidence.
• It integrates calibration, pragmatics, and personalization to improve reliability, trustworthiness, and user-centric interaction in AI systems.
• Methodologies such as prompt elicitation and reinforcement fine-tuning address issues like overconfidence and inconsistent uncertainty signaling in current models.

Searching arXiv for the specified papers to ground the article in current literature. {"query": "\"Anthropomimetic Uncertainty\" (Ulmer et al., 11 Jul 2025)", "max_results": 5, "sort_by": "submittedDate", "sort_order": "descending"} {"query": "\"Human Uncertainty in Concept-Based AI Systems\" (Collins et al., 2023)", "max_results": 5, "sort_by": "submittedDate", "sort_order": "descending"} {"query": "\"The Ghost in the Grammar\" (Costa, 24 Feb 2026)", "max_results": 5, "sort_by": "submittedDate", "sort_order": "descending"} {"query": "\"Modeling Human Behavior Part II -- Cognitive approaches and Uncertainty\" (Fuchs et al., 2022)", "max_results": 5, "sort_by": "submittedDate", "sort_order": "descending"} Anthropomimetic uncertainty denotes a research program in which LLMs are expected not only to quantify “how sure” they are, but to communicate uncertainty in ways that adhere to norms in human uncertainty communication and are user-centric, with the stated aim of improving reliability, trustworthiness, and personalization in human–machine dialogue (Ulmer et al., 11 Jul 2025). In this formulation, uncertainty is not treated solely as a scalar confidence estimate; it is expressed through linguistic form, pragmatic timing, interpersonal positioning, and contextual adaptation. The topic therefore sits at the intersection of calibration, verbalized uncertainty, pragmatics, human–computer interaction, and broader debates about anthropomorphism in AI.

1. Definition and conceptual scope

Ulmer et al. formally define Anthropomimetic Uncertainty as “the adherence of LLMs to norms in human uncertainty communication that is user-centric in order to improve reliability, trustworthiness, and personalization” (Ulmer et al., 11 Jul 2025). The proposal is explicitly motivated by a mismatch between the confidence with which LLMs often present outputs and the questionable accuracy of many of those outputs. In this setting, verbalized uncertainty is the expression of confidence with linguistic means, and the central claim is that language-based interfaces are especially well suited to this mode of uncertainty signaling.

Within this framework, an AMU-aware model does not merely emit a number such as “60 % certain.” It selects among numerical formats, verbal hedges, and integrated phrasings such as “It seems likely” or “I could be mistaken here, but…” according to pragmatic and interpersonal cues documented in human communication research (Ulmer et al., 11 Jul 2025). The proposal therefore expands the target of uncertainty modeling from calibration alone to a broader communicative competence.

A related but distinct use of the same phrase appears in concept-based AI. In that literature, “modeling human-style uncertainty in concept-based models” concerns uncertain human interventions on interpretable concepts, soft labels, and interactive prediction rather than verbal hedging by LLMs (Collins et al., 2023). This suggests that anthropomimetic uncertainty can function as a family resemblance term across AI subfields, but the primary contemporary usage centers on verbalized uncertainty in LLMs.

2. Foundations in human uncertainty communication

The AMU proposal is grounded in a long line of work on human uncertainty communication. Ulmer et al. locate its foundations in Gricean cooperation, epistemic markers, politeness, epistemic vigilance, and the distinction between numerical and verbal registers (Ulmer et al., 11 Jul 2025). In conversational theory, speakers balance truthfulness, informativeness, relevance, and clarity. Under uncertainty, a cooperative speaker signals limited commitment so that the hearer neither overreacts nor underreacts.

Epistemic markers such as might, maybe, certainly, and probably are treated as mechanisms for hedging or strengthening claims. Politeness norms further predict more hedging toward higher-status interlocutors or when stakes are high. In parallel, listeners are described as exercising “epistemic vigilance,” weighing source credibility and content plausibility; a phrase such as “I’m not entirely sure” can trigger scrutiny, whereas “Definitely” can invite acceptance until contradicted (Ulmer et al., 11 Jul 2025).

A key point in this literature is that uncertainty expression is register-dependent. People may choose numerical forms, such as “there’s a 40 % chance,” or verbal hedges, such as “it’s possible.” Both are described as inherently imprecise, and the data block notes that people map “possible” anywhere from 17 %–89 %. At the same time, the two registers carry different pragmatic stances and are preferred in different domains, including medicine and intelligence analysis (Ulmer et al., 11 Jul 2025). The AMU agenda extends this observation into NLP by asking whether models can emulate the same sensitivity to relationship, stakes, and conversational context.

Broader cognitive modeling work supplies an adjacent theoretical background. Fuchs et al. describe uncertainty in human reasoning as a dynamic interplay of memory decay, heuristic shortcuts, attentional filters, and quantum-like interference effects, rather than a single unambiguous distribution (Fuchs et al., 2022). This broader account does not address verbalized uncertainty in LLMs directly, but it reinforces the general premise that human uncertainty is structured by bounded rationality, resource constraints, and context-sensitive cognition.

3. Failures of current LLMs

The principal empirical motivation for anthropomimetic uncertainty is that current LLMs miscommunicate uncertainty in systematic ways. The survey summarized by Ulmer et al. identifies three recurrent problems: models overstate confidence, vary unpredictably with topic and prompt, and mismatch verbal forms to actual accuracy (Ulmer et al., 11 Jul 2025).

Several concrete findings illustrate the problem. In the journalism-review study by Jazwińska and Chandrasekar, ChatGPT erred 67 % of the time when identifying news-article snippets, yet used hedges in only 7.5 % of its responses and never declined to answer. Krause et al. annotated English, Amharic, German, Spanish, Hindi, and Dutch outputs and found that markers such as “probably” and “might” appeared with nearly uniform frequency regardless of whether the answer was correct. Ulmer et al. further report that, when the same quiz questions were posed under different social “roles” such as Employee→Boss, Citizen→Monarch, and Suspect→Interrogator, six commercial models alternated between never expressing uncertainty, appending a numerical estimate, or adding a phrase such as “I could be wrong.” In some hierarchical scenarios, the models hedged almost always; in others, almost never (Ulmer et al., 11 Jul 2025).

The paper attributes part of this behavior to data-inherent biases. In pretraining corpora including The Pile, news, Reddit, books, and legal texts, round multiples of 5 such as 50 % and 100 % dominate numeral use. In instruction-tuning datasets, strengtheners such as “certain” and “definitely” overwhelmingly outnumber weakeners. RLHF reward models are reported to amplify these tendencies by assigning higher reward to unhedged, confident statements (Ulmer et al., 11 Jul 2025). A plausible implication is that uncertainty miscommunication is not merely a decoding artifact; it is also a corpus and optimization artifact.

The concept-based modeling literature shows an analogous issue on the human side of human–AI interaction. Prior work often assumes that humans are oracles who are always certain and correct, whereas real-world decision-making is prone to occasional mistakes and uncertainty (Collins et al., 2023). This parallel matters because AMU is not only about making models sound more human; it also concerns building systems that can interact with the actual uncertainty profiles of human collaborators.

4. Methodological approaches

Existing work on verbalized uncertainty in LLMs falls into two main paradigms: prompt elicitation and model fine-tuning or reinforcement (Ulmer et al., 11 Jul 2025). Prompt elicitation includes simple prompts asking for a 0–100 confidence score or instructing the model to use words like “maybe” or “probably,” chain-of-thought variations that interleave reasoning steps with self-assessed confidence, and post-answer injection in which a generated answer is followed by a phrase such as “I am X % sure” or “I could be wrong, but…”.

Fine-tuning approaches first estimate $\hat p$ through an external method such as token-level log-probabilities, true/false token probabilities, or self-consistency sample variance; map $\hat p$ to discrete markers or numerals; include those in training pairs; and then train on augmented answer-plus-uncertainty examples. Reinforcement-based approaches optimize directly for better alignment between expressed uncertainty and eventual correctness. LACIE uses a “listener” model to prefer responses whose expressed uncertainty best matches eventual correctness, with those preferences driving Direct Preference Optimization. Band et al. train with a PPO reward that penalizes overconfidence and rewards calibrated hedging. Xu et al.’s “Sayself” combines SFT with PPO on a reward measuring alignment between $\hat p$ and answer correctness, while also generating token-level rationales for why the model is uncertain (Ulmer et al., 11 Jul 2025).

Ulmer et al. state that none of these systems yet attempt full anthropomimetic personalization, meaning adaptation of hedging style to a specific user’s background or communication history, but they do show that models can learn to insert lexical hedges in fluent answers (Ulmer et al., 11 Jul 2025). This is a limited but important distinction: present systems mainly learn better uncertainty insertion, not yet the broader user-centric communicative competence implied by AMU.

In concept-based AI, the methodological analogue is “training with uncertainty.” The same joint CBM/CEM loss is used, but concept labels $c_i$ may be soft values in $[0,1]$ rather than hard binary labels. Test-time interventions may likewise be uncertain, replacing $\hat c_i$ with a human-supplied probability rather than an oracle value. On UMNIST, CheXpert, and CUB-S, this setup is used to study how interactive models behave when human collaborators provide uncertain concept interventions rather than certain corrections (Collins et al., 2023). Although this is not a language-model framework, it enlarges the methodological landscape of anthropomimetic uncertainty by treating human uncertainty as a first-class training and inference object.

5. Metrics and empirical assessment

Progress toward AMU is evaluated along multiple axes rather than a single benchmark dimension. The main paper emphasizes calibration, linguistic alignment, register usage, and context sensitivity (Ulmer et al., 11 Jul 2025).

Metric or axis	Definition from the literature	Reported significance
Frequentist Calibration	$p(y=\hat y \mid \hat p)=\hat p$	Perfect calibration requires reported confidence to match empirical correctness
ECE	$ECE=\sum_k |acc(B_k)-conf(B_k)|\cdot |B_k|/N$	Measures calibration error over confidence bins
Yule’s Y	$Y=(\sqrt{ad}-\sqrt{bc})/(\sqrt{ad}+\sqrt{bc})$	Measures alignment of weakeners and strengtheners with wrong and right answers
Register usage rates	Fraction using no marker, numerical percentages, discrete verbal markers, or fully integrated hedging	Distinguishes post-hoc uncertainty from fluent integration
Domain and context sensitivity	Variation across topics and conversational roles	Tests whether uncertainty expression is stable under reframing

Yule’s Y is defined using $a=$ the number of weakeners with wrong answers, $\hat p$0 the number of weakeners with right answers, $\hat p$1 the number of strengtheners with wrong answers, and $\hat p$2 the number of strengtheners with right answers. It ranges from $\hat p$3 to $\hat p$4, where $\hat p$5 corresponds to all strong language on wrong answers and $\hat p$6 to perfect alignment of strengtheners with correct answers. Ulmer et al. report that across contexts, $\hat p$7 frequently hovers near zero or becomes negative, indicating weak or anti-correlated linguistic calibration (Ulmer et al., 11 Jul 2025).

The same paper reports that, in a survey of 40 papers since 2022, 61 % rely solely on numerical registers, 23 % tack verbal markers on post hoc, and only 16 % attempt fluent integration. It also reports that, across TriviaQA and MMLU questions spanning eight topics and ten conversational roles, models show ECE swings from 10 % to 50 % when only the social frame is changed, and that even when one context yields $\hat p$8, another yields $\hat p$9; Yule’s Y likewise varies between $\hat p$0 and $\hat p$1 (Ulmer et al., 11 Jul 2025). These values support the claim that uncertainty behavior is highly frame-sensitive.

For large-scale correctness labeling, the paper combines two heuristics: a ROUGE-L threshold $\hat p$2 against the gold answer, and a secondary GPT-4o prompt that judges whether the model response matches the reference using an ultra-lenient “YES/NO” quiz-judge prompt (Ulmer et al., 11 Jul 2025). In the concept-based literature, calibration is also central: annotator ECE is computed as $\hat p$3, and many annotators were observed to over-assign weight to 0 % or 100 % and to default to 50 % when uncertain, which is described as an interface artifact (Collins et al., 2023).

6. Related interpretations, controversies, and research directions

Anthropomimetic uncertainty is associated with at least two different interpretive directions. In the LLM communication literature, it is a constructive proposal: models should emulate human uncertainty communication so that their output becomes more intuitive and trustworthy (Ulmer et al., 11 Jul 2025). In Costa’s philosophical analysis of AI safety evaluations, by contrast, a related formulation designates an epistemic opacity introduced when researchers project human-like agency onto LLMs through grammar and experimental design (Costa, 24 Feb 2026). There, “anthropomimetic uncertainty” arises not from insufficiently human uncertainty expression, but from excessive anthropomorphic framing.

Costa argues that subject–predicate grammar can encourage researchers to describe models as if they “want” or “intend,” despite the fact that LLMs generate text via conditional probabilities. The essay treats outcomes such as the “Alex” blackmailer and “Claudius” shopkeeper cases as products of prompt-engineered coherence and structural incoherence rather than evidence of genuine self-preservation or identity crisis. It proposes alternative concepts such as the Global Field, Normative Layer, Local Field, coherence $\hat p$4, and structural incoherence, and recommends field-based metrics, prompt auditing, de-anthropomorphization, and an uncertainty term $\hat p$5 for variance under small prompt perturbations (Costa, 24 Feb 2026). A central controversy therefore concerns the boundary between useful emulation of human communicative norms and misleading attribution of human-like interiority.

The future directions proposed by Ulmer et al. decompose AMU into six research avenues: consistency across utterances, personalization to user profiles, dynamic register selection, multilingual and cross-cultural hedging, explanations of uncertainty, and movement beyond classical calibration toward human-centered notions such as trust trajectories over time and synergy with empathy or accountability cues (Ulmer et al., 11 Jul 2025). In concept-based AI, corresponding open challenges include complementarity between model and human uncertainty, miscalibration of annotators, and elicitation cost for rich soft labels (Collins et al., 2023). In the broader cognitive modeling literature, limitations include domain dependence, parameter sensitivity, partial coverage of human uncertainty, and brittleness of individual frameworks, while potential extensions include meta-learning of cognitive parameters, hybrid architectures, deep quantum-inspired networks, and lifelong modeling of shifting uncertainty under expertise acquisition (Fuchs et al., 2022).

Taken together, these strands portray anthropomimetic uncertainty as a technically heterogeneous but conceptually coherent research area. Its unifying concern is that uncertainty in AI systems should be treated not only as a latent probabilistic state, but as a phenomenon embedded in interaction, representation, and interpretation. This suggests that the mature form of the field will likely require simultaneous advances in calibration, pragmatics, personalization, multilinguality, interface design, and methodological restraint regarding anthropomorphic projection (Ulmer et al., 11 Jul 2025).