Causal Stories from Sensor Traces: Auditing Epistemic Overreach in LLM-Generated Personal Sensing Explanations

Abstract: LLMs are increasingly used to explain personal sensing data, translating traces of activity and mood into natural-language accounts of why an anomalous day may have occurred. However, such explanations can sound coherent and personally meaningful even when the underlying evidence is sparse or missing. We introduce epistemic overreach (EO) as a measure for cases where a generated explanation implies more than the available sensing evidence can justify. To audit how often and in what forms EO occurs, we obtained anomalous-day scenarios from three longitudinal sensing datasets of college students: StudentLife, GLOBEM, and CollegeExperience. Across activity, sleep, and affect anomalies, we generated 14,922 explanations using three LLM families -- Llama, Qwen, and GPT -- under two prompting conditions: one minimally constrained prompt and another prompt explicitly instructing models to bound claims to the data. For each scenario, we varied the amount of behavioral evidence available to the model to examine whether more evidence reduces EO. We evaluated each explanation using a structured rubric, decomposing EO into the dimensions of unsupported causal attribution, unacknowledged data gaps, overconfident language, temporal inconsistency, and diagnostic inference. We find that LLMs routinely attribute anomalous days to causes without sufficient support from the data, and that this pattern replicates across datasets, anomaly types, and model families. Further, providing richer context does not reliably reduce EO; bounded prompting helps but does not eliminate it. These findings suggest that evidential grounding should be a first-order evaluation criterion for LLM-generated personal sensing explanations, alongside fluency and plausibility. We argue that personal sensing explanations require evidential discipline: systems must distinguish what is observed, what is inferred, and what remains unknown.

• The paper introduces epistemic overreach, showing that LLMs generate plausible yet unsupported causal and diagnostic narratives from sensor data.
• It employs a tiered evidence framework and dual prompting strategies to reveal how richer context can both ground and spur speculative inferences.
• Findings indicate that even evidence-bounded prompts only partially reduce overreach, highlighting the need for rigorous, evidence-aware explanation models.

Causal Stories from Sensor Traces: Auditing Epistemic Overreach in LLM-Generated Personal Sensing Explanations

Introduction

This work interrogates the evidential validity of explanations generated by LLMs describing personal sensor data—specifically, when these explanations ascribe meaning, causality, or psychological states to behavioral anomalies (e.g., in sleep, activity, mood) absent adequate or unambiguous evidence. The paper introduces and formalizes epistemic overreach (EO), defined as the phenomenon where model-generated accounts imply more than the underlying observational data can substantiate. The central insight is that, while LLMs can produce explanations that are fluent and plausible, these explanations often project causal or psychological narratives not justifiable from the available traces, thus raising significant interpretability, trustworthiness, and user impact concerns in personal informatics.

Study Design and Methodology

The authors construct a rigorous empirical audit framework for EO, leveraging three longitudinal behavioral sensing datasets from college student cohorts ("StudentLife," "GLOBEM," "CollegeExperience"). For each dataset, they detect anomalous participant-relative days in core behavioral metrics (activity, sleep, affect) and define three nested evidence tiers:

• E1: Core behavioral signals (activity, sleep, affect)
• E2: E1 plus interaction features (phone use, conversation, various environmental sensors)
• E3: E1/E2 plus contextual information (calendar events, academic context, GPS-derived features)

They systematically vary both the amount of accessible evidence (tiered) and the prompting policy: either (i) unconstrained, or (ii) “evidence-bounded,” which instructs the model to restrict causal/diagnostic inferences to what the evidence directly supports, acknowledge missingness, and distinguish observation from interpretation.

Explanations are generated for each scenario using three representative LLM architectures (Llama-3.2-3B, Qwen-2.5-7B, and GPT-5-nano). The total experimental matrix spans 14,922 explanations.

Critically, outputs are audited via a fine-grained rubric structured into five EO dimensions:

1. Causal attribution overreach (unsupported causes, treating associations as causal, speculative mechanisms)
2. Missing-context overreach (failure to acknowledge missing data, treating missingness as normal, unobserved-context assumptions)
3. Confidence overreach (overstated certainty, insufficient hedging, overgeneralized conclusions)
4. Temporal inference overreach (temporally incoherent claims, cause-effect errors)
5. Diagnostic inference overreach (unsupported inferences about mental or physical conditions, psychological state attribution, or unwarranted clinical escalation)

An automated LLM-as-judge framework, supplemented with human validation, assigns binary scores for each rubric item, aggregated into a normalized EO score per explanation.

Main Results

(1) Prevalence and Character of Epistemic Overreach

EO is pervasive: mean EO scores under unconstrained prompting range from 0.139 (GLOBEM) to 0.168 (StudentLife), corresponding to over two dimensions of overreach per explanation (out of 16). Notably, EO is not limited to rare hallucinations or single failure types but is routinely distributed across multiple dimensions—predominantly unwarranted causal attribution and overconfident language.

Model-wise, Llama-3 series exhibits the lowest EO, while Qwen and especially GPT-5-nano tend to construct more expansive, narrative-styled explanations prone to overreach, especially when more context is available.

(2) Effects of Evidence Tier and Prompt Policy

Increased evidence availability does not monotonically reduce EO. While richer evidence (E2, E3) at times anchors explanations and marginally reduces some forms of overreach, it also enables new unsupported attributions—particularly when additional context yields more opportunities for plausible but unsubstantiated inference.

Evidence-bounded prompting reliably suppresses EO overall (up to 62% reduction for GPT-5-nano, moderate effect for Qwen and Llama-3.2), but does not eliminate the problem. Residual EO persists, especially in causal and diagnostic dimensions, even when models explicitly hedge and acknowledge uncertainty.

(3) Breakdown by Anomaly Type, Dataset, and Model

EO occurs across all anomaly types (activity, sleep, affect) and datasets. The highest EMI (Epistemic Misinterpretation Index) risk is observed not in the sparsest data settings, but when context is sufficiently rich to support fluent narrative constructions yet still underdetermines causal or psychodiagnostic claims.

Dimension analyses reinforce that temporal coherence is rarely violated, and missing-context errors are infrequent. The operational risk lies in the assertion of internal states or causal relations not warranted by the direct evidence.

Theoretical and Practical Implications

The findings have significant implications for both the design and assessment of LLM-mediated personal informatics:

• Evidential grounding must be a primary evaluation criterion. Surface-level qualities (plausibility, fluency) are insufficient; systems must be audited for how explanations map onto observable evidence versus inference. Treating LLM output as authoritative or reflective of underlying causes or states when such evidence is lacking is inappropriate, especially in clinical or behavioral health contexts.
• More context and stricter prompts are only partial solutions. While both reduce EO, neither suffices to guarantee evidential discipline. Systems must explicitly structure the separation of observation versus inference, possibly adopting response templates that enumerate observed, missing, weakly suggested, and non-inferable elements.
• The main epistemic failure mode is the (over)integration of partial and ambiguous behavioral data into causal narratives or psychodiagnostic claims, which are likely to influence user self-attribution, recall, and potentially reinforce unwarranted beliefs.
• Design of user interfaces should expose evidential boundaries, allowing users to distinguish between what is supported, what is plausible, and what is not inferable. Interaction designs supporting contestation and annotation/feedback are warranted.
• Metrics for LLM explanations in personal informatics should incorporate dimension-level EO auditing rather than rely solely on conventional quality or user satisfaction outcomes.

Limitations and Directions for Future Research

The analysis is limited to three LLMs and three student-centric datasets. The study does not target the ultimate “truth” of the cause of behavioral anomalies (often unknowable), but rather the evidential discipline of the generated explanation. The generalizability of findings to other populations, sensor setups, or real-time user interactions remains to be empirically established.

Future work should examine:

• More aggressive or programmatic mitigation (e.g., response post-processing, claim verification, retrieval-augmented/guided generation, structured explanation formats).
• Longitudinal user studies quantifying real-world effects on reflection, self-understanding, and trust calibration.
• Expansion to richer or less-structured behavioral domains, additional LLMs, and more complex causal inference targets.

Conclusion

This paper establishes epistemic overreach as a prevalent and multi-faceted failure mode in LLM-generated explanations for personal sensing data. The deployment of LLMs in personal informatics should be contingent on the explicit auditing and mitigation of EO. The work highlights the necessity for evidence-aware explanation generation, structured model-evaluation frameworks tailored to evidential validity, and transparent user interface designs that distinguish observation from inference.

The argument is clear: providing plausible, narrative explanations is not the same as providing truth-conditional or evidence-justifiable accounts, and the distinction is paramount when algorithmic explanations concern an individual's own life and wellbeing.

---

Citation:

Causal Stories from Sensor Traces: Auditing Epistemic Overreach in LLM-Generated Personal Sensing Explanations (2605.08590)