Offloading Score: Measuring AI Reliance Through Counterfactual Workflows
Abstract: AI tools are increasingly integrated into real-world workflows. However, existing measures of reliance on these tools focus on AI output adoption or on self-reported indicators, rather than how task effort is distributed between users and tools. Here, we introduce offloading score, a measure of reliance that quantifies the fraction of cognitive effort offloaded to an AI tool. Offloading Score is simulation-based -- we construct a counterfactual workflow by estimating how the user would have completed the task without the tool, and then computing the fraction of steps saved by using the tool. We validate offloading score through intrinsic evaluations of metric validity, and a controlled user study ($n=40$) with developers performing programming tasks using AI tools. We vary time pressure to test whether reliance measures capture the known increase in reliance under time pressure. We show that offloading score detects significantly higher reliance in time-constrained settings ($+43\%$, $p=0.018$), while usage-based and self-reported baseline measures of reliance do not distinguish the conditions. We complement this with descriptive insights showing that higher reliance manifests as greater delegation of subtasks to the tool and more direct reuse of AI outputs. Finally, we demonstrate an approach of using offloading score in combination with target outcomes of a task (e.g., code understanding) to identify when reliance may be (in)appropriate. Our framework offers two contributions: an instrument users can apply to measure and reflect on their own reliance, and a quantitative signal that agent designers can utilize to mitigate overreliance.
Summary
- The paper introduces the Offloading Score metric that measures AI reliance by comparing observed workflows with counterfactual human-only sequences.
- The paper employs rigorous counterfactual simulation to evaluate reliance differences under varying conditions, validated through empirical studies on programming tasks.
- The paper demonstrates that the Offloading Score differentiates cognitive effort levels and informs design strategies to mitigate overreliance on AI.
Process-Oriented Measurement of AI Reliance: Offloading Score and Counterfactual Workflows
Motivation and Context
The proliferation of AI tools across cognitive domains, notably in software engineering, necessitates refined methodologies for quantifying user reliance. Existing approaches—primarily output-based adoption metrics and subjective self-report instruments—are inadequate for multi-turn, tool-mediated workflows characterized by non-trivial delegation of cognitive effort. These limitations impede the design of robust, human-compatible AI agents and exacerbate risks related to deskilling, overreliance, and degradation of independent problem-solving.
The paper "Offloading Score: Measuring AI Reliance Through Counterfactual Workflows" (2605.29392) introduces a rigorous process-oriented metric, Offloading Score, designed to capture the fraction of cognitive effort offloaded to an AI tool in realistic, non-deterministic workflows. This metric leverages counterfactual simulation: it estimates, for each AI-assisted step, the sequence of human-only workflow steps that would have been required, then computes the proportion of steps saved due to AI intervention.
Figure 1: Offloading score conceptualization, computing the scalar measure of cognitive effort offloaded to AI via counterfactual workflow expansion.
Formalization and Measurement Framework
Workflow Induction and Counterfactual Construction
Workflows are operationalized as ordered sequences of steps encompassing planning, execution, and verification. Each step is annotated with its cognitive process type (e.g., planning, execution, feedback, control) and its mode of engagement with AI outputs (reuse, adapt, pushback, reject).
For a given workflow , the Offloading Score is calculated by replacing every AI-assisted step with a simulated human-only alternative sequence, yielding a counterfactual workflow . The score is defined as , where and are the number of steps in the observed and counterfactual workflows, respectively.
Figure 2: Offloading score computation pipeline—expansion of AI-assisted steps into counterfactual human-only sequences and the resulting proportion of steps saved.
Workflow induction is instantiated through action trace segmentation, VLM annotation, and LLM-based grouping, producing natural language descriptions of user activity aligned with cognitive process categories.
Figure 3: Detailed calculation of Offloading Score from raw interaction traces with workflow induction and counterfactual reconstruction.
Validity and Empirical Evaluation
Metric Validity
Offloading Score undergoes comprehensive validation:
- Content validity: Human raters agree (>85% plausible ratings) with AI-generated counterfactual step mappings.
- Construct validity: The metric is sensitive to reductions in AI-assisted steps and stable across irrelevant perturbations (e.g., paraphrasing, model variation).
- Criterion validity: In controlled experimental manipulations (short vs. long time constraints), Offloading Score displays significant discrimination (+43%, ), outperforming baseline measures (AI output adoption, self-reported cognitive load).
Baseline Comparisons
Empirical studies with 40 experienced developers on programming tasks reveal that Offloading Score is the only metric to significantly differentiate reliance between high-pressure and low-pressure conditions. AI code fraction approaches significance but is less robust (). Self-reported cognitive load lacks discriminatory power ().
Figure 4: Comparison of reliance measures—Offloading Score versus baseline metrics across experimental conditions.
Behavioral Characterization
Descriptive dimensions supplement the scalar metric. High Offloading Score manifests as increased direct delegation and output reuse, whereas lower scores correspond to selective engagement (output rejection/adaptation, increased planning-oriented use).
Figure 5: Process and output-use distributions—high time pressure yields execution-centric and reuse-dominated behavior, low pressure fosters selective adaptation.
Association with Task Outcomes: Normative Judgment
Reliance vs. System Understanding
A critical test of reliance appropriateness is its association with task outcome, exemplified by post-task code understanding ("system recall"). Offloading Score exhibits a general negative correlation with recall (–0.145), stronger than other reliance measures. Excluding high-recall outlier clusters, the correlation strengthens to –0.440, enabling operational thresholds for overreliance (Offloading Score > 0.33).
Figure 6: Reliance (Offloading Score) versus system recall—identification of overreliance boundaries and outlier effective users.
Notably, a distinct cluster demonstrates high reliance coupled with high recall, predominantly among users employing the tool for educational augmentation rather than mere execution. This finding underscores the necessity for context-sensitive thresholds; high reliance is not universally problematic when user intent is skill acquisition.
Technical Breakdown and Supplementary Findings
Workflow and Tool Usage Statistics
Long condition workflows are marginally longer, with lower AI-assisted step proportions, and a broader distribution of process engagement. Tool diversity is significant, with Claude and ChatGPT dominating, but no clear pattern links tool choice to reliance—user behavior is the primary determinant.
Figure 7: Workflow step statistics—distribution of AI-assisted versus human-only steps across conditions.
Figure 8: Primary tool usage per workflow—distribution by keyword mentions.
Figure 9: Workflow mentions of AI coding tools—tool adoption frequency across datasets.
Embedding and Clustering
t-SNE visualization of workflow embeddings validates that synthetic counterfactual workflows cluster with human workflows from the same task, substantiating content validity.
Figure 10: t-SNE workflow embeddings—synthetic counterfactuals cluster with human workflows by task.
Robustness Analyses
Metric robustness is supported by minimal variance across repeated runs and model settings, and clear decrement in Offloading Score with systematic reduction in AI-assisted steps.
Figure 11: Distribution variation in Offloading Score across 5 robustness runs.
Figure 12: Sensitivity—Offloading Score value decrease with controlled perturbations to reduce AI usage.
Correlations and Interactions
Offloading Score is strongly correlated with usage-based baselines but outperforms them in capturing condition effects and outcome associations.
Figure 13: Correlation with baseline measures—Offloading Score outperforms usage metrics.
Figure 14: Offloading Score correlation with experimental condition label—strongest among tested reliance measures.
Figure 15: Correlation with binary task outcome (system recall ≥ 0.33)—Offloading Score yields highest predictive validity.
Task-wise and Interaction-wise Patterns
Offloading Score reliably assigns lower mean values in longer conditions across all tasks.
Figure 16: Task-specific Offloading Score distributions—consistent lower scores in low-pressure settings.
Process and output-use label fractions confirm behavioral shifts.
Figure 17: Fraction of workflow steps with process/output-use labels—short condition dominated by execution and reuse.
Toolwise reliance shows substantial variability, with no dominant trend.
Figure 18: Offloading Score variation across AI tools—user behavior rather than tool determines reliance.
Counterfactual workflow length varies by process type and output label.
Figure 19: Counterfactual workflow length—planning steps yield longer alternatives.
Figure 20: Counterfactual workflow length—debug/pushback steps correspond to longer, more involved alternatives.
Implications and Future Work
The introduction of Offloading Score delivers dual impact: as a self-reflective instrument for users to audit reliance, and as a quantitative signal for agent designers to mitigate overreliance risks in interactive systems. Its computation from interaction traces, independent of reference outputs, facilitates portability across tools and interfaces.
Practically, this metric enables granular analysis of workflow allocation, supporting fine-tuned intervention strategies (e.g., cognitive forcing functions, dynamic agent behavior adaptation [buccinca2021trust]). Theoretically, Offloading Score opens avenues for personalized, model-driven counterfactual estimation as user modeling advances [shaikh2026learning], enhancing precision and applicability.
Formal empirical validation demonstrates its superiority in capturing reliance variations relevant to behavioral manipulation and outcome tracking—critical for designing accountability mechanisms in AI-augmented environments.
Conclusion
The Offloading Score framework constitutes a significant advance for measurement of AI reliance in complex workflows. Through rigorous counterfactual simulation and empirical validation, it addresses longstanding gaps in behavioral assessment, outperforming conventional adoption-based and subjective metrics. Its adaptability to context, outcome, and user goals imparts practical utility for both research and applied agent design in cognitive work domains. Future developments in user modeling are poised to further refine this metric, generating increasingly personalized, actionable insights into AI-human collaboration.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
What is this paper about?
This paper introduces a simple way to measure how much people “lean on” AI tools when doing thinking-heavy work like coding. The authors call their measure the Offloading Score. It estimates what share of the mental work a person hands over to an AI, compared to what they would have done on their own.
What questions did the researchers ask?
- Can we build a measure that captures not just whether someone used AI, but how much thinking the AI did for them?
- Can this measure help tell when people are relying on AI in helpful ways versus relying too much (for example, when it hurts their understanding of their own code)?
How did they do it? (Methods explained simply)
Think of solving a big homework problem. If you use an AI assistant, you might:
- Ask it to plan the steps,
- Have it write chunks of code,
- Or ask it to check your work.
The researchers studied people’s real computer activity (like keystrokes, clicks, and screenshots) while they worked on programming tasks with AI tools. They turned these raw logs into “workflows,” which are just the step-by-step paths people took to finish the task.
They then invented the Offloading Score using a “what if” idea (that’s the counterfactual part):
- Imagine the same person did the task without AI. How many steps would it take?
- Compare that to the steps they actually took with AI.
- The Offloading Score is the fraction of steps saved thanks to AI.
In everyday terms:
- If doing it alone would take 100 steps, and with AI it took 60 steps, then 40 steps were “offloaded” to AI.
- Offloading Score = 40 out of 100 = 0.40 (or 40%).
To make this “what if” estimate, they built a short, reasonable alternate workflow for each AI-assisted step—basically: “How would an average person have handled this part without AI?”
They also looked at:
- What kinds of thinking were handed to the AI (planning, execution, feedback, etc.).
- How people used AI outputs (directly reused, adapted, questioned/debugged, or rejected them).
Finally, they ran a controlled user study with 40 experienced freelance developers:
- Everyone built small web apps with any AI tools they liked.
- Half had tight time limits (1 hour), and half had more time (4 hours).
- After finishing, participants took a short quiz on how their code worked, to measure how well they understood what they built.
They checked that the Offloading Score “makes sense” by:
- Seeing if it reacts in the right direction when they remove AI-assisted steps (the score goes down).
- Seeing if it stays steady when things that shouldn’t affect reliance change (like paraphrasing a step).
- Checking whether it picks up known behavior (time pressure usually makes people rely more on tools).
What did they find, and why is it important?
- The Offloading Score went up under time pressure. With only 1 hour, people offloaded more mental work to AI than with 4 hours. The increase was large (about +43%) and statistically significant. This matches what we expect: when rushed, people rely more on help.
- Older, simpler “usage” measures—like “how many lines of AI-generated code were kept,” “how many AI interactions happened,” or “how much time was spent with the AI”—didn’t reliably tell the two time conditions apart. In other words, just counting AI usage didn’t capture true reliance as well as Offloading Score did.
- How people used AI changed with time pressure:
- Short time: more delegation of execution to AI and more direct reuse of AI outputs.
- Long time: more planning with AI, more rejecting/adapting outputs, and more user control.
- Reliance and understanding:
- In general, higher Offloading Scores were linked to worse understanding of the final system (people knew less about how their own code worked).
- But there was a notable group who used AI a lot and still had high understanding. They seemed to use the AI to learn unfamiliar things—asking questions, exploring options, and maintaining awareness—so their reliance was still “appropriate.”
Why this matters: the Offloading Score can spot when reliance changes, even when usage counts can’t. It also helps connect reliance to outcomes like understanding—so you can tell when “more help” is too much help.
What could this change in the real world?
- For students and professionals: The Offloading Score can help you reflect on your own habits. Are you delegating the hard thinking to AI so much that you stop learning? Or are you using AI to learn and still keeping track of what’s going on?
- For tool designers: The score can act like a “reliance signal.” Tools could gently nudge people if they’re slipping into overreliance (for example, by prompting them to verify or explain a step).
- For teachers and team leads: You can combine the Offloading Score with goals (like maintaining code understanding) to set healthy norms. For example, it might be fine to rely heavily on AI when exploring something new, but not fine if it means you can’t explain your own code later.
In short, this paper shows a clearer, fairer way to measure how much thinking people hand over to AI, and how to use that information to encourage better, more mindful AI use.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a focused list of unresolved issues and concrete directions for future research stemming from the paper.
- Counterfactual validity and personalization: The method assumes an “average user” to construct human-only counterfactual workflows. How does offloading score change when counterfactuals are conditioned on user expertise, prior behavior, or domain familiarity? Develop and evaluate personalized counterfactual generators.
- Step granularity dependence: Offloading score depends on the workflow induction algorithm and its step segmentation. How invariant is the metric across different segmentation schemes or tools? Benchmark against human-coded steps and alternative process-mining pipelines.
- Effort weighting vs step counting: The metric treats all steps equally, ignoring time-on-task, cognitive difficulty, or complexity. Can steps be weighted by time, keystrokes, code churn, or validated cognitive-load proxies to better reflect “effort saved”?
- AI-induced overhead and negative offloading: The formulation assumes m ≥ n (tool use saves steps). What if AI causes detours (n > m)? Define and validate a complementary “overhead score” and clarify how offloading score behaves in such cases.
- Detection of AI-assisted steps: Identification relies on an LLM over traces/screenshots; precision/recall of this detection is not reported. Build and validate robust detectors (e.g., IDE telemetry, on-device logging, code provenance) with labeled ground truth.
- LLM dependence for labels and counterfactuals: Counterfactual construction and descriptive labels (process and output-use) are produced by gpt-5.2. Quantify measurement error, inter-model variance, and drift across model versions; triangulate with expert annotators.
- Generalization beyond coding: The framework is validated on programming tasks. How well do process/output-use taxonomies transfer to writing, analysis, design, decision-making, or scientific workflows? Develop domain-specific rubrics and validate in non-coding contexts.
- Sample and context limitations: Participants were U.S.-based Upwork freelancers (N=40). Replicate with novices, students, enterprise engineers, and non-U.S. populations; assess cultural and organizational moderators.
- Manipulation scope: Only time pressure was varied. Test other drivers of reliance (e.g., accountability, auditing requirements, feedback visibility, error costs, incentives, trust calibration).
- Task difficulty confounds: Differences in reliance may reflect task difficulty or spec clarity. Pre-register difficulty ratings, randomize task–condition assignment, and include task fixed effects and power analyses.
- Outcome measures for “appropriate reliance”: The normative analysis uses system recall as a proxy for understanding with an ad hoc threshold (0.33). Validate thresholds empirically, test alternative outcomes (e.g., maintainability, defect rates, security, long-term productivity), and perform multi-objective trade-off analyses.
- LLM-as-judge validity for understanding: System recall is scored by an LLM. Compare against expert human grading, code comprehension exams, timed debugging tasks, and delayed retention tests to establish criterion validity.
- Longitudinal effects: The study is single-session. Measure how offloading score relates to skill acquisition, deskilling, retention, and future maintenance performance over weeks or months.
- Tool and interface diversity: Participants used a mix of tools (Claude, ChatGPT, Cursor, Gemini). Quantify how offloading score varies across IDE-integrated assistants, autonomous agents, and multi-tool workflows; assess tool-specific artifacts.
- Scalability and compute cost: Counterfactual generation and labeling may be expensive on long or numerous workflows. Report runtime/cost, propose amortized or retrieval-augmented approaches, and evaluate scalability to enterprise-scale telemetry.
- Reproducibility and model drift: Results may depend on gpt-5.2 behavior. Establish test harnesses with version pinning, evaluate across multiple LLMs, and report robustness to prompt and seed variations.
- Goodhart’s law and gaming: If offloading score is surfaced or optimized, users or tools may game step counts (e.g., batching or re-segmentation). Design adversarial evaluations and anti-gaming safeguards.
- Interpreting “learning mode” outliers: The high-offloading/high-recall cluster is described qualitatively. Identify behavioral signals that distinguish “learning use” from “production use,” and derive context-specific reliance thresholds for each.
- Alternative baselines and multimodal signals: Compare against process-mining metrics (e.g., control-flow complexity), information-theoretic measures (e.g., entropy reduction), eye-tracking, pupillometry, or think-aloud protocols for convergent validity.
- Silent cognition gap: The method cannot observe internal reasoning (reading, planning, mental simulation). Incorporate proxies (e.g., dwell time, scrolling, file navigation) and test how unobserved cognition biases offloading estimates.
- Nonlinear workflows and branching: Offloading score is defined over linearized step sequences. Extend to DAGs with backtracking/parallelism and define “steps saved” over graph structures.
- Partial assistance within steps: Many steps mix human and AI effort. Develop within-step fractional attribution (e.g., token-level, edit distance, code provenance) to refine offloading estimation.
- Team settings: The measure is individual-centric. Extend to teams (e.g., pair programming, code review), defining shared reliance and attribution across collaborators.
- Privacy and ethics of trace logging: Workflow induction captures sensitive screen and keystroke data. Specify privacy-preserving collection, minimization, on-device computation, and governance for real-world deployment.
- Weak correlation with self-reports: NASA-TLX did not differentiate conditions; triangulate offloading score with richer subjective measures (e.g., trust calibration, perceived agency) and physiological data to validate construct coverage.
- Behavioral interventions: The paper proposes using offloading score as a feedback instrument but does not test it. Run RCTs to evaluate whether surfacing the metric (and actionable guidance) reduces overreliance without harming productivity.
- Cross-domain thresholding: Provide procedures to set reliance thresholds per domain/task and user goal (e.g., safety-critical vs exploratory), accounting for different acceptable trade-offs between speed, accuracy, and understanding.
Practical Applications
Practical Applications of “Offloading Score: Measuring AI Reliance Through Counterfactual Workflows”
Below are concrete, real-world applications that leverage the paper’s findings, measurements, and methodology. Each item indicates sectors, potential tools/products/workflows, and feasibility assumptions or dependencies.
Immediate Applications
- Reliance analytics in developer tooling (software)
- What to do: Integrate offloading score into IDEs (VS Code/JetBrains) and AI coding assistants (e.g., Copilot, Cursor, ChatGPT/Claude plugins) to show per-session and per-PR reliance badges; flag PRs with high offloading for extra review; track reliance trends at team/org level.
- Sectors: Software, DevOps
- Tools/products/workflows: “Reliance Meter” IDE extension; CI hook that adds an offloading-score badge to PRs; dashboard in engineering analytics (e.g., in Linear/Jira/GitLab).
- Assumptions/dependencies: Access to interaction traces (editor diffs, chat logs) with user consent; stable workflow induction; LLM budget for counterfactual step expansion; clear thresholds tied to outcomes (e.g., code understanding).
- Reliance-aware UX nudges and guardrails (software, HCI)
- What to do: Trigger cognitive forcing prompts or “verify before accept” flows when offloading exceeds a threshold; encourage task decomposition and verification steps; adapt assistant autonomy level in real time.
- Sectors: Software, Productivity apps
- Tools/products/workflows: “Reliance Guardrails” SDK; prompt middleware that inserts verification checklists; adjustable autonomy slider driven by offloading score.
- Assumptions/dependencies: Real-time or near-real-time scoring from incremental traces; validated thresholds for a given task domain; minimal friction so nudges don’t harm productivity.
- A/B testing and product evaluation using offloading score (industry R&D)
- What to do: Add offloading score as a key outcome in experiments for new AI features (e.g., code-completion strategies, prompt templates) to ensure productivity gains don’t drive overreliance under time pressure.
- Sectors: Software platforms, SaaS
- Tools/products/workflows: Experimentation platform integration (e.g., LaunchDarkly/Optimizely) with offloading KPIs; time-pressure testing cells.
- Assumptions/dependencies: Instrumentation in beta programs; outcome metrics (quality, understanding) to interpret reliance shifts.
- Team training and enablement diagnostics (HR/L&D)
- What to do: Use reliance reports to tailor training—identify when developers skip planning/verification; run workshops to practice balanced workflows; track deskilling risk longitudinally.
- Sectors: Software, Professional services
- Tools/products/workflows: Quarterly “Reliance x Understanding” cohort reports; custom learning paths triggered by high offloading in execution steps.
- Assumptions/dependencies: Privacy-preserving aggregation; clear comms to avoid punitive use; buy-in from managers and ICs.
- Contact center QA and agent coaching (customer support)
- What to do: Measure how much agents offload to AI suggesters and how they engage with outputs (direct reuse vs adapt); flag conversations for review when offloading is high and understanding/accuracy KPIs are low.
- Sectors: Customer support/BPO
- Tools/products/workflows: “Agent Reliance Dashboard” tied to QA; auto-coaching prompts encouraging scrutiny before sending.
- Assumptions/dependencies: Logged interactions; mapping of process/output-use labels to call/chat quality; data governance for customer PII.
- Education: assignment policies and student feedback (education)
- What to do: Require an offloading report with AI-permitted assignments; grade understanding directly (e.g., system recall or viva) and use reliance to trigger remediation; teach AI usage patterns (planning, verification).
- Sectors: Higher-ed, Bootcamps, Corporate training
- Tools/products/workflows: LMS plugin that ingests editor/chat traces and outputs offloading + process/output-use profiles; “AI usage reflection” worksheet.
- Assumptions/dependencies: Student consent; feasible capture in BYOD settings; institution policy alignment; low-cost LLM evaluation.
- Procurement and vendor evaluation of AI tools (enterprise governance)
- What to do: Include offloading-score telemetry and “reliance x outcome” evidence in pilot evaluations; prefer vendors supporting reliance measurement and guardrails.
- Sectors: Enterprise IT, Compliance
- Tools/products/workflows: RFP criteria for reliance analytics; pilot scorecards comparing productivity, quality, and reliance.
- Assumptions/dependencies: Short proof-of-concept with trace capture; cross-vendor comparability.
- Personal digital wellbeing around AI use (daily life)
- What to do: Provide a browser/editor extension that reports weekly offloading trends across writing/coding/research tasks; set personal goals (e.g., “increase planning, reduce direct reuse”).
- Sectors: Consumer productivity
- Tools/products/workflows: “AI Diet” personal analytics; reflection prompts when under time pressure.
- Assumptions/dependencies: Opt-in telemetry; lightweight on-device processing or affordable cloud compute.
- Research instrumentation across HCI/AIED studies (academia)
- What to do: Use offloading score plus process/output-use labels as standard dependent variables; replicate time-pressure manipulations; relate reliance to multiple outcomes (learning, creativity, error rates).
- Sectors: Academia, Research labs
- Tools/products/workflows: Open-source analysis pipeline; pre-registered protocols that include reliance metrics.
- Assumptions/dependencies: Access to the released code/website; IRB and data protections; consistent step granularity.
- Internal audits for high-volume AI writing (marketing/legal drafting)
- What to do: Audit campaigns and templates where offloading is persistently high and rejection/scrutiny low; add review checklists for regulated claims.
- Sectors: Marketing, Legal ops
- Tools/products/workflows: “Reliance x Risk” map for content types; automated insertion of verification steps.
- Assumptions/dependencies: Content platform integration; policy-specific outcome checks (brand safety, compliance).
Long-Term Applications
- Reliance-aware autonomy control in high-stakes domains (healthcare, finance, energy, aviation)
- What to do: Adjust decision support autonomy based on offloading and outcome proxies; require additional verification or human sign-off when offloading crosses thresholds linked to risk.
- Sectors: Healthcare (CDS, documentation), Finance (advice, trading ops), Energy/Utilities (control rooms), Aviation (dispatch/maintenance)
- Tools/products/workflows: “Reliance-Gated” workflows; safety cases showing that guardrails reduce overreliance incidents.
- Assumptions/dependencies: Strong domain-specific outcome metrics; validated counterfactuals; rigorous safety/QA; regulatory approvals.
- Regulatory standards and audits for reliance (policy, standards)
- What to do: Incorporate offloading/reliance measures into AI risk management frameworks and audit trails; disclose reliance characteristics to end-users; set sector-specific thresholds or controls.
- Sectors: Policy, Standards bodies, Compliance
- Tools/products/workflows: Reliance sections in conformity assessments; attestations under AI governance frameworks; industry benchmarks.
- Assumptions/dependencies: Consensus on definitions and thresholds; privacy-preserving, interoperable telemetry standards; enforcement mechanisms.
- Personalized counterfactuals and skill-aware measurement (advanced user modeling)
- What to do: Move from “average user” counterfactuals to personalized ones that account for an individual’s skills/history; tailor nudges and training to the person.
- Sectors: Software, Education, HR tech
- Tools/products/workflows: User modeling services that estimate human-only step sequences; personalized reliance coach.
- Assumptions/dependencies: Longitudinal data; fairness and privacy safeguards; robust modeling of user ability without bias.
- Cross-domain generalization beyond coding and text (multimodal work)
- What to do: Extend workflow induction and counterfactuals to design, data science notebooks, BI dashboards, CAD, and creative tools; include non-text interactions (pen, voice, AR/VR).
- Sectors: Design, Data/Analytics, Manufacturing, Media
- Tools/products/workflows: Multimodal workflow recorders; domain-specific process labels; reliance-aware creative assistants.
- Assumptions/dependencies: High-quality interaction capture across tools; domain ontologies for steps; evaluation datasets.
- Enterprise skill maintenance and workforce planning (strategy/HR)
- What to do: Use reliance trends to anticipate deskilling risks; design rotational assignments and “human-in-the-loop” practice to preserve critical capabilities; track ROI balancing productivity and maintainability.
- Sectors: Enterprise, Public sector
- Tools/products/workflows: “Skill Health” dashboards; policies that cap offloading on critical tasks or mandate periodic manual runs.
- Assumptions/dependencies: Causal links between reliance and long-term performance; change management; cultural buy-in.
- Marketplace signals and pricing tied to reliance (platform economics)
- What to do: Price or quota AI features by offloading (e.g., cheaper for planning help, more expensive for end-to-end execution); align incentives with user learning goals.
- Sectors: SaaS platforms, Marketplaces
- Tools/products/workflows: Usage-based billing that incorporates offloading classes; SLA terms around reliance and review requirements.
- Assumptions/dependencies: Accurate classification of process/output-use; acceptance by customers; avoidance of perverse incentives.
- Education policy and accreditation around AI use (policy/education)
- What to do: Define acceptable reliance bands for different course objectives; require reporting of reliance alongside learning outcomes; accreditation bodies include reliance measures in reviews.
- Sectors: K–12, Higher-ed, Professional certification
- Tools/products/workflows: Standardized “AI-usage statement” with offloading metrics; program-level dashboards.
- Assumptions/dependencies: Agreement on pedagogical goals; accessible tooling for institutions; equity considerations.
- Privacy-preserving reliance analytics (platform and standards)
- What to do: Develop on-device or federated computation of offloading score and labels; differentially private aggregation for org-wide reports.
- Sectors: Platform infrastructure, Privacy tech
- Tools/products/workflows: Federated workflow induction; DP guarantees for reliance reporting.
- Assumptions/dependencies: Efficient on-device models; acceptable utility-privacy tradeoffs; standard APIs across tools.
- Benchmarking and certification for AI tools’ impact on reliance (ecosystem)
- What to do: Create public benchmarks where tools are scored on productivity, accuracy, and reliance (including under time pressure); certify tools that support balanced workflows.
- Sectors: AI vendors, Evaluations
- Tools/products/workflows: Open benchmark suites; third-party certifications; “Reliance Impact” labels for tooling.
- Assumptions/dependencies: Community-maintained tasks and traces; reproducibility; multi-model comparability.
- Multi-agent and orchestration systems that manage offloading explicitly (advanced AI systems)
- What to do: Orchestrators that choose when to plan vs execute vs verify, targeting a desired offloading profile; teach agents to prompt users for oversight when needed.
- Sectors: Software, Robotics, Ops automation
- Tools/products/workflows: Reliance-aware planners; self-regulating agent loops that keep users “in the circuit.”
- Assumptions/dependencies: Reliable detection of process stages; alignment with user goals; robust human-agent interfaces.
Cross-cutting assumptions and risks to feasibility
- Data access and consent: Many applications require capturing interaction traces (keystrokes, screenshots, chat logs). Strong privacy and security measures are essential and may limit granularity.
- Validity external to coding: The method is proven in coding; extensions to other domains need new task ontologies, ground truth outcomes, and validation.
- Step granularity and counterfactual quality: Workflow induction must be consistent; counterfactual expansion depends on LLM reliability and may need human calibration in high-stakes settings.
- Thresholds depend on outcomes: “Appropriate” vs “overreliant” must be tied to domain-specific outcomes (e.g., understanding, safety, accuracy). Threshold setting requires empirical validation.
- Potential gaming and incentives: If offloading affects rewards, users or vendors may game the metric. Design audits and randomized spot-checks can mitigate.
- Cost and latency: LLM-based annotation/scoring can be expensive; consider batching, caching, or on-device models for scale.
Glossary
- AI Code Fraction: A usage-based baseline metric estimating how much of the final code was generated by the AI. "baseline measures (e.g., AI Code Fraction) (\Cref{sec:validation_user_study})"
- automation complacency scales: Psychometric instruments that quantify reduced vigilance or overtrust in automation. "automation complacency scales \citep{merritt2019automation}"
- between-subjects experiment: An experimental design where different participants are assigned to different conditions. "We conduct a between-subjects experiment to validate the proposed scalar measure of reliance, offloading score."
- Bloom's taxonomy: A hierarchical framework of cognitive processes used here to categorize how users engage with AI outputs. "using a four-point scale inspired by Bloom's taxonomy"
- construct validity: The extent to which a measure behaves consistently with the theoretical construct it is intended to capture. "Construct Validity. Construct validity asks whether a metric behaves consistently with the construct it is intended to measure."
- content validity: The degree to which a measure adequately represents all aspects of a concept. "Content Validity. Content validity assesses whether a metric captures the construct it is intended to measure."
- counterfactual formulation: Modeling approach that evaluates what would have happened under an alternative scenario, here without AI assistance. "We adopt a counterfactual formulation to capture not just how much the tool is used, but how it is used within the workflow."
- counterfactual workflow: A simulated sequence of steps estimating how a user would complete a task without AI. "we construct a counterfactual workflow by estimating how the user would have completed the task without the tool"
- criterion validity: How well a metric correlates with an external standard or outcome. "Criterion Validity. Criterion validity measures the extent to which the proposed metric aligns with an external reference standard."
- deskilling: Erosion of a person’s abilities due to overreliance on tools. "including deskilling and reduced independent problem-solving ability"
- effect size: A quantitative measure of the magnitude of a phenomenon, used here via correlation with condition labels. "and reporting the correlation with the binary condition label as an effect size."
- Flower model of cognitive processes: A framework (from writing research) used to categorize types of cognitive tasks offloaded to AI. "inspired by the Flower model of cognitive processes"
- human-only counterfactual sequence: The simulated set of steps a human would take to complete an AI-assisted sub-goal without the tool. "we estimate a corresponding human-only counterfactual sequence"
- interaction traces: Recorded user interactions (e.g., keystrokes, screenshots) used to reconstruct workflows and compute metrics. "computed directly from interaction traces (e.g., screenshots and keystrokes)"
- intrinsic evaluations: Validity tests that examine a metric’s behavior and components without relying on external benchmarks. "We validate offloading score through intrinsic evaluations of metric validity"
- Likert scale: A psychometric response scale used to measure attitudes or judgments. "( on a $5$-point Likert scale)"
- LLM-as-judge: Using a LLM to evaluate responses or outputs against a reference. "which we evaluate using an LLM-as-judge against the reference implementation."
- NASA-TLX: A standardized instrument for measuring perceived workload (Task Load Index). "using the NASA-TLX scale"
- normative goal: The user-defined objective that determines what level of reliance is appropriate. "depends on the userâs normative goal, i.e., what outcome reliance is meant to support."
- offloading score: The scalar metric quantifying the fraction of cognitive effort offloaded to AI by comparing observed and counterfactual workflows. "we introduce offloading score, a measure of reliance that quantifies the fraction of cognitive effort offloaded to an AI tool."
- output-based methods: Reliance measures based on acceptance or rejection of AI outputs, often treated as binary. "Output-based methods, developed in prior work focusing on classification settings, treat reliance as a binary measurement (accept/reject)"
- output-use labels: Categories describing how users engage with AI outputs (e.g., reuse, adapt, reject). "These are referred to as output-use labels."
- overreliance: Excessive dependence on AI that can reduce scrutiny, oversight, or understanding. "risks of overreliance, including deskilling and reduced independent problem-solving ability"
- Pearson correlation: A statistic measuring linear association between two variables. "the Pearson correlation of offloading score and system recall increases to ."
- permutation test: A nonparametric significance test based on label shuffling. " with a permutation test"
- process labels: Categories indicating the types of cognitive processes offloaded to AI (e.g., planning, execution). "These are referred to as process labels."
- short-horizon simulation: A limited-step simulation to estimate alternative workflow paths for metric computation. "This short-horizon simulation is inspired by past work"
- switch fraction: A usage-based metric indicating how often users change decisions after seeing AI suggestions. "switch fraction \citep{yin2019understanding}"
- system recall: A measure of users’ understanding of the system they built, assessed via post-task questions. "We include system recall to capture how well users understand the systems they co-create"
- technical debt: The implied cost of additional rework caused by choosing expedient solutions over better approaches. "increase maintenance burden and technical debt when developers must later understand, review, or repair generated artifacts"
- two-sample t-tests: Statistical tests comparing the means of two independent groups. "using two-sample -tests"
- underreliance: Insufficient use of AI assistance, such as rejecting correct outputs. "underreliance (reject + correct)"
- usage-based metrics: Measures derived from how much or how often AI is used in a workflow. "Other related usage-based metrics, such as switch fraction \citep{yin2019understanding} and weight of advice \citep{logg2019algorithm}, measure whether users update their decisions with AI suggestions."
- weight of advice: A metric capturing how much users adjust decisions toward AI recommendations. "weight of advice \citep{logg2019algorithm}"
- Wilcoxon test: A nonparametric statistical test for comparing paired or independent samples. " with a Wilcoxon test"
- workflow induction toolkit: A tool that reconstructs structured workflows from raw interaction data. "using the workflow induction toolkit \citep{wang2025ai}"
Collections
Sign up for free to add this paper to one or more collections.