IntentionTest: Methods in Testing and AI

• IntentionTest is a family of methods that treat intent as an explicit object for specification, detection, and evaluation in system testing.
• It spans various domains such as software testing, legal analysis, robotics, and AI, using approaches from behavioral verification to latent goal inference.
• Empirical studies report improvements in test coverage, reduced evaluation time, and enhanced system reliability, demonstrating practical benefits across applications.

In the literature represented by these works, IntentionTest is not a single standardized benchmark. A plausible unifying characterization is a family of methods that treat intent, intentionality, or test intent as an explicit object of specification, detection, or evaluation: some methods formalize what a test is supposed to validate, some measure whether an AI system behaves like an intentional actor, and some infer latent goals from indirect signals rather than explicit labels (Assi et al., 2014, Chiappetta et al., 6 May 2026, Feng et al., 9 Jun 2025).

1. Conceptual scope and competing meanings

The term is used across several technical traditions, and the underlying object of analysis changes with the domain. In software testing, “intent” usually denotes the behavior a test case is meant to cover or the validation goal a developer or tester is trying to encode. In legal-algorithmic analysis, intent is divided into direct, oblique, and ulterior forms, with the crucial requirement that the agent have alternative actions, some capacity to represent outcomes, and a subjective model of how actions cause results; the paper explicitly treats foreseeability and side effects “according to D’s estimate” rather than by an external reasonable-person standard (Ashton, 2021). In governance-oriented AI work, functional intentionality is defined behaviorally rather than mentally, through purpose, foresight, volition, temporal commitment, and coherence, and is explicitly separated from consciousness, sentience, moral agency, autonomy in the metaphysical sense, and legal personhood (Chiappetta et al., 6 May 2026).

In robotics and forecasting, “intention” is often a structured latent variable rather than a normative status. In industrial human-robot collaboration, the relevant distinction is between low-level task intention and high-level interaction intention, such as whether a human currently prefers coexistence or cooperation (Huang et al., 2022). In pedestrian trajectory prediction, intention is decomposed into short-term local motion tendency and long-term endpoint or destination intent, both of which are treated as predictive structure inside a generative model rather than as psychological self-report (Liu et al., 10 Aug 2025). This diversity of usage is itself a central fact about the topic.

2. Test intent in software engineering

One major lineage treats IntentionTest as a problem of making the purpose of a test explicit and checkable. UCov defines a user-defined coverage criterion in which a test requirement is “an execution pattern of program elements and predicates,” introduced specifically for test case intent verification (Assi et al., 2014). Its requirement language includes basic, conditional, sequential, and repeated test requirements, allowing a tester to specify not merely that a statement or branch is covered, but that a particular behavior, state condition, execution order, or repetition pattern is exercised. The stated motivation is that even 100% branch coverage does not guarantee that a bug-fix behavior or important application scenario is still being tested.

A second lineage raises the abstraction level of test authoring. AppIntent proposes a high-level mobile-app automation specification language in which the initial key denotes the app name, terms connected by \rightarrow denote pages or actions, . denotes page-local conditions or operations, and {} stores parameters or key-value pairs (Gopi, 2018). High-level intent specifications are processed by an intent composition engine, resolved through a manually maintained mapping file, translated into executable Appium code, and run through Appium Studio. The framework is explicitly positioned as capturing what a tester intends to test “with in and across multiple apps,” while leaving low-level tool scripting to the backend. The language is only semi-formally defined, and the paper emphasizes that mappings and object identification remain manual.

A third lineage concerns understanding and generating tests from intent. TestIntention formalizes an Appium script as an operation sequence

$$OpSeq = \{Op_1, Op_2, \dots, Op_n\}, \qquad Op_i=\{type, selector, operation, content\},$$

then reconstructs per-operation intent through two channels: GUI Intent for XPath-selected widgets and Code Intent for ID-linked callbacks (Yu et al., 2021). GUI-side inference uses runtime layout dumps, OCR, widget screenshots, and an image-captioning model based on VGG-16; code-side inference uses Android-specific response-binding templates and a code2seq-based summarizer. On a user study, the average understanding time dropped from 76.33 s to 22.27 s, a reported 70.83% reduction.

Recent work extends intent from understanding tests to generating, migrating, and mutating them. IntentionTest for project-specific unit tests takes a focal method plus a structured validation intention—with Objective mandatory and Preconditions and Expected Results optional—retrieves a referable project test, ranks crucial facts from a code graph, and edits the reference test toward the target intention (Qi et al., 28 Jul 2025). On 4,146 test cases from 13 open-source projects, it improves Common Mutation Score over ChatTester by 39.03%, coverage overlap by 40.14%, and successful passing tests by 21.30%. IntentTester performs cross-library and cross-language migration by abstracting source tests into a Test Description Language (TDL), aligning them to a repository graph with multiple agents, and iterating through planning, synthesis, and verification; it reports 2,776 syntactically correct tests with 85% correctness and 74% effectiveness (Gao et al., 24 Jun 2026). Intent-Based Mutation Testing moves mutation from code syntax to natural-language programming intent; the paper reports that 55% of intent-based mutations are not subsumed by traditional mutations, framing intent variants as a complement to syntax-based mutation rather than a replacement (Hamidi et al., 6 Jul 2026).

3. Intent integrity, functional intentionality, and accountable AI

A separate literature treats IntentionTest as an instrument for evaluating AI systems themselves. The clearest example is the Functional Intentionality Test (FIT), which defines intentionality as a behavioral profile over five dimensions—purpose, foresight, volition, temporal commitment, and coherence—each scored from 0 to 4, with composite score

$\text{FIT-Score}=\frac{P+F+V+T+C}{5}.$

The average is mapped to provisional Intentionality Levels $IL0$–$IL4$, and the associated FIT-Eval protocol is described as behavioral, model-agnostic, reproducible in principle, and governance-relevant (Chiappetta et al., 6 May 2026). The paper’s central claim is that intentionality is design-contingent: architectural choices such as memory persistence, planning depth, and tool autonomy can increase or decrease measured intentional-like behavior, so the framework is meant not only for diagnosis but for autonomy calibration.

A more deployment-oriented use appears in IntenTest, a stress-testing framework for API-calling LLM agents (Feng et al., 9 Jun 2025). Here the concern is intent integrity: whether the agent’s behavior faithfully preserves the user’s true intent. The task space is partitioned into VALID, INVALID, and UNDERSPEC regions for each API parameter; seed tasks are generated from toolkit documentation; and the central search objective is to find an intent-preserving mutation $u'$ such that $I(u')=I(u)$ but $\pi(u')\ne\pi(u)$. The framework then ranks candidate mutations with a lightweight likelihood model and stores high-yield mutation strategies in a datatype-aware memory. Across 80 toolkit APIs, it outperforms the SelfRef baseline on both Error-Exposing Success Rate (EESR) and Average Queries to First Failure (AQFF).

The same general idea—intent as an explicit intermediate variable rather than a latent byproduct—also appears in safety classification. The AIMS dataset contains 1,724 difficult safety annotations, corresponding to 1,275 unique prompts, each paired with a human-written intent description and harm label (Ferrao et al., 25 Jun 2026). The paper uses AIMS across supervised fine-tuning, DPO, reasoning distillation, and GRPO, and reports that directly rewarding intent faithfulness with GRPO yields the strongest average performance across five external safety benchmarks, while the intent-aware models form the inference latency–F1 Pareto frontier. In this setting, intent is not merely explanatory text; it is a compact supervision signal that sits explicitly between prompt and safety label.

Legal-formal work supplies a different but related template for accountability. The paper on algorithm-suitable definitions of intent introduces capacity requirements for autonomous algorithms and then defines direct, oblique, and ulterior intent in terms of free agency, knowledge, foreseeable causality, aim, side effects that are “almost certainly” true according to the agent’s own estimate, and commitment to future conditional action (Ashton, 2021). This provides a white-box evaluative structure for cases where one wants to ask not only whether an outcome occurred, but whether the system selected or tolerated it in a way analogous to doctrinal intent.

4. Embodied, multimodal, and interactive intention evaluation

In embodied systems, IntentionTest often becomes a problem of recovering hidden goals from motion, context, or indirect instruction. Hierarchical Intention Tracking for industrial human-robot collaboration models intention as a Markov process and recursively estimates

$P(G_t \mid X_{1:t}^{h,r}),$

with a two-level hierarchy: low-level task intention and high-level interaction intention (Huang et al., 2022). In the demonstrated assembly task, the high-level state distinguishes coexistence from cooperation, while the low-level state tracks which assembly region the human is pursuing or whether the human is engaged in failure recovery. In a pilot study with a UR5e-based system, the proposed HIT architecture reduced average failures from 1.2 to 0.0 relative to the coexistence-only baseline and reduced human energy from 21.04 J to 5.71 J relative to the cooperation-only baseline.

For vision-LLMs, intention is evaluated through open-ended social inference. The benchmark on VLM Theory-of-Mind reasoning uses 30 images and asks three questions per image: infer the intention/mental state, identify the visual cues, and predict what might happen next (Wen et al., 28 Mar 2025). Scores are assigned with a manual $0/0.5/1$ rubric. GPT-4 obtains 27 / 27 / 28 across the three categories, while GPT-4o-mini obtains 27.5 / 27 / 27.5; smaller models perform markedly worse. The authors highlight a recurrent failure mode: models often describe visible actions while missing latent social intentions, with cheating during an exam identified as a case where all four tested VLMs failed.

A still more task-complete benchmark is IntentionNav, which studies intent-driven object navigation from implicit human instruction (Qian et al., 22 May 2026). Each episode provides free-text intent, RGB-D observations, and pose, but withholds the target category; the benchmark contains 500 intents, 176 scenes, and 64 target categories, with each intent rewritten in four controlled styles and annotated with one of four intent modes. Evaluation separates intent matching, navigation success, neighborhood reachability, and grounded success rather than collapsing them into a single score. Across three VLM backends, models identify the intended target in 48.3% of episodes, enter its 2 m neighborhood in 68.7%, terminate successfully in only 24.9%, and achieve grounded 1 m success in 5.5%. The benchmark’s most important finding is that the largest gap is not only between language and search, but between reaching the right vicinity and stopping correctly with visual grounding.

5. Intent as an inferred latent variable in prediction and analysis

Another substantial body of work uses intention as a latent variable for prediction rather than as a normative label. In Intention-Aware Diffusion for pedestrian trajectory prediction, intention is decomposed into short-term and long-term components (Liu et al., 10 Aug 2025). Short-term intention is represented continuously in polar form as direction and magnitude derived from second-order temporal derivatives, while long-term intention is modeled as a set of endpoint hypotheses with probabilities. These intention signals condition a diffusion-based trajectory generator through adaptive guidance and a residual noise predictor. On ETH, UCY, and SDD, the strongest evidence for the design comes from ablation: removing short-term or long-term intention degrades performance, indicating that explicit intention variables materially contribute to prediction quality.

The paper INTENT takes a related but lighter-weight approach (Tang et al., 6 Mar 2025). It models road-agent intention from trajectory alone, with four coarse classes—straight, left, right, and static—and uses contrastive clustering to handle the claim that intention is fuzzy and abstract. The model is deliberately MLP-based and omits maps and interaction graphs. On ETH-UCY it reports average ADE/FDE = 0.48 / 0.96, and on runtime it reports average training time of 277.5 s, compared with 5437.92 s for the transformer baseline and 521.3 s for the LSTM baseline. Here “intention” is operationalized as a soft motion tendency that constrains plausible futures.

In program analysis, intention can also denote developer-controlled semantic capability. SmartIntentNN detects risky smart-contract intents from Solidity source using a three-stage pipeline: Universal Sentence Encoder embeddings for function snippets, K-means highlighting of distinctive intent-bearing functions, and a BiLSTM multi-label classifier over 10 intent categories such as fee, blacklist, mint, honeypot, and maxSell (Huang et al., 2022). The abstract reports an F1-score of 0.8633 across the 10 categories, while the detailed baseline table reports 0.8212 for the best Scale×2 dropout configuration. The conceptual move is consistent with the broader IntentionTest theme: the target is not vulnerability in the usual exploit sense, but purposeful control logic embedded by the developer.

A more general and more philosophical analysis appears in the work on intentionality in knowledge representation (Burgess, 14 Jul 2025). There intentionality is treated as an observer-relative property detectable in streams through anomalous multi-scale structure, coherence intervals, and estimated work required to produce a fragment. The paper proposes low-cost scores over symbolic fragments and argues that a Tiny LLM can partition ambient context from intended content without large-scale training. This suggests a very different style of IntentionTest: not a benchmark with fixed labels, but a process for identifying structured, effort-bearing signals that stand out from background regularity.

6. Limitations, controversies, and common misconceptions

A recurring limitation is that the topic lacks a single settled formalism. Across these papers, “intent” ranges from validation intention and test case intent to functional intentionality, indirect human needs, trajectory-derived destination tendency, and developer-controlled smart-contract capability. This suggests that IntentionTest is presently a problem family rather than a standardized scientific construct.

Several misconceptions are addressed explicitly in the literature. FIT is not a test for consciousness, sentience, moral agency, or legal personhood; it is a behavioral index with provisional, domain-dependent thresholds, and FIT-Eval is described as a scaffold rather than a validated benchmark (Chiappetta et al., 6 May 2026). AppIntent does not learn user behavior patterns from data; behavior is captured through manually authored high-level scenarios, and the current implementation depends on manually maintained mappings and manually identified page objects (Gopi, 2018). Intention-aware trajectory models do not infer rich psychological goals; they operationalize intent as motion tendency, destination tendency, or clustered trajectory structure (Liu et al., 10 Aug 2025).

Empirical coverage is also often limited. The open-ended VLM Theory-of-Mind benchmark contains only 30 images, with no reported inter-annotator agreement statistics for scoring (Wen et al., 28 Mar 2025). SmartIntentNN faces extreme class imbalance, with categories such as maxSell reported at 0.05%, and the dataset is restricted to BSC and Solidity (Huang et al., 2022). IntenTest relies on access to API-calling trajectories and on LLM-based judgments for intent preservation and strategy novelty, which leaves observability and evaluator reliability as open issues (Feng et al., 9 Jun 2025).

A final controversy concerns what exactly counts as evidence of intention. Some papers adopt a behavioral stance, where intent is whatever can be reconstructed from action organization, persistence, and consequence modeling. Others require white-box access to goals, plans, or causal models. Still others reduce intention to a predictive latent variable. A plausible implication is that future IntentionTest frameworks will need to make this choice explicit: whether they are testing what a system is meant to do, what a human user meant, what a model can infer about another agent’s goal, or how much an AI system functionally resembles an intentional actor.