AgentFuel: Benchmark & Fuel Analytics Framework

• AgentFuel is a framework for generating customized benchmarks that evaluate timeseries agents on stateful and incident-specific queries.
• It formalizes dataset and query expressivity gaps to ensure domain-faithful evaluation with realistic enterprise and IoT data.
• Broader applications extend to fuel analytics, inverse design, and explainable decision support in operational settings.

AgentFuel is most explicitly introduced as a framework for generating customized, expressive evaluation benchmarks for conversational and LLM-based data analysis agents that operate over enterprise timeseries data (Maddi et al., 12 Mar 2026). In the surrounding literature, the same label is also used more broadly as a design lens for fuel and energy agents that combine interpretable fleet analytics, multimodal reporting, surrogate property prediction, inverse molecular or blend design, and operational control (Barbado et al., 2021, Ma et al., 17 Nov 2025, Almomtan et al., 2 Jun 2025, Larrañaga et al., 9 Jun 2026). Taken together, these usages define AgentFuel as both a concrete benchmark-generation system and a broader agent-systems pattern for fuel-domain reasoning, where domain-faithful data models, grounded tool or simulator access, and auditable outputs are central.

1. Core conception and scope

In its primary and explicit sense, AgentFuel is a modular pipeline for creating customized timeseries evaluations. Its purpose is to help practitioners test whether a “talk to your data” agent can correctly answer domain-relevant questions before deployment, especially in settings such as IoT sensors, observability and monitoring data, telecommunications network telemetry, cybersecurity events and host or network measurements, and product analytics or user clickstream event data (Maddi et al., 12 Mar 2026). The motivating claim is that existing text-to-SQL and general data-agent benchmarks are dominated by static, snapshot-like aggregation tasks, whereas operational users ask stateful, incident-centered, and cross-entity questions.

The framework is organized around two expressivity gaps. The first is a dataset expressivity gap: generic or public datasets often do not encode realistic entity types, expected temporal structure, domain-specific normal behavior, or abnormal patterns such as KPI degradation, outages, flash crowds, latency spikes, or cascades across dependent entities. The second is a query expressivity gap: current evals underrepresent stateful questions about event sequences and durations, incident-specific analytics, and persona-specific or linguistically varied questions (Maddi et al., 12 Mar 2026). This dual diagnosis is foundational, because AgentFuel is designed to make evaluation data and evaluation questions jointly domain-faithful.

A broader AgentFuel interpretation is suggested by later work that treats fuel analytics as a multi-stage agent problem rather than a single predictive task. In that reading, AgentFuel denotes an end-to-end stack that begins with telemetry, spectra, chemistry, or plant design variables; passes through interpretable models or surrogates; and ends with stakeholder-facing explanations, recommendations, or operational actions (Ma et al., 17 Nov 2025, Almomtan et al., 2 Jun 2025, Larrañaga et al., 9 Jun 2026). This suggests a family resemblance rather than a single standardized implementation.

2. Formal structure of the benchmark-generation framework

AgentFuel has three logical phases: dataset generation, question-answer generation, and test integration (Maddi et al., 12 Mar 2026). The data model is defined through entities, static attributes, dynamic keys, measurements, and time series. An entity has static attributes $A$, dynamic keys $K$, measurements $Meas(e,t)$, and an ordered time series $TS(e)$. Each entity is modeled as having a static profile, a state machine governing behavior, and measurement-generation rules conditioned on state. State trajectories are described as a semi-Markov process, which allows dwell times in states and probabilistic transitions.

Dataset generation proceeds through entity specifications, exemplar datasets, and global assembly by epoch blending. An exemplar is a synthetic dataset representing one named behavior, such as normal browsing sessions, cart abandonment behavior, degraded sensor behavior, router link degradation, or outage patterns. The global dataset is assembled by partitioning time into epochs and blending exemplars with varying weights over time, which enables gradual drift, seasonal oscillations, sudden regime changes, incident onset, and recovery (Maddi et al., 12 Mar 2026). The system also includes an extensible library of SQL-based data quality and data pattern checkers to verify that the generated data actually satisfies the intended scenario.

Question generation is explicitly formalized. A query is represented as

$$Q=\bigl(EntityFilter(\cdot),\,EFSet,\,\mathbf{F}\bigr),$$

where $EntityFilter(\cdot)$ selects a subpopulation based on static profiles, $EFSet$ is a set of event predicates over measurements, and $\mathbf{F}$ is an analysis function producing the result (Maddi et al., 12 Mar 2026). Stateless queries treat events in a window as an unordered collection, while stateful queries depend on reconstructed temporal occupancy. The framework defines the state occupancy record as

$$OCC(e,s,W)=\bigl\{[t^{\mathrm{in}_j},\,t^{\mathrm{out}_j})\mid state_e(t)=s,\; t\in W\bigr\},$$

which supports reachability, duration in state, transition counts, sequence matching, time between events, and counts conditioned on state.

Representative operators are given explicitly. Event count is

$$COUNT(e,EventFilter,W)=\bigl|\{t\in M(e)\cap W : EventFilter(m(e,t))\}\bigr|,$$

attribute mean is

$K$0

state reached is

$K$1

and state duration is

$K$2

These definitions make AgentFuel a formal benchmark-construction framework rather than a prompt template library (Maddi et al., 12 Mar 2026).

3. Empirical findings, task difficulty, and failure modes

The published AgentFuel benchmarks comprise three domain-specific test suites: E-commerce with 2 tables, 6K rows, and 12 stateless plus 12 stateful queries; IoT with 1 table, 50K rows, and 12 stateless plus 12 stateful queries; and Telecom with 3 tables, 23.5K rows, and 12 stateless plus 12 incident-specific queries (Maddi et al., 12 Mar 2026). Several existing agents were evaluated, including Databricks Genie, Snowflake Cortex Analyst, PandasAI, and Nao, with PandasAI paired to multiple underlying models.

The main quantitative result is a sharp degradation from easy to operationally realistic query types. Across agents and datasets, performance was around 73% on simple stateless aggregation queries, about 34% on stateful queries, and about 10% on incident-specific queries (Maddi et al., 12 Mar 2026). Existing public benchmarks were also analyzed and found to be overwhelmingly dominated by stateless questions: Spider2-Snow at 92.32% stateless, BIRD at 96.33% stateless, and Beaver at 93.78% stateless, with incident-style queries almost absent. This establishes the central empirical argument for AgentFuel: prevailing evals overestimate agent competence in timeseries domains.

The failure analysis is correspondingly concrete. On even stateless tasks, agents showed schema confusion, wrong table selection, wrong interval selection, and missing columns due to poor table choice. On stateful tasks, they failed to track state across events, counted all events in a session once a trigger appeared, and confused exact sequence matching with subsequence matching. On incident-specific tasks, they assumed a fixed global incident window, averaged over all data rather than the incident interval, used arbitrary thresholds not derived from entity-specific baselines, and failed to identify the affected entities (Maddi et al., 12 Mar 2026). The reported mean self-consistency of around 76% indicates that these are systematic errors rather than random fluctuations.

The paper also provides preliminary evidence that AgentFuel can drive improvement, not only measurement. Using GEPA on telecom incident-specific tasks with 30 additional AgentFuel-generated queries, a budget of 200 evaluations, GPT-4.1 as reflection model, and GPT-4o-mini for automated scoring in that experiment, the reported result was 17% performance improvement overall, with one optimized prompt for PandasAI/O4Mini giving 25% improvement (Maddi et al., 12 Mar 2026). This suggests that expressive eval generation can function as optimization infrastructure for agent iteration.

4. AgentFuel as fuel-analytics interpretation and explainable decision support

A major extension of the AgentFuel idea appears in operational fuel analytics, where the core problem is no longer only “can an agent answer a timeseries query,” but also “can it turn model outputs into actionably grounded guidance.” In vehicle-fleet fuel optimization, an Explainable Boosting Machine was used to predict fuel consumption of different types of industrial vehicles under real-world operation, and to explain the relationship between input factors and fuel consumption by quantifying the individual contribution of each factor (Barbado et al., 2021). The paper reports that 70% of the categories associated to the fuel-factors are similar to the previous literature, and estimates that optimizing driving behaviour decreases fuel consumption between 12% and 15% in a large fleet of more than 1000 vehicles. The paper is explicit that this is an operational implication from the explainability analysis, not a closed-loop field trial result.

The same decision-support theme is developed further in public transportation fuel-efficiency analytics. A multi-agent multimodal LLM framework coordinates three specialized agents—a data narration agent, an LLM-as-a-judge agent, and an optional human-in-the-loop evaluator—to convert charts, tables, clustering artifacts, and post hoc analyses into stakeholder-oriented reports (Ma et al., 17 Nov 2025). The case study uses 4006 bus trips in Northern Jutland, Denmark, with fuel efficiency measured in liters per 100 km and analyzed using Gaussian Mixture Model clustering. The pipeline is explicitly four-stage: raw data description, data modeling, post hoc analytics, and integration and narration.

The judge agent scores narratives on clarity, relevance, insightfulness, and contextualization, each from 0 to 4, and returns a structured JSON object. Comparative experiments across five multimodal-capable LLMs and three prompting paradigms found GPT-4.1 mini with Chain-of-Thought prompting to be the preferred configuration, achieving 97.31 ± 1.295 percent accuracy, summarized as 97.3% narrative accuracy (Ma et al., 17 Nov 2025). This is relevant to AgentFuel because it defines an interpretation layer that is neither raw model inference nor generic summarization: it is a modular narration-and-review process for fuel analytics, with explicit scoring contracts and optional expert escalation.

Taken together, these works imply an AgentFuel stack in which interpretable models, artifact-grounded narration, and machine-judge validation sit between raw fleet data and human operational decisions. This suggests that fuel agents are most useful when they can separate controllable behavioral penalties from structural route or mission effects, and when they can communicate that distinction in a form that survives operational scrutiny.

5. Fuel property prediction, inverse design, and formulation

Another major branch of the AgentFuel literature concerns property estimation and design. In ATR-FTIR-based fuel characterization, Fuelprop predicts Research Octane Number, Motor Octane Number, and Derived Cetane Number directly from spectra using a multi-target 1D CNN with three convolutional layers and three output heads (Almomtan et al., 2 Jun 2025). The dataset contains 757 fuel entries—104 pure components, 518 surrogate blends, and 135 real fuels—and uses a deliberately out-of-distribution protocol in which training uses pure components and surrogate blends while all real fuels are held out exclusively for test. The full system combines synthetic spectra blending, pseudo-labeling, semi-supervised synthetic data generation, and consistency-enforcing unsupervised data augmentation; the reported overall improvement is a 23.9% MAE reduction over the baseline CNN, with aggregated parity-plot performance of $K$3 on held-out real fuels (Almomtan et al., 2 Jun 2025).

A complementary surrogate-modeling line focuses on thermophysical properties rather than ignition metrics. For liquid-fuel density, Gaussian Process and probabilistic conditional generative models are trained on MD-generated data across five n-alkanes over $K$4–$K$5 K and $K$6–$K$7 MPa, with multi-fidelity fusion against NIST data near critical conditions (Freitas et al., 2021). The crucial design pattern is recursive low-/high-fidelity coupling,

$K$8

or, in the generative version,

$K$9

so that broad MD coverage can be corrected by sparse experimental anchors. This provides an AgentFuel blueprint for fast property oracles with uncertainty estimates and explicit fidelity correction.

At the molecular scale, inverse design is addressed by both graph-based and latent-space generative models. A modular graph-ML CAMD framework combines graph generators, higher-order GNN predictors for RON, MON, and DCN, applicability-domain filtering via one-class SVMs, and latent-space optimization with Bayesian optimization or genetic algorithms (Rittig et al., 2022). The design objective is

$Meas(e,t)$0

with promising molecules operationally defined by $Meas(e,t)$1 and $Meas(e,t)$2. The framework successfully recovers known high-octane molecules such as MTBE and ETBE, but the experimental follow-up on 2,2-dimethoxypropane shows a strong failure case, illustrating the danger of sparse and chemically unbalanced RON and MON training data (Rittig et al., 2022). A related Co-VAE framework for inverse design of high-RON molecules couples a VAE latent space to a property predictor with the loss

$Meas(e,t)$3

then refines scoring with CatBoost and searches latent space with differential evolution (Yalamanchi et al., 16 Apr 2025). It reports 1189 unique and valid SMILES, corresponding to 1185 unique chemical species with predicted $Meas(e,t)$4, of which 921 were unseen by the Co-VAE model.

At the formulation level, The Fuel Optimizer casts sustainable aviation fuel design as an inverse problem over 51 pure species and 2508 simulated multi-component blends, with a neural surrogate replacing a 1D reactor network model of a CFM56-7B27 turbofan (Larrañaga et al., 9 Jun 2026). Merit functions are normalized weighted sums relative to Jet A; for example,

$Meas(e,t)$5

and optimization is performed with a genetic algorithm under ASTM D1655-related property constraints, optional oxygenate exclusion, an olefin cap of 5%, and a seal swelling constraint (Larrañaga et al., 9 Jun 2026). The framework reports that optimized candidates outperform the training database across all merit functions and validates the best candidates through reactor simulations. This broadens AgentFuel from analytics into constrained fuel composition search, where user-defined objectives, feasibility filters, and surrogate-assisted exploration are coequal components.

6. Architectural patterns, control extensions, and outlook

A broader systems reading of AgentFuel is supported by several papers on agentic energy operation that are not nominally about fuel analytics alone, but formalize patterns directly reusable in fuel-domain agents. FleetAgent is a cloud-hosted multimodal assistant for autonomous fleets that consumes compact vectorized vehicle-to-network messages and produces structured natural-language narration, explanation, evaluation, and an intervention urgency score (Peng et al., 19 Jun 2026). Its VecFormer interface compresses vectors into embeddings and uses differentiable top-$Meas(e,t)$6 context selection, reducing uplink payload by up to 625 times compared with raw images and reducing KV-cache memory by 16.54 times compared with original text descriptions. On VecEval, it improves Lingo-Judge score by 16.8% and reduces intervention failure rate by 19.9% relative to Qwen2.5-VL-7B using language descriptions. This suggests a general AgentFuel principle: semantically compressed structured messages can be superior to raw streams for large-scale operational monitoring.

MasCOR extends the agent concept to stochastic co-optimization of e-fuel system design and operation by learning from globally optimal operational trajectories of a power-to-methanol plant (Kim et al., 3 Mar 2026). It conditions its actor on design and long-horizon return-to-go and carbon-to-go targets, and its critic on design plus renewable-trend tokens. In runtime terms, it acts as a fast operational surrogate inside a higher-level Bayesian design loop. Reported runtime for 1000 scenarios is 17.6 s on one NVIDIA H200 GPU versus 84.8 s for Gurobi, with a similar speed advantage at 10,000 scenarios (Kim et al., 3 Mar 2026). A plausible implication is that AgentFuel can serve not only as an evaluator of fixed systems, but as a reusable controller inside design search.

At the real-time control layer, sequence-aware reinforcement learning for a heavy-duty series hybrid electric vehicle shows that a Soft Actor-Critic agent with a Decision Transformer actor and GRU critic can come within 1.8% of Dynamic Programming on the HFET cycle, while generalized sequence-aware agents outperform feedforward baselines on unseen drive cycles (Jaleel et al., 6 Aug 2025). This suggests that temporal context and route-progress representation are critical when fuel minimization is coupled to state-of-charge management and long horizons.

Finally, OptAgent provides a transferable organizational pattern for domain-specific agent systems: a physics-informed runtime environment coupled to an agentic layer with 11 specialist agents and 72 Model Context Protocol tools (Jiang et al., 27 Jan 2026). Across about 4000 runs, the paper finds that a centralized two-stage planning mode yields the best balance of accuracy, token consumption, execution time, and cost, and that orchestrator quality matters more than specialist quality. This suggests that future AgentFuel systems may benefit from a similar decomposition: a strong orchestrator over typed tools and narrower specialists over physics-grounded or simulator-grounded execution.

Across these lines of work, the main limitations are consistent. Current systems remain sensitive to prompt quality, schema design, null or missing values, sparse fuel-property labels, incomplete chemistry coverage, and reward misspecification (Ma et al., 17 Nov 2025, Rittig et al., 2022, Larrañaga et al., 9 Jun 2026, Jiang et al., 27 Jan 2026). They also often lack calibrated uncertainty estimates, stronger cross-stage consistency checks, or closed-loop live deployment evidence. The most plausible near-term trajectory for AgentFuel is therefore not a single monolithic agent, but a layered architecture in which expressive evaluation, interpretable analytics, surrogate property models, inverse design, and simulator-grounded orchestration are combined under explicit validation and human-auditable control.