Beyond Static Snapshots: A Grounded Evaluation Framework for Language Models at the Agentic Frontier

Abstract: We argue that current evaluation frameworks for LLMs suffer from four systematic failures that make them structurally inadequate for assessing deployed, agentic systems: distributional invalidity (evaluation inputs do not reflect real interaction distributions), temporal invalidity (evaluations are post-hoc rather than training-integrated), scope invalidity (evaluations measure single-turn outputs rather than long-horizon trajectories), and process invalidity (evaluations assess outputs rather than reasoning). These failures compound critically in RLHF, where reward models are evaluated under conditions that do not hold during RL training, making reward hacking a predictable consequence of evaluation design rather than a training pathology. We propose the Grounded Continuous Evaluation (GCE) framework and present ISOPro, a simulation-based fine-tuning and evaluation system. ISOPro replaces the learned reward model with a deterministic ground-truth verifier, eliminating reward hacking by construction in verifiable-reward domains, and operates on LoRA adapter weights updatable on CPU, reducing the hardware barrier by an order of magnitude. We validate ISOPro on a resource-constrained scheduling domain with six difficulty tiers, demonstrating capability emergence visible only through continuous evaluation, an implicit curriculum that forms without researcher curation, and a 3x accuracy improvement over zero-shot baselines, all on consumer hardware with 0.216% trainable parameters.

• The paper introduces the Grounded Continuous Evaluation (GCE) framework to overcome static evaluation failures in agentic language models.
• It details ISOPro, a simulation-based system that integrates interaction-grounded prompt sampling and continuous training for precise multi-step assessments.
• Empirical results on consumer-grade hardware reveal clear capability emergence and honest performance reporting while eliminating reward hacking.

A Grounded Continuous Evaluation Framework for Agentic LLMs

Motivation and Taxonomy of Evaluation Failures

Standard LLM evaluation protocols—benchmarks, static accuracy, and post-hoc metrics—fail to capture the dynamics and complexities of agentic deployment, especially for long-horizon, multi-step scenarios. The paper introduces a well-specified taxonomy of structural validity failures in existing LLM evaluation frameworks, grounded in measurement validity theory. Four critical validity gaps are formalized:

• Distributional invalidity: Evaluation distributions are artificially researcher-constructed, diverging from real-world deployment exposure, causing reward models (RMs) to generalize poorly under reinforcement learning (RL) policy shift.
• Temporal invalidity: Static, point-in-time evaluations obscure training dynamics, missing abrupt phase transitions or capability emergence during fine-tuning.
• Scope invalidity: Single-turn assessments ignore multi-step reasoning or trajectory-level coherence, critical in agentic utility and sequential decision making.
• Process invalidity: Output-based evaluations reward superficially fluent or preferred answers, regardless of underlying reasoning—a driver of reward hacking and sycophancy.

These failure modes become compounding, particularly under RLHF, where the reward model is misaligned with post-deployment input distributions and diverges further as the policy learns to exploit annotation-induced blind spots.

The Grounded Continuous Evaluation (GCE) Framework

Addressing these intertwined validity failures, the authors propose the Grounded Continuous Evaluation (GCE) framework, built on three foundational principles:

1. Interaction-grounded prompt sampling: Evaluation problems are sampled proportional to deployment distributions, validated via empirically-matched marginals and coverage, addressing distributional validity.
2. Training-integrated, continuous evaluation: Evaluation is tightly coupled to training iterations, enabling rapid feedback on evolving model capabilities and the detection of dynamics invisible to snapshot-based assessment.
3. Simulation-based agentic assessment: Models are assessed on realistic, multi-step tasks in structured environments, with trajectory-level metrics, counterfactual probes, and granular checkpointing, directly targeting scope and process validity.

ISOPro: Reference Implementation and Experimental Results

To operationalize GCE, ISOPro is introduced—a simulation-based fine-tuning and continuous evaluation system. It demonstrates the GCE principles over resource-constrained scheduling (RCPSP) with six progressively challenging tiers, using Qwen 2.5 3B Instruct as the backbone and leveraging LoRA adapters for memory- and compute-efficient parameter updates. ISOPro’s core mechanisms include:

• Process-level gradient supervision: Training loss is accumulated only over reasoning steps leading to verified correctness, ensuring the gradient directly shapes reasoning, not only output accuracy.
• Verifier-based rejection sampling loop: Rollouts are filtered using ground-truth deterministic verifiers, architecturally precluding reward hacking.
• Implicit-curriculum buffer: The replay buffer accumulates correct traces, naturally evolving the training distribution as the model’s capabilities progress—negating the need for researcher-curated curricula.
• Activation-guided LoRA targeting: LoRA updates concentrate on layers empirically shown to mediate constraint reasoning.

ISOPro operates fully on consumer hardware: all experiments are conducted on an Apple M1 laptop with <8GB RAM for 3B parameter models, training 0.216% of parameters. This markedly lowers the resource barrier relative to standard RLHF pipelines, which require loading two full model copies in VRAM.

Empirical evaluation Figure 1 reveals strong gains: ISOPro with LoRA attains the highest per-tier accuracy for T0 (100%), T1 (66.7%), and T3 (66.7%). All methods (including zero-shot and in-context learning baselines) stall at 0% for resource-intensive T2 and generalization tier T5, exposing honest capability limits—documented granularity not recoverable from snapshot metrics.

Figure 1: Per-tier accuracy across all evaluation conditions; ISOPro + LoRA (red) achieves top results for T0, T1, and T3, with GCE revealing explicit capability boundaries.

Continuous evaluation uncovers inflection dynamics Figure 2: the rate of successful rollouts doubles sharply after three training iterations, corresponding to a marked drop in cross-entropy loss. This temporal emergence would be entirely invisible in conventional checkpointed evaluation.

Figure 2: Training dynamics showing rollout hit rate and loss; continuous evaluation captures the learning inflection that discrete checkpoints miss.

Capability emergence mapping Figure 3 illustrates which competencies manifest at which training iteration. T0 and T3 are present initially; T1 enters at iteration 2, T4 at 3. Importantly, T2 and T5 are never reached, giving precise characterization of current agentic frontiers and highlighting ISOPro’s honest reporting of capability bounds.

Figure 3: Heatmap depicting the first appearance of correct traces per tier; T2 Resource and T5 Full Composition remain unsolved after all training iterations.

Ablation Analysis and Mechanism Insights

Ablation studies clarify the contributions of key mechanisms. Removing chain-of-thought (CoT) reasoning or buffer accumulation substantially degrades mean accuracy and increases variance. Without buffer accumulation, accuracy drops by 12 percentage points and variance increases fourfold. CoT ablation especially hurts deadline-constrained tiers. Interestingly, randomizing LoRA layer selection achieves performance statistically indistinguishable from activation-guided placement at this scale and task.

ISOPro vs. RLHF: Structural and Practical Implications

Crucially, ISOPro completely removes the learned reward model, replacing it with a deterministic verifier. This architectural change eliminates reward hacking by construction when the reward is verifiable, distinguishing reward hacking as an evaluation design fault, not an inherent training pathology. ISOPro also abolishes RLHF’s “dual-model” memory penalty: LoRA updates proceed directly (on CPU), and the base model remains frozen.

However, some RLHF strengths are not matched: SFT cold start, partial credit assignment, and coverage of non-verifiable objectives. ISOPro and GCE are thus optimal for verifiable-reward scenarios where ground-truth evaluation can be mechanized.

The paper’s methodological convergence with DeepSeek-R1’s GRPO is notable—both frameworks independently arrive at the core architectural principle: in verifiable-reward domains, the verifier itself is the reward signal, and a learned proxy is unnecessary. ISOPro demonstrates that this insight is valid even at consumer-hardware, small-model scales.

Broader Impact and Future Research

Practically, ISOPro and GCE make agentic LLM evaluation and fine-tuning dramatically more accessible and reproducible, lowering hardware barriers for academic and non-profit labs. Theoretically, honest, granular, and continuous evaluation supports more reliable detection of overoptimization and reward specification drift, with direct implications for alignment and safety-critical deployment.

Important future directions include validating GCE in other domains (formal logic, code, scientific reasoning), scaling to larger models, and extending evaluation to non-verifiable behaviors via rubric-based segment scoring. The framework’s continuous curriculum and fine-grained emergence tracking are also promising tools for characterizing agent development and understanding scaling laws in reasoning tasks.

Conclusion

This paper presents a rigorous framework, GCE, that directly targets the systematic validity failures of static evaluation protocols in agentic LLM settings. Through ISOPro, the framework is demonstrated to be practical, efficient, and accessible, yielding measurable improvements in agentic capability without reward hacking. The architectural principle—that for verifiable tasks, a ground-truth verifier renders the reward model obsolete—finds independent support in frontier-scale work and provides a sturdy foundation for future methodological improvements and the democratization of agentic LLM evaluation.