Suboptimal-Value-Bench Benchmark

• Suboptimal-Value-Bench is a robotic manipulation benchmark that defines suboptimal segments (hesitation and retry) with precise, human-verified annotations.
• It uses a two-stage annotation pipeline combining automated VLM proposals with detailed human refinement to create accurate per-frame value curves over 800 trajectories.
• The benchmark employs specialized metrics like Hesitation-RMSE and Retry-VOC to assess temporal calibration and directional value estimation, guiding improved policy learning.

Searching arXiv for papers that explicitly mention “Suboptimal-Value-Bench” and closely related work. Suboptimal-Value-Bench is a multi-embodiment benchmark for evaluating robotic value models on suboptimal manipulation trajectories, introduced alongside the World Value Model (WVM) work and referred to in that paper as “mygreen” (Wang et al., 23 Jun 2026). It was designed to address a specific blind spot in existing value-estimation suites: standard evaluations largely contain expert-only trajectories with monotonic reward progressions, whereas real robotic data is dominated by hesitations, retries, and other non-progress segments. The benchmark therefore provides dense, human-labeled frame annotations on suboptimal segments, enabling direct measurement of whether a value model can track task progress when behavior stalls, regresses, and then recovers (Wang et al., 23 Jun 2026).

1. Origin and benchmark objective

Suboptimal-Value-Bench was introduced to complement expert-only evaluations of robotic value models (Wang et al., 23 Jun 2026). The motivating claim is that value estimation requires temporal grounding over past context and forward-looking planning over future outcomes, but many existing robotic value models are built on Vision-LLM backbones pretrained on static or temporally sparse observations. In that setting, a model can perform well on monotonic expert demonstrations yet fail to recognize hesitation plateaus or retry regressions in mixed-quality data.

The benchmark operationalizes two suboptimal modes. A hesitation segment is a period in which the robot does not advance the task state, including stationary behavior or task-irrelevant micro-movements. A retry segment is a failed attempt followed by release or retraction and then a subsequent new attempt. These cases are central because ordinary Value-Order Correlation (VOC) on expert trajectories measures monotonicity, whereas Suboptimal-Value-Bench evaluates stability on plateaus and correct ordering on regressions (Wang et al., 23 Jun 2026).

A common misconception is that any value benchmark based on frame ordering is sufficient for mixed-quality data. Suboptimal-Value-Bench directly contradicts that assumption: expert-only suites primarily test whether values increase on successful trajectories, while this benchmark asks whether a model remains constant during non-progress and decreases appropriately during backtracking. That distinction is the benchmark’s defining contribution.

2. Dataset composition and task coverage

The benchmark contains 800 human-annotated trajectories, each with dense per-frame value targets focused on hesitation and retry (Wang et al., 23 Jun 2026). It spans three embodiments: AgileX, ARX, and RoboSuite. Every trajectory contains a human-verified suboptimal segment and resumed progress; the annotation JSON includes a task_completed boolean, so both completed and non-completed rollouts appear, although the completion distribution is not reported.

Embodiment	Tasks	Trajectories
AgileX	carrot off plate; carrot on plate; mickey box; sausage pot	200
ARX	flip bottles; open box; split cups; stack bowls; stack cups	300
RoboSuite	lift; pick & place bread, can, cereal, milk; stack	300

The appendix-level counts are further partitioned by suboptimal type. AgileX contains 100 hesitation and 100 retry trajectories over 42.0 min total; ARX contains 150 hesitation and 150 retry trajectories over 51.3 min; RoboSuite contains 150 hesitation and 150 retry trajectories over 119.9 min; the full benchmark totals 400 hesitation, 400 retry, and 213.2 min (Wang et al., 23 Jun 2026).

The modalities per trajectory are single-view RGB video, a natural-language task description, dense frame-level value targets, and suboptimal segment metadata including the type, segment endpoints $(m,n)$, total length $T$, task_completed, and a short summary. Evaluation resamples videos to 2 fps for simulation and 3 fps for real data so that per-trajectory frame counts are comparable; precise per-trajectory frame count distributions are not reported (Wang et al., 23 Jun 2026).

In relation to prior resources, the benchmark is explicitly contrasted with expert-only suites such as OXE, RoboCOIN, and EgoDex. Those suites primarily measure monotonic progress and success and do not isolate hesitation plateaus or retry regressions, nor do they generally provide frame-level annotations for suboptimal intervals (Wang et al., 23 Jun 2026).

3. Annotation pipeline and construction of frame-level value targets

The annotation protocol is a two-stage pipeline (Wang et al., 23 Jun 2026). First, a proprietary VLM produces a coarse proposal of non_progress_segments from sampled frames and the task description. Second, human annotators refine those intervals at frame-level resolution in a custom interface, including cases in which the VLM proposal is empty. The final labels per trajectory are the suboptimal type, endpoints $(m,n)$, total length $T$, task_completed, and a one-sentence summary.

From these labels, the ground-truth per-frame value curve $v_t \in [0,1]$ is deterministically constructed as a four-point piecewise-linear curve through

$(0,0),\ (m,v_m),\ (n,v_n),\ (T-1,1),$

with linear interpolation between control points and clamping outside $[0,T-1]$ (Wang et al., 23 Jun 2026).

For hesitation, if the plateau length is $x=n-m$, progress remains constant across $[m,n]$, and the remaining effective frames cover the unit interval at the same rate:

$v_m = v_n = \frac{m}{T-1-x}.$

The corresponding slopes are $T$0, then $T$1, then $T$2 (Wang et al., 23 Jun 2026).

For retry, with uniform-speed retraction of length $T$3 and forward/backward rate

$T$4

the control values are

$T$5

When $T$6, the middle segment is a symmetric negative-slope leg of a “V”; when $T$7, the value would become negative and is clamped to $T$8 (Wang et al., 23 Jun 2026).

This construction makes the benchmark unusually explicit about the semantics of suboptimality. Hesitation is encoded as a plateau with invariant progress, whereas retry is encoded as a regression followed by renewed ascent. A plausible implication is that the benchmark evaluates not merely temporal smoothness but mode-specific temporal semantics: constancy under stalls and reversal under failed attempts.

4. Evaluation metrics and protocol

The benchmark uses a mixed metric suite because a single correlation measure is insufficient across both suboptimal modes (Wang et al., 23 Jun 2026). Expert-VOC is standard VOC on expert demonstrations over the full trajectory, where ground-truth progress is strictly increasing. For hesitation windows, correlation is ill-defined because the target is constant, so the benchmark uses Hesitation-RMSE:

$T$9

where $(m,n)$0 is the set of frames in a hesitation segment (Wang et al., 23 Jun 2026).

For retries, the benchmark defines Retry-VOC as VOC restricted to the monotonically decreasing retraction interval. VOC is implemented as Spearman rank correlation over an evaluation window $(m,n)$1:

$(m,n)$2

In Expert-VOC, $(m,n)$3 is the full trajectory; in Retry-VOC, $(m,n)$4 over the decreasing segment (Wang et al., 23 Jun 2026).

The evaluation is reported separately on AgileX, ARX, and RoboSuite and then averaged. The baseline harness resamples trajectories to a common target frame rate and applies each baseline’s recommended sampling protocol. The listed baselines are GVL, VLAC, Robometer, TopReward, RoboReward, and Robo-Dopamine; the paper notes that most are VLM-based value or reward models, while Robometer and RoboReward use multi-anchor prefix evaluation (Wang et al., 23 Jun 2026).

The protocol-level importance of these metrics is that they test different failure modes. Hesitation-RMSE penalizes instability on plateaus, whereas Retry-VOC penalizes incorrect ordering during regressions. This suggests that benchmark success requires a value model to be both temporally calibrated and temporally directional.

5. Reported results and use in policy learning

On Suboptimal-Value-Bench, the paper reports that WVM (“myblue”) achieves the best average Hesitation-RMSE and Retry-VOC among the compared methods (Wang et al., 23 Jun 2026). The average Hesitation-RMSE values are 0.05 for myblue, 0.14 for GVL, 0.14 for Robometer, 0.31 for TopReward, 0.21 for RoboReward, 0.49 for Robo-Dopamine, and 0.51 for VLAC. The average Retry-VOC values are 0.78 for myblue, 0.62 for GVL, −0.16 for Robometer, 0.00 for TopReward, and −0.37 for VLAC, with VLAC marked as ill-defined in this setting for some embodiment-specific cases (Wang et al., 23 Jun 2026).

The paper also gives embodiment-specific results. For example, on AgileX hesitation, myblue attains 0.07 RMSE versus 0.11 for GVL and 0.13 for Robometer; on ARX hesitation, myblue attains 0.05 versus 0.14 and 0.12; on RoboSuite hesitation, myblue attains 0.04 versus 0.16 and 0.16 (Wang et al., 23 Jun 2026). For retry, myblue attains 0.79 on AgileX, 0.79 on ARX, and 0.75 on RoboSuite, exceeding the corresponding GVL scores of 0.73, 0.76, and 0.43 (Wang et al., 23 Jun 2026).

The benchmark is also tied to downstream policy extraction. The paper uses myblue’s per-frame values to guide filtered behavioral cloning and advantage-weighted regression on suboptimal datasets. For a chunk anchored at frame $(m,n)$5 with

$(m,n)$6

the chunk-level advantage proxy is

$(m,n)$7

The weighted behavioral cloning objective is

$(m,n)$8

with three reported weight schemes:

• Filtered BC: $(m,n)$9
• Filtered BC (top-70%): $T$0, where $T$1 is the 30th percentile of $T$2
• AWR: $T$3 (Wang et al., 23 Jun 2026)

Empirically, all three myblue-guided variants outperform vanilla BC in both simulation and real AgileX tasks. In the benchmark’s intended workflow, low Hesitation-RMSE and high Retry-VOC are therefore not only descriptive metrics; they are treated as evidence that a value model is reliable enough to serve as a data filter or advantage proxy for mixed-quality policy learning (Wang et al., 23 Jun 2026).

6. Scope, limitations, and broader use of the name

Suboptimal-Value-Bench is not presented as a universal benchmark for all forms of suboptimal sequential decision making. Its current scope is robotic manipulation, and the paper notes that current coverage is primarily pick-and-place and short-horizon manipulations; broader dexterous and longer-horizon tasks remain future work (Wang et al., 23 Jun 2026). The annotation pipeline relies on a proprietary VLM for pre-segmentation, human verification mitigates proposal bias, and inter-annotator agreement is not reported. These points matter because they delimit what can be concluded from benchmark scores.

The name also appears more broadly as a proposed benchmark label in several neighboring research areas, although those works do not introduce the same released robotic dataset. In in-context reinforcement learning, the Decision Importance Transformer paper states that its datasets, splits, and baselines can serve as the backbone for a “Suboptimal-Value-Bench” centered on learning from suboptimal historical trajectories (Dong et al., 27 Jan 2026). In learning to branch for MILP solvers, SORREL presents an experimental design that can directly inform a benchmark for learning from suboptimal-value demonstrations (Feng et al., 2024). In explainability, “Suboptimal Shapley Value Explanations” states that its theory and protocol can be turned into such a benchmark for detecting and mitigating suboptimal baselines in Shapley explanations (Lu, 17 Feb 2025). In multi-agent reinforcement learning, the MRVF paper frames its stable-point diagnostics as directly relevant to a benchmark focused on suboptimal convergence in value-factorization methods (Tao et al., 7 Apr 2026).

This suggests that “Suboptimal-Value-Bench” functions in two distinct senses. In the strict sense, it denotes the released robotic evaluation suite introduced with WVM (Wang et al., 23 Jun 2026). In a broader editorial sense, it has become a reusable label for benchmark designs that evaluate value estimation or value-guided learning under suboptimal data, suboptimal demonstrations, or suboptimal convergence regimes across multiple subfields (Dong et al., 27 Jan 2026, Feng et al., 2024, Lu, 17 Feb 2025, Tao et al., 7 Apr 2026).