---
title: Shift × Lag Benchmark for Robotic Retargeting
url: https://www.emergentmind.com/topics/shift-x-lag-benchmark
type: topic
---

# Shift × Lag Benchmark for Robotic Retargeting

The Shift × Lag Benchmark is a reproducible stress test for robotic manipulation under combined scene perturbation and sensing delay. In the U-LAG framework, it is used to evaluate mid-execution goal retargeting when an object is abruptly displaced in the table plane and perception is made stale by an injected lag during approach. The benchmark is defined in simulation for pick, push, stacking, and peg insertion, and is specifically designed to test whether a retargeting layer can leave the low-level controller unchanged while re-aiming task goals $(g^{pre}, g^{contact}, g^{post})$ as new observations arrive [2510.02526].

## 1. Definition and conceptual scope

In its original formulation, the benchmark is a “Shift × Lag stress test” in PyBullet/PandaGym in which “the object undergoes abrupt in-plane shifts while synthetic perception lag is injected during approach” [2510.02526]. The term “shift” denotes a discrete perturbation of object pose in the table plane, and the term “lag” denotes a synthetic delay during which the controller continues to act on stale goals. The benchmark therefore isolates a specific failure mode of manipulation systems: action under percepts that are “late, noisy, or stale.”

The benchmark is tightly coupled to the U-LAG problem formulation. U-LAG is described as “a mid-execution goal-retargeting layer that leaves the low-level controller unchanged while re-aiming task goals (pre-contact, contact, post) as new observations arrive.” This positioning distinguishes the benchmark from motion retargeting and from generic visual servoing, because the evaluated object is not merely trajectory adjustment in the abstract, but “in-flight goal re-aiming” treated as a “first-class, pluggable module between perception and control” [2510.02526].

A common misconception is to read the benchmark as a generic manipulation benchmark. The underlying design is narrower and more technical: it is a controlled perturbation grid over shift magnitude and sensing lag, intended to quantify the behavior of retargeters under a single abrupt disturbance per trial and a single post-lag retargeting event. This suggests that its principal scientific role is not broad task coverage per se, but controlled stress testing of retargeting under delayed perception.

## 2. Simulation environment, tasks, and perturbation protocol

The benchmark is implemented in a simulation stack comprising “Physics engine: PyBullet (v3.x),” “Robot: Franka Panda via PandaGym [39],” “Sensor: single fixed RGB–D pinhole camera (60° FOV), known intrinsics & extrinsics,” and a “100 Hz Cartesian position servo $(\Delta x_{max} = 2.5$–$3 \text{ mm per tick})$” [2510.02526]. The use of a fixed RGB–D camera and a Cartesian position servo makes the benchmark explicitly about retargeting under stale perception rather than about active sensing.

The four benchmarked tasks are as follows.

| Task | Benchmark description | Key task detail |
|---|---|---|
| Pick-and-place (“Pick”) | grasp a 4 cm cube with parallel-jaw end-effector, lift, and place on a target location | prehensile manipulation |
| Planar pushing (“Push”) | non-prehensile push of the cube from current proxy center to a target $(x^*, y^*)$ on the table plane | planar non-prehensile manipulation |
| Block stacking (“Stack”) | pick a cube, then accurately place it on top of a base cube within tolerances $(tol_{xy}, tol_z)$ | placement with tolerance constraints |
| Peg-insertion (“Peg”) | pick a peg, approach a socket, and insert to specified depth; sensing via rim point-cloud | insertion with rim point-cloud sensing |

The perturbation protocol is triggered “when EE reaches a standoff waypoint.” At that predefined “approach trigger,” the benchmark performs three operations in sequence: “1) teleport the object, 2) hold the low-level controller against stale goals for a synthetic delay $L \in \{0,100,200,300,400\} \text{ ms}$, 3) after $L$, re-capture point cloud and invoke the retargeter once” [2510.02526]. A “simple scheduler blocks perception updates and retargeting until $L$ has elapsed,” and the “latency distribution” is “deterministic per trial (no random jitter).”

The object shift itself is also injected at the same trigger. The object pose is “instantaneously shifted in the table plane by a vector of magnitude $r \in \{0,2,4,6,8,10\} \text{ cm}$ and random direction $\theta \sim \mathrm{Uniform}[0,2\pi)$.” The synthetic shift function is

$$
S(o_t, t)=
\begin{cases}
o_t, & t < t_{trig} \\
o_t + [r \cos \theta,\; r \sin \theta,\; 0]^T, & t = t_{trig}.
\end{cases}
$$

Only one shift occurs per trial, and there are “no further disturbances thereafter” [2510.02526]. This single-shift design makes the benchmark a sharply delimited probe of mid-execution recovery, rather than a benchmark for persistent disturbance rejection.

## 3. Mathematical formulation and retargeting model

The benchmark formalizes lag at the point-cloud level. Let $C_t$ denote the true point-cloud at time $t$. The observed cloud at control time $t$ is defined as

$$
L_L(C_t) \equiv C_{t-L}.
$$

The latent object pose $o_t$ evolves quasi-statically and may be shifted by $S$ at $t_{trig}$ [2510.02526]. This formulation makes the benchmark a state-estimation-and-control problem under delayed observation rather than a purely geometric registration problem.

The retargeting module is written as

$$
\mathcal{R} : (C_{t-L}, C_t, state) \mapsto (g^{pre}, g^{contact}, g^{post}).
$$

In the benchmark, this map is the interface between perception and control. Because the low-level controller is intentionally held fixed, benchmark performance is attributable to the quality of goal retargeting rather than to controller redesign. That modularity is central to the benchmark’s stated contribution as a “pluggable retargeting interface” [2510.02526].

The main uncertainty-aware instantiation is UAR–PF. It “maintains $p(o_t \mid C_{0:t-L})$ via particles $\{x_i,w_i\}$ over planar state $s=[x,y,\theta,v_x,v_y,\omega]$.” The prediction step is

$$
s_i^{t+1} = F(s_i^t) + \eta,\qquad \eta \sim N(0,\Sigma_{process}),
$$

and the measurement update with $z_t =$ proxy from $C_t$ is

$$
w_i \propto \exp\!\left(-\frac12 \left\| [x_i,y_i]^T - z_t \right\|_{\Sigma_{obs}}^2 \right).
$$

Resampling occurs “when $\mathrm{ESS}<N/2$” [2510.02526]. The paper characterizes UAR–PF as an “uncertainty-aware retargeter that maintains a distribution over object pose under sensing lag and selects goals that maximize expected progress.” This implies that the Shift × Lag Benchmark is not merely evaluating retargeting latency or registration error; it is evaluating whether belief-aware retargeting materially improves task success under delayed and stale sensing.

## 4. Evaluation protocol, metrics, and reproducibility

The benchmark grid is a full factorial product of “shift magnitudes $r \in \{0,2,4,6,8,10\} \text{ cm}$ (6 levels) × lag $L \in \{0,100,200,300,400\} \text{ ms}$ (5 levels).” With “50 random seeds per $(r,L)$ cell,” this yields “$6\times 5 \times 50 = 1{,}500$ trials per task, 6,000 total across four tasks” [2510.02526]. The benchmark is therefore explicitly designed as a grid-based stress test rather than as an unconstrained randomized evaluation.

The reported performance metrics are:
- “Success rate (%) = # successful trials / 50”
- “Abort rate (%) = # trials aborted by slip/stall monitors / 50”
- “Mean end-effector travel $S = \sum_t \|\Delta x_t\|_2$ (logged per tick)”
- “Mean retarget latency (ms) = time between end of $L$ and new-goal selection”
- “Mean replan count = # additional re-sense/replan events per trial (budget $\le 1$, zero for no-retarget)”
- “Task-specific residual: final XY error, insertion depth error, etc.” [2510.02526]

Measured retarget latency is reported as “$O(1$–$30 \text{ ms})$ and added to injected $L$,” while the aggregate result is that retarget latency “tracks injected $L$ within $\pm 20 \text{ ms}$ overhead” [2510.02526]. These statements matter because they separate algorithmic overhead from the synthetic sensing delay that defines the benchmark axis.

Reproducibility is also specified in operational terms. The listed dependencies are “Python 3.7+; PyBullet; PandaGym; NumPy; SciPy; Open3D or equivalent for ICP,” with “Optionally PyTorch for UAR–PF belief updates.” The reproduction procedure includes cloning the environments, installing requirements, and running

```bash
python run_benchmark.py --task pick/push/stack/peg --shifts 0 2 … 10 --lags 0 100 … 400 --seeds 0–49
```

with results “saved in CSV/JSON for plotting” [2510.02526]. This emphasis on explicit grid execution and logged outputs is part of the benchmark’s claim to be “controlled” and “reproducible.”

## 5. Empirical behavior and benchmark results

At the hardest reported cell, $(r,L) = (10 \text{ cm}, 400 \text{ ms})$, the benchmark reports the following representative success rates [2510.02526].

| Task | no-retarget | ICP | Nearest | UAR | UAR–PF |
|---|---:|---:|---:|---:|---:|
| Pick | 0.0 | 0.24 | 0.10 | 0.10 | 0.86 |
| Push | 0.04 | 0.04 | 0.12 | 0.72 | 0.70 |
| Stack | 0.00 | 0.00 | 0.00 | 0.00 | 0.64 |
| Peg-insert | 0.085 | 0.271 | 0.297 | 0.339 | 0.343 |

Averaged over the full 30-cell grid, the benchmark reports:
- Pick: “UAR–PF 0.80+, UAR $\approx 0.65$, ICP $\approx 0.35$, Nearest $\approx 0.10$, no-retarget $\approx 0.05$”
- Push: “UAR 0.84, UAR–PF 0.82, ICP 0.45, Nearest 0.33, no-retarget 0.23”
- Stack: “UAR–PF 0.773, UAR 0.745, ICP 0.513, Nearest 0.123, no-retarget 0.250”
- Peg-insertion: “UAR–PF 0.343, UAR 0.339, Nearest 0.297, ICP 0.271, no-retarget 0.085” [2510.02526]

The benchmark also reports bounded end-effector travel across modes: “Pick: ~22–25 m,” “Push: ~21 m,” “Stack: ~44 m,” and “Peg: ~30 m.” In the paper’s summary, “Across 0–10 cm shifts and 0–400 ms lags, UAR–PF and ICP degrade gracefully relative to a no-retarget baseline, achieving higher success with modest end-effector travel and fewer aborts; simple operational safeguards further improve stability” [2510.02526].

Several patterns follow directly from these numbers. First, Pick and Stack show the clearest advantage for UAR–PF under severe shift and lag. Second, Push is unusual in that UAR slightly exceeds UAR–PF on the grid average while both strongly exceed no-retarget. Third, Peg-insertion remains difficult for all methods, with average success rates clustered in a narrower band. A plausible implication is that the benchmark exposes not only perception lag sensitivity but also task-dependent sensitivity to contact precision and residual geometric uncertainty.

## 6. Terminological scope and related uses of “shift” and “lag”

The phrase “shift” and the concept of “lag” appear in several distinct arXiv lines of work, but they do not denote the same object. In high-frequency finance, Hoffmann, Rosenbaum, and Yoshida study the “lead–lag parameter” $\vartheta$ between two non-synchronously observed asset processes and estimate it by maximizing a shifted Hayashi–Yoshida covariance contrast over a grid [1303.4871]. There, the central object is a continuous-time semimartingale model with an unknown temporal offset, not a manipulation stress test.

In social-platform forecasting, “Benchmark Datasets for Lead-Lag Forecasting on Social Platforms” defines Lead-Lag Forecasting (LLF) as predicting a temporally shifted lag channel from early observations of a lead channel, using large-scale arXiv and GitHub datasets with long-horizon outcomes such as accesses $\rightarrow$ citations and pushes/stars $\rightarrow$ forks [2511.03877]. The same summary explicitly frames this LLF setting as the “Shift × Lag Benchmark.” In macro-financial LLM evaluation, “Leakage-Aware Benchmarking of LLM Forecasting” uses “lag-shifted FRED macro variables” and a 12-month embargo to enforce “decision-time cleanliness” in factor ranking [2606.22719].

This suggests that “Shift × Lag Benchmark” is not yet a single field-wide standardized name. In robotics, it refers to a reproducible perturbation benchmark for mid-execution goal retargeting [2510.02526]. In the LLF summary, it denotes cross-channel, long-horizon forecasting on social platforms [2511.03877]. In leakage-aware macro forecasting, it denotes a shift-and-lag protocol for input construction under observability constraints [2606.22719]. The shared idea is controlled temporal misalignment, but the scientific questions differ: robustness of manipulation control, estimation of lead–lag structure, cross-channel forecasting, and leakage-controlled prediction.

For the robotics benchmark specifically, the most precise definition remains the one given by U-LAG: a “controlled, reproducible stress test of mid-execution retargeting under combined perception delay and abrupt scene changes” [2510.02526].

Source: https://www.emergentmind.com/topics/shift-x-lag-benchmark