Cube Bench: Dual Benchmark for Robotics and Optimization

• Cube Bench is a dual-purpose benchmark that evaluates robotic manipulation using precise Rubik’s Cube tasks, measuring accuracy and speed.
• It serves as a canonical unit-cube packing instance for testing integer programming formulations and comparing solver performance.
• The framework underscores the importance of standardized, reproducible evaluation protocols for both AI in robotics and operations research.

Cube Bench is a term denoting two distinct computational benchmarking frameworks unified by their cube-based structure: (1) a physical manipulation benchmark for robotics based on the Rubik’s Cube, assessing precision and sequential dexterity in embodied agents (Yang et al., 2022); and (2) a prominent decision problem in combinatorial optimization, where "Cube Bench" refers to a canonical unit-cube packing instance that serves as a litmus test for the strength of mathematical programming formulations (Allen et al., 2021). Both instantiations of Cube Bench explicitly stress the need for robust, generalizable, and quantifiable evaluation protocols in artificial intelligence and operations research.

1. Cube Bench in Robot Manipulation

The robotic Cube Bench is formalized as a task suite for general-purpose manipulators involving the sequential execution of face rotations on a standard $3 \times 3$ Rubik’s Cube under strictly controlled conditions (Yang et al., 2022). The setup requires the cube to rest on a planar surface, with robot end-effectors initiating each trial at a defined clearance and orientation. No human intervention is permitted post-initiation; all trials must be conducted autonomously. Protocol tiers are denoted Rubiks-M-N, with $M$ representing consecutive trials and $N$ the prescribed number of cube moves per trial. The protocol repeats standardized random-move sequences for robust cross-system comparison.

2. Performance Metrics and Formal Definitions

The benchmark employs both accuracy and speed as standardized evaluation criteria. Accuracy, $A$, is defined as the proportion of correctly faced stickers post-sequence:

$$A = \frac{1}{N_{\rm stickers}} \sum_{i=1}^{N_{\rm stickers}} \mathbf{1}[S_i = G_i]$$

where $S_i$ and $G_i$ are observed and goal sticker states, with $N_{\rm stickers} = 54$. Speed is quantified by the mean elapsed time per trial, $\bar{T}$. Compound performance can be represented via a single metric, $P$, e.g., $P = \bar{A}/\bar{T}$, for unambiguous system ranking. These formalizations uniquely enable direct comparison across heterogeneous robotic platforms and strategies (Yang et al., 2022).

3. Robotic Baselines and Empirical Evaluation

Baseline implementations on the PR2 manipulator demonstrate Cube Bench’s ability to illuminate comparative system strengths:

• Pose-Based (Dead-Reckoning): Utilizing a one-time pose estimate followed by open-loop execution mapped via a finite state machine, this approach is prone to accumulative error in long move sequences, with substantial failures at higher $N$.
• Pre-Touch Sensor-Aided: By incorporating optical pre-touch proximity sensors, this baseline enables real-time pose correction at re-grasp points ($\sim 5$ mm accuracy), improving robustness for longer sequences.

A summary table of mean times ($\bar{T}$, in seconds) for two Rubiks-M-N tiers is shown below:

Tier	Dead Reckoning (PR2)	Pre-Touch Aided (PR2)
Rubiks-1-10	248.4	215.4
Rubiks-1-20	463.5	448.0

In all high-performance cases, pre-touch sensing outperforms dead-reckoning, particularly at higher $N$. The benchmark was further ported to the HERB manipulator using a push–grasp approach, attesting to its adaptability (Yang et al., 2022).

4. Generalization, Limitations, and Extensions

Cube Bench is designed for any multi-fingered or multi-arm system capable of face-level grasp and rotation. It supports a variety of sensory modalities—vision, tactile, pre-touch, or hybrid—and evaluates both sub-centimeter precision and the integrity of long-horizon sequential execution. However, all sequences originate from a fixed cube orientation, lacking randomized scrambles, and there is no per-move error evaluation. Proposed extensions include additional randomization (Rubiks-M-N-R), move-level error tracking, robustness under adversarial environmental variations, and dynamic disturbance response (Yang et al., 2022).

5. Cube Bench in Combinatorial Packing: The Pigeonhole Cube Instance

In optimization, the Cube Bench instance serves as a computational stress test for three-dimensional box-packing formulations (Allen et al., 2021). The canonical instance asks whether $12$ unit cubes can fit into a $1 \times 1 \times 11$ box (with $\varepsilon \to 0^+$ slack), a case trivially infeasible via the pigeonhole principle but nontrivial for integer programming solvers.

The benchmark exposes the limitations of the Chen/Padberg precedence-based formulation, whose LP relaxations yield weak fractional solutions (integrality gap $\sim$8%) and are empirically intractable for $n \geq 11$ (no major solver closes in 1 hour for $n=12$). In contrast, the space-indexed formulation discretizes the container, introducing a binary variable $\mu_{x,y,z}^1$ for placing a cube at grid cell $(x,y,z)$. This yields both tight LP relaxations and decisive infeasibility proofs at the root node, with Gurobi solving Pigeon-12 instances in under 0.1 s—orders of magnitude faster than the classical approach (Allen et al., 2021).

6. Computational Results and Formulation Comparison

A summary of solver performance on the Cube Bench packing instance is shown below:

$n$	Chen/Padberg (s)	Space-Indexed (s)
10	1381.4	<1
11	>3600	<1
12	>3600	<1

On large-scale instances discretized to $10^6$ cells, space-indexed models maintain sub-minute solve times and sub-1% optimality gaps. The approach generalizes efficiently to non-cube, multi-type box-packing instances provided all objects are integer-aligned (Allen et al., 2021).

7. Significance and Impact

Cube Bench serves as a reproducible, quantifiable probe for embodied spatial reasoning in robotics and a decisive testbed for packing model strength in integer programming. Its dual usage illustrates both the demand for rigorous manipulation benchmarks and the importance of modeling discipline in computational optimization. In robot manipulation, Cube Bench enables fine-grained diagnosis of perception, planning, and actuation modules under repeatable conditions. In mathematical optimization, it offers a canonical instance for benchmarking new formulations, with space-indexing now recommended as the default modeling paradigm for integer-aligned 3D packing (Yang et al., 2022, Allen et al., 2021).