Resolve selection bias in the supporting ChainBench manifest

Determine whether selection bias introduced by rebuilding the 143-chain ChainBench manifest affects the reported memory-related task-success gains and their generalizability beyond that manifest.

Background

The supporting ChainBench study used a rebuilt manifest that removed and added chains according to a criterion not frozen in advance. Although sensitivity analyses remain positive, the undocumented manifest reconstruction may have selected chains with different motor feasibility or difficulty, so the population-level interpretation remains unresolved.

References

The rebuild was not documented contemporaneously and its criterion, motor feasibility rather than observed outcome, was not frozen in advance, so selection bias remains unresolved and we make no population-level claim beyond this manifest.

— Beyond Success Rate: Interventional Evaluation of Memory in Vision-Language-Action Systems  (2610.23372 - Luo et al., 16 Oct 2026) in Appendix B, Section “ChainBench E1B and E1”