Evaluate generalization and advantage over competent external implementations

Establish, using independent labels and broader held-out workflows, whether Matoi provides a material advantage over competent external protected-incremental implementations and whether its reported behavior generalizes beyond the finite same-project experiments.

Background

The experiments use bounded finite languages, authored workflows, same-project verification, and small prescribed task populations. The paper explicitly states that these studies do not establish broad generalization or superiority over competent external implementations. Independent semantic labels, broader held-out workflows, and a matched external baseline are therefore required to determine whether the observed cost and correctness results extend beyond the demonstrated prototypes and workloads.

References

Independent labels, broader held-out workflows and a material advantage over competent external implementations remain unestablished; their evaluation is scientific work, not an inference from larger test counts.

— Matoi: Selective Revalidation of Consequence Evidence for Agent Authorization  (2610.21394 - Wang, 15 Oct 2026) in Section 18, Discussion and Limitations, paragraph beginning “Limits of the demonstrated result”