Resolve the generalization and evaluation limitations of STRATA

Establish the robustness and generality of STRATA by replicating its results across language-model backbones, recovering contrast relations without a predefined vocabulary of opposite operators, and conducting a controlled experiment that fixes the evaluation set while varying only the number of accumulated experiences.

Background

STRATA is evaluated only with the DeepSeek-V4 model family, and the effectiveness of its contrast mechanism depends on whether the underlying backbone follows an injected warning. The contrast edges are currently generated from a hand-specified lexicon of opposing operators, which limits the mechanism's domain generality. In addition, the benchmark comparisons confound the number of accumulated experiences with task difficulty and distribution shift. The authors therefore explicitly identify replication across backbones, vocabulary-independent opposition discovery, and a controlled scaling study as unresolved directions.

References

Several directions remain open. Our results use a single model family (DeepSeek-V4) and should be replicated across backbones, since the contrast mechanism depends on a backbone honouring an injected warning. Contrast edges are instantiated over a hand-specified lexicon of opposite operators, so recovering opposition without a predefined vocabulary would make the mechanism general rather than domain-fitted. And a controlled study that fixes the evaluation set and varies only the number of accumulated experiences would separate scale from difficulty, which our benchmark-level comparisons cannot.

— STRATA: Self-Maintaining Structured Experience Graphs for Self-Evolving Language Agents  (2610.19386 - Zhu et al., 14 Oct 2026) in Section Conclusions and Future Work