Specialization on truly private codebases

Ascertain whether repository specialization with SERA yields comparable performance improvements on truly private codebases that were not included in base-model or teacher-model training data and that models have never seen, given the lack of directly testable evaluation instances.

Background

Specialization experiments target public repositories (Django, Sympy, Sphinx) using evaluation instances from SWE-bench Verified, which may overlap with model pretraining data.

The authors note potential bias and explicitly state they have not verified specialization on truly private codebases due to the absence of evaluation data.