Optimal throughput–efficiency–archival trade-off in asynchronous ShinkaEvolve

Determine the optimal trade-off between throughput, sample efficiency, and archival fidelity ("off-archiveness") in a fully asynchronous implementation of ShinkaEvolve that leverages both a job queue and a proposal queue, where proposals may be generated ahead of completed evaluations and faster-to-query models' proposals are executed earlier.

Background

The paper describes an alternative fully asynchronous architecture for ShinkaEvolve that uses both a job queue and a proposal queue to increase throughput. In this setup, new code proposals can be generated in advance, potentially not reflecting the most recent archive state, and proposals from faster-to-query models may be executed earlier, affecting search dynamics.

The authors note that while this design can improve throughput, it introduces “off-archiveness,” raising unresolved questions about how to balance speed, sample efficiency, and fidelity to the evolving archive. Identifying the optimal balance is crucial for maximizing search quality without sacrificing efficiency.