Churn-aware routing-table replanning

Develop and evaluate a churn-aware replanning algorithm for CacheRoute that limits unnecessary key-to-destination movement while balancing stale-plan penalties against cache rewarming costs.

Background

CacheRoute recomputes its routing table as estimated per-key rates drift, but recomputing LPT from scratch can change most key-to-destination assignments even when assignment counts remain unchanged. In the reported drift experiment, fresh replanning caused substantial cache churn, while installing a fresh table also produced a transient KV-hit drop during rewarming.

The paper identifies placement hysteresis and a churn-aware replanner as possible ways to reduce unnecessary movement, but does not evaluate such an algorithm. The unresolved problem is therefore to design and assess replanning that trades off stale-plan degradation against rewarming cost.

References

A deployment should therefore preserve placements until the measured staleness penalty exceeds the warm-up penalty. We have not implemented a churn-aware replanner.

— CacheRoute: Planned Prefix-Affinity Routing for Large-Scale LLM Serving  (2608.19677 - Cheng, 20 Aug 2026) in Section 4.3, "Rate drift and remapping"; Appendix, Section "Replanning Under Drift"