Attribution of the unexplained residency-curve shape

Determine whether batching, KV-cache block quantization, or another serving-engine mechanism causes the observed residency curve that drops sharply and then plateaus, thereby explaining why the tested single-characteristic-time occupancy model fails to predict served KV-hit rates.

Background

The paper evaluates a single-characteristic-time occupancy model for predicting served KV-cache hit rates on the 70B fp8 serving engine. Despite passing component-level instrumentation checks, the model misses the measured hit rate substantially and fails to reproduce the observed residency curve.

The measured curve falls sharply and then plateaus, but the paper does not establish the underlying engine mechanism. It specifically identifies batching, block quantization, and other possible mechanisms as unconfirmed explanations, leaving causal attribution open.