Adapt FOLD’s inference-time hyperparameters efficiently

Develop an efficient inference-time method for adapting the FOLD window size W and concentration factor α during decoding instead of keeping both hyperparameters manually selected and fixed.

Background

FOLD requires two inference-time hyperparameters: the local-window size W and the far-memory concentration factor α. In the experiments, these values are manually selected and held constant throughout decoding.

The limitations section explicitly leaves efficient dynamic adaptation unresolved, indicating a concrete direction for improving robustness across target models, tasks, and changing input conditions.

References

The window size $W$ and concentration factor $\alpha$ are set manually and remain fixed during decoding. Efficient adaptation of these parameters during inference is left for future work.

— FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding  (2610.20392 - Li et al., 14 Oct 2026) in Limitations