Precise natural-language control of AutoDeco decoding

Determine whether joint training of the base transformer and the AutoDeco temperature and top-p heads enables precise, absolute control of token-level decoding parameters via natural-language commands (such as directives for low diversity or no randomness), and characterize the mechanism underlying AutoDeco’s emergent prompt-driven adjustments to predicted temperature and top-p values.

Background

The paper reports an emergent capability in AutoDeco: the model can interpret high-level natural-language commands about creativity or certainty and accordingly adjust its predicted temperature and top-p on a token-by-token basis. While targeted training with a ranking loss improves the consistency of these adjustments, the authors observe that the resulting control is not yet precise or absolute.

They explicitly state that they lack a conclusive understanding of the phenomenon and hypothesize that achieving fine-grained control may require joint training of the base LLM and the AutoDeco heads. The released models therefore omit this experimental control feature pending further investigation.