Resolve the released model’s quantisation, channel-utilisation, and vocabulary inefficiencies

Address the approximately 6% perplexity penalty from post-training 4-bit quantisation, the roughly 47.9% of short-convolution channels that remain inert, and the oversized 49,152-entry vocabulary in the Daedalus-150M model.

Background

The conclusion identifies three unresolved engineering problems in the released Daedalus-150M model. Quantisation-aware training failed with a non-finite loss and was disabled, leaving a substantially larger 4-bit quality penalty than observed at smaller scale. Nearly half of the short-convolution channels are inactive, but export-time structural pruning is incompatible with the stock inference runtime. In addition, the 49,152-entry vocabulary is larger than scaling-law estimates for a 150-million-parameter model and consumes parameters that could otherwise be allocated to computational layers. The authors characterize these issues as engineering problems rather than fundamental architectural research questions, while noting that they remain unresolved in the released model.

References

What remains open is not the architecture but the engineering around it: a 4-bit penalty that quantisation-aware training was meant to remove, roughly half the convolution channels sitting inert because nothing in training discourages that, and a vocabulary inherited rather than chosen.

— Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference  (2608.20210 - Koutsiaris, 20 Aug 2026) in Section 12, Conclusion