Dice Question Streamline Icon: https://streamlinehq.com

Conjecture: Clock algorithm functions as an internal error-correcting code

Establish whether the Clock algorithm used by transformer-based large language models to perform addition serves as a robust internal error-correcting code that mitigates representational noise during numerical computation.

Information Square Streamline Icon: https://streamlinehq.com

Background

The paper’s conclusion and appendix discuss potential reasons for the adoption of helix-based addition, including robustness and redundancy akin to error-correcting codes.

Preliminary experiments reported in the appendix show that ‘linear addition’ using a high-R2 linear fit underperforms GPT-J, suggesting that redundancy in helix-based computation may confer fault tolerance.

References

We conjecture that LLMs use the Clock algorithm as a form of robust, error correcting code.

— Language Models Use Trigonometry to Do Addition (2502.00873 - Kantamneni et al., 2 Feb 2025) in Appendix, Section "Why Use the Clock Algorithm at All?"