2000 character limit reached
A remark on a paper of Krotov and Hopfield [arXiv:2008.06996] (2105.15034v2)
Published 31 May 2021 in q-bio.NC, cs.AI, cs.CV, and cs.LG
Abstract: In their paper titled "Large Associative Memory Problem in Neurobiology and Machine Learning" [arXiv:2008.06996] the authors gave a biologically plausible microscopic theory from which one can recover many dense associative memory models discussed in the literature. We show that the layers of the recent "MLP-mixer" [arXiv:2105.01601] as well as the essentially equivalent model in [arXiv:2105.02723] are amongst them.