Cochlear Tokens: Auditory Processing Elements

• Cochlear tokens are discrete, physiologically salient elements that encapsulate frequency-specific and temporally structured auditory information.
• They are modeled using biophysical frameworks such as Hopf oscillators and fluid-structure interactions to replicate nonlinear auditory responses and pitch perception.
• Engineered systems, including digital silicon cochleae and autoregressive models, leverage cochlear tokens to enhance speech representation and biomimetic audio processing.

Cochlear tokens are discrete, physiologically or computationally salient elements in auditory processing that emerge from the interaction of cochlear mechanics, neurophysiology, and modeling frameworks. They encapsulate the localized, frequency-specific, and temporally structured information within the cochlea and are considered to form the substrate upon which higher-level auditory perception and downstream machine learning tasks are built. Recent work has formalized and operationalized cochlear tokens via neurobiologically realistic models, mathematical abstractions, engineered metamaterials, digital hardware implementations, and autoregressive neural architectures.

1. Biophysical and Mathematical Foundations

The concept of cochlear tokens arises from the partitioning of the cochlea into frequency-selective units that respond preferentially to specific stimulus properties. Classical and contemporary physiological models, particularly those based on cascades of nonlinear Hopf oscillators, provide a rigorous mathematical framework for these units (Gomez et al., 2013, Hayton et al., 2018).

In these models, each cochlear section is abstracted as a Hopf relaxation oscillator near bifurcation, governed (for instance) by:

$$\frac{dz}{dt} = (u + i)\,\omega_{\mathrm{ch}}\,z - \omega_{\mathrm{ch}}\,|z|^2 z - \omega_{\mathrm{ch}}\,F(t)$$

where $z(t)$ is the oscillator state, $u$ measures distance to bifurcation, $\omega_{\mathrm{ch}}$ is characteristic frequency, and $F(t)$ is the input. The response of each section, especially after considering the viscous, low-pass filtering characteristics of the cochlear fluid, captures the emergence of combination tones and nonlinear frequency interactions that constitute canonical cochlear tokens.

Nonhyperbolic network models extend this abstraction by assembling arrays of such critical oscillators and coupling them unidirectionally, producing complex spectral-temporal response patterns:

$$\frac{dX}{dt} = A X - |X|^2 X + I(t), \quad X \in \mathbb{C}^{2N}$$

and enabling phenomena such as high-order compressive nonlinearities and traveling wave formation (Hayton et al., 2018). In these settings, bound states, scaling laws, and phase relations among oscillators quantitatively define the distributed structure of tokens across the cochlea.

2. Physical, Fluidic, and Structural Basis

The cochlea's physical structure—in particular its coiled, conical duct and the mechanical properties of the basilar and Reissner’s membranes—underpins the anatomical realization of cochlear tokens (Semotiuk et al., 2023, Ammari et al., 2019). Transverse resonance phenomena, engendered by the shape and boundary conditions of the cochlear spiral, create standing waves with spatially stable nodes and antinodes. These correspond mathematically to amplitude maxima given by:

$$S = 2A \cos\left( \frac{\theta - \theta(D)}{2} \right) \cos\left( \omega t + \frac{\theta + \theta(D)}{2} \right)$$

with the phase shift $\theta(D)$ reflecting local duct geometry.

Spatial locations—resonant "tokens"—are mapped directly to input frequencies, thereby creating a quasi-linear topographic representation of the spectral content along the cochlear spiral. The organ of Corti, via mechanotransduction at the hair cell array, samples these distributed pressure patterns and transduces them into neural signals, preserving the spatial and spectral granularity imparted by standing wave structure (Semotiuk et al., 2023).

Artificial cochleae using fluid-coupled arrays of Hopf resonators replicate this behavior by engineering size-graded resonator chains and modal decompositions:

$$p(x, t) = \mathrm{Re} \left\{\sum_n \alpha_n(t)\,u_n(x)\right\}$$

with nonlinear amplification and physically derived coupling mirroring biological selectivity and gain (Ammari et al., 2019).

3. Cochlear Tokens in Psychoacoustics and Neurophysiology

Psychoacoustic experiments, notably the pitch-shift paradigms of Smoorenburg, demonstrate that the perception of pitch and residue is determined by cochlear-level biophysical interactions rather than exclusively by cortical computation (Gomez et al., 2013). Experimental findings reveal two classes of pitch shifts—"first" and "second" pitch shifts—where the "second" is only accurately reproduced when the cochlear fluid's feed-forward coupling and viscous damping are included in the model. These mechanisms promote the emergence and amplification of subharmonic combination tones, effectively shifting the perceptual "center of gravity" of the auditory input.

Physiological recordings from the cat cochlear nucleus show that characteristic frequency-specific neuronal firing is tightly aligned with model predictions derived from cochlear preprocessing. This supports the hypothesis that essential pitch and spectral information is encoded pre-cortically in the pattern of cochlear tokens transmitted along the auditory nerve (Gomez et al., 2013).

4. Engineering, Digital Implementation, and Feature Extraction

The engineering of silicon cochleae and FPGA-based models leverages cascades of asymmetric resonators with active compression (e.g., as embodied in the CAR-FAC model) to synthesize multichannel output streams, each serving as a channel-local cochlear token (Xu et al., 2022, Lyon et al., 26 Apr 2024). These outputs are often post-processed as spike streams by leaky integrate-and-fire neurons, facilitating event-driven analysis.

Spectrotemporal feature extraction techniques such as FEAST (Feature Extraction using Adaptive Selection Thresholds) exploit the structure of these outputs to form higher-level tokens (e.g., event context windows and spectrotemporal receptive fields):

Extraction Stage	Representation	Mechanistic Basis
1-D FEAST	Temporal event context	Spike timing in one channel, matched with learned filters
2-D FEAST	Spectrotemporal window	Joint time-frequency structure across multiple channels

In benchmark speech recognition tasks (e.g., TIDIGITS), these engineered tokens enable high classification rates, with accuracies up to 97.71% using adaptive spiking feature representations (Xu et al., 2022).

Versioned upgrades in CAR-FAC (as in v2) introduce physiologically grounded modifications, such as improved highpass filtering to suppress DC quadratic distortion and secondary capacitance for more realistic neural synchrony, further refining the informational content and interpretability of cochlear tokens for auditory modeling (Lyon et al., 26 Apr 2024).

5. Autoregressive and Machine Learning Representations

Recent advances in biologically inspired speech representation learning operationalize cochlear tokens as the input layer for autoregressive models (Tuckute et al., 15 Aug 2025). This two-stage framework—AuriStream—first generates time-frequency "cochleagrams," discretizes them into a vocabulary of tokens (using quantization modules with, e.g., $2^{13}$ codebooks), and models token sequences with large-context autoregressive architectures akin to GPT.

The formal tokenization is described as:

$\text{token}_t = Q( C(t, \cdot) )$

where $C(t, f)$ is the cochleagram and $Q(\cdot)$ a quantization operator, yielding a sequence $\{\text{token}_1, \text{token}_2, ...\}$ that succinctly encodes the spectral-temporal structure of audio.

These tokens serve as the substrate for learning phoneme, word, and lexical semantic representations, yielding strong performance on benchmark tasks such as SUPERB with efficient model utilization (approximately 200 tokens/second). The autoregressive sequence model supports audio generation via token sequence continuation and reconstruction to approximate waveforms or cochleagram visualizations (Tuckute et al., 15 Aug 2025).

6. Significance, Modeling Implications, and Human Auditory Cognition

Cochlear tokens consolidate important insights for both scientific and engineering applications. Biophysically, they reflect the unimodal, frequency-specific, and nonlinear mechanics of the cochlea, providing the neural system with robust, noise-resilient input representations. In machine learning, their use as discrete units in autoregressive prediction frameworks aligns computational models with human auditory hierarchies, facilitating explainable and efficient speech modeling.

One recognized implication is that the generation and selection of auditory tokens is primarily a pre-cortical phenomenon, challenging prevailing views that assign complex feature extraction solely to central processing. Models demonstrate that both the emergence of pitch and higher-order spectral features are determined by peripheral nonlinear dynamics and fluid-structure interactions. This has significant consequences for the understanding of auditory perception, the diagnosis and remediation of sensorineural hearing loss, and the development of biomimetic auditory devices (Gomez et al., 2013, Lyon et al., 26 Apr 2024).

A plausible implication is that the abstraction of the auditory stream into discrete, context-sensitive cochlear tokens is a foundational operation that bridges biological and artificial paradigms, suggesting avenues for further investigations into adaptive auditory coding, auditory scene analysis, and neuro-inspired audio processing.

7. Comparative Table of Cochlear Token Realizations

Model / System	Nature of Cochlear Token	Key Functionality
Hopf oscillator cascade (Gomez et al., 2013, Hayton et al., 2018)	Local nonlinear oscillator state	Pitch encoding, residue, nonlinearity, subharmonics
Fluid-coupled metamaterial (Ammari et al., 2019)	Resonator mode response	Frequency separation, nonlinear gain
CAR-FAC model (Xu et al., 2022, Lyon et al., 26 Apr 2024)	Filter stage channel output (BM, IHC)	Dynamic filterbank coding, spike representation
Autoregressive framework (Tuckute et al., 15 Aug 2025)	Quantized cochleagram token	Input to sequence model for speech representation
Standing wave model (Semotiuk et al., 2023)	Stable amplitude node/antinode	Spatially mapped spectral marker

These realizations, whether physiological, mathematical, engineered, or computational, illustrate the diversity and centrality of cochlear tokens in auditory processing research.