Phonosemantic Attention Patterns

• Phonosemantic attention patterns are quantifiable tendencies where models and humans prioritize specific phonetic elements to infer semantic or affective meanings.
• Empirical studies using regression and attention-fraction analyses reveal pronounced first- and last-phoneme effects linked to emotion judgments across languages.
• Neural architectures and human annotation studies demonstrate actionable insights for enhancing multimodal models despite limitations in handling complex phoneme interactions.

Phonosemantic attention patterns refer to structured regularities by which models—or human annotators—prioritize particular phonetic elements within a word when associating it with semantic or affective dimensions. These patterns reveal how both artificial and human systems leverage non-arbitrary sound–meaning correspondences, especially in the absence of established lexical semantics. Recent computational studies have quantified these phenomena through attention mechanisms in neural models and through crowd-annotated emotion judgments, underscoring the extent to which sublexical cues (e.g., individual phonemes) can modulate semantic inferences across languages and modalities.

1. Definitions and Theoretical Motivation

The concept of phonosemantic attention emerges at the intersection of sound symbolism—the idea that specific sounds are preferentially linked to particular meanings—and attention mechanisms in neural network architectures. "Phonosemantic attention patterns" denote the quantifiable tendencies in models or humans to focus on distinctive phonological features (such as initial or final phonemes) when making inferences about semantics (categories such as emotional valence, size, or shape).

In neural architectures, this is operationalized by examining model-internal attention weights assigned to phoneme tokens in relation to semantic-feature representations. In human annotation studies, it is realized through the statistically robust association between specific phonetic positions (e.g., word-onset /p/) and perceived affect or meaning. Such patterns are consistently non-uniform and can be measured through regression or attention-fraction analyses (Sabbatino et al., 2022, Jeong et al., 13 Nov 2025).

2. Empirical Evidence from Human and Model Judgments

Annotation-based investigations have established that even in the absence of lexical semantics, humans infer emotion intensity from the sound shapes of nonwords. Sabbatino et al. conducted a large-scale best–worst scaling study in which participants scored 272 nonsense words and 68 real words for six basic emotions. Distinct phonosemantic regularities were found: first-phoneme effects (e.g., onset /p/ or /s/ increasing perceived joy; /sh/ increasing surprise) and last-phoneme effects (e.g., coda /p/ increasing disgust), with split-half reliability for the nonsense set reaching Pearson $r=0.60$–$0.72$ (Sabbatino et al., 2022).

Neural regression models further confirmed the functional relevance of these surface cues. Intensity regressors based on phoneme embedding sequences (phn2vec representations) showed that models trained on real words generalize weakly but nontrivially to nonwords ($r=0.17$ for char-2gram models). Models trained only on nonwords did not recover real-word emotion structure ($r<0.05$), suggesting a dominant role for entrenched lexical semantics but also a measurable sublexical effect.

Layer-wise probing of transformer models provides complementary evidence. Multimodal LLMs, given IPA-encoded or audio-word prompts, systematically align their attention to iconic phonemes associated with semantic feature pairs, as measured by attention-fraction metrics ($\mathrm{Frac}^{(l)}_{p,f}$), with late layers amplifying the effect (Jeong et al., 13 Nov 2025).

3. Quantifying Phonosemantic Attention: Models and Metrics

Phonosemantic patterns are computationally formalized via analysis of attention weights $\alpha_{i \to j}^{(l,h)}$ in transformer networks. For a given word $w$ and phoneme $p$, the mean attention a model assigns from phoneme tokens $i \in \mathcal{I}_p(w)$ to semantic-feature tokens $j \in \mathcal{J}_{f_k}(w)$ at layer $l$ is aggregated as

$$A^{(l)}_{p,f_k} = \frac{1}{H |W| |\mathcal{I}_p|} \sum_{w \in W} \sum_{h=1}^H \sum_{i \in \mathcal{I}_p(w)} \sum_{j \in \mathcal{J}_{f_k}(w)} \alpha^{(l,h)}_{i \to j}$$

Attention-fraction scores such as

$$\mathrm{Frac}^{(l)}_{p, f_1} = \frac{A^{(l)}_{p, f_1}}{A^{(l)}_{p, f_1} + A^{(l)}_{p, f_2}}$$

capture the model's degree of focus on feature $f_1$ versus $f_2$ conditioned on the presence of phoneme $p$. Fraction values above 0.5 reflect preferential attention consistent with sound-symbolic mappings (e.g., /p/ cues "sharp," /m/ cues "round") (Jeong et al., 13 Nov 2025).

In annotation studies, position-specific groupings of ARPAbet phonemes (first/last/any position) enable statistical tests (Welch’s $t$) of median emotion intensity, isolating the strongest acoustic predictors of sublexical emotional judgments (Sabbatino et al., 2022).

Position	Phoneme	Emotion Effect
First	/p/	Joy $>$ Anger, Fear, Disgust
First	/s/	Joy $>$ Anger
First	/sh/	Surprise $>$ All Others
Last	/p/	Disgust $>$ Anger, Fear
Last	/sh/	Joy $>$ Anger

4. Cross-Linguistic and Multimodal Perspectives

The LEX-ICON corpus underpins cross-linguistic exploration of phonosemantic attention, containing over 8,000 natural words from English, French, Japanese, and Korean, alongside 2,930 systematically constructed pseudo-words. Words are annotated on up to 25 semantic differentials (e.g., sharp/round, big/small). MLLMs evaluated on these datasets demonstrate robust, above-baseline accuracy (macro-F1 $>0.50$ in 84.2% of semantic dimensions for natural words). Model–human alignment is substantial (Pearson $r \approx 0.579$ for best models) (Jeong et al., 13 Nov 2025).

Key findings include amplified attention-fraction scores for iconic phoneme–meaning pairs in constructed word settings versus natural lexica, due to the latter’s arbitrary historical mappings. Notably, audio modalities and phoneme-level IPA representations both support these effects, though attention specialization emerges most prominently in the higher (8–24) transformer layers.

5. Model Architecture and Attention Mechanisms

In grounded speech models, such as RNNs with post-recurrent attention layers, phonological features are encoded in earlier processing stages, while attention layers suppress low-level phonology in favor of semantic invariance, especially to synonyms (Alishahi et al., 2017). This mechanism is mirrored in transformer-based MLLMs: lower layers reflect fine-grained phonological salience, while top layers concentrate attention on iconic phonemes if the mapping is meaningful for the current task (Jeong et al., 13 Nov 2025).

In neural regression frameworks (e.g., CNN + BiLSTM followed by linear outputs), models exploit n-gram character or phoneme vectors for emotion intensity prediction. Their performance degrades when deprived of real-word training signals, indicating that exposure to broad lexical semantics is crucial for robust phonosemantic generalization (Sabbatino et al., 2022).

6. Implications, Limitations, and Future Directions

These studies collectively demonstrate that both human annotators and large-scale neural models attend to sublexical sound patterns when inferring semantic dimensions of unfamiliar words. Early positions (first phoneme) are most diagnostic, particularly for affective meaning. However, in natural vocabularies semantic arbitrariness attenuates model attention to iconicity. Attention mechanisms in deep networks appear to instantiate emergent form–meaning circuits, with specialization peaking in higher layers.

Several limitations are acknowledged: restricted nonword lexicons, uncomplicated phoneme-level groupings, and lack of explicit modeling for phoneme–phoneme interactions or prosodic/acoustic cues in current approaches. Future work is anticipated to expand nonlexical corpora, exploit multi-speaker and cross-linguistic variation, and pursue richer structural features, including biphone/triphone interactions and compositional prosody (Sabbatino et al., 2022, Jeong et al., 13 Nov 2025).