Phonological Vector Arithmetic

• Phonological vector arithmetic is defined by representing phonological features as linear, interpretable directions in embedding spaces.
• It uses operations like addition, subtraction, and thresholding on phone embeddings to simulate feature transformations for tasks such as ASR and multilingual analysis.
• Key methodologies include metric learning, structured feature spaces, and self-supervised representations that yield robust, compositional phonological structures.

Phonological vector arithmetic is the use of vector-space operations to represent, compare, and manipulate phonological structure. In the strongest formulation, phonological features such as voicing, nasality, vowel height, or place of articulation correspond to approximately linear directions in a learned representation space, so that relations such as $[b] = [d] - [t] + [p]$ hold as analogies over phone embeddings (Choi et al., 21 Feb 2026). In broader usage, the term also covers structured operations over phonological posterior vectors, feature-based phone embeddings, and word-level phonetic embeddings, including thresholding, concatenation, nearest-neighbor retrieval, and feature-conditioned transformations (Cernak et al., 2016, Zhu et al., 2021, Sharma et al., 2021). The literature does not treat all such spaces as equally phonologically well formed: some representations support robust linear analogies and continuous feature control, whereas others encode only weak or moderate correlations with phonological similarity (Choi et al., 21 Feb 2026, Abdullah et al., 2021).

1. Early vectorial formulations of phonological structure

A precursor to later arithmetic-based accounts is the phonological posterior vector. For a short-time segment of speech, a phonological posterior is defined as

$$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$

where each component is the posterior probability of a phonological class such as [vocalic], [consonantal], [high], [round], or nasal. In that framework, the vector is sparse because only a few classes comprise a short term speech signal; a binary first-order sparsity structure is obtained by thresholding at $0.5$, and higher-order structures are obtained by concatenating first-order binary vectors across time (Cernak et al., 2016). These operations already constitute a restricted algebra over phonological vectors: thresholding projects continuous posteriors into a discrete support pattern, concatenation composes segmental structure across time, similarity is computed by binary pattern matching, and majority vote aggregates frame-level decisions into supra-segmental labels (Cernak et al., 2016).

The empirical motivation for this formulation was that phonological posteriors, although estimated at segmental level, convey supra-segmental information (Cernak et al., 2016). Using simple binary pattern matching of first-order or high-order structures, linguistic parsing was reported for consonant-vowel detection, stress detection, and accent detection. On the Nancy corpus, for example, C–V detection rose from $53.5$ at context size $0$ to $96.7$ at context size $6$; stress detection rose from $75.4$ to $99.5$; accent detection rose from $78.4$ to $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$0 (Cernak et al., 2016). This line of work did not frame the problem as analogy or offset learning, but it established that phonological structure can be encoded as vectors whose support, concatenation, and similarity relations are linguistically informative.

A second foundational strand is metric learning over phonological feature vectors. In that setting, each phoneme is represented as a binary feature vector, and perceptual distance is modeled as

$$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$1

with $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$2 (Lakretz et al., 2018). In the diagonal case, the metric becomes a feature-weighted squared difference, so that each learned weight is interpretable as a perceptual saliency of a phonological feature (Lakretz et al., 2018). This makes vector arithmetic over phonemes metric-sensitive: the difference vector between two phones encodes a set of feature substitutions, and the learned metric assigns different costs to movements along different phonological dimensions. For English, the derived saliencies place voicing first and nasality second across datasets, while the framework also shows cross-linguistic variation by comparing English and Hebrew (Lakretz et al., 2018). A plausible implication is that later work on linear phonological directions inherits not only a vector-space view of phonology, but also a learned geometry over those directions.

2. Explicit phonological feature spaces and phone embeddings

A direct and explicit realization of phonological vector arithmetic appears in phonology-driven phone embeddings. In JoinAP, each phone in the IPA table is encoded as a phonological-vector derived from PanPhon features: $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$3 phonological features are binary encoded as $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$4 components, and three special tokens <blk>, <spn>, and <nsn> are added, yielding $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$5 (Zhu et al., 2021). Phone embeddings are then computed either linearly,

$$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$6

or nonlinearly,

$$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$7

and used in acoustic scoring through

$$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$8

inside a CTC-CRF recognizer (Zhu et al., 2021). In the linear case, the embedding is literally a sum of active phonological-feature directions; in the nonlinear case, feature interactions are permitted while preserving a phonology-grounded parametrization (Zhu et al., 2021).

This explicit construction was motivated by multilingual and crosslingual speech recognition. The same PF-to-embedding transformation is shared across languages, which allows phones with similar PFs to share similar embeddings and unseen phones to receive principled embeddings through their phonological vectors (Zhu et al., 2021). The paper reports that JoinAP with nonlinear phone embeddings is superior to JoinAP with linear phone embeddings and the traditional method with flat phone embeddings in multilingual and crosslingual experiments on German, French, Spanish, Italian, Mandarin, and Polish (Zhu et al., 2021). In the multilingual setting without finetuning, average WER across the four CommonVoice languages is $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$9 for Flat-Phone, $0.5$0 for JoinAP-Linear, and $0.5$1 for JoinAP-Nonlinear; in Mandarin zero-shot transfer the corresponding WERs are $0.5$2, $0.5$3, and $0.5$4 (Zhu et al., 2021). These results make the arithmetic operational: phonological composition is not only interpretable but directly tied to parameter sharing and zero-shot insertion of unseen phones.

A related but frame-level representation is PhonoQ-2.0, which predicts a structured $0.5$5-dimensional feature vector per frame encoding manner, vowel quality, place, and voicing (Hernandez et al., 25 May 2026). The inventory comprises Manner $0.5$6, Vowel height $0.5$7, Vowel backness $0.5$8, Place $0.5$9, and Voicing $53.5$0, and the model uses a manner-conditioned gating mechanism so that “the manner head determines silence, consonant, or vowel, and vowel features (height, backness) and place are only predicted for the relevant manner class, enforcing phonologically coherent outputs” (Hernandez et al., 25 May 2026). Because logits, probabilities, and argmax-decoded feature bundles are all available, the representation can be treated as one-hot, probabilistic, or continuous for subsequent arithmetic (Hernandez et al., 25 May 2026).

The PhonoQ-2.0 results quantify the usefulness of such a structured feature space. Built on a frozen multilingual wav2vec 2.0 model with a two-layer Conformer, it achieves an average macro-F1 of $53.5$1 in-domain and $53.5$2 out-of-domain; compared to a strong CTC phoneme baseline, it delivers consistent gains of $53.5$3 F1 in-domain and $53.5$4 out-of-domain on average (Hernandez et al., 25 May 2026). In unseen-language evaluation, macro-F1 improves from $53.5$5 to $53.5$6, with French rising from $53.5$7 to $53.5$8, Italian from $53.5$9 to $0$0, and Russian from $0$1 to $0$2 (Hernandez et al., 25 May 2026). Because every segment in all involved languages lives in the same $0$3-D space, this representation is particularly well suited to cross-linguistic phonological comparisons and feature-difference vectors.

3. Self-supervised speech models and linear phonological directions

The strongest empirical evidence for phonological vector arithmetic comes from self-supervised speech models. A comprehensive study across $0$4 languages shows that there exist linear directions within the representation space of wav2vec 2.0 Large, HuBERT Large, and WavLM Large that correspond to phonological features, and that the scale of these phonological vectors correlates to the degree of acoustic realization of their corresponding phonological features in a continuous manner (Choi et al., 21 Feb 2026). Phone embeddings are obtained by average-pooling model representations over phonetically segmented intervals, and analogy quadruplets are built automatically from PanPhon feature vectors so that

$0$5

in feature space (Choi et al., 21 Feb 2026). The item-based analogy test asks whether

$0$6

with success measured against same-phone and different-phone cosine baselines (Choi et al., 21 Feb 2026).

The discrete results are strong. On TIMIT, MelSpec and MFCC are poor, with success around $0$7–$0$8, whereas WavLM and HuBERT at the final layer achieve approximately $0$9–$96.7$0 success (Choi et al., 21 Feb 2026). On VoxAngeles, which covers $96.7$1 languages and includes $96.7$2 of $96.7$3 quadruplets with at least one non-English phone, WavLM achieves approximately $96.7$4 success, compared with approximately $96.7$5 for MFCC and $96.7$6 for MelSpec (Choi et al., 21 Feb 2026). The paper further reports that $96.7$7 phonological features are tested across datasets, and that layer trends are largely feature-agnostic and distance-agnostic (Choi et al., 21 Feb 2026). These figures support the claim that phonological relations are not merely local coincidences but stable vector directions.

The continuous version of the arithmetic is based on global feature vectors

$96.7$8

computed separately for consonants and vowels (Choi et al., 21 Feb 2026). A feature is then added to a phone segment by modifying the representation frames within that segment: $96.7$9 Using a WavLM-based representation and a vocoder inverse, the paper shows monotonic acoustic effects for eight features—high, low, back, round, nasal, sonorant, strident, and voice—on both TIMIT and VoxAngeles (Choi et al., 21 Feb 2026). Increasing the round vector on $6$0 lowers $6$1, $6$2, and $6$3; increasing the voice vector on $6$4 shrinks voice onset time and can yield negative VOT; increasing strident on $6$5 introduces sustained high-frequency frication; increasing nasal on $6$6 adds a low-frequency nasal murmur (Choi et al., 21 Feb 2026). The paper’s conclusion is explicit: self-supervised speech models encode speech using phonologically interpretable and compositional vectors, demonstrating phonological vector arithmetic (Choi et al., 21 Feb 2026).

4. Context, superposition, and position-dependent subspaces

A further development concerns how a single frame-level representation can encode phones and their surrounding context. Transformer-based self-supervised speech models are often described as contextualized, and one proposal is that vectors corresponding to previous, current, and next phones are superposed within a single frame-level representation (Choi et al., 13 Mar 2026). This extension treats phonological vector arithmetic not only as phone-internal composition but also as contextual superposition: $6$7 (Choi et al., 13 Mar 2026). The crucial structural claim is that the mappings for different relative positions occupy approximately orthogonal subspaces.

The empirical basis is a set of difference-of-means phonological vectors $6$8, where $6$9 is one of eight features—high, low, back, round, nasal, sonorant, strident, voicing—and $75.4$0 is the relative position (Choi et al., 13 Mar 2026). The study compares mean pooling and center pooling and finds that center pooling matches or outperforms mean pooling across TIMIT and VoxAngeles, showing that a single central frame already supports phonological analogy structure (Choi et al., 13 Mar 2026). It then asks whether the center frame of a middle phone $75.4$1 encodes enough information to recover phonological contrasts in the previous and next phones $75.4$2 and $75.4$3. Later Transformer layers, especially in WavLM and HuBERT, show nonzero success rates for previous and next positions, which indicates that a single frame contains enough information to perform phonological vector arithmetic on neighboring phones (Choi et al., 13 Mar 2026).

The position-dependent geometry is described in terms of within-position and across-position cosine similarity. Within-position phonological vectors show linguistically coherent structure—high versus low strongly negative, nasal/sonorant/voice positive, vowel versus consonant near orthogonal—while vectors from different positions have substantially lower cosine similarity, often close to zero (Choi et al., 13 Mar 2026). Layerwise analysis shows that within-position similarity is consistently higher than across-position similarity, and vector norms decay with distance according to

$75.4$4

(Choi et al., 13 Mar 2026). In a boundary-based cosine similarity experiment on TIMIT, the curves for $75.4$5 and $75.4$6 cross near frame index $75.4$7, which is the annotated boundary, for all eight features (Choi et al., 13 Mar 2026). This provides a concrete account of contextualization: phonological vectors for different positions are superposed in one frame, but stored in position-dependent orthogonal subspaces that switch at phonetic boundaries.

5. Word-level embeddings: explicit success and empirical failure

At the lexical level, one line of work constructs phonetic word embeddings from an explicit phonetic similarity function. In that approach, words are represented as sequences of phonemes; phoneme similarity is defined by Jaccard similarity over feature sets,

$75.4$8

and word similarity is computed by a dynamic programming recurrence, including a vowel-weighted bi-gram variant $75.4$9 (Sharma et al., 2021). A word-by-word similarity matrix $99.5$0 is then factorized as

$99.5$1

with $99.5$2 dimensions, yielding a phonetic embedding space in which inner products approximate word-level phonetic similarity (Sharma et al., 2021). The reported lexicons contain $99.5$3 English words from CMUdict and $99.5$4 Hindi words from IndicNLP, and similar sounding words occur together in t-SNE visualizations for both languages (Sharma et al., 2021).

This space explicitly supports analogy-like operations. For a sound analogy $99.5$5, the paper approximates $99.5$6 by

$99.5$7

where $99.5$8 returns the nearest neighbor word to vector $99.5$9 (Sharma et al., 2021). The same framework is evaluated on human similarity judgments and a heterographic pun dataset derived from SemEval-2017 Task 7. The final pun benchmark contains $78.4$0 word pairs, and the paper reports examples such as “mutter” / “mother”, where PSSVec similarity is approximately $78.4$1 and the proposed method yields $78.4$2 (Sharma et al., 2021). In this regime, phonological vector arithmetic is a designed property of the embedding construction rather than an emergent by-product of an acoustic discrimination task.

A contrasting result comes from acoustic word embeddings. In a controlled study on German and Czech, acoustic word embeddings are trained with CNN or BGRU encoders and with phone $78.4$3-gram detection, word-to-phones, or Siamese triplet objectives, and then evaluated for both word discrimination and phonological similarity (Abdullah et al., 2021). Phonological similarity is measured by agreement between cosine-similarity rankings in embedding space and rankings induced by a phonologically weighted Levenshtein distance (PWLD), using Kendall’s $78.4$4 (Abdullah et al., 2021). All models yield $78.4$5, but the best correlations are only moderate: approximately $78.4$6 for BGRU PhoneDetect on German and approximately $78.4$7 for BGRU Word2Phones on Czech (Abdullah et al., 2021).

The central negative result is that improving the standard word discrimination objective does not necessarily yield models that better reflect word phonological similarity (Abdullah et al., 2021). BGRU Siamese with hard negatives gives the best mAP for discrimination—$78.4$8 on German and $78.4$9 on Czech—but only $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$00 and $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$01 respectively (Abdullah et al., 2021). By contrast, BGRU PhoneDetect obtains lower mAP but a much higher $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$02 on German (Abdullah et al., 2021). The paper therefore concludes that contrastive objectives “emphasize word separability in the embedding space which hinders the ability of the emerging distance to reflect word similarity” (Abdullah et al., 2021). For phonological vector arithmetic, this is a substantive limitation: a space may support retrieval of same-word tokens while failing to encode the smooth, linear, feature-wise organization needed for robust vector offsets.

6. Multilingual scope, adjacent extensions, and unresolved issues

The multilingual literature suggests that phonological vector arithmetic is most stable when the underlying space is explicitly grounded in phonological features or when self-supervised models have discovered such features. JoinAP uses a universal phone embedding mechanism over German, French, Spanish, and Italian, and transfers it to Polish and Mandarin; PhonoQ-2.0 uses a shared $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$03-dimensional space over English, German, Spanish, and Czech and evaluates zero-shot on French, Italian, and Russian (Zhu et al., 2021, Hernandez et al., 25 May 2026). In both cases, the same axes or transformation functions are reused across languages. This suggests that a “voicing difference” vector or a place-of-articulation contrast is meaningful beyond a single inventory, provided the representation space itself is language-general.

The applications discussed in the literature are correspondingly diverse. Phonetic word embeddings are proposed for keyword detection, keyword spotting, poetry generation, pun detection, limited vocabulary ASR, and wake-word detection, all using nearest neighbors, cosine similarity, or sound analogies as the principal vector-space operations (Sharma et al., 2021). JoinAP uses phonological-vector based phone embeddings to improve multilingual and crosslingual ASR without requiring inversion from acoustics to phonological features (Zhu et al., 2021). PhonoQ-2.0 provides a frame-level phonological feature recognizer with strong out-of-domain and unseen-language performance, which makes it a practical basis for segment-level distances, feature-difference vectors, and rule-like transformations in a shared multilingual space (Hernandez et al., 25 May 2026).

A related extension appears in LLMs, where many word-form variations are captured by transformation vectors in both the input and output spaces (Reif et al., 19 Oct 2025). The compositional scheme is

$$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$04

so that forms such as “walked” are represented as “walk” plus a past-tense vector (Reif et al., 19 Oct 2025). Across five languages, the method removes up to $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$05 of vocabulary entries while expanding vocabulary coverage to out-of-vocabulary words, with minimal impact on downstream performance, and without modifying model weights (Reif et al., 19 Oct 2025). This work concerns vocabulary design and word-form variation rather than segmental speech representations, but it shows that additive form transformations can extend beyond phone or speech spaces into general language-model embeddings.

The unresolved issues are substantial. Acoustic word embeddings show that positive correlation with phonological distance can remain weak or moderate even when discrimination is strong (Abdullah et al., 2021). PhonoQ-2.0 explicitly notes that its $$\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,$$06-dimensional inventory does not encode French vowel nasality, Russian palatalization contrasts, or Italian gemination (Hernandez et al., 25 May 2026). The self-supervised speech literature notes that behavior differs by architecture, that wav2vec 2.0 can exhibit anisotropic collapse in later layers, and that some synthesis effects may depend on the vocoder used to invert representations (Choi et al., 21 Feb 2026). Taken together, these findings indicate that phonological vector arithmetic is not a generic property of any embedding space over speech or words. It emerges most clearly when the geometry is either explicitly constrained by phonological features or empirically shown to encode linear, compositional, and continuous feature directions (Choi et al., 21 Feb 2026, Choi et al., 13 Mar 2026).