---
title: Phonological Vector Arithmetic
url: https://www.emergentmind.com/topics/phonological-vector-arithmetic
type: topic
---

# Phonological Vector Arithmetic

Phonological vector arithmetic is the use of vector-space operations to represent, compare, and manipulate phonological structure. In the strongest formulation, phonological features such as voicing, nasality, vowel height, or place of articulation correspond to approximately linear directions in a learned representation space, so that relations such as $[b] = [d] - [t] + [p]$ hold as analogies over phone embeddings [2602.18899]. In broader usage, the term also covers structured operations over phonological posterior vectors, feature-based phone embeddings, and word-level phonetic embeddings, including thresholding, concatenation, nearest-neighbor retrieval, and feature-conditioned transformations [1601.05647][2107.05038][2109.14796]. The literature does not treat all such spaces as equally phonologically well formed: some representations support robust linear analogies and continuous feature control, whereas others encode only weak or moderate correlations with phonological similarity [2602.18899][2106.08686].

## 1. Early vectorial formulations of phonological structure

A precursor to later arithmetic-based accounts is the phonological posterior vector. For a short-time segment of speech, a phonological posterior is defined as
\[
\vec{z}_n = \big[ p(c_1 \mid \vec{x}_n), \ldots, p(c_K \mid \vec{x}_n) \big]^\top,
\]
where each component is the posterior probability of a phonological class such as [vocalic], [consonantal], [high], [round], or [nasal] [1601.05647]. In that framework, the vector is sparse because only a few classes comprise a short term speech signal; a binary first-order sparsity structure is obtained by thresholding at $0.5$, and higher-order structures are obtained by concatenating first-order binary vectors across time [1601.05647]. These operations already constitute a restricted algebra over phonological vectors: thresholding projects continuous posteriors into a discrete support pattern, concatenation composes segmental structure across time, similarity is computed by binary pattern matching, and majority vote aggregates frame-level decisions into supra-segmental labels [1601.05647].

The empirical motivation for this formulation was that phonological posteriors, although estimated at segmental level, convey supra-segmental information [1601.05647]. Using simple binary pattern matching of first-order or high-order structures, linguistic parsing was reported for consonant-vowel detection, stress detection, and accent detection. On the Nancy corpus, for example, C–V detection rose from $53.5$ at context size $0$ to $96.7$ at context size $6$; stress detection rose from $75.4$ to $99.5$; accent detection rose from $78.4$ to $99.5$ [1601.05647]. This line of work did not frame the problem as analogy or offset learning, but it established that phonological structure can be encoded as vectors whose support, concatenation, and similarity relations are linguistically informative.

A second foundational strand is metric learning over phonological feature vectors. In that setting, each phoneme is represented as a binary feature vector, and perceptual distance is modeled as
\[
d_{ij} = (\bar p^i - \bar p^j)^T W (\bar p^i - \bar p^j),
\]
with $W \succeq 0$ [1809.07824]. In the diagonal case, the metric becomes a feature-weighted squared difference, so that each learned weight is interpretable as a perceptual saliency of a phonological feature [1809.07824]. This makes vector arithmetic over phonemes metric-sensitive: the difference vector between two phones encodes a set of feature substitutions, and the learned metric assigns different costs to movements along different phonological dimensions. For English, the derived saliencies place voicing first and nasality second across datasets, while the framework also shows cross-linguistic variation by comparing English and Hebrew [1809.07824]. A plausible implication is that later work on linear phonological directions inherits not only a vector-space view of phonology, but also a learned geometry over those directions.

## 2. Explicit phonological feature spaces and phone embeddings

A direct and explicit realization of phonological vector arithmetic appears in phonology-driven phone embeddings. In JoinAP, each phone in the IPA table is encoded as a phonological-vector derived from PanPhon features: $24$ phonological features are binary encoded as $48$ components, and three special tokens `<blk>`, `<spn>`, and `<nsn>` are added, yielding $p_i \in \mathbb{R}^{51}$ [2107.05038]. Phone embeddings are then computed either linearly,
\[
e_i = A p_i,
\]
or nonlinearly,
\[
e_i = A_2 \,\sigma(A_1 p_i),
\]
and used in acoustic scoring through
\[
z_{t,i} = e_i^\top h_t
\]
inside a CTC-CRF recognizer [2107.05038]. In the linear case, the embedding is literally a sum of active phonological-feature directions; in the nonlinear case, feature interactions are permitted while preserving a phonology-grounded parametrization [2107.05038].

This explicit construction was motivated by multilingual and crosslingual speech recognition. The same PF-to-embedding transformation is shared across languages, which allows phones with similar PFs to share similar embeddings and unseen phones to receive principled embeddings through their phonological vectors [2107.05038]. The paper reports that JoinAP with nonlinear phone embeddings is superior to JoinAP with linear phone embeddings and the traditional method with flat phone embeddings in multilingual and crosslingual experiments on German, French, Spanish, Italian, Mandarin, and Polish [2107.05038]. In the multilingual setting without finetuning, average WER across the four CommonVoice languages is $19.25\%$ for Flat-Phone, $19.06\%$ for JoinAP-Linear, and $18.75\%$ for JoinAP-Nonlinear; in Mandarin zero-shot transfer the corresponding WERs are $97.10\%$, $89.51\%$, and $88.41\%$ [2107.05038]. These results make the arithmetic operational: phonological composition is not only interpretable but directly tied to parameter sharing and zero-shot insertion of unseen phones.

A related but frame-level representation is PhonoQ-2.0, which predicts a structured $22$-dimensional feature vector per frame encoding manner, vowel quality, place, and voicing [2605.25596]. The inventory comprises Manner $(9)$, Vowel height $(3)$, Vowel backness $(3)$, Place $(5)$, and Voicing $(2)$, and the model uses a manner-conditioned gating mechanism so that “the manner head determines silence, consonant, or vowel, and vowel features (height, backness) and place are only predicted for the relevant manner class, enforcing phonologically coherent outputs” [2605.25596]. Because logits, probabilities, and argmax-decoded feature bundles are all available, the representation can be treated as one-hot, probabilistic, or continuous for subsequent arithmetic [2605.25596].

The PhonoQ-2.0 results quantify the usefulness of such a structured feature space. Built on a frozen multilingual wav2vec 2.0 model with a two-layer Conformer, it achieves an average macro-F1 of $91.3\%$ in-domain and $88.9\%$ out-of-domain; compared to a strong CTC phoneme baseline, it delivers consistent gains of $+8.8$ F1 in-domain and $+8.6$ out-of-domain on average [2605.25596]. In unseen-language evaluation, macro-F1 improves from $66.9\%$ to $73.6\%$, with French rising from $70.4$ to $76.3$, Italian from $64.8$ to $75.6$, and Russian from $65.5$ to $68.8$ [2605.25596]. Because every segment in all involved languages lives in the same $22$-D space, this representation is particularly well suited to cross-linguistic phonological comparisons and feature-difference vectors.

## 3. Self-supervised speech models and linear phonological directions

The strongest empirical evidence for phonological vector arithmetic comes from self-supervised speech models. A comprehensive study across $96$ languages shows that there exist linear directions within the representation space of wav2vec 2.0 Large, HuBERT Large, and WavLM Large that correspond to phonological features, and that the scale of these phonological vectors correlates to the degree of acoustic realization of their corresponding phonological features in a continuous manner [2602.18899]. Phone embeddings are obtained by average-pooling model representations over phonetically segmented intervals, and analogy quadruplets are built automatically from PanPhon feature vectors so that
\[
\mathbf{h}_{p_1} - \mathbf{h}_{p_2} = \mathbf{h}_{p_3} - \mathbf{h}_{p_4}
\]
in feature space [2602.18899]. The item-based analogy test asks whether
\[
\mathbf{r}_{p_1} \simeq \mathbf{r}_{p_2} + \mathbf{r}_{p_3} - \mathbf{r}_{p_4},
\]
with success measured against same-phone and different-phone cosine baselines [2602.18899].

The discrete results are strong. On TIMIT, MelSpec and MFCC are poor, with success around $0$–$19\%$, whereas WavLM and HuBERT at the final layer achieve approximately $92$–$94\%$ success [2602.18899]. On VoxAngeles, which covers $95$ languages and includes $316$ of $468$ quadruplets with at least one non-English phone, WavLM achieves approximately $93\%$ success, compared with approximately $19\%$ for MFCC and $0\%$ for MelSpec [2602.18899]. The paper further reports that $19$ phonological features are tested across datasets, and that layer trends are largely feature-agnostic and distance-agnostic [2602.18899]. These figures support the claim that phonological relations are not merely local coincidences but stable vector directions.

The continuous version of the arithmetic is based on global feature vectors
\[
\mathbf{v}_i = \mathbb{E}_{\mathbf{h}[i]=+1}[\mathbf{r}] - \mathbb{E}_{\mathbf{h}[i]=-1}[\mathbf{r}],
\]
computed separately for consonants and vowels [2602.18899]. A feature is then added to a phone segment by modifying the representation frames within that segment:
\[
\tilde{\mathbf{R}}_t =
\begin{cases}
\mathbf{R}_t + \lambda \mathbf{v}_i & \text{if } t'_s \le t < t'_e \\
\mathbf{R}_t & \text{otherwise}.
\end{cases}
\]
Using a WavLM-based representation and a vocoder inverse, the paper shows monotonic acoustic effects for eight features—high, low, back, round, nasal, sonorant, strident, and voice—on both TIMIT and VoxAngeles [2602.18899]. Increasing the round vector on $[i]$ lowers $F1$, $F2$, and $F3$; increasing the voice vector on $[b]$ shrinks voice onset time and can yield negative VOT; increasing strident on $[b]$ introduces sustained high-frequency frication; increasing nasal on $[b]$ adds a low-frequency nasal murmur [2602.18899]. The paper’s conclusion is explicit: self-supervised speech models encode speech using phonologically interpretable and compositional vectors, demonstrating phonological vector arithmetic [2602.18899].

## 4. Context, superposition, and position-dependent subspaces

A further development concerns how a single frame-level representation can encode phones and their surrounding context. Transformer-based self-supervised speech models are often described as contextualized, and one proposal is that vectors corresponding to previous, current, and next phones are superposed within a single frame-level representation [2603.12642]. This extension treats phonological vector arithmetic not only as phone-internal composition but also as contextual superposition:
\[
\mathbf{h}_t \approx \mathbf{f}^{(-1)}(p_{t-1}) + \mathbf{f}^{(0)}(p_t) + \mathbf{f}^{(+1)}(p_{t+1}) + \dots
\]
[2603.12642]. The crucial structural claim is that the mappings for different relative positions occupy approximately orthogonal subspaces.

The empirical basis is a set of difference-of-means phonological vectors $\mathbf{v}^{k}_{F}$, where $F$ is one of eight features—high, low, back, round, nasal, sonorant, strident, voicing—and $k \in \{-2,-1,0,+1,+2\}$ is the relative position [2603.12642]. The study compares mean pooling and center pooling and finds that center pooling matches or outperforms mean pooling across TIMIT and VoxAngeles, showing that a single central frame already supports phonological analogy structure [2603.12642]. It then asks whether the center frame of a middle phone $p^0$ encodes enough information to recover phonological contrasts in the previous and next phones $p^{-1}$ and $p^{+1}$. Later Transformer layers, especially in WavLM and HuBERT, show nonzero success rates for previous and next positions, which indicates that a single frame contains enough information to perform phonological vector arithmetic on neighboring phones [2603.12642].

The position-dependent geometry is described in terms of within-position and across-position cosine similarity. Within-position phonological vectors show linguistically coherent structure—high versus low strongly negative, nasal/sonorant/voice positive, vowel versus consonant near orthogonal—while vectors from different positions have substantially lower cosine similarity, often close to zero [2603.12642]. Layerwise analysis shows that within-position similarity is consistently higher than across-position similarity, and vector norms decay with distance according to
\[
\|\mathbf{v}^{0}_{F}\| > \|\mathbf{v}^{\pm1}_{F}\| > \|\mathbf{v}^{\pm2}_{F}\|
\]
[2603.12642]. In a boundary-based cosine similarity experiment on TIMIT, the curves for $\cos(\mathbf{v}^{0}_{F},\mathbf{r}[i])$ and $\cos(\mathbf{v}^{\pm1}_{F},\mathbf{r}[i])$ cross near frame index $5$, which is the annotated boundary, for all eight features [2603.12642]. This provides a concrete account of contextualization: phonological vectors for different positions are superposed in one frame, but stored in position-dependent orthogonal subspaces that switch at phonetic boundaries.

## 5. Word-level embeddings: explicit success and empirical failure

At the lexical level, one line of work constructs phonetic word embeddings from an explicit phonetic similarity function. In that approach, words are represented as sequences of phonemes; phoneme similarity is defined by Jaccard similarity over feature sets,
\[
S(P_a, P_b) = \frac{|F(P_a)\cap F(P_b)|}{|F(P_a)\cup F(P_b)|},
\]
and word similarity is computed by a dynamic programming recurrence, including a vowel-weighted bi-gram variant $W_S^V(a,b)$ [2109.14796]. A word-by-word similarity matrix $M$ is then factorized as
\[
\min_V \ \| M - V V^\top \|^2,
\]
with $d=50$ dimensions, yielding a phonetic embedding space in which inner products approximate word-level phonetic similarity [2109.14796]. The reported lexicons contain $133{,}859$ English words from CMUdict and $22{,}877$ Hindi words from IndicNLP, and similar sounding words occur together in t-SNE visualizations for both languages [2109.14796].

This space explicitly supports analogy-like operations. For a sound analogy $W_a : W_b :: W_c : W_d$, the paper approximates $W_d$ by
\[
W_d = N\big( V(W_b) - V(W_a) + V(W_c) \big),
\]
where $N(v)$ returns the nearest neighbor word to vector $v$ [2109.14796]. The same framework is evaluated on human similarity judgments and a heterographic pun dataset derived from SemEval-2017 Task 7. The final pun benchmark contains $778$ word pairs, and the paper reports examples such as “mutter” / “mother”, where PSSVec similarity is approximately $-0.0123$ and the proposed method yields $0.8993$ [2109.14796]. In this regime, phonological vector arithmetic is a designed property of the embedding construction rather than an emergent by-product of an acoustic discrimination task.

A contrasting result comes from acoustic word embeddings. In a controlled study on German and Czech, acoustic word embeddings are trained with CNN or BGRU encoders and with phone $n$-gram detection, word-to-phones, or Siamese triplet objectives, and then evaluated for both word discrimination and phonological similarity [2106.08686]. Phonological similarity is measured by agreement between cosine-similarity rankings in embedding space and rankings induced by a phonologically weighted Levenshtein distance (PWLD), using Kendall’s $\tau$ [2106.08686]. All models yield $\overline{\tau} > 0$, but the best correlations are only moderate: approximately $0.237$ for BGRU PhoneDetect on German and approximately $0.226$ for BGRU Word2Phones on Czech [2106.08686].

The central negative result is that improving the standard word discrimination objective does not necessarily yield models that better reflect word phonological similarity [2106.08686]. BGRU Siamese with hard negatives gives the best mAP for discrimination—$0.757$ on German and $0.842$ on Czech—but only $\overline{\tau}=0.044$ and $0.077$ respectively [2106.08686]. By contrast, BGRU PhoneDetect obtains lower mAP but a much higher $\overline{\tau}$ on German [2106.08686]. The paper therefore concludes that contrastive objectives “emphasize word separability in the embedding space which hinders the ability of the emerging distance to reflect word similarity” [2106.08686]. For phonological vector arithmetic, this is a substantive limitation: a space may support retrieval of same-word tokens while failing to encode the smooth, linear, feature-wise organization needed for robust vector offsets.

## 6. Multilingual scope, adjacent extensions, and unresolved issues

The multilingual literature suggests that phonological vector arithmetic is most stable when the underlying space is explicitly grounded in phonological features or when self-supervised models have discovered such features. JoinAP uses a universal phone embedding mechanism over German, French, Spanish, and Italian, and transfers it to Polish and Mandarin; PhonoQ-2.0 uses a shared $22$-dimensional space over English, German, Spanish, and Czech and evaluates zero-shot on French, Italian, and Russian [2107.05038][2605.25596]. In both cases, the same axes or transformation functions are reused across languages. This suggests that a “voicing difference” vector or a place-of-articulation contrast is meaningful beyond a single inventory, provided the representation space itself is language-general.

The applications discussed in the literature are correspondingly diverse. Phonetic word embeddings are proposed for keyword detection, keyword spotting, poetry generation, pun detection, limited vocabulary ASR, and wake-word detection, all using nearest neighbors, cosine similarity, or sound analogies as the principal vector-space operations [2109.14796]. JoinAP uses phonological-vector based phone embeddings to improve multilingual and crosslingual ASR without requiring inversion from acoustics to phonological features [2107.05038]. PhonoQ-2.0 provides a frame-level phonological feature recognizer with strong out-of-domain and unseen-language performance, which makes it a practical basis for segment-level distances, feature-difference vectors, and rule-like transformations in a shared multilingual space [2605.25596].

A related extension appears in large language models, where many word-form variations are captured by transformation vectors in both the input and output spaces [2510.17001]. The compositional scheme is
\[
e_w = e_{b_w} + \sum_{t_i \in T(w)} e_{t_i},
\]
so that forms such as “walked” are represented as “walk” plus a past-tense vector [2510.17001]. Across five languages, the method removes up to $10\%$ of vocabulary entries while expanding vocabulary coverage to out-of-vocabulary words, with minimal impact on downstream performance, and without modifying model weights [2510.17001]. This work concerns vocabulary design and word-form variation rather than segmental speech representations, but it shows that additive form transformations can extend beyond phone or speech spaces into general language-model embeddings.

The unresolved issues are substantial. Acoustic word embeddings show that positive correlation with phonological distance can remain weak or moderate even when discrimination is strong [2106.08686]. PhonoQ-2.0 explicitly notes that its $22$-dimensional inventory does not encode French vowel nasality, Russian palatalization contrasts, or Italian gemination [2605.25596]. The self-supervised speech literature notes that behavior differs by architecture, that wav2vec 2.0 can exhibit anisotropic collapse in later layers, and that some synthesis effects may depend on the vocoder used to invert representations [2602.18899]. Taken together, these findings indicate that phonological vector arithmetic is not a generic property of any embedding space over speech or words. It emerges most clearly when the geometry is either explicitly constrained by phonological features or empirically shown to encode linear, compositional, and continuous feature directions [2602.18899][2603.12642].

Source: https://www.emergentmind.com/topics/phonological-vector-arithmetic