Phonological Vectors in Speech Research

• Phonological vectors are numerical representations that encode phonological structure through posterior probabilities, feature bundles, and latent embeddings.
• They facilitate operations like metric comparison, composition, and interpolation to capture segmental, contextual, and supra-segmental organization.
• Their applications span speech parsing, ASR, TTS, and clinical diagnostics, demonstrating measurable improvements across various performance metrics.

Phonological vectors are vector-valued representations in which phonological structure is encoded numerically rather than only as categorical symbols. In contemporary research, the term is used for several related objects: frame-level posterior probability vectors over phonological classes, explicit feature bundles for phones or frames, learned phone or word embeddings whose geometry is intended to reflect phonological similarity, linear directions in self-supervised speech-model spaces corresponding to features such as voicing or nasality, and continuous latent underlying forms in differentiable models of phonology (Cernak et al., 2016, Choi et al., 21 Feb 2026, Wu et al., 2021). Across these formulations, the common objective is to represent segmental, contextual, and sometimes supra-segmental organization in a form that supports metric comparison, composition, interpolation, and downstream inference.

1. Conceptual scope

The literature does not use a single canonical definition of a phonological vector. In one line of work, a phonological vector is a posterior probability vector over a predefined inventory of phonological classes estimated from short-time speech segments. In another, it is an explicit symbolic code: a phone may be represented by a 51-dimensional phonological-vector built from PanPhon features, a speech frame by a 22-dimensional structured feature vector, or a CLTS transcription by a 39-feature ternary vector. In still another, phonological vectors are not explicit labels at all but linear directions or subspaces in learned hidden representations, extracted by difference-of-means procedures or analogy tests. A more abstract variant replaces discrete underlying forms with continuous vectors in $\mathbb{R}^d$ (Zhu et al., 2021, Hernandez et al., 25 May 2026, Rubehn et al., 2024, Choi et al., 13 Mar 2026, Wu et al., 2021).

These variants differ in granularity and ontology. Some operate at the frame level, some at the phone or word level, and some at the level of morphemes or latent underlying forms. Some are directly interpretable feature bundles, while others are geometric structures inferred from model behavior. A plausible implication is that “phonological vector” functions less as the name of a single representation than as a family of representational strategies that preserve phonological structure under vector operations.

The idea also extends beyond spoken-language segment inventories. In sign-language work, signs are decomposed into phonological parameters such as handshape, orientation, and location, and these parameters are predicted with a multi-label model rather than treated as a single opaque class label. This suggests a broader view in which phonological vectors are structured bundles of sublexical features regardless of modality (Mocialov et al., 2022).

2. Posterior vectors and explicit feature encodings

A canonical explicit formulation is the phonological posterior. If acoustic input is represented as $X=\{\vec{x}_1,\ldots,\vec{x}_N\}$, a DNN-based bank of phonological class analyzers produces posterior vectors $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$, with

$\vec{z}_n=[p(c_1|x_n),\ldots,p(c_K|x_n)]^\top .$

Each component is the estimated probability that class $c_k$ is present in the analyzed short segment. In the English setup, 39-dimensional MFCCs with 9 successive frames of context yield an input dimension of 351, and each class detector uses a $351 \times 1024 \times 1024 \times 1024 \times 2$ architecture with softmax outputs trained by cross-entropy using stochastic gradient descent. Because only a few phonological classes are strongly active in any short speech segment, the resulting posterior is sparse; the central claim is that this sparsity is structured rather than random (Cernak et al., 2016).

That work operationalizes structured sparsity by binarizing each posterior vector segmentwise: probabilities above 0.5 are set to 1 and probabilities below 0.5 are set to 0. The resulting binary vector is the first-order sparsity structure. Higher-order structures are obtained by concatenating adjacent first-order vectors, producing super-vectors whose supports can be matched against class-specific codebooks by binary similarity measures. The paper defines Jaccard, inner product, Hamming, Ample, Simpson, and Hellinger measures, and reports that inner product is the most effective and efficient for parsing experiments. In this framework, the support of the vector, rather than the exact posterior magnitudes, is treated as the carrier of linguistic structure (Cernak et al., 2016).

Other systems encode phonological structure directly rather than via posterior estimation. JoinAP represents each IPA phone by 24 PanPhon features, encodes each feature value with 2 bits (+ $\rightarrow$ 10, - $\rightarrow$ 01, 0 $\rightarrow$ 00), appends 3 one-hot bits for <blk>, <spn>, and <nsn>, and thereby obtains a 51-dimensional phonological-vector. A linear map $e_i=Ap_i$ or a nonlinear map $X=\{\vec{x}_1,\ldots,\vec{x}_N\}$0 then generates the phone embedding used in the final classifier (Zhu et al., 2021). PhonoQ-2.0 instead predicts a structured 22-dimensional feature vector per frame, decomposed into manner, vowel height and backness, place, and voicing, and uses a manner-conditioned gating mechanism that restricts vowel and place predictions to compatible manner classes, explicitly enforcing phonological coherence (Hernandez et al., 25 May 2026). In cross-linguistic transcription work, CLTS sound descriptions are dynamically mapped to a 39-feature vector with values $X=\{\vec{x}_1,\ldots,\vec{x}_N\}$1, $X=\{\vec{x}_1,\ldots,\vec{x}_N\}$2, and $X=\{\vec{x}_1,\ldots,\vec{x}_N\}$3, allowing clicks, tones, diphthongs, contour tones, and diacritics to be handled compositionally rather than through a fixed lookup table (Rubehn et al., 2024).

3. Metric structure and phonological geometry

A central use of phonological vectors is to define a geometry of similarity. In metric learning for phoneme perception, each phoneme $X=\{\vec{x}_1,\ldots,\vec{x}_N\}$4 is represented as a binary feature vector $X=\{\vec{x}_1,\ldots,\vec{x}_N\}$5, and perceptual distance is modeled by a Mahalanobis-style quadratic form

$X=\{\vec{x}_1,\ldots,\vec{x}_N\}$6

with $X=\{\vec{x}_1,\ldots,\vec{x}_N\}$7 constrained to be positive semidefinite. The learned metric outperforms hand-designed baselines and yields interpretable saliency weights on phonological dimensions. For English, voicing and nasality receive high saliency; for Hebrew, nasality and approximant-related features remain prominent but voicing is much less salient and labial is relatively more salient. This shows that the same feature coordinates can support different perceptual geometries across languages (Lakretz et al., 2018).

At the word level, the relation between vector geometry and phonological similarity is less direct. In supervised acoustic word embeddings, distance in embedding space correlates positively with phonological distance only weakly to moderately in the best cases. The strongest word-discrimination models are Siamese systems with semi-hard negatives, achieving mAP scores of 0.757 for German and 0.842 for Czech, but these same models perform poorly on phonological similarity, with mean Kendall correlations of only 0.044 and 0.077. By contrast, recurrent models with symbolic grounding preserve phonological structure better. This establishes an important distinction: discriminability and phonological adequacy are not equivalent properties of an embedding space (Abdullah et al., 2021).

A more explicitly phonetic-word-vector approach constructs pairwise word similarity from articulatory feature sets, order-sensitive recurrences over phoneme or phoneme-bigram sequences, and vowel weighting, then learns a low-dimensional embedding matrix that approximates the resulting similarity matrix. The method is evaluated on English and Hindi, using about 133,859 words from CMU Pronouncing Dictionary 0.7b and 22,877 words from IndicNLP, and is designed for sound-based retrieval, analogy, and pun-related tasks rather than lexical identity alone (Sharma et al., 2021). In descriptive cross-linguistics, dynamically generated CLTS vectors provide a different kind of geometry: 8,684 unique sounds in CLTS version 2.1.0 collapse into 5,285 distinct feature vectors, yet 2,376 language varieties, or 81.8%, have 0 confused sounds, and 2,841 varieties, or 97.8%, have at most 4, indicating that a compact vector space can still preserve inventory-level distinctiveness at scale (Rubehn et al., 2024).

4. Compositionality, context, and temporal organization

Recent self-supervised speech-model work treats phonological vectors as linear directions in hidden representation space. For a phonological feature $X=\{\vec{x}_1,\ldots,\vec{x}_N\}$8, a vector can be estimated as

$X=\{\vec{x}_1,\ldots,\vec{x}_N\}$9

where $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$0 is the phone representation and $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$1 the PanPhon feature vector. This supports phonological vector arithmetic such as $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$2, interpreted as adding a voicing vector to [p]. Across 96 languages, the best self-supervised models substantially outperform MelSpec and MFCC baselines on analogy success; on TIMIT, HuBERT last layer reaches 94%, WavLM last layer 92%, wav2vec 2.0 middle layer 61%, MFCC 19%, and MelSpec 0%. The same work further reports that scaling these vectors continuously affects acoustic realization in resynthesized speech, using Vocos and acoustic proxies such as F1, F2, F1 bandwidth, HNR, and COG (Choi et al., 21 Feb 2026).

A more fine-grained account of contextualization proposes that previous, current, and next phones are encoded in position-dependent orthogonal subspaces within a single frame-level representation. In this view, a frame aligned to $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$3 contains compositional phonological information about $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$4, $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$5, and $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$6, with vectors from different relative positions approximately orthogonal. Center pooling performs comparably or better than mean pooling for phonological analogy tasks, nonzero analogy success is observed for previous, current, and next phones, and vector norms decline with distance from the center phone, with strongest signal at position 0, weaker at $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$7, and near zero at $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$8. Similarity curves for position-specific vectors cross near manually annotated boundaries, producing a staircase-like transition that suggests implicit phonetic segmentation (Choi et al., 13 Mar 2026).

Context sensitivity also appears in controlled psycholinguistic paradigms. On ambiguous /l/-/r/ continua, Wav2Vec2 hidden states shift toward the phonotactically admissible category: after /t/ the model favors /r/, after /s/ it favors /l/, and the bias emerges around Transformer layer 4 in both ASR-finetuned and self-supervised speech-pretrained models, while acoustic-scenes pretraining and random initialization do not show the effect (Kloots et al., 2024). In place-assimilation experiments, layerwise probes indicate that early layers remain surface-like, whereas later layers in compensating cases increasingly encode the underlying consonant; causal interventions show that replacing activations corresponding to the critical context phone can flip the model’s prediction from the surface consonant to the underlying one. At the same time, sentential semantic context does not increase compensation rate, and the probability of the underlying phoneme does not rise under biasing context relative to neutral context (Pouw et al., 2024).

5. Downstream applications

One of the earliest downstream uses of phonological vectors is linguistic parsing. Structured sparsity codebooks derived from binary phonological posteriors are used for consonant-vowel, stress, and accent detection. On the English Nancy database, accuracy rises from 53.5% to 96.7% for consonant-vowel detection, from 75.4% to 99.5% for stress detection, and from 78.4% to 99.5% for accent detection as context size increases; on the French SIWIS database, the corresponding improvements are 64.5% to 90.3%, 96.9% to 98.5%, and 91.6% to 94.8%. The same paper reports that stressed-structure codebooks can be used for accent detection and vice versa, indicating strong structural correlation between stress and accent (Cernak et al., 2016).

In multilingual and crosslingual ASR, phonological vectors provide structured output parameterization. JoinAP replaces flat phone embeddings with phone embeddings generated from phonological vectors, and reports that JoinAP-Nonlinear is best on average in multilingual experiments and consistently outperforms both JoinAP-Linear and Flat-Phone in zero-shot recognition on Polish and Mandarin (Zhu et al., 2021). PhonoQ-2.0 performs direct structured phonological feature recognition from frozen XLSR-ft representations with a shared linear projection, a 2-layer Conformer, and four prediction heads. It achieves average macro-F1 of 91.3% in-domain and 88.9% out-of-domain, compared to 82.5% and about 80.3% for a strong CTC phoneme baseline, and improves unseen-language macro-F1 from 66.9% to 73.6% (Hernandez et al., 25 May 2026).

Production-oriented applications use phonological vectors as controllable targets. In articulatory trajectory simulation, phonetic target sequences are encoded as vectors in generative-phonology or articulatory-phonology feature spaces, interpolated, and then linearly projected to articulatory trajectories derived from MOCHA-TIMIT EMA recordings. The best result, 0.679 Pearson correlation, is obtained with a generative-phonology feature set enriched with one-hot phoneme encodings and linear interpolation; linear interpolation consistently outperforms piecewise constant and cubic methods in this setup (Tandazo et al., 2024). In multilingual TTS, FUL-derived binary phonological feature vectors replace phone IDs as input to a FastSpeech-based system for English and Mandarin. The system uses 19 phonological features in total and is reported to support native, non-native, and code-switched speech, though tone and alignment remain limiting factors (Zhang et al., 2022). In differentiable generative phonology, underlying forms themselves become continuous latent vectors in $Z=\{\vec{z}_1,\ldots,\vec{z}_N\}$9, and the position-independent UF model achieves average accuracy 93.0 on UniMorph, outperforming position-dependent UF at 75.7 and the UF-less joint model at 73.7 (Wu et al., 2021).

Clinical and diagnostic applications treat phonological vectors as measurable subspaces in frozen self-supervised representations. A training-free dysarthria severity method aligns speech with Montreal Forced Aligner, extracts HuBERT-base-ls960 phone embeddings, estimates phonological contrast directions from healthy controls, and computes a 12-dimensional phonological profile including nine $\vec{z}_n=[p(c_1|x_n),\ldots,p(c_K|x_n)]^\top .$0 features, boundary sharpness, cross-position cosine similarity, and vowel triangle area. Across 890 speakers from 10 corpora, 5 languages, and 3 primary aetiologies, all five consonant $\vec{z}_n=[p(c_1|x_n),\ldots,p(c_K|x_n)]^\top .$1 features correlate negatively with severity, with pooled Spearman correlations from about $\vec{z}_n=[p(c_1|x_n),\ldots,p(c_K|x_n)]^\top .$2 to $\vec{z}_n=[p(c_1|x_n),\ldots,p(c_K|x_n)]^\top .$3, and all 12 features distinguish controls from severely dysarthric speakers with $\vec{z}_n=[p(c_1|x_n),\ldots,p(c_K|x_n)]^\top .$4 (Muller et al., 11 Apr 2026). In sign-language processing, a multi-label Fast R-CNN predicts handedness, handshape, orientation, and location from OpenPose-derived representations; the final co-dependent model reports 92% for handedness, 87% for handshape, 68% for orientation, and 60% for location, illustrating a parameter-bundle analogue of phonological vectors in the visual-gestural modality (Mocialov et al., 2022).

6. Debates, limitations, and research trajectory

A persistent issue is that phonological vectors do not designate a single representational object. Some formulations are explicitly symbolic and sparse, some are probabilistic outputs, some are learned embeddings evaluated by similarity, and some are latent directions in high-dimensional hidden spaces. This heterogeneity complicates comparison across tasks and models. A common misconception is that any high-performing speech embedding is therefore phonologically meaningful; the AWE literature directly contradicts this, showing that excellent same-word retrieval can coexist with poor phonological similarity structure (Abdullah et al., 2021).

A second debate concerns universality versus language-specificity. Feature systems such as PanPhon, PHOIBLE, FUL, and CLTS are intended to provide shared coordinates across languages, and multilingual systems such as JoinAP and PhonoQ-2.0 exploit exactly this property (Zhu et al., 2021, Hernandez et al., 25 May 2026). Yet learned perceptual saliencies differ across English and Hebrew, and cross-linguistic vector generation sometimes deliberately collapses narrow contrasts that are predictable or non-contrastive in context. This suggests that a universal coordinate system does not imply a universal metric or a universal notion of relevance (Lakretz et al., 2018, Rubehn et al., 2024).

Methodological constraints also delimit interpretation. Structured-sparsity parsing assumes known segment or event boundaries (Cernak et al., 2016). Dysarthria profiling requires an MFA model for the target language and healthy-control speech for direction estimation (Muller et al., 11 Apr 2026). Phonological vector arithmetic studies only a small number of self-supervised architectures, assumes PanPhon, and notes that synthesis behavior depends partly on the vocoder (Choi et al., 21 Feb 2026). Neural compensation for assimilation is entangled with lexical and distributional information and does not replicate human use of long-range semantic context (Pouw et al., 2024). FUL-based TTS remains limited by forced-alignment mismatch and by lexical tone generation, especially for crosslingual transfer into Mandarin (Zhang et al., 2022).

The current trajectory of the field is nevertheless coherent. Posterior-vector work emphasizes sparse support patterns and temporal concatenation; metric-learning and embedding work emphasizes distance structure and feature saliency; self-supervised representation studies emphasize linear directions, compositionality, and position-specific subspaces; and generative models emphasize continuous latent underlying forms. Taken together, these strands suggest that phonological structure can be encoded in vector spaces at multiple descriptive levels, but that the adequacy of any such vectorization depends on the task, the feature inventory, the temporal scope, and the evaluation criterion.