Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent

Abstract: This paper argues that the relationship between lexical identity and prosody -- one well-studied parameter of linguistic variation -- can be characterized using information theory. We predict that languages that use prosody to make lexical distinctions should exhibit a higher mutual information between word identity and prosody, compared to languages that don't. We test this hypothesis in the domain of pitch, which is used to make lexical distinctions in tonal languages, like Cantonese. We use a dataset of speakers reading sentences aloud in ten languages across five language families to estimate the mutual information between the text and their pitch curves. We find that, across languages, pitch curves display similar amounts of entropy. However, these curves are easier to predict given their associated text in the tonal languages, compared to pitch- and stress-accent languages, and thus the mutual information is higher in these languages, supporting our hypothesis. Our results support perspectives that view linguistic typology as gradient, rather than categorical.

Prosodic Typology and Information Theory: An Analytical Overview

The paper "Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent" introduces a compelling application of information theory to linguistic typology, specifically focusing on prosody. The authors aim to quantify the mutual information between lexical identity and prosody to elucidate the prosodic characteristics of various languages, classifying them into tonal, pitch-accent, and stress-accent categories.

The central thesis is that tonal languages, which utilize prosody to make lexical distinctions, should exhibit higher mutual information between word identity and prosody than non-tonal languages. To test this hypothesis, the authors harness a dataset comprising spoken sentences from ten languages spanning five language families, analyzing the mutual information between text and pitch curves. Their findings endorse the predictive hypothesis, demonstrating that tonal languages indeed feature greater mutual information between lexical identity and pitch when compared to pitch- and stress-accent languages.

Methodological Contributions

A significant contribution of the paper is the methodological pipeline developed for estimating mutual information between prosody and text across languages. The approach involves:

1. Data Collection and Processing: The study leverages the Common Voice dataset, comprising cross-linguistic audio recordings, facilitating the estimation of prosodic features across various language types. For each language, pitch estimations are derived from the pitch curves of spoken words, parameterized using Discrete Cosine Transform (DCT) coefficients.
2. Mutual Information Estimation: The estimation hinges on conditional and unconditional entropy calculations, adjusted to multilingual contexts. The researchers improve existing pipelines, addressing limitations and biases in current models, proposing both Gaussian Kernel Density Estimators (KDE) and Mixture Density Networks (MDN) as methods for estimating conditional distributions.
3. Analytical Framework: Different models, including fastText, mGPT, and mBERT, are employed to represent text inputs, each offering distinct contextual scopes. FastText represents non-contextual information, mGPT provides previous context, and mBERT affords bidirectional context, allowing for nuanced insights into how lexical and contextual information correlate with recorded pitch.

Empirical Observations and Results

The analysis yields compelling evidence favoring a typological ordering: tonal languages feature higher mutual information relative to pitch-accent and stress-accent languages. This pattern holds across the various models employed, with tonal languages prominently displaying the most significant predictability from prosody, which aligns with traditional typological distinctions.

Interestingly, while tonal languages consistently show higher mutual information, the data also challenges strict categorical distinctions, suggesting a gradient typology. The absence of multimodal distributions in mutual information values across language types implies that prosodic typology could be better conceived as existing on a continuum rather than in discrete categories.

Linguistic Implications and Future Directions

The findings make substantial contributions to the understanding of linguistic typology in an information-theoretic context, offering a gradient perspective to prosodic classification that challenges traditional, rigid categorizations. This approach underscores how information theory can effectively quantify complex linguistic attributes, suggesting that many language properties might universally adhere to underlying information-theoretic principles.

In terms of future research directions, the application of the proposed framework to additional prosodic features beyond pitch, such as length or loudness, could yield further insights into crosslinguistic typological variation. Moreover, incorporating more diverse language datasets with balanced speaker representation and expanded sample sizes could refine typological characterizations, potentially broadening the explanatory power of information-theoretic approaches in linguistics. Additionally, further exploration of how phonotactic complexity and syllable structure might interact with mutual information estimates warrants attention.

Overall, this paper exemplifies the integration of quantitative methods into typological studies, enriching our theoretical frameworks and potentially guiding the development of neural models that better capture the rich diversity of human language.