Temporal Semantic Entropy (TSE)

• Temporal Semantic Entropy is a quantitative measure that captures the diversity of local geometric shapes in time series using a physical analogy of particle trajectories.
• It encodes discrete time series into 13 geometric symbols (or 17 in the analog limit) through finite differences and sign patterns to facilitate precise entropy estimation.
• Its application in neurophysiology, acoustics, and language processing demonstrates its sensitivity, noise robustness, and utility in detecting dynamic signal features like seizure onset.

Temporal Semantic Entropy (TSE) is a quantitative measure for capturing the diversity and complexity of local geometric shapes in time series, grounded in a physical analogy between signals and the trajectories of a particle in a force field with one degree of freedom. TSE, along with the physically motivated "information power," provides a framework for analyzing, describing, and distinguishing the semantic content inherent in signal shape, with applications in fields as diverse as neurophysiology, acoustics, and regular language processing (Majumdar et al., 2018, Majumdar et al., 2016).

1. Physical Analogy and Definition

A real-valued time series $s[n]$ (or its continuous counterpart $s(t)$) is interpreted as the trajectory of a unit-mass particle under the influence of a force field. The force is given by $F(t) = s''(t)$, and the instantaneous power—representing the rate at which the particle's kinetic energy is dissipated to encode information in the signal’s local geometric structure—is $P(s(t)) = s''(t)\,s'(t)$. The cumulative semantic information in a continuous window $[a,b]$ is the integral $\int_a^b |P(s(t))|\,dt$. This physical framework assigns semantic meaning to the local geometric features of the signal, formalizing the intuition that "the meaning of the signal is in its shape" (Majumdar et al., 2016).

2. Discrete Geometric Encoding and Symbolization

The digitization of $s(t)$ yields a discrete time series $s[n]$, from which geometric "shapes" are extracted locally using finite differences:

• Backward difference: $s'[n] = s[n] - s[n - 1]$
• Forward difference: $s'[n+1] = s[n+1] - s[n]$
• Second difference: $s(t)$0

The triple $s(t)$1 captures the 3-point local geometric configuration. A detailed combinatorial analysis shows that, under the physicality constraints of a traceable trajectory, only 13 of the 27 ($s(t)$2) possible sign-triples actually arise—these are the "13 digital configurations" or "geometric symbols" (Majumdar et al., 2018). Each time point $s(t)$3 (for $s(t)$4) in the series is thus mapped to one of these 13 symbols, generating a symbolic string representative of the signal's evolving local geometry.

3. Semantic Entropy: Mathematical Formulation

TSE applies a Shannon-type entropy to the empirical distribution $s(t)$5 of these 13 geometric symbols over a window of the signal: $s(t)$6 where $s(t)$7 is the relative frequency of symbol $s(t)$8 and $s(t)$9 its count in the encoding. When all 13 symbols occur equally ($F(t) = s''(t)$0), $F(t) = s''(t)$1 nats (or $F(t) = s''(t)$2 bits). Conversely, $F(t) = s''(t)$3 when the signal is either constant or monotonic and only one symbol is present. The entropy thus quantifies the "shape diversity"—higher TSE indicates a richer repertoire of local geometric features, while lower TSE reflects repetition or regularity (Majumdar et al., 2018).

In the continuous-time (analog) limit, the number of theoretically distinct infinitesimal geometric shapes increases to 17 (Majumdar et al., 2016). The analog semantic entropy is then defined over a 17-symbol alphabet,

$F(t) = s''(t)$4

where $F(t) = s''(t)$5 denotes relative duration (probability) of each analog shape type.

4. Information Power and the E/P Synchrony Indicator

The "information power" $F(t) = s''(t)$6 is defined as the absolute value of the P-operator, in discrete form as $F(t) = s''(t)$7 or $F(t) = s''(t)$8, averaged or summed over the series. $F(t) = s''(t)$9 quantifies the total geometric "activity" or vigor of the signal's shape dynamics.

The ratio $P(s(t)) = s''(t)\,s'(t)$0 serves as a synchrony indicator. Empirical studies report that, in highly regular or synchronous behavior (e.g., epileptic seizures recorded in intracranial EEG), the entropy $P(s(t)) = s''(t)\,s'(t)$1 drops due to a reduction in observed geometric configurations, while the power $P(s(t)) = s''(t)\,s'(t)$2 rises as the waveform exhibits stronger curvature. This yields a pronounced reduction in $P(s(t)) = s''(t)\,s'(t)$3; for instance, in 72 out of 87 seizures (across 21 subjects), $P(s(t)) = s''(t)\,s'(t)$4 attains its minimum during seizure onset, outperforming classical spectral power and permutation entropy in uniformity of detection (Majumdar et al., 2018).

5. Theoretical Foundations and Properties

• Completeness: The 13 digital configurations fully enumerate all possible 3-point geometric neighborhoods compatible with the constraints of traceable motion (Majumdar et al., 2018), while the analog case contains exactly 17.
• Sensitivity: TSE’s construction ensures sensitivity to local curvature and monotonicity, enabling rapid response to transient structural features.
• Noise Robustness: While high-frequency noise introduces random sign changes (inflating $P(s(t)) = s''(t)\,s'(t)$5), genuine structured features are mapped consistently, and rare symbol types remain infrequent.
• Comparison with Other Metrics: Unlike amplitude-valued Shannon entropy, which ignores temporal ordering, and permutation entropy (order-3), which admits only $P(s(t)) = s''(t)\,s'(t)$6 patterns and thus omits half the possible geometric shapes, TSE’s 13-symbol encoding provides substantially finer discrimination of local signal shapes (Majumdar et al., 2018).

6. Algorithmic Realization and Practical Considerations

The conversion of $P(s(t)) = s''(t)\,s'(t)$7 to its symbol string is amenable to efficient regular language processing. A deterministic finite automaton (DFA) is constructed to parse the time series and recognize the sequence of admissible patterns. For integration of amplitude information or detection of complex patterns (such as action potentials or speech phonemes), the DFA can be extended to a weighted finite state transducer (WFST), which accumulates weights based on difference magnitudes (Majumdar et al., 2016).

For computational estimation of TSE:

1. Slide a window of chosen length across $P(s(t)) = s''(t)\,s'(t)$8, mapping each to a symbol based on local differences.
2. Tally symbol frequencies to obtain $P(s(t)) = s''(t)\,s'(t)$9.
3. Compute TSE by summing $[a,b]$0 over all 13 symbols.
4. For monitoring dynamic signals, TSE can be tracked in an online or blockwise fashion.

Care is required in sampling: undersampling may spuriously reduce TSE by collapsing patterns, while oversmoothing (e.g., by spline fitting) may erase genuine features. In the presence of high noise, post-hoc filtering or sign hysteresis can reduce the creation of spurious configurations. Blockwise or streaming implementations support application to long or continuous data streams (Majumdar et al., 2018, Majumdar et al., 2016).

7. Applications and Empirical Insights

TSE and its associated framework have been applied in multiple domains:

• Neural spike trains: In the Hodgkin-Huxley model, the distribution of 13 local shape patterns—and thus TSE—remains stable across large sampling rate variations, evidencing the invariance of the spike’s geometric code (Majumdar et al., 2016).
• Speech analysis: Subtle interspeaker differences in phoneme articulation produce observable changes in local pattern distribution; a WFST incorporating amplitude weights discriminates speakers uttering identical phonemes (Majumdar et al., 2016).
• Epileptology: TSE, and particularly the $[a,b]$1 ratio, is highly effective in localizing periods of neural synchrony, surpassing both permutation entropy and classical spectral features in intra-seizure tracking (Majumdar et al., 2018).

TSE’s physical grounding and interpretability in terms of local geometric configuration distinguish it as a powerful, theoretically rigorous approach to signal shape analysis and semantic information quantification.

---

References:

• Majumdar, R., & Jayachandran, U. "A Geometric Analysis of Time Series Leading to Information Encoding and a New Entropy Measure" (Majumdar et al., 2018)
• Majumdar, R., & Jayachandran, U. "Semantic Information Encoding in One Dimensional Time Domain Signals" (Majumdar et al., 2016)