--- title: Probability Matching Interval Coding (PMATIC) url: https://www.emergentmind.com/topics/probability-matching-interval-coding-pmatic type: topic --- # Probability Matching Interval Coding (PMATIC) Probability Matching Interval Coding (PMATIC) is a family of schemes for reliable communication and lossless data compression that operate by aligning or quantizing interval probabilities in the encoding and decoding process. PMATIC spans two main research lines: (1) randomized feedback schemes that achieve channel capacity for memoryless channels via sequential interval refinement, and (2) robust, model-agnostic coding for lossless compression under bounded predictor mismatch, especially in the context of neural network-driven codecs. Both classes leverage probability synchronization and interval-based representation to ensure exact decoding with strong theoretical guarantees while accommodating practical implementation constraints [1507.08929][2601.10678][1901.02523]. ## 1. Mathematical Foundations and Core Principles PMATIC schemes center on expressing message information through a random interval in the unit interval $[0,1]$, which is iteratively refined based on channel feedback or model predictions. In canonical feedback communication, the encoder views the message as a point $\Theta_0 \sim \text{Uniform}[0,1]$ and, at each time step, updates a posterior interval based on channel output or, analogously, the predicted probability distribution in a compression scenario. For channel coding, the encoder and decoder share common randomness (e.g., a sequence $V_n \sim \text{Uniform}[0,1]$). The encoder transmits $X_n = F_X^{-1}(\Theta_n)$, with posterior update $\Theta_{n+1} = (F_{\Theta|Y}(\Theta_n | Y_n) + V_n) \bmod 1$, where $F_X$ is the CDF of the chosen input distribution $P_X$ and $F_{\Theta|Y}$ the posterior-matching kernel induced by $P_{XY}$ [1507.08929]. The decoder applies the reversed iterated function system (RIFS) to reconstruct the shrinking interval $J_n$ such that $\Pr(\Theta_0 \notin J_n) = p_e$ for each $n$. The instantaneous decoded rate is $R_n = -(1/n)\log_2|J_n|$. For model-driven lossless compression, PMATIC quantizes the predicted per-bit probabilities $\{p_i(j)\}$ to robust centers to synchronize encoder and decoder even under bounded model mismatch. The approach ensures that both parties select identical quantized probabilities for each prefix despite discrepancies in the underlying probability vectors, with a helper bit per generated code bit to resolve near-boundary ambiguity [2601.10678]. ## 2. Encoder and Decoder Algorithms ### Randomized Posterior Matching (Feedback Channel) - **Encoder**: Initializes with $\Theta_1 = \Theta_0$; at iteration $n$, computes $X_n = F_X^{-1}(\Theta_n)$; receives $Y_n$ via noiseless feedback; updates state to $\Theta_{n+1} = (F_{\Theta|Y}(\Theta_n | Y_n) + V_n) \bmod 1$. - **Decoder (RIFS)**: Sets initial interval $J_0$ of length $1-p_e$; iteratively applies $J_{k+1} = F_{\Theta|Y}^{-1}((J_k - V_{n-k}) \bmod 1 | Y_{n-k})$ for $k=0,\dots,n-1$. These operations require evaluation of the CDF and its inverse for both marginals and posteriors at each step; each update has constant computational complexity assuming fast inversion routines [1507.08929]. ### Model-Driven Lossless Compression (Bounded Predictor Mismatch) - **Encoder**: For each token $x_i$ (mapped to bits $b_i$), computes model probabilities $p_i(j)$ for $j=1,\dots,\ell$, where $\ell$ is token bit width. Each $p_i(j)$ is quantized: if $p_i(j)$ lies safely within a bin, encode helper bit $b'=0$ and use the bin center, else $b'=1$ and use nearest boundary point. Both helper and data bits are arithmetic encoded using the quantized probability [2601.10678]. - **Decoder**: For each position, computes prediction $q_i(j)$, uses received helper bit $b'$ to select quantization bin/boundary identical to encoder’s choice, then decodes corresponding bit. This guarantees exact token reconstruction when $\|\text{logits}_{\text{Enc}} - \text{logits}_{\text{Dec}}\|_\infty \leq \epsilon$, with helper-bit and quantization overhead controlled by parameter $r > 2\delta$, $\delta = \epsilon/2$ [2601.10678]. ## 3. Theoretical Properties and Performance Guarantees ### Channel Feedback Coding - **Capacity Achievement**: For any memoryless channel satisfying mild regularity (absolute continuity, finite moments), and for any target error $p_e>0$, PMATIC achieves \[ \lim_{n \to \infty} \Pr[R_n > I(X;Y) - \delta] = 1 \] for any $\delta>0$, where $I(X;Y)$ is the mutual information for the chosen $P_X$. Optimizing $P_X$ over the capacity-achieving input gives $R \to C$ [1507.08929]. - **Error Control**: The error probability $\Pr(\Theta_0 \notin J_n)$ is exactly $p_e$ for all $n$. - **Random Walk Interpretation**: The shrinkage of $J_n$ is governed by a Markov random walk $\{S_n\}$ with increments $L_k = \log(|J_{k-1}|/|J_k|)$, converging to mean $I(X;Y)$ in the limit [1507.08929]. ### Compression under Prediction Mismatch - **Decoding Correctness**: For $d_{\text{cond-TV}}(p(i),q(i)) \leq \delta$ at all $i$, encoder and decoder always agree on quantized per-bit probabilities, guaranteeing exact reconstruction [2601.10678]. - **Redundancy and Overhead**: Overhead per encoded bit is $O(\sqrt{\delta \log(1/\delta)})$, balancing helper-bit entropy and Bernoulli-KL divergence due to quantization. - **Empirical Performance**: For example, with $\delta=10^{-5}, r=0.005$, PMATIC achieves $3.52$ bits/token on text, decoding accurately under logit noise, outperforming standard compressors such as gzip [2601.10678]. ## 4. Higher-dimensional and Optimal Transport Extensions PMATIC generalizes to higher-dimensional message spaces using optimal transport theory. For parameter estimation/message transmission in $\mathbb{R}^d$, at each step $n$: - Construct the optimal transport map $T_{n-1}: \Omega \to \Omega$ that pushes the current posterior density $p_{n-1}$ to uniform, then select $U_n = T_{n-1}(W)$ with $W$ the message point. - Transmit $X_n = \Phi(U_n)$, $\Phi$ the OT map to the optimal input distribution on $\mathcal{X}^d$. - The decoder refines an estimate $J_n = T_n^{-1}([{\varepsilon}/2,1-{\varepsilon}/2]^d)$, guaranteeing $P(W \in J_n | Y^{1:n}) \geq 1-\varepsilon$ and $\text{Vol}(J_n)\to 0$ [1901.02523]. A key result is that reliability and positive rate transmission are equivalent to Birkhoff-ergodicity of the induced Markov process $(U_n)$, resulting in an "all-or-nothing" property: either no rate is possible or all $R