--- title: Expected Codeword Overlap Analysis url: https://www.emergentmind.com/topics/expected-codeword-overlap type: topic --- # Expected Codeword Overlap Analysis Expected codeword overlap is the quantitative study of prefix–suffix coincidences between codewords, usually for ordered pairs of equal-length words over a finite alphabet. In the recent literature, this notion is formalized through several distinct but closely related objects: borders and correlations of ordered pairs, shortest and longest overlap lengths, extremal code families that forbid prescribed overlap lengths, and, in overlapped arithmetic coding, the expected number of same-coset neighbors at a given Hamming distance. Taken together, these models show that “expected overlap” is not a single invariant: the expected shortest overlap is bounded, the expected longest overlap diverges, and zero-overlap design imposes sharp cardinality penalties on codebooks [2405.09393], [2010.14663], [2411.02899]. ## 1. Core definitions and overlap models Fix a finite alphabet \(\Sigma\) of size \(\sigma\), and let \(\Sigma^n\) denote the set of words of length \(n\). For an ordered pair \((u,v)\in\Sigma^n\times\Sigma^n\), a border of the pair is a non-empty word \(z\) that is both a suffix of \(u\) and a prefix of \(v\). The overlap relation is directional: borders of \((u,v)\) and \((v,u)\) are counted separately. The longest-border random variable is \[ L_n(u,v):=\max\{k\in\{0,\dots,n\}:u[n-k..n-1]=v[0..k-1]\}, \] with \(k=0\) representing the absence of a non-empty overlap [2405.09393]. The same overlap structure can be encoded by the correlation vector \[ c(u,v)[i]= \begin{cases} 1,&\text{if }u[i..n-1]=v[0..n-i-1],\\ 0,&\text{otherwise}, \end{cases} \qquad i=0,\dots,n-1. \] A \(1\) at position \(i\) records a border of length \(n-i\), and the leftmost \(1\) determines the longest border. A central structural fact is \[ \Delta_n=\{\,0^{\,n-j}s \mid j\in\{0,\dots,n\},\ s\in\Gamma_j\,\}, \] where \(\Gamma_j\) is the set of autocorrelations of words of length \(j\). Thus every pair-correlation is a shifted autocorrelation of its longest border [2405.09393]. A complementary formalization focuses on the shortest overlap. For words \(u,v\in\Sigma_k^n\), \(\so(u,v)\) is the shortest right-border of \((u,v)\) if one exists, and \(\epsilon\) otherwise; \(\lso(u,v)=|\so(u,v)|\), with \(\lso(u,v)=0\) if there is no right-border. Gabric also distinguishes left- and right-borders, and calls a pair mutually bordered if it has both a right-border and a left-border [2010.14663]. These viewpoints are not interchangeable. The longest-border model emphasizes the full overlap profile of an ordered pair, while the shortest-overlap model isolates the first non-trivial prefix–suffix coincidence. A common conflation is to treat them as having the same asymptotic behavior; the current literature shows that they do not. ## 2. Expected shortest overlap For the shortest-overlap model, the relevant combinatorial quantity is the number \(u_i\) of unbordered words of length \(i\). A key lemma states that the shortest overlap word is unbordered: if \(w\) is both a proper suffix of \(u\) and a proper prefix of \(v\), then \(w=\so(u,v)\) if and only if \(w\) is unbordered. This reduces the distribution of shortest overlaps to unbordered-word counting [2010.14663]. Let \(X_n=\lso(u,v)\) for two independently and uniformly chosen words \(u,v\in\Sigma_k^n\). If \(S_k(i,n)\) denotes the number of ordered pairs with \(\lso(u,v)=i\), then \[ S_k(i,n)=u_i\,k^{2(n-i)}. \] Consequently, \[ \Pr[X_n=i]=u_i\,k^{-2i}, \] and \[ \mathbb{E}[X_n]=\sum_{i=1}^{n-1} i\,u_i\,k^{-2i}. \] As \(n\to\infty\), \[ \mathbb{E}[\lso(u,v)]\to \sum_{i=1}^{\infty} i\,u_i\,k^{-2i}, \] and the series converges. Hence the expected shortest overlap is \(O(1)\), with the constant depending on the alphabet size \(k\) [2010.14663]. The paper also gives numerical values for the limiting expectation. | \(k\) | \(\sum_{i=1}^{\infty} i\,u_i\,k^{-2i}\) | |---|---:| | 2 | 1.156 | | 3 | 0.605 | | 4 | 0.395 | | 5 | 0.290 | | 10 | 0.121 | | 100 | 0.010 | These values show that the expected shortest suffix–prefix overlap is small even for binary alphabets and decreases rapidly with alphabet size. The same work also proves that the number \(M_k(n)\) of mutually bordered pairs satisfies \[ M_k(n)\sim c_k\,k^{2n}, \qquad c_k=\left(\sum_{i=1}^{\infty}u_i\,k^{-2i}\right)^2, \] so a constant fraction of pairs are mutually bordered, although the shortest overlap itself has bounded expectation [2010.14663]. ## 3. Expected longest overlap and correlation populations The longest-overlap problem is substantially richer because it depends on the entire correlation vector rather than on the first admissible border alone. For a correlation \(t\in\Delta_n\), the population \[ POP(t)=\{(u,v)\in\Sigma^n\times\Sigma^n:c(u,v)=t\}, \qquad pop(t)=|POP(t)| \] counts ordered pairs with exactly that overlap pattern. If \(t=0^{n-j}s\) with \(s\in\Gamma_j\), then the one-sided population size is \[ pop_r(t)=pop(s)\,\sigma^{\,n-j}, \] because the first \(j\) symbols of the right word must realize the autocorrelation \(s\), while the remaining \(n-j\) symbols are free [2405.09393]. The key reduction embeds a pair into a single word \(w=vu\) of length \(2n\). If \(G(t)\) is the set of length-\(2n\) words whose autocorrelation has \(t\) as a suffix, then for \(t\in\Delta_n\setminus\Gamma_n\), \[ pop(t)=g(t):=|G(t)|. \] This transfers pair-overlap enumeration to autocorrelation counting, where Guibas–Odlyzko recurrences apply. The resulting framework yields explicit formulas for the number \(L_j\) of pairs whose longest border has exactly length \(j\), and therefore for \[ \mathbb{E}[L_n] =\sum_{j=1}^{n-1} j\,\frac{L_j}{\sigma^{2n}}. \] This resolved an open question raised by Gabric in 2022 [2405.09393]. The asymptotic conclusion is striking. Let \(X\) be the length of the longest border of a random pair \((u,v)\in\Sigma^n\times\Sigma^n\). Then \[ \mathbb{E}[X]=\mathbb{E}[L_n]\xrightarrow[n\to\infty]{}\infty. \] Thus the expected longest overlap between two random words diverges with \(n\). The proof uses asymptotic bounds for population ratios of the form \[ c\,\sigma^{2n}+O((\sigma+\mu)^n) \;\le\; \frac{pop(t)}{pop(s)} \;<\; \frac{c\,\sigma}{\sigma-1}\,\sigma^{2n}+O(n(\sigma+\mu)^n), \] which imply that, for each fixed overlap structure \(s\), the corresponding correlation class occupies a positive asymptotic fraction of all ordered pairs [2405.09393]. This creates a sharp contrast with the shortest-overlap theory. In the same random-pair model, the expected shortest overlap converges to a finite constant, while the expected longest overlap diverges. The divergence is slower for larger alphabets, but it persists for every fixed \(\sigma\ge 2\). ## 4. Zero-overlap codes and codes with restricted overlaps A block code \(C\subseteq E^n\) is non-overlapping if for every pair of not necessarily distinct codewords \(c_1,c_2\in C\) and every \(k\in\{1,\dots,n-1\}\), no length-\(k\) prefix of one equals a length-\(k\) suffix of the other in either direction. In such a code, if two codewords are chosen independently and uniformly from \(C\), the probability of a non-trivial prefix–suffix overlap is exactly zero [2307.12593]. The central extremal parameter is \[ S(q,n)=\max\{|C|:C\subseteq E^n\text{ non-overlapping}\}. \] Known exact results include \[ S(q,2)=\left\lfloor \frac{q^2}{4}\right\rfloor, \qquad S(q,3)=\left\lfloor\frac{q}{2}\right\rfloor\left(q-\left\lfloor\frac{q}{2}\right\rfloor\right), \] and the Blackburn construction is maximum when \(n\mid q\). Recent work characterizes maximal non-overlapping codes through a recursive partition system \((L_i,R_i)\) and formulates the maximum-size problem as an integer optimization problem [2307.12593]. Restricted-overlap codes interpolate between unrestricted random codebooks and non-overlapping codes. A code is \((t_1,t_2)\)-overlap-free if no two not necessarily distinct codewords have a \(t\)-overlap for \(t_12t_2\), \[ S_{t_1}^{t_2}(q,n)=q^{n-2t_2}S_{t_1}^{t_2}(q,2t_2). \] For \((1,k)\)-overlap-free codes with \(k\ge n/2\), one has \[ S_k(q,n)\le P_q(n), \] where \(P_q(n)\) is the number of primitive words of length \(n\) [2411.02899]. Earlier bounds already showed the same tradeoff in interval-forbidden models. If \(C\) is \((k,n-1)\)-overlap-free, then \[ |C|\le \frac{q^n}{2n-2k+1}, \] and if \(C\) is \((1,k)\)-overlap-free with \(k\le n/2\), then \[ |C|\le \frac{q^n}{2k}. \] In the binary case, explicit constructions give \[ C(n,k)\ge \frac{2}{9k}\,2^n, \] and for \(k\) a power of two, \[ C(n,k)\ge \frac{1}{4k}\,2^n. \] These results show that one can eliminate all overlaps in a prescribed length range while keeping the code size within a controlled factor of the ambient space [2211.10309]. ## 5. Density limits for overlap-free codebook pairs A bipartite version of the problem considers two sets \(A,B\subseteq\Sigma^n\) such that no suffix of a word in \(A\) coincides with a prefix of a word in \(B\). Writing \[ \alpha=\frac{|A|}{q^n}, \qquad \beta=\frac{|B|}{q^n}, \] Zakharov proves that \[ \alpha\beta\le \frac{1}{n}. \] More precisely, if \(U(A)\) is the set of words that do not overlap with any element of \(A\), and \[ \gamma(\alpha,n)=\sup\{p(U(A)):A\subseteq\Sigma^n,\ p(A)=\alpha\}, \] then \[ \gamma(\alpha,n)\le \frac{(1-\alpha)\big(1-(1-\alpha)^2\big)}{\alpha n}, \] which yields the product bound above [2602.20143]. This inequality is an isoperimetric constraint on zero-overlap design. If \(A\) and \(B\) are viewed as two codebooks and a random pair \((W,U)\) is drawn uniformly from \(A\times B\), then the overlap probability is exactly zero by construction, but the joint density of the support satisfies \[ \Pr[W\in A,\ U\in B]=\alpha\beta\le \frac{1}{n}. \] In the balanced case \(\alpha\approx\beta\), each codebook can occupy at most about a \(1/\sqrt{n}\) fraction of the ambient space. The bound is sharp up to a factor of \(e\). The paper gives constructions for which \[ p(U(A))\approx \frac{1}{e}\,\frac{1}{\alpha n}, \] so that \[ \alpha\,p(U(A))\approx \frac{1}{en}. \] A plausible implication is that the \(1/n\) law is the correct asymptotic scale for two-codebook zero-overlap design, even though the exact optimal constant remains between \(1/e\) and \(1\) in the regimes analyzed. ## 6. Hamming-domain overlap in overlapped arithmetic codes In overlapped arithmetic coding, “expected codeword overlap” is studied in a different sense. A binary source block \(X^n\) is mapped many-to-one into a syndrome \(m(X^n)\), producing cosets \[ \mathcal{C}_m=\{x^n\in\mathbb{B}^n:m(x^n)=m\}. \] The code-level Hamming Distance Spectrum is \[ \psi(d;n)=\mathbb{E}_{X^n}[k(X^n,d)], \] where \(k(x^n,d)\) counts the number of sequences in the same coset as \(x^n\) and at Hamming distance \(d\). This quantity is the expected number of same-coset neighbors at distance \(d\), and it is therefore a natural expected-overlap metric in the Hamming domain [2309.04309]. The Coset Cardinality Spectrum \(f(u)\) describes asymptotic normalized coset sizes. Its relation to HDS is summarized by \[ \sum_{d=0}^n \psi(d;n)\approx 2^{n(1-r)}\int_0^1 f^2(u)\,du. \] Hence \(\int_0^1 f^2(u)\,du\) is a scalar measure of coset unevenness, and larger values increase total expected overlap across Hamming distances. For large distances \(d\approx n\), the bridge result gives \[ \psi(d;n)\approx \binom{n}{d}\,2^{\alpha-nr-1}f(1/2), \qquad \alpha=\mathbf{1}_{(d=n)}. \] Thus CCS controls HDS both globally, through \(\int f^2\), and in the high-distance tail, through \(f(1/2)\) [2309.04309]. This model is distinct from prefix–suffix overlap, but it serves the same analytical purpose: it quantifies how strongly codewords collide under a specified decoding structure. In Slepian–Wolf applications, large \(\psi(d;n)\) at small \(d\) signals many near-collisions within a coset, while large \(\int f^2\) signals heavy overall collision concentration. ## 7. Conceptual synthesis The current theory supports three main conclusions about expected codeword overlap. First, the answer depends on which overlap statistic is being averaged. For random pairs of words, the expected shortest overlap converges to a finite constant, while the expected longest overlap diverges with blocklength [2010.14663], [2405.09393]. Second, zero-overlap guarantees are combinatorially expensive but quantitatively tractable. Non-overlapping and restricted-overlap codes suppress overlap events exactly, and the corresponding extremal functions \(S(q,n)\) and \(S_{t_1}^{t_2}(q,n)\) specify how much codebook size must be sacrificed to do so [2307.12593], [2411.02899], [2211.10309]. Third, overlap can be studied either as a prefix–suffix phenomenon or as a same-coset Hamming collision phenomenon. The former dominates the combinatorics-on-words and synchronization literature; the latter arises in overlapped arithmetic coding. The underlying methodological theme is the same in both cases: expected overlap is controlled by exact counting of admissible coincidence patterns, whether encoded as borders, correlations, forbidden overlap intervals, or coset spectra. Source: https://www.emergentmind.com/topics/expected-codeword-overlap