Entropy-Complexity Pair Overview

• The entropy-complexity pair is defined as a joint measure that quantifies both uncertainty (entropy) and structure (complexity) across diverse mathematical frameworks.
• It is applied to fields such as time series analysis, computational mechanics, logic-based models, and coherent-state geometry, each using tailored entropy and complexity measures.
• The framework aids in distinguishing stochastic, chaotic, and critical processes while providing tools for statistical inference and predictive modeling.

An entropy-complexity pair is a joint characterization in which an entropy quantity is considered together with a complexity quantity, so that randomness, disorder, or logarithmic class size is not treated in isolation from structure, predictability, disequilibrium, or description cost. Across the cited literature, the pair appears in multiple, non-equivalent forms: as a point or curve in a complexity-entropy plane for time series, as the relation between excess entropy and statistical complexity in computational mechanics, as a correspondence between Boltzmann entropy and formula-length complexity in logic, as an equality between entropy rate and algorithmic complexity rate in ergodic dynamics, and as a Kähler-geometric pairing for coherent states (Ribeiro et al., 2017, 0905.2918, Jaakkola et al., 2022, Ray, 2024).

1. Conceptual range of the pair

The recurring pattern is that entropy quantifies uncertainty, disorder, or class size, whereas complexity quantifies structure, stored predictive information, disequilibrium, description length, or state-to-state separation. The literature therefore uses the same phrase for mathematically different objects. This suggests that “entropy-complexity pair” is best understood as a family resemblance across frameworks rather than a single invariant construction.

Domain	Entropy quantity	Complexity quantity
Ordinal time-series analysis	$H_1(P)$, $H_q(P)$, $H_\alpha(P)$	$C_1(P)$, $C_q(P)$, $C_\alpha(P)$
Computational mechanics	$E=I[\overleftarrow{X};\overrightarrow{X}]$	$C_\mu=H[\mathcal{S}_0]$
Logic-based model classes	$S_B(C_i)=\log_2|C_i|$	$DC_L(M)$
Coherent-state geometry	Shannon entropy $H_q(P)$0	$H_q(P)$1

In the time-series setting, the entropy-complexity pair usually combines a normalized entropy with a disequilibrium factor relative to the uniform distribution, producing either a single point or a parametric curve (Ribeiro et al., 2017, Jauregui et al., 2018). In computational mechanics, the pair $H_q(P)$2 and $H_q(P)$3 separates predictive information stored in the process from memory required by a predictive model, and their difference is interpreted as information erasure (0905.2918). In the logic-based setting, Boltzmann entropy is the logarithm of the size of an equivalence class of structures, whereas description complexity is the minimum formula size needed to define that class (Jaakkola et al., 2022). In coherent-state geometry, entropy and complexity both descend from the same Kähler potential of the Fubini-Study metric (Ray, 2024).

A recurrent misconception is that entropy itself is a direct measure of complexity. One paper explicitly disputes this view and introduces a nonlinear transformation of time-dependent entropy,

$H_q(P)$4

with $H_q(P)$5 at both $H_q(P)$6 and $H_q(P)$7, and a unique maximum at

$H_q(P)$8

In that formulation, the most complex states are “optimally mixed” rather than maximally entropic (Klamut et al., 2020).

2. Ordinal-pattern time series and complexity-entropy curves

In the complexity-entropy plane for time series, one begins with a probability distribution $H_q(P)$9 over ordinal patterns. For Shannon entropy,

$H_\alpha(P)$0

The corresponding classical statistical complexity is

$H_\alpha(P)$1

where $H_\alpha(P)$2 is the uniform distribution, $H_\alpha(P)$3 is the Jensen-Shannon divergence,

$H_\alpha(P)$4

and $H_\alpha(P)$5 is its maximum over all $H_\alpha(P)$6 (Ribeiro et al., 2017).

The Tsallis generalization replaces Shannon entropy by

$H_\alpha(P)$7

with normalized entropy

$H_\alpha(P)$8

The corresponding $H_\alpha(P)$9-complexity is

$C_1(P)$0

where

$C_1(P)$1

and

$C_1(P)$2

The $C_1(P)$3-complexity-entropy curve is the parametric set $C_1(P)$4 as $C_1(P)$5 varies (Ribeiro et al., 2017).

The distribution $C_1(P)$6 is typically estimated by the Bandt-Pompe method. Given a scalar time series $C_1(P)$7 and embedding dimension $C_1(P)$8, one forms the overlapping vectors $C_1(P)$9, determines the permutation that sorts each vector in ascending order, and counts how many times each of the $C_q(P)$0 possible permutations occurs. The empirical frequencies

$C_q(P)$1

capture the “ordinal dynamics” at scale $C_q(P)$2 (Ribeiro et al., 2017).

The interpretation of the resulting curves is highly specific. Stochastic processes such as white noise, fractional Brownian motion, and harmonic noise eventually sample all $C_q(P)$3 patterns, so $C_q(P)$4 has full support and the $C_q(P)$5-curve is a closed loop starting and ending at $C_q(P)$6 as $C_q(P)$7 or $C_q(P)$8. Chaotic maps often forbid some ordinal patterns, so $C_q(P)$9 has zeros and the curve is open. Long-range correlations produce broader loops; short-range or uncorrelated signals produce narrower loops; oscillatory correlations distort loop width as a function of $C_\alpha(P)$0. The extremal values $C_\alpha(P)$1, where $C_\alpha(P)$2 is minimal, and $C_\alpha(P)$3, where $C_\alpha(P)$4 is maximal, serve as discriminating scales. In the reported applications, the curves distinguish simulated stochastic processes and chaotic maps, yield open curves for chaotic laser-intensity pulsations, show closed or partly open loops for crude-oil prices and the sunspot index, and improve heart-rate classification from approximately $C_\alpha(P)$5 at $C_\alpha(P)$6 to approximately $C_\alpha(P)$7 when $C_\alpha(P)$8 is used in a k-NN classifier (Ribeiro et al., 2017).

A parallel construction uses Rényi entropy,

$C_\alpha(P)$9

together with a Rényi-based disequilibrium $E=I[\overleftarrow{X};\overrightarrow{X}]$0 and normalized complexity

$E=I[\overleftarrow{X};\overrightarrow{X}]$1

As $E=I[\overleftarrow{X};\overrightarrow{X}]$2 varies, $E=I[\overleftarrow{X};\overrightarrow{X}]$3 traces a Rényi complexity-entropy curve. Its local geometry near $E=I[\overleftarrow{X};\overrightarrow{X}]$4 is used for classification: stochastic processes yield positive curvature near $E=I[\overleftarrow{X};\overrightarrow{X}]$5, chaotic processes yield negative curvature from an initial point inside the unit square when forbidden patterns are present, and periodic time series produce vertical straight lines because $E=I[\overleftarrow{X};\overrightarrow{X}]$6 is independent of $E=I[\overleftarrow{X};\overrightarrow{X}]$7 when the realized patterns are equally likely (Jauregui et al., 2018).

3. Asymptotic theory and statistical inference

The entropy-complexity pair became a “popular tool for summarizing the time-series dynamics,” but its inferential theory was developed later. For ordinal patterns of length $E=I[\overleftarrow{X};\overrightarrow{X}]$8, let $E=I[\overleftarrow{X};\overrightarrow{X}]$9 be the true ordinal-pattern distribution and

$C_\mu=H[\mathcal{S}_0]$0

its empirical estimator. Under weak dependence conditions, one has

$C_\mu=H[\mathcal{S}_0]$1

where the long-run covariance matrix has entries

$C_\mu=H[\mathcal{S}_0]$2

From this, a delta-method analysis yields the asymptotic distribution of the entropy-complexity pair (Silbernagel et al., 23 Jul 2025).

A key distinction is between the non-uniform case $C_\mu=H[\mathcal{S}_0]$3 and the uniform case $C_\mu=H[\mathcal{S}_0]$4. In the non-uniform case, the first-order delta method applies to the smooth map $C_\mu=H[\mathcal{S}_0]$5, giving

$C_\mu=H[\mathcal{S}_0]$6

In the uniform case, the first-order delta method collapses, and a second-order Taylor expansion of entropy is required. Then

$C_\mu=H[\mathcal{S}_0]$7

where $C_\mu=H[\mathcal{S}_0]$8 is a quadratic form determined by the nonzero eigenvalues of $C_\mu=H[\mathcal{S}_0]$9. A corresponding result holds jointly for $S_B(C_i)=\log_2|C_i|$0 (Silbernagel et al., 23 Jul 2025).

These limit theorems support formal inference. Under the i.i.d. null, the ordinal-pattern distribution is uniform, and the pair concentrates on the line

$S_B(C_i)=\log_2|C_i|$1

Two one-dimensional tests were proposed,

$S_B(C_i)=\log_2|C_i|$2

both converging to $S_B(C_i)=\log_2|C_i|$3 under the null. For $S_B(C_i)=\log_2|C_i|$4, $S_B(C_i)=\log_2|C_i|$5, and sample sizes $S_B(C_i)=\log_2|C_i|$6, the reported finite-sample results show size approximately $S_B(C_i)=\log_2|C_i|$7 for i.i.d. $S_B(C_i)=\log_2|C_i|$8, power tending to $S_B(C_i)=\log_2|C_i|$9 for AR(1), and moderate power for QMA(1) and TEAR(1). In those experiments, complexity added no appreciable power gain beyond entropy alone. The same asymptotic theory also yields approximate standard errors and confidence ellipsoids for $DC_L(M)$0 when $DC_L(M)$1 (Silbernagel et al., 23 Jul 2025).

4. Predictive information, statistical complexity, and criticality

In computational mechanics, the canonical entropy-complexity pair is excess entropy and statistical complexity. For a stationary stochastic process with infinite past $DC_L(M)$2 and infinite future $DC_L(M)$3, excess entropy is

$DC_L(M)$4

where $DC_L(M)$5 and $DC_L(M)$6. Causal states $DC_L(M)$7 partition pasts according to equality of future conditional distributions, and statistical complexity is

$DC_L(M)$8

The central relation is

$DC_L(M)$9

so the gap between stored memory and predictive information is exactly the information erased during forecasting. The efficiency ratio

$H_q(P)$00

satisfies $H_q(P)$01. In this framework, models with small $H_q(P)$02 or large $H_q(P)$03 are the efficient ones, and $H_q(P)$04 corresponds to “crypticity zero” (0905.2918).

For finite symbolic data, one may use an approximate complexity proxy. In the analysis of the single-spin time series of the $H_q(P)$05 Ising ferromagnet, the $H_q(P)$06-block Shannon entropy is

$H_q(P)$07

and the entropy rate is

$H_q(P)$08

An “approximate complexity” is then defined by

$H_q(P)$09

with $H_q(P)$10, or operationally by estimating $H_q(P)$11 from a shuffled sequence that preserves symbol frequencies but destroys temporal correlations. Three estimators were used: block entropy, NSRPS, and zlib/LZ77 compression (Melchert et al., 2012).

When the spin orientation of a fixed site is recorded under Metropolis dynamics on a $H_q(P)$12 lattice, the entropy-complexity pair exhibits the characteristic critical pattern. The entropy rate $H_q(P)$13 grows from $H_q(P)$14 at $H_q(P)$15, passes through an inflection near $H_q(P)$16, and saturates at $H_q(P)$17 for $H_q(P)$18. The approximate complexity $H_q(P)$19 is small in both the ordered and disordered phases but has a pronounced peak at $H_q(P)$20. The parametric diagram $H_q(P)$21 versus $H_q(P)$22 forms a loop rising from $H_q(P)$23, turning back near $H_q(P)$24, and returning to $H_q(P)$25. A finite-size scaling fit for the peak location,

$H_q(P)$26

gave

$H_q(P)$27

showing convergence of the complexity maximum to the true critical temperature (Melchert et al., 2012).

5. Algorithmic, logical, and dynamical correspondences

A different use of the entropy-complexity pair arises when entropy rate is compared with algorithmic or description complexity rate. In the classical dynamical setting, Brudno’s theorem states that for an ergodic system the Kolmogorov complexity rate of almost every symbolic trajectory equals the Kolmogorov-Sinai entropy rate. In the notation of prefix-free Kolmogorov complexity $H_q(P)$28,

$H_q(P)$29

for $H_q(P)$30-almost every orbit. Using Gács’ machine-independent complexity, the same equality is recovered in the classical ergodic case, and a quantum analogue is proved densely for ergodic quantum spin chains: $H_q(P)$31 for minimal projectors inside the typical subspaces provided by the quantum Shannon-McMillan theorem (Benatti et al., 2017).

The amenable-group generalization extends this correspondence beyond $H_q(P)$32-actions. For a computable amenable group $H_q(P)$33, a modest, tempered, computable Følner sequence $H_q(P)$34, and an ergodic invariant measure $H_q(P)$35 on $H_q(P)$36,

$H_q(P)$37

for $H_q(P)$38-almost every $H_q(P)$39. In the topological case, every point satisfies

$H_q(P)$40

and there exists $H_q(P)$41 attaining equality. This makes entropy a precise rate of algorithmic information production for typical or entropy-maximizing orbits of amenable group actions (Alpeev, 2018).

In the logic-based scenario, the entropy side is Boltzmann entropy of equivalence classes,

$H_q(P)$42

and the complexity side is minimal formula length,

$H_q(P)$43

For MLU over a finite unary vocabulary, the class realizing all $H_q(P)$44-types has maximal Boltzmann entropy and also maximal description complexity. For GMLU,

$H_q(P)$45

so

$H_q(P)$46

By contrast, for first-order logic over vocabularies with at least one relation of arity $H_q(P)$47, expected description complexity grows asymptotically faster than expected Boltzmann entropy. The paper therefore shows that the entropy-complexity correspondence depends sharply on the expressive power of the description language (Jaakkola et al., 2022).

6. Geometric and thermodynamic realizations

For spin-$H_q(P)$48 coherent states of $H_q(P)$49 on $H_q(P)$50, entropy and complexity both descend from the Fubini-Study Kähler potential

$H_q(P)$51

The coherent-state probabilities are

$H_q(P)$52

with Shannon entropy

$H_q(P)$53

After Legendre transforming to the symplectic coordinate $H_q(P)$54, the Guillemin potential becomes

$H_q(P)$55

and one has

$H_q(P)$56

For two coherent states $H_q(P)$57 and $H_q(P)$58, complexity is defined by

$H_q(P)$59

so that

$H_q(P)$60

where $H_q(P)$61 is Calabi’s diastasis. In this construction, entropy and log-complexity are two different manifestations of the same Kähler potential. Non-trivial deformations of the Kähler potential break the precise equality $H_q(P)$62, which is why the Fubini-Study metric is described there as “optimal” (Ray, 2024).

Thermodynamic versions of the pair link entropy or sequence complexity to extractable work. In the Mandal-Jarzynski model, a two-state system interacts with a tape of bits, and the average work obeys

$H_q(P)$63

where $H_q(P)$64 is the Shannon-entropy increase of the tape. The same argument extends to any concave entropy-like functional $H_q(P)$65, including the predictability functional $H_q(P)$66 and the squared-error functional $H_q(P)$67. For an individual incoming bit-string $H_q(P)$68, Shannon entropy can be replaced by the Lempel-Ziv complexity

$H_q(P)$69

yielding, in the slow-mixing limit,

$H_q(P)$70

This gives LZ complexity the role of an individual-sequence entropy in a physical work-extraction bound (Merhav, 2015).

A related classical Markov-chain model of $H_q(P)$71 independent $H_q(P)$72-dits defines configuration entropy by

$H_q(P)$73

and classical absolute complexity by the minimum of the complexity relative to all-uniform reference states. In that model, states of maximal entropy coincide with states of maximal absolute complexity, and a “Second Law of Classical Absolute Complexity” is conjectured: in an isolated system, $H_q(P)$74 tends to grow on average until it reaches its maximum. For bits, the average complexity satisfies

$H_q(P)$75

which defines a parametric trajectory in the $H_q(P)$76-plane (Shangnan, 2019).

Specialized physical applications also use paired information-theoretic measures. For hydrogenic Rydberg atoms, Shannon entropy and Fisher information are combined with Cramér-Rao, Fisher-Shannon, and López-Mancini-Calbet complexities in both position and momentum space. For large principal quantum number $H_q(P)$77, the Cramér-Rao and Fisher-Shannon complexities grow as $H_q(P)$78 for several families of states, whereas some LMC complexities saturate to $H_q(P)$79 constants, such as $H_q(P)$80 and $H_q(P)$81 for circular states (López-Rosa et al., 2013).

Taken together, these formulations show that entropy-complexity pairs are not interchangeable. Some pairs diagnose forbidden ordinal patterns, some quantify predictive inefficiency, some express equality theorems between entropy rate and algorithmic complexity rate, and some derive both quantities from a single geometric or thermodynamic structure. A plausible implication is that the pair is most informative when the chosen complexity notion is matched to the operative structure of the problem: ordinal support for time series, causal states for prediction, formula size for definability, or Kähler potential for coherent-state geometry.