Hidden Complexity in Systems

• Hidden Complexity is a concept where latent layers—such as temporal order, hidden states, or input compositions—govern system behaviors not apparent in raw observations.
• It influences predictive modeling and learning by revealing gaps between observable outputs and the underlying structures that drive spreading, growth, and computational performance.
• Practical insights include improved network design, enhanced economic analysis, and refined physical modeling achieved by reconstructing and measuring these concealed organizational layers.

Hidden complexity denotes a recurring research motif in which the decisive organization of a system is not directly visible in raw observations, aggregate outputs, or compact formal descriptions, but is instead encoded in latent structure: temporal order, hidden states, internal representations, concealed inputs, occupational skill bundles, or initial conditions. Across contemporary work, the phrase does not refer to a single invariant. It names a family of problems in which observed behavior is governed by a concealed structural layer that must be reconstructed, approximated, or measured indirectly in order to explain spreading, prediction, learning, growth, or computability (Dekker et al., 2021, Russo et al., 8 Jul 2025, Janik et al., 2020, Manin, 2013).

1. Hidden versus revealed structure

A common distinction across the literature is between an observable layer and a concealed organizing layer. In temporal networks, the observable aggregate contact graph suppresses the temporal ordering of interactions that determines which spreading paths are causally possible (Dekker et al., 2021). In economic complexity, “revealed complexity” is inferred from outputs such as exports or production, whereas “hidden complexity” is built bottom-up from occupations, skills, and employment patterns (Russo et al., 8 Jul 2025). In hidden-state stochastic processes, the emitted symbols are observable but the predictive object is the observer’s evolving posterior over hidden states (Grassberger, 2024). In deep networks, parameter count and depth are structural descriptors, but the proposed complexity is the entropy of the network’s realized nonlinear switching behavior on a dataset (Janik et al., 2020). In Manin’s account of scientific discourse, laws have relatively small Kolmogorov complexity, while initial conditions, boundary conditions, and databases may carry indefinitely large complexity (Manin, 2013).

Domain	Hidden locus	Observable contrast
Temporal networks	temporal ordering of interactions	aggregated contact graph
Economic complexity	occupations, job skills, employment patterns	exports or production
Hidden processes	posterior or mixed states	emitted symbols
Deep networks	nonlinear activation patterns, hidden states	parameter count, depth, next-token loss
Scientific discourse	initial conditions, boundary conditions, databases	compact laws and equations

This suggests that hidden complexity is not merely “unseen detail.” It is a structural layer whose omission changes the relevant notion of explanation. In the cited work, the concealed layer is what governs spreading vulnerability, predictive sufficiency, wage and growth associations, or the compressibility of scientific description.

2. Temporal order, hidden states, and predictive geometry

In temporal-network science, hidden complexity is tied to the fact that spreading must obey time order and causality. The paper on spreading vulnerability emphasizes that temporal networks are not static graphs repeated over time: many paths visible in an aggregate graph are temporally impossible. To preserve the order of interactions, it defines the entropy of temporal entanglement by constructing events as connected components at each sampled time, introducing agent-to-event and event-to-agent propagators, forming the agent-to-agent propagator $$P(t)=Q^{(\mathrm{in})}(t)\cdot Q^{(\mathrm{out})}(t)$$, and then composing these into the forward propagation operator

$$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$

The associated normalized entropy $S(t,\Delta t)\in[0,1]$ is parameter-free, grows monotonically with $\Delta t$, varies with the starting time $t$, and was shown on real-world temporal networks to synchronize strongly with standardized spreading-vulnerability measures for majority-vote, transport-delay, and susceptible-infected dynamics (Dekker et al., 2021). A central claim of that work is therefore that topological complexity hides a systems-level vulnerability to spreading.

For hidden Markov models and related formalisms, the concealed object is the predictive belief state rather than the observation sequence. The observer tracks a posterior distribution over hidden states, and that posterior is the effective predictive state. The paper on entropy and forecasting complexity stresses that the “mixed-state” update is the standard forward algorithm: belief states are updated by multiplication with symbol-conditioned transition matrices and renormalization. It distinguishes entropy rate, which measures irreducible randomness in the outputs, from forecasting complexity, which measures how much hidden structure must be retained for entropy-optimal prediction. It also argues that forecasting complexity is meaningful only after minimizing equivalent predictive states, and criticizes claims of novelty, countability, and dimension that ignore this minimization step (Grassberger, 2024).

The structural consequences can be extreme. Finite-state hidden generators can induce infinite or uncountable predictive-state sets, so the minimal predictive model is generically infinite-state. The paper on divergent predictive states formalizes this through the mixed-state presentation and the statistical complexity dimension $d_\mu$, defined as the information dimension of the predictive-state measure. This quantity tracks the divergence rate of the minimal memory resources required for optimal prediction. In that framework, hidden complexity is not exhausted by entropy rate $\hmu$; it includes the scaling of predictive memory when the causal-state set is infinite (Jurgens et al., 2021).

A related line of work shows that, for output processes of finite-state finite-alphabet hidden Markov models with ergodic internal processes, many standard information-theoretic quantities coincide asymptotically with their permutation analogues. Entropy rate, excess entropy, transfer entropy, momentary information transfer, and directed information can all be captured in the permutation framework under the stated ergodicity assumptions (Haruna et al., 2012). A plausible implication is that some hidden structure can be represented either through values or through orderings without losing asymptotic rate information, although the underlying predictive state remains latent.

3. Internal computation and representational structure in learning systems

In deep neural networks, one influential proposal relocates complexity from architecture size to realized computation. For a ReLU neuron, the relevant nonlinear event is whether the unit is on or off, encoded by a binary nonlinearity variable $z=\theta(W_i x_i+b)$. The complexity of a neuron is then the Shannon entropy of $z$ over the dataset, and the complexity of a layer is the joint entropy of the binary activation pattern. This definition is explicitly not parameter count, not depth, not VC or Rademacher capacity, and not worst-case expressivity. It characterizes the task-dependent internal computation actually used by the trained network. The same work introduces an effective dimension of feature representations via the entropy of the PCA spectrum, and reports that complexity and effective dimension generally increase with depth in early and middle layers, while normalized total correlation decreases during training according to a power law (Janik et al., 2020).

A more recent proposal measures in-context computation complexity by predicting future hidden states rather than future tokens. The “prediction of hidden states” layer inserts an information bottleneck into the main hidden-state pathway, using a posterior encoder $q_\psi(z_t\mid h_t)$, a learned autoregressive prior $$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$0, and a decoder $$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$1. The resulting PHi loss is a KL divergence between posterior and prior and is interpreted as the “nats of novel information learned in timestep $$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$2 that is not predictable from the past.” Empirically, this metric tracks the description length of probabilistic finite automata learned in-context, correlates with mathematical problem difficulty on MATH, and helps predict the correctness of self-generated reasoning chains on GSM-8k and MATH beyond next-token loss alone (Herrmann et al., 17 Mar 2025). That work also states a limitation: copying repeated random subsequences can inflate PHi loss.

The paper on hidden factorial structure addresses a related question in high-dimensional discrete learning. It studies conditional distributions

$$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$3

where input and output spaces factorize into unknown discrete components and each output factor depends only on a small subset of input factors. The paper defines an approximation-complexity proxy

$$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$4

and contrasts an unstructured sample complexity $$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$5 with a conjectured structured rate $$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$6, where $$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$7. Its controlled experiments indicate that multilayer perceptrons can exploit such latent decomposition and learn more efficiently than an unstructured analysis would predict, while also noting that low rank alone does not imply a genuine factorization (Arnal et al., 2024).

A complementary body of work studies hidden complexity as sample complexity or norm-based capacity. For one-hidden-layer convolutional neural networks with non-overlapping filters, approximate gradient descent attains statistical error of order

$$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$8

with sample complexity essentially $$\wp(t,\Delta t)=P(t)\cdot P(t+1)\cdot \cdots \cdot P(t+\Delta t).$$9, matching the information-theoretic lower bound in the linear case (Cao et al., 2019). For one-hidden-layer networks with equivariance, locality, and weight sharing, empirical Rademacher complexity yields dimension-independent bounds of order $S(t,\Delta t)\in[0,1]$0 under the stated assumptions, while showing that locality can improve the bound further and that weight sharing alone does not automatically imply the same gain as equivariance (Behboodi et al., 2024).

4. Capability layers, structural organization, and compressed description

In economic complexity, the term “hidden complexity” is used explicitly to denote an input-based alternative to output-based revealed complexity. The proposed framework is multipartite, with four layers—skills, occupations, industries, and US counties—and uses O*NET, BLS OEWS/QCEW, and UN-COMTRADE data. Job fitness is computed with the nonlinear fitness-complexity algorithm on the skill-job network, then averaged across occupations to define industry-level Job-Based Complexity $S(t,\Delta t)\in[0,1]$1, and finally aggregated across locally present industries to obtain county-level Job-Based Fitness $S(t,\Delta t)\in[0,1]$2. The measure differs from classic complexity metrics in being input-based rather than output-based and in naturally including services. Empirically, it is positively associated with wage levels and labor productivity growth, while classic revealed complexity is not; at the territorial level it covers almost all counties and retains significance for county GDP growth once diversification and initial GDP per capita are controlled for (Russo et al., 8 Jul 2025).

A different usage appears in the biological paper on longevity. There, the “potential of longevity” is defined as the maximum lifespan of individuals of a species living in an ideal environment, and the main thesis is that this potential is hidden in structural complexity. Structural complexity is defined by the number of hierarchical levels of sub-structures, the diversity and amount of sub-structures at each level, and the number of communicating pathways between sub-structures. The proposed causal chain is that higher structural complexity implies longer development time and greater maintenance capacity through basic functionality, network-like functional compensation, and regeneration. Actual lifespan, however, is said to depend strongly on damage exposure, living habits, and random factors (Wang-Michelitsch et al., 2015).

Manin’s treatment of Kolmogorov complexity generalizes the theme to scientific description itself. Scientific theories, on this view, are bipartite: a relatively small Kolmogorov-complexity part consisting of laws, equations, and symmetry principles, and a large or potentially unbounded part consisting of initial conditions, boundary conditions, empirical databases, populations, and measured outcomes. Newtonian mechanics, the Standard Model, and natural selection are used as examples of compact law-like descriptions paired with highly complex realized states. The hidden complexity of science is therefore not in the elegance of the law but in the empirical content needed to instantiate it (Manin, 2013).

5. Quantum many-body states and foundational no-go results

In variational many-body physics, hidden complexity appears as the mismatch between a target state and the simplifying structure exploitable by an ansatz. The study of hidden fermion determinant states examines five fermionic models, all displaying volume-law bipartite entanglement, and correlates ansatz performance with three diagnostics: Shannon entropy of the Born distribution $S(t,\Delta t)\in[0,1]$3, one-particle reduced-density-matrix entropy $S(t,\Delta t)\in[0,1]$4, and bipartite entanglement entropy. Its main conclusion is that volume-law entanglement is not the decisive obstacle. The ansatz works efficiently when the target lies near a low-complexity region of theory space—weakly correlated Slater-determinant-like regimes, fragmented configuration-interaction-like regimes, or Gutzwiller-like regimes—and becomes ineffective when no such exploitable structure exists, as in the fully connected SYK model (Wurst et al., 2024).

At the level of hidden-variable theories, computational complexity is used to rule out broad classes of models. The no-go theorem for sequential and retro-causal hidden-variable theories defines sampling classes $S(t,\Delta t)\in[0,1]$5, $S(t,\Delta t)\in[0,1]$6, and $S(t,\Delta t)\in[0,1]$7, then argues that if a theory reproduces quantum mechanics, sampling from its models must be at least as hard as sampling from quantum circuits. Under standard conjectures such as $S(t,\Delta t)\in[0,1]$8, $S(t,\Delta t)\in[0,1]$9, and non-collapse of the polynomial hierarchy, a theory with $\Delta t$0 in any of those classical or post-selection classes is not valid. Sequential causal models are identified with $\Delta t$1, retro-causal post-selection-based models with $\Delta t$2 or $\Delta t$3, and simplified Schulman-type theories are ruled out in this sense, although the exact finely tuned Schulman construction is discussed as a possible loophole (Brogioli, 2024).

These results indicate that hidden complexity in quantum theory can mean either concealed exploitable structure, as in ansatz design, or irreducible computational hardness, as in hidden-variable reconstruction.

6. Hidden inputs, oracle access, and equilibrium structure

Several papers study hidden complexity by restricting access to the underlying object. In the hidden subgroup problem over finite groups, the goal is to identify an unknown subgroup $\Delta t$4 from i.i.d. uniform examples $\Delta t$5. The paper derives general lower and upper bounds on classical sample complexity in terms of $\Delta t$6, $\Delta t$7, and the subgroup rank of the candidate family, and gives a complete characterization for generalized Simon’s problem: $\Delta t$8 In this setting, the hidden subgroup structure is precisely what determines learning difficulty (Ye et al., 2021).

The hidden satisfiability work studies a related oracle model. Instead of querying a visible formula, one proposes a probability distribution over assignments and receives the clause that is most likely to be violated. This stronger oracle suffices to solve hidden $\Delta t$9 in time $t$0, even with repeated clauses, and to learn an equivalent repetition-free hidden $t$1 instance in polynomial time. The quantum extension shows that hidden $t$2 can be solved in polynomial time up to constant precision and that hidden $t$3 can be learned in polynomial time up to inverse-polynomial precision under a non-star-like interaction-graph condition (Arad et al., 2016).

Game theory supplies a different manifestation. The paper on Nash equilibrium argues that a concept with a deceptively simple textbook definition can conceal heteroclinic Hamiltonian dynamics, complex asymptotic structure in two-player bimatrix games, explosive growth in the number of equilibria, and computational intractability. The review cites, among other results, nondegenerate $t$4 games with asymptotically more than $t$5 equilibria and a $t$6 game with $t$7 equilibria, as well as chaotic learning dynamics in generalized rock-paper-scissors (0707.0891).

Taken together, these works show that hidden complexity often enters through restricted observability: one sees samples, violated constraints, or best-response conditions, but the underlying subgroup, formula, or equilibrium geometry remains concealed.

7. Recurring themes and common misconceptions

Several misconceptions recur in the literature. One is that output difficulty is the right proxy for hidden structure. The PHi work argues that next-token loss is a poor indicator because low loss can come from trivially predictable sequences and high loss can come from random but irrelevant information (Herrmann et al., 17 Mar 2025). Another is that hidden complexity reduces to observation entropy. The HMM literature distinguishes entropy rate from forecasting complexity and shows that predictive memory can diverge even when the output process has a well-defined entropy rate (Grassberger, 2024). A third is that large entanglement alone determines hardness; the HFDS study explicitly rejects this by showing that learnability varies strongly across volume-law states depending on the presence or absence of exploitable low-complexity structure (Wurst et al., 2024).

Other papers caution against conflating compressed representations with the specific hidden organization of interest. The factorial-structure study notes that low rank alone does not imply a genuine factorization (Arnal et al., 2024). The equivariant-network analysis shows that weight sharing can match equivariant bounds only for suitably structured sharing mechanisms; arbitrary sharing does not guarantee the same generalization benefit (Behboodi et al., 2024). The critique of recent mixed-state work on HMMs argues that rebranding the forward algorithm does not create a new theory of hidden complexity (Grassberger, 2024).

These studies collectively suggest that hidden complexity is best understood as a relation between an observable layer and a latent structural layer, together with a method for quantifying the gap between them. The quantification may take the form of entropy, information dimension, Rademacher complexity, sample complexity, variational overlap error, or oracle learnability. What unifies the literature is not a single formula, but the insistence that the relevant structure of a system is often concealed in time order, predictive state, internal representation, input composition, or hidden-instance geometry rather than in the immediately visible output alone.