Knowledge Equations (KEs): Formalizing Knowledge

• Knowledge Equations (KEs) are explicit formal constructions that encode, measure, or constrain knowledge through mathematical relations across various domains.
• They are applied in fields such as economic complexity, scientific machine learning, and epistemic logic to derive operational measures like the Economic Complexity Index and knowledge entropy.
• KE frameworks unify diverse approaches by structurally encoding dependencies, enabling rigorous inference, measurement, and regularization in complex systems.

Searching arXiv for the cited papers and related work on Knowledge Equations. Searching arXiv for "Knowledge Equations" and the specified arXiv IDs. Knowledge Equations (KEs) denote a heterogeneous family of formal constructions for representing, measuring, or constraining knowledge. In one line of work, knowledge is treated as productive knowledge and modeled by self-consistent relations between locations and activities under the axiom of non-fungibility. In another, a KE is any user-specified governing relation in “open-form” (ASCII) that can be parsed into a computational graph and embedded into the loss of a neural emulator. Other formulations use differential equations linking uncertainty, knowledge, and ignorance, or logical operators expressing knowledge of functional dependencies between variables. The cited literature therefore treats KEs less as a single canonical object than as a recurrent strategy: encode knowledge in explicit formal relations whose structure constrains inference, measurement, or learning (Hidalgo, 2022, Du et al., 2022, Hou, 2018, Ding, 2017).

1. Terminological range and conceptual commonalities

Across the cited works, the label “Knowledge Equations” is attached to several distinct mathematical programs. The common thread is not a shared notation, but the use of explicit equations or dependency formalisms to make knowledge operational.

Domain	KE object	Core formal ingredients
Economic complexity	Self-consistent knowledge system	$K_c$, $K_p$, $M_{cp}$, eigenvalue problems, ECI
Scientific machine learning	User-specified governing relation	ASCII equation, AST, computational graph, autodiff, composite loss
Recognition theory	Evolution process equations	$U$, $K$, $I$, closed-form measures, knowledge entropy
Epistemic logic	Functional-dependency knowledge operator	$K(d)$, $K(C,d)$, function-domain $F$, S5-style semantics

A common misconception is that KEs name a single standardized framework. The cited literature instead shows domain-specific meanings. In economics, the emphasis falls on non-fungible combinations of productive capabilities. In scientific machine learning, the emphasis falls on automating the embedding of governing equations. In recognition theory, the emphasis falls on deriving measures of knowledge and ignorance from uncertainty dynamics. In epistemic logic, the emphasis falls on expressing knowledge of relations between variables rather than only knowledge of individual values.

This suggests that the unifying role of a KE is structural rather than disciplinary. A KE specifies how knowledge-related quantities, constraints, or dependencies co-determine one another, and the resulting formalism is then used either to derive a measure, characterize an epistemic state, or regularize a learning system.

2. Non-fungible productive knowledge

In the economic-complexity formulation, “knowledge” is shorthand for productive knowledge: the know-how expressed in productive activities, from exporting a product to creating cultural goods. It parallels procedural knowledge but extends to collective forms, such as the assembly-line knowledge of a factory versus the individual skill of a craftsman. This framework attributes four fundamental properties to knowledge: it is non-rival; it may be tacit or explicit; organizations differ in absorptive capacity; and, centrally, it is non-fungible (Hidalgo, 2022).

Non-fungibility is the chapter’s central axiom. Pieces of knowledge carry unique “flavors” or “letters” that cannot be freely exchanged or aggregated like scalar quantities. The thought experiments are designed to make this axiom concrete. Swapping a surgeon into a piano concert or a pianist into surgery fails because the mismatch concerns specific knowledge. Two pianists can substitute for each other only if both know the same piece; otherwise fungibility is partial. A pianist and a guitarist can form a band because complementary “letters” combine, whereas a surgeon and a pianist generally cannot.

The formal system indexes locations by $c$ and activities by $K_p$0. Let $K_p$1 be the measure of knowledge available in location $K_p$2, $K_p$3 the measure of knowledge required by activity $K_p$4, and $K_p$5 an indicator that activity $K_p$6 is present in location $K_p$7. The self-consistent definition is

$K_p$8

Equivalently,

$K_p$9

The alphabet analogy clarifies why non-fungibility changes the geometry of economic evolution. If words are activities and letters are atoms of knowledge, then in a fungible world all 3-letter words are equivalent and the product-space is a simple chain by word length. In a non-fungible world, similarity between “dog” and “log” arises from shared letters $M_{cp}$0, so economies evolve by entering activities that reuse some existing letters, forming a network rather than a line. The surgeon/pianist analogy gives the same point in matrix form: when no “letters” are shared, the relevant similarity is zero and there is no eigen-driven upgrade; when enough musical letters are shared, transition to a complementary activity becomes possible.

3. Eigenvalue constructions and economic complexity

For simple choices of $M_{cp}$1 and $M_{cp}$2, the self-consistent system reduces to matrix eigenvalue equations. In the extensive construction, $M_{cp}$3 and $M_{cp}$4 are sums: $M_{cp}$5 Defining the location-location similarity matrix

$M_{cp}$6

the system implies

$M_{cp}$7

Here $M_{cp}$8 counts the number of shared activities between $M_{cp}$9 and $U$0 (Hidalgo, 2022).

In the intensive construction, diversity and ubiquity are introduced as

$U$1

and the equations become

$U$2

The implied similarity matrix is

$U$3

which is row-stochastic, and again one has

$U$4

The leading eigenvector of the intensive similarity matrix is trivial, namely the all-ones vector. The second eigenvector captures non-fungible combinatorial structure, and this second eigenvector is the well-known Economic Complexity Index (ECI). Within this interpretation, eigenvectors identify the intrinsic alphabet of adjacency matrices, analogously to normal modes in physics, while the second eigenvector provides a nontrivial measure of knowledge intensity orthogonal to mere size or diversity.

The empirical significance of this construction is that the ECI correlates strongly with future GDP growth, income inequality dynamics, and carbon-emissions trajectories across countries and cities. The framework also identifies several limitations and open questions: large disparities in country and product sizes require multiple normalizations such as location quotients (RCA), binarization, and stochastic scaling; existing metrics typically use one layer at a time; more general choices of $U$5 and $U$6 may combine extensive and intensive terms or higher-order interactions; and future work is directed toward multi-layer or tensor-based knowledge equations that integrate trade, technology, scientific publications, and human capital in a single framework.

4. Recognition, ignorance, and knowledge entropy

A different KE framework, developed by Fujun Hou, begins from three bounded variables: the level of knowledge for recognition $U$7, the level of ignorance $U$8, and the level of uncertainty $U$9, with $K$0. The underlying assumptions are that knowledge reduces uncertainty, ignorance increases uncertainty, and the rates of change are proportional to the current uncertainty. This yields two evolution process equations: $K$1 with $K$2 and $K$3 (Hou, 2018).

Integrating gives

$K$4

In the finite-cardinality setting, one considers a universe $K$5 with an equivalence relation, class sizes $K$6, and

$K$7

Because $K$8, the extremes correspond to full knowledge, where each class is singleton and $K$9, and no knowledge, where all items lie in a single class and $I$0. Imposing these as boundary conditions yields the closed-form measure

$I$1

and, analogously,

$I$2

In this model,

$I$3

From the ignorance formula, Hou defines knowledge entropy

$I$4

For preference-sequence partitions, the same construction applies after replacing $I$5 by $I$6. The interpretation is that $I$7 is the logarithm of the number of indistinguishable groupings, normalized by $I$8, so that $I$9.

The framework explicitly compares $K(d)$0 with Boltzmann’s entropy and Shannon’s entropy. The similarity to Boltzmann is formal: both are logarithmic measures of a count of states or groupings. The difference from Shannon is structural: $K(d)$1 is based on a single integer count rather than a probability distribution, does not satisfy sub-additivity, and quantifies how coarse-grained a partition or ranking is rather than expected surprise under known probabilities. Two principles follow. First, the knowledge level of a group is not necessarily a simple sum of the individuals’ knowledge levels. Second, an individual’s knowledge entropy never increases if the individual’s thirst for knowledge never decreases.

5. Functional dependency in epistemic logic

In Yifeng Ding’s epistemic logic, the relevant “knowledge equation” idea is expressed not by a differential or matrix equation but by an operator for knowledge of functional dependency. The language $K(d)$2 contains propositional atoms $K(d)$3, data-variables $K(d)$4, a set of possible values $K(d)$5, and formulas generated using $K(d)$6, propositional atoms, the knowing-value operator $K(d)$7, the functional-dependency operator $K(d)$8, Boolean connectives, and the standard S5 knowledge operator $K(d)$9. Here $K(C,d)$0 means “knowing-value $K(C,d)$1,” while $K(C,d)$2 means “knowing a functional dependency of $K(C,d)$3 on the finite set $K(C,d)$4” (Ding, 2017).

In the single-agent semantics, an $K(C,d)$5-model is $K(C,d)$6, where $K(C,d)$7, $K(C,d)$8, and $K(C,d)$9. A fixed function-domain $F$0 determines which functions count as admissible dependencies. Truth is defined by the usual clauses for propositional logic and S5 knowledge together with

$F$1

and

$F$2

Soundness requires two conditions on $F$3: projection functions must belong to $F$4, and $F$5 must be closed under composition.

The base system $F$6 extends propositional logic and S5 with axioms for $F$7 and $F$8, including $F$9, $c$0, $c$1, $c$2, $c$3, $c$4, and $c$5. The last three capture the Armstrong axioms for functional dependence. Three single-agent logics arise from three choices of the function-domain. With the full function-domain $c$6, one gets the additional axiom $c$7, since any constant function is available, and $c$8 is complete under the stated cardinality bound. With the minimal function-domain $c$9 consisting of all projections, one instead obtains $K_p$00 and $K_p$01, and $K_p$02 is complete under its own cardinality bound. An intermediate function-domain $K_p$03 requires no additional axiom beyond $K_p$04.

The framework is then unified in a multiagent setting by allowing each agent $K_p$05 to have a world-relative function-domain $K_p$06. A multiagent model is

$K_p$07

with S5 accessibility relations $K_p$08, pointwise soundness conditions on each $K_p$09, and the condition that $K_p$10 implies $K_p$11. The semantics generalize directly to $K_p$12, $K_p$13, and $K_p$14, and a multiagent completeness theorem is given under the stated bound on $K_p$15.

The examples motivate why $K_p$16 cannot be reduced to knowing-value. In the encryption example, one may know the value of $K_p$17 without knowing the admissible functional dependency from $K_p$18 to $K_p$19. In the hair-color/fingers example, one may know hair color while rejecting that it functionally depends on the number of fingers. The operator therefore distinguishes mere logical functionality from context-sensitive understanding of dependencies.

6. Scientific machine learning and AutoKE

In AutoKE, a Knowledge Equation is any user-specified governing relation—ordinary, partial, high-order or stochastic—in “open-form” (ASCII) that encodes physical laws such as conservation, diffusion, reaction, or constitutive relations. AutoKE automatically parses this equation into an abstract syntax tree, converts it into a PyTorch or TensorFlow computational graph, and embeds its residuals into a composite loss for training a neural emulator. KEs therefore serve as soft constraints that inform the neural network of known physics without hand-coding derivatives or loss terms (Du et al., 2022).

The generic formulation begins from

$K_p$20

with boundary and initial conditions

$K_p$21

For the two-dimensional single-phase flow example,

$K_p$22

The parser can read an ASCII expression such as $K_p$91 build an AST, and emit a sequence of PyTorch operations defining the same computational graph. Once the AST has been converted, every derivative is obtained automatically via autodifferentiation; no hand-coded symbolic derivatives are needed.

Training uses mean-squared losses for the KE residual, labeled data, initial conditions, and boundary conditions. A traditional composite loss has fixed weights $K_p$23, $K_p$24, and $K_p$25. AutoKE replaces manual choice of these weights with a Lagrangian dual saddle-point problem: $K_p$26 Two update strategies are implemented: a step update

$K_p$27

and a gradient-ascent update

$K_p$28

The full pipeline includes equation parsing, construction of a prediction net and any auxiliary nets, grid-search NAS with early stopping, loss assembly, training with the Lagrangian dual, and optional transfer learning.

The implementation details emphasize support for stochastic fields through auxiliary DNNs, high-order derivatives such as $K_p$29, and Dirichlet, Neumann, and Robin boundary conditions expressed in ASCII. Memory and stability issues for very high orders or deep networks are mitigated by gradient-checkpointing and layer-wise regularization.

The reservoir-flow experiments quantify the effect of the framework. With hard-coded weights, an emulator using 8 layers and 64 units with Softplus attains relative $K_p$30 error $K_p$31, $K_p$32, and convergence in approximately 2000 epochs. Using adaptive $K_p$33 via the Lagrangian dual and Strategy B yields faster convergence and final error approximately $K_p$34 in 2000 epochs, eliminating manual weight-tuning. NAS improves performance further: switching activation from Softplus to GeLU reduces error from $K_p$35 to $K_p$36, and a larger $K_p$37 network reduces it to approximately $K_p$38; early stopping cuts NAS search time by approximately $K_p$39. Transfer learning also changes both speed and accuracy. In a reaction-diffusion example, the baseline relative $K_p$40 error $K_p$41 falls to $K_p$42 with transfer. In reservoir flow, pretraining on simpler groundwater flow gives error $K_p$43 after 500 epochs, compared with $K_p$44 from scratch, corresponding to $K_p$45 speedup and $K_p$46 accuracy gain.

7. Adjacent LLM knowledge estimation and cross-cutting open problems

A plausible implication of the broader literature is that KE-style thinking extends beyond explicit equations to internal estimators of model knowledge. KEEN, introduced as “Knowledge Estimation of Entities,” does not define a KE in the same sense as the economic, logical, or scientific-machine-learning formalisms, but it addresses a closely related problem: whether one can estimate how knowledgeable a LLM is about an entity before the model generates any text (Gottesman et al., 2024).

KEEN trains a simple probe over internal subject representations. For an entity $K_p$47, the model is prompted with “This document describes [$K_p$48],” and hidden states at the final subject-token position are extracted from a set of layers $K_p$49. Three feature constructions are defined: the hidden-state probe $K_p$50, the vocabulary-projection probe $K_p$51, and the interpretable top-$K_p$52 variant $K_p$53-$K_p$54. The probe is

$K_p$55

and, because the targets are real-valued scores in $K_p$56, training minimizes mean-squared error

$K_p$57

Targets are either average QA accuracy per subject or factuality of open-ended responses measured by FActScore.

The reported results show significant Pearson correlations at $K_p$58. For QA, the full $K_p$59 probe reaches $K_p$60 on GPT2-XL, $K_p$61 on Pythia 12B, and $K_p$62 on Vicuna 13B; the $K_p$63 probe reaches $K_p$64, $K_p$65, and $K_p$66, respectively. For open-ended generation factuality, $K_p$67 reaches $K_p$68 on Pythia 12B and $K_p$69 on Vicuna 13B, while $K_p$70 reaches $K_p$71 and $K_p$72. Cross-task transfer from QA training to FActScore testing yields $K_p$73 and $K_p$74 on Pythia 12B for $K_p$75 and $K_p$76, and $K_p$77 and $K_p$78 on Vicuna 13B. In twenty fine-tunes of LLaMA2-7B on Wikipedia paragraphs for one target entity, the fine-tuned subject set shows $K_p$79Gold QA $K_p$80 and $K_p$81KEEN$K_p$82, while 256 non-target entities show $K_p$83Gold QA $K_p$84 and $K_p$85KEEN$K_p$86. The token-based $K_p$87-$K_p$88 variant selects vocabulary items with the largest absolute probe weights, yielding a compact feature subspace whose positive and negative coefficients indicate clusters and gaps in the model’s knowledge.

Across the broader KE landscape, several open problems recur. In economic complexity, the open questions concern multi-layer or tensor-based equations, higher-order interactions, and the existence, uniqueness, and stability of solutions for more complex $K_p$89 and $K_p$90 (Hidalgo, 2022). In LLM knowledge estimation, the limitations concern the use of a single summary score per entity, the focus on named entities rather than predicates, and the reliance on linear probes rather than more expressive but less interpretable alternatives. Read together, these programs suggest a shared research agenda: move from scalar or single-layer summaries of knowledge toward structured, multi-relational formalisms that preserve dependencies, combinatorial overlap, and domain constraints.