Copy Lemma in Information Theory

• Copy Lemma is an extension mechanism for entropy functions that adjoins auxiliary variables to replicate specified marginals while enforcing conditional independence.
• It underpins the derivation of non-Shannon inequalities by distinguishing entropic regions from standard polymatroid constraints.
• Iterated and symmetric variants of the lemma facilitate computational exploration and linear programming applications in information theory.

Searching arXiv for papers on “Copy Lemma” and closely related usages to ground the article. Searching for the cited arXiv records and topic variants: “Copy Lemma”, “lambda-Lemma”, and direct applications of the copy lemma. The Copy Lemma is a term used most explicitly in information theory and the study of the entropic region, where it denotes an extension principle for entropy functions and polymatroids: auxiliary variables can be adjoined so that they reproduce specified marginals while satisfying prescribed conditional-independence constraints. In that setting, the lemma is a central mechanism for deriving non-Shannon information inequalities and for computational exploration of entropy cones (Csirmaz, 15 Sep 2025). In several other arXiv literatures, closely analogous ideas appear without that exact name: as a transported copy of dynamics on foliation leaves in gradient flows, as a semantic guarantee for secure cloning, as a cube-copy sampler inside Boolean slices, or as a counting lemma ensuring many copies of a configuration in a reduced object (Weber, 2015).

1. Terminological scope and historical role

In the entropic-region literature, the Copy Lemma is presented as the mechanism distilled from the original Zhang–Yeung construction that produced the first non-Shannon information inequality (Csirmaz, 15 Sep 2025). The historical point is precise: Shannon inequalities define the polyhedral cone $\Gamma_N$ of polymatroids, whereas the entropy region $\Gamma_N^*$ is strictly smaller, and the Copy Lemma provides an operation that preserves entropic and almost-entropic points but not all polymatroids. That asymmetry is exactly what allows one to derive inequalities valid for entropy functions but not implied by the polymatroid axioms alone (Csirmaz, 15 Sep 2025).

The same phrase is not uniformly standardized across other fields. In finite-dimensional gradient dynamics, the relevant paper does not name a “Copy Lemma”; its terminology is a backward $\lambda$-Lemma, stable foliations, induced semi-flow, and dynamical thickening, although its leafwise transported dynamics is explicitly the nearest equivalent of a copying principle (Weber, 2015). In secure object cloning, no explicit “Copy Lemma” is named either; the closest formal analogue is a theorem turning an abstract copy-policy type into a semantic non-sharing guarantee for a clone result (Jensen et al., 2012). In extremal binary matroid theory, the corresponding result is explicitly called a Counting Lemma, but it plays the graph-theoretic role often associated with a copy lemma: if the reduced object contains the target configuration, then the original dense object contains many copies of it (Luo, 2016).

A useful consequence is that the term has both a specific and a family-resemblance meaning. Specifically, it refers to the information-theoretic extension lemma. More broadly, it often denotes a mechanism by which one transfers a structure, a dynamics, or a configuration into an enlarged or reduced setting.

2. Formal information-theoretic statement

In Csirmaz’s formulation, let $f$ be an entropy function or polymatroid on a finite base $N$, written as a partition $N=ED$ with $E,D\neq\emptyset$. For $A\subseteq E$, let $A'$ be a disjoint copy of $A$, and let $\Gamma_N^*$0 be the canonical permutation on $\Gamma_N^*$1 that swaps each element of $\Gamma_N^*$2 with its copy and fixes $\Gamma_N^*$3 pointwise. Then $\Gamma_N^*$4 is an $\Gamma_N^*$5-copy of $\Gamma_N^*$6 over $\Gamma_N^*$7 if three conditions hold: extension on $\Gamma_N^*$8, copied marginal/isomorphism on $\Gamma_N^*$9, and conditional independence $\lambda$0 (Csirmaz, 15 Sep 2025).

The three defining clauses are: $\lambda$1

$\lambda$2

$\lambda$3

Using extension and copied-marginal isomorphism, the conditional-independence clause is equivalently expressible as

$\lambda$4

The case $\lambda$5 is called a full copy (Csirmaz, 15 Sep 2025).

The main theorem states that if $\lambda$6 is entropic (or aent, or linear), then it has an entropic (or aent, or linear, respectively) $\lambda$7-copy over $\lambda$8 (Csirmaz, 15 Sep 2025). This preservation statement is the structural core of the lemma. Its force comes from the fact that the extension is valid for entropy functions and almost-entropic limits, but not for arbitrary polymatroids.

For actual random variables, a tuple form appears in direct LP applications. If $\lambda$9, then $f$0 has the same distribution as $f$1, while

$f$2

Entropy-level encodings of this construction include equalities such as

$f$3

for relevant $f$4, together with

$f$5

which is the linear entropy form of the copy-independence constraint (Gürpınar et al., 2019).

The entropic proof is constructive. For a distribution $f$6, the paper constructs a full copy $f$7 on $f$8 by

$f$9

This enforces agreement of the $N$0 and $N$1 marginals while making the copied part conditionally independent of the original part over $N$2 (Csirmaz, 15 Sep 2025).

3. Deriving non-Shannon inequalities

The standard use of the Copy Lemma is to derive entropy inequalities that are valid for entropic or almost-entropic points but not derivable from Shannon inequalities alone. The mechanism is linear-algebraic. Given an aent polymatroid $N$3 on $N$4 and a copy extension $N$5, one imposes four families of constraints in the enlarged entropy space: original entropies on $N$6, copied-marginal equalities, conditional-independence equalities, and all Shannon inequalities on the extended variable set. Eliminating the auxiliary coordinates yields inequalities on the original variables alone (Csirmaz, 15 Sep 2025).

The canonical example is the derivation of the Zhang–Yeung inequality. One introduces a $N$7-copy over $N$8, denoted $N$9, so that $N=ED$0 is isomorphic to $N=ED$1 and

$N=ED$2

One then applies the five-variable Shannon inequality

$N=ED$3

with $N=ED$4. The last term vanishes by the copy condition, and the remaining terms are replaced using the copy identities. The نتیجه is

$N=ED$5

which is the Zhang–Yeung inequality (Csirmaz, 15 Sep 2025).

The same pattern excludes the Vámos polymatroid from the almost-entropic region. For the Vámos vector $N=ED$6, the relevant conditional mutual informations vanish, while the Ingleton expression is $N=ED$7. Assuming $N=ED$8 were almost entropic, a $N=ED$9-copy over $E,D\neq\emptyset$0 would again force the terms needed to invoke the same five-variable Shannon inequality, leading to a contradiction. This is an explicit separation of $E,D\neq\emptyset$1 from $E,D\neq\emptyset$2 using a single copy step (Csirmaz, 15 Sep 2025).

A general methodological point follows. The Copy Lemma does not itself state a non-Shannon inequality; rather, it creates an entropic extension whose Shannon-type consequences project to non-Shannon-type statements in the original variables.

4. Iteration, symmetry, and related variants

The lemma can be iterated. An iterated copy is specified by a sequence

$E,D\neq\emptyset$3

where each $E,D\neq\emptyset$4 and $E,D\neq\emptyset$5 is taken in the current base at stage $E,D\neq\emptyset$6. Csirmaz emphasizes that partial copies are not theoretically stronger than full copies: an $E,D\neq\emptyset$7-copy of a minor is a minor of a full copy, and similarly for factors (Csirmaz, 15 Sep 2025). This makes iteration the natural strength parameter of the method.

The practical reason iteration matters is also stated explicitly. A $E,D\neq\emptyset$8-iterated copy has the same consequences as taking only the first copy step and augmenting the LP with all linear inequalities needed to ensure that the remaining $E,D\neq\emptyset$9 copy steps can be carried out; conversely, such extra inequalities can be simulated by further copy steps. Theoretical equivalence thus coexists with a computational asymmetry: iteration is often more feasible than explicit high-dimensional vertex enumeration (Csirmaz, 15 Sep 2025). The paper notes that iterated copy methods have been used successfully to generate hundreds of four-variable non-Shannon inequalities.

Symmetry is another major refinement. The construction yields a copy in which the canonical map $A\subseteq E$0 is a symmetry, and for full copies the original and copied sides can be made fully symmetric by averaging $A\subseteq E$1 and $A\subseteq E$2. For partial copies, however, imposing $A\subseteq E$3-symmetry can be genuinely stronger, and the paper cites earlier work where such extra symmetry assumptions were needed (Csirmaz, 15 Sep 2025). If the original polymatroid has a symmetry $A\subseteq E$4 preserving $A\subseteq E$5, that symmetry extends to the full copy and can be propagated through iterated constructions.

Two broader variants are discussed. First, the Maximum Entropy Method preserves chosen marginals and forces conditional independence across every 3-partition separating them; the paper states that the Copy Lemma is equivalent to the special case in which independence is guaranteed for a single 3-partition only (Csirmaz, 15 Sep 2025). Second, the Ahlswede–Körner method is shown to contain a hidden application of the Copy Lemma and to be strictly weaker, because the AK operation by itself preserves all polymatroids and therefore cannot generate non-Shannon inequalities (Csirmaz, 15 Sep 2025).

5. Polyhedral and computational formulations

A major shift in later work is to use the Copy Lemma directly inside linear programs, without first deriving an explicit universal inequality. The enlarged entropy space carries all Shannon inequalities together with affine copy constraints, and the optimization objective is a linear functional of the original variables. This means that the LP can implicitly exploit known or still-undiscovered non-Shannon inequalities without ever writing them down explicitly (Gürpınar et al., 2019).

Formally, if $A\subseteq E$6 denotes the original entropy coordinates and $A\subseteq E$7 the auxiliary copy coordinates, then after adjoining copy-isomorphism equalities, conditional-independence equalities, and Shannon inequalities on the enlarged space, one obtains a system

$A\subseteq E$8

A linear inequality $A\subseteq E$9 is a consequence of the chosen copy instance precisely when it belongs to the projected cone

$A'$0

This realizes the Copy Lemma as a polyhedral elimination procedure (Csirmaz, 15 Sep 2025).

The direct-LP framework is illustrated by the best known lower bound on the Ingleton score. In the normalized problem $A'$1, one minimizes

$A'$2

subject to Shannon inequalities, symmetry constraints, and three copy specifications: $A'$3

$A'$4

$A'$5

The LP optimum is

$A'$6

and the appendix gives an explicit dual certificate yielding

$A'$7

equivalent to the same bound (Gürpınar et al., 2019).

The same approach improves the lower bound for the optimal information ratio of the Vámos-matroid access structure. With $A'$8 and $A'$9, two tuple copies $A$0 and $A$1 are taken over $A$2, enforcing

$A$3

and

$A$4

The resulting 12-variable LP yields

$A$5

The paper stresses that duplicating a pair of correlated random variables in one shot was crucial in both applications (Gürpınar et al., 2019).

6. Copy-lemma–type principles in other areas

Outside information theory, the exact phrase often disappears while the underlying idea survives.

In finite-dimensional gradient flows, the relevant result is a backward $A$6-Lemma. Near a nondegenerate critical point $A$7, preimages of codimension-$A$8 disks under the time-$A$9 map are represented as graphs

$\Gamma_N^*$00

that converge in $\Gamma_N^*$01 to the local stable-manifold graph $\Gamma_N^*$02, with quantitative bounds

$\Gamma_N^*$03

and, under $\Gamma_N^*$04,

$\Gamma_N^*$05

The resulting foliation equips each leaf with an induced semi-flow

$\Gamma_N^*$06

so that the neighborhood becomes, in the paper’s own words, a “disjoint union of copies of the dynamical system $\Gamma_N^*$07.” This is the clearest non-information-theoretic instance of a genuine copying mechanism (Weber, 2015).

In secure object cloning, the analogous question is semantic separation rather than entropy extension. A copy policy $\Gamma_N^*$08 specifies which fields are deep and which are shallow, and the core semantic condition is

$\Gamma_N^*$09

meaning that any location reachable from $\Gamma_N^*$10 along a path following only deep fields of $\Gamma_N^*$11 is not reachable from any path rooted at a different variable. The central theorem says that if the clone result is typed by the graph $\Gamma_N^*$12 and the freshly allocated region is unreachable from all other variables, then the returned value satisfies the copy policy semantically (Jensen et al., 2012). This is not a “Copy Lemma” by name, but it plays that role for verified cloning.

In the analysis of Boolean slices, the nearest equivalent is again not named that way. The paper constructs a natural embedding of a copy of the Boolean cube into the balanced slice $\Gamma_N^*$13 by pairing zero- and one-coordinates of a point $\Gamma_N^*$14 and defining

$\Gamma_N^*$15

The resulting random-walk operator $\Gamma_N^*$16 has the property that for any set $\Gamma_N^*$17 of density $\Gamma_N^*$18,

$\Gamma_N^*$19

while for the nonzero set of a degree-$\Gamma_N^*$20 polynomial,

$\Gamma_N^*$21

This transfers the cube ODLSZ bound to the balanced slice up to lower-order terms (Amireddy et al., 3 Jul 2025). The paper’s own summary describes this as proving that a random embedded cube is a good sampler for the slice.

In binary matroid theory, the corresponding theorem is explicitly a Counting Lemma. After decomposing the indicator $\Gamma_N^*$22 into a structured part, a Gowers-uniform part, and a small $\Gamma_N^*$23-error, one forms a reduced matroid from atoms with high density and small error. If a homomorphism from a fixed matroid $\Gamma_N^*$24 exists into the reduced matroid, then the original matroid contains at least

$\Gamma_N^*$25

copies of $\Gamma_N^*$26 (Luo, 2016). This is precisely the configuration-copying role played by graph counting lemmas.

A more distant use of “copy” appears in low-resource morphological reinflection. The paper “Align and Copy” is not about a lemma, but its central claim is that “the transduction of the lemma into the inflected form is dominated by copying over lemma characters,” motivating hard monotonic attention and explicit COPY actions in sequence models (Makarov et al., 2017). This usage is architectural rather than theorematic, but it shows how the copy concept migrates from mathematical existence statements to inductive biases in neural transduction.

Taken together, these literatures suggest a stable conceptual core. Whether as entropy extension, leafwise conjugation, secure non-sharing, embedded-cube sampling, reduced-structure counting, or edit-script transduction, a “copy lemma” is typically a device that preserves a designated local structure while imposing additional constraints that make the copied object analytically useful.