---
title: 'VC-Tendency: Thresholds in Graphs and Learning'
url: https://www.emergentmind.com/topics/vc-tendency
type: topic
---

# VC-Tendency: Thresholds in Graphs and Learning

Searching arXiv for the cited papers and the phrase "VC-tendency" to ground the article in the relevant literature.
Searching for "The Twin-Width of Graphs of Bounded VC-Dimension" and related VC-tendency papers.
VC-tendency denotes a recurring phenomenon in which VC-theoretic complexity parameters act as threshold variables for structural, algorithmic, and statistical behavior. In the cited literature, the phenomenon appears in several technically distinct forms: bounded versus unbounded VC-dimension in hereditary graph classes determines whether twin-width is sub-linear or linear and whether identifying codes are polynomially large or logarithmic; bounded color complexity or bounded graph order constrains the VC dimension of graph neural networks through Weisfeiler–Leman distinguishability; bounded primal or dual VC-dimension does not by itself guarantee parameterized tractability for Hitting Set; and, in model theory, VC density tracks dp-rank up to an additive constant and exhibits essentially linear growth in the number of parameters [2606.21640] [1407.5833] [2301.11039] [1108.4398] [1512.00481].

## 1. Formal meanings of VC-based complexity

The term combines several related notions rather than a single invariant. In graph theory, one definition starts from open neighborhoods. For a simple graph $G=(V,E)$, let
$$
\mathcal{F}(G)=\{N(v):v\in V\}.
$$
A subset $S\subseteq V$ is shattered by $\mathcal{F}(G)$ if for every $A\subseteq S$ there is some $N(v)\in\mathcal{F}(G)$ with $N(v)\cap S=A$. The VC-dimension of $G$, denoted $vcdim(G)$, is the maximum size of a shattered subset of $V$ (or $\infty$ if arbitrarily large shattered sets exist) [2606.21640].

For identifying codes, the relevant hypergraph is built from closed neighborhoods. If $G=(V,E)$ is twin-free, an identifying code is a set $C\subseteq V$ such that
$$
\forall v\in V:\quad N[v]\cap C\neq\emptyset
\quad\text{and}\quad
u\neq v\;\Longrightarrow\;N[u]\cap C\neq N[v]\cap C.
$$
The closed-neighborhood hypergraph is
$$
H(G)=\bigl(V,\{N[v]:v\in V\}\bigr),
$$
and the VC-dimension of a hereditary class $\mathcal C$ is
$$
\VCdim(\mathcal C)=\sup_{G\in\mathcal C}\VCdim\bigl(H(G)\bigr)\in\{0,1,2,\dots\}\cup\{\infty\}.
$$
This change from open to closed neighborhoods is technically important, but the same threshold logic reappears [1407.5833].

In learning theory, VC-dimension is defined through growth functions. For a real-valued hypothesis class $\mathcal H$ on domain $\mathcal F$,
$$
\Pi_{\mathcal H}(m)=\max_{f_1,\dots,f_m\in\mathcal F}\bigl|\{(\operatorname{sign}(h(f_1)),\dots,\operatorname{sign}(h(f_m))):h\in\mathcal H\}\bigr|.
$$
The class shatters $\{f_1,\dots,f_m\}$ if $\Pi_{\mathcal H}(m)=2^m$. For real-valued classes one often passes to the pseudodimension via
$$
\operatorname{sign}(\mathcal H)=\{f\mapsto \operatorname{sign}(h(f)-b):h\in\mathcal H,\ b\in\mathbb R\},
$$
and writes $VC(\mathcal H)$ for this pseudodimension [2410.15800].

In model theory, VC behavior is captured by growth exponents rather than shattering numbers alone. For a partitioned formula $\phi(x;y)$ and finite $A\subseteq M^m$,
$$
\mathcal I_\phi(A)=\{\phi(A,b):b\in M^n\},\qquad
\pi_\phi(A)=|\mathcal I_\phi(A)|.
$$
The VC-density of $\phi$ is
$$
vc(\phi)=\inf\{d\in\mathbb R_{\ge 0}:\pi_\phi(A)=O(|A|^d)\},
$$
and
$$
dens(n)=\sup\{vc(\phi(x;y)):\ |y|=n\}.
$$
This is paired with dp-rank through ICT-patterns and the quantity $dpR(n)$ [1108.4398].

A further variant occurs in hypergraph algorithms. For a set system $(X,\mathcal R)$, VC-dimension is the largest $|A|$ such that the trace
$$
PR_{\mathcal R}(A)=\{R\cap A:R\in\mathcal R\}
$$
realizes all subsets of $A$. One also studies the dual system and the shatter function
$$
\pi_{\mathcal R}(m)=\max\{|PR_{\mathcal R}(A)|:A\subseteq X,\ |A|=m\},
$$
as well as finer $(\alpha,\beta)$-system constraints [1512.00481].

## 2. Hereditary graph classes: the threshold between linear and sub-linear twin-width

In structural graph theory, VC-tendency appears as a threshold for twin-width growth. Twin-width is defined through contraction sequences in trigraphs whose edges are colored black or red. A contraction identifies two vertices $u,v$ into a single new vertex $w$; every edge from $u$ or $v$ to a third vertex $x$ becomes black if both $ux$ and $vx$ were black, and red otherwise, including the case where one was a non-edge. The twin-width $tw(G)$ is the minimum integer $d$ such that $G$ admits a contraction sequence in which every intermediate trigraph has maximum red degree at most $d$ [2606.21640].

The decisive result is that bounded VC-dimension forces sub-linear twin-width. If $k=vcdim(G)$, then for every $n$-vertex graph $G$ with $vcdim(G)\le k$,
$$
tw(G)=O\bigl(n^{1-1/(2k+1)}\bigr),
$$
and in particular there is some constant $C(k)$ such that
$$
tw(G)\le C(k)\cdot n^{1-1/(2k+1)}.
$$
This gives a general sub-linear upper bound for every bounded-VC-dimension graph class [2606.21640].

The converse threshold is hereditary-class theoretic. A classical result reported there states that a hereditary class has bounded VC-dimension if and only if it excludes, as induced subgraphs, all large split graphs, all large bipartite graphs, and all large co-bipartite graphs. Since each of these three types can have linear twin-width by modifying conference-graph constructions, any hereditary class of unbounded VC-dimension contains graphs of twin-width $\Omega(n)$. The resulting equivalence is the core of the graph-theoretic VC-tendency:
- unbounded VC-dim $\Rightarrow$ linear twin-width through split, bipartite, and co-bipartite obstructions;
- bounded VC-dim $\Rightarrow\ tw(G)=o(n)$ [2606.21640].

The proof strategy passes through a contraction mechanism based on partitions by distinct neighborhoods. Suppose $V(G)$ is partitioned into disjoint sets $X_1,\dots,X_t,X_{t+1}$ such that $t\le \alpha$, $|X_{t+1}|\le \beta$, and for each $i\le t$, only $\gamma$ vertices outside $X_i$ are mixed on $X_i$. Then
$$
tw(G)\le \alpha+\max\{\gamma,\beta\}.
$$
The partition itself is extracted via a combinatorial lemma based on Haussler’s packing bound: in a VC-dimension-$k$ graph on $N$ vertices, for any subset $A$ of size $a$ one cannot have more than $M(a,\delta)=O(\delta^{-k})$ vertices whose neighborhoods in $A$ differ pairwise by at least $\delta\cdot a$. Iterating this produces blocks with controlled mixed sets and yields the exponent $1-1/(2k+1)$ [2606.21640].

Interval graphs provide a sharper special case. If $G$ is an $n$-vertex interval graph, then
$$
tw(G)\le 2\lceil\sqrt{2n}\rceil\in O(\sqrt n).
$$
There is also a lower-bound construction of $n$-vertex interval graphs with twin-width in $\Omega(n^{1/4})$; concretely, for $n=3r^2-2$ the construction yields twin-width at least
$$
\left(\frac{n+2}{12}\right)^{1/4}-1.
$$
This leaves an explicit gap even at VC-dimension $2$ [2606.21640].

A common misconception is that bounded VC-dimension should force bounded twin-width. The results do not state this. They state that graphs of bounded VC-dimension can have unbounded twin-width, but not linear twin-width; the guaranteed behavior is sub-linear, not constant or uniformly bounded [2606.21640].

## 3. Identifying codes: logarithmic versus polynomial size

For identifying codes, VC-tendency takes the form of a dichotomy theorem for hereditary classes. Let $\mathcal C$ be hereditary and write $\dim(\mathcal C)=\VCdim(\mathcal C)$. Exactly one of the following holds. If $\dim(\mathcal C)=\infty$, then for every integer $k\ge 1$ there is a graph $G_k\in\mathcal C$ with
$$
n_k=|V(G_k)|\ge 2^k-1
\quad\text{and}\quad
\gamma_{ID}(G_k)\le 2k=O(\log n_k).
$$
If $\dim(\mathcal C)=d<\infty$, then there is an exponent
$$
\varepsilon=\tfrac1d>0
$$
such that every twin-free $G\in\mathcal C$ on $n$ vertices satisfies
$$
\gamma_{ID}(G)\ge (n-1)^{1/d}=\Omega(n^\varepsilon).
$$
Thus VC-dimension completely determines whether identifying codes can be logarithmic or must be polynomially large [1407.5833].

The finite-dimension side is an application of Sauer’s lemma. Any identifying code $C$ in a twin-free graph must induce $n$ distinct traces of the $n$ closed neighborhoods on $C$, so
$$
n\le \sum_{i=0}^d \binom{|C|}{i}\le |C|^d+1,
$$
whence $|C|\ge (n-1)^{1/d}$. The infinite-dimension side is witnessed by shattered sets: if a graph in $\mathcal C$ has a shattered set $X$ of size $k$, then there are $2^k-1$ further vertices whose closed neighborhoods realize all nonempty subsets of $X$, and one constructs an identifying code of size $2k$ [1407.5833].

This threshold behavior has direct algorithmic consequences. If $\VCdim(\mathcal C)=\infty$, then $\mathcal C$ must contain, as induced subgraphs, all bipartite graphs or all split graphs or all co-bipartite graphs, and Min ID Code is log-APX-hard on $\mathcal C$. If $\VCdim(\mathcal C)$ is finite, one does not obtain a uniform approximation theorem. Interval graphs, which have $\VCdim=2$, admit a polynomial-time $6$-approximation algorithm. In contrast, on $C_4$-free bipartite graphs, also of VC-dimension $2$, Min ID Code cannot be approximated within a factor $c\log n$ for some $c>0$, unless $NP\subseteq ZTIME(n^{O(\log\log n)})$ [1407.5833].

This establishes an important limitation of the VC-tendency viewpoint. Finite VC-dimension controls the asymptotic minimum size of identifying codes, but it does not by itself force constant-factor approximability. The interval-graph and $C_4$-free bipartite cases demonstrate that identical VC-dimension can coexist with sharply different optimization complexity [1407.5833].

## 4. Learning-theoretic manifestations: GCNNs, GNNs, and WL color complexity

In deep learning, VC-tendency concerns how architectural parameters and combinatorial graph invariants govern capacity. For group convolutional neural networks, let $\mathcal H(k,m_0,\dots,m_L,r)$ denote the class of GCNNs with kernel-basis dimension $k$, layer widths $m_0,\dots,m_L$, and discretization resolution $r$. If
$$
W_\ell=\sum_{j=1}^{\ell} m_j\cdot (k\cdot m_{j-1}+1),
$$
then the layer-wise upper bound is
$$
VC(\mathcal H)\le L+1+4\Bigl(\sum_{\ell=1}^L W_\ell\Bigr)\log_2\!\Bigl(8e\,r\sum_{\ell=1}^L m_\ell\Bigr).
$$
For the class $\mathcal H_{W,L,r}$ of GCNNs with at most $L$ layers, at most $W$ weights, and resolution $r$, the upper and lower bounds imply, up to constant factors,
$$
VC(\mathcal H_{W,L,r})=\Theta\bigl(WL\log W+W\log r\bigr).
$$
The $WL\log W$ term matches the familiar scaling for fully connected ReLU networks, while the additional $W\log r$ term reflects the dependence on group discretization resolution [2410.15800].

The same study makes the interpretation explicit: weight sharing does not reduce VC significantly compared to DNNs for fixed $W$, but it controls the number of weights $W$ itself via shared filters. Input resolution enters only logarithmically, so doubling $r$ adds only $O(W)$ to VC. In the limit $r\to\infty$ for continuous groups, VC becomes infinite, matching known infinite VC for continuous two-layer GCNNs [2410.15800].

For graph neural networks analyzed through Weisfeiler–Leman, the threshold is expressed in three regimes. In the unbounded-order regime, for bit-length $b$ one has $VC\asymp b$, and fixed-width, fixed-depth classes $GNN(d,L)$ have infinite VC dimension once arbitrarily high bit-length is allowed. In the bounded-order regime, if
$$
m_{n,d,L}=\max\{|S|:S\subseteq G_{d,n}^B\ \text{and}\ 1\text{-}WL_L\ \text{colors distinguishes every pair in}\ S\},
$$
then
$$
{}_{G_{d,n}^B}(GNN(L))=m_{n,d,L}.
$$
On subclasses of bounded color complexity $u$, a Bartlett-style bound yields, for piece-wise-linear activations,
$$
VC\le O\bigl(L\cdot P\cdot \log(p\,u\,P)\bigr),
$$
where $P=d(2dL+L+1)+1$. Thus VC grows only logarithmically in the $1$-WL color complexity $u$ [2301.11039].

Empirical results in the same work align with these theorems. Increasing feature dimension $d$ at fixed depth enlarges the train–test gap, increasing the number of distinct $1$-WL color histograms enlarges the gap until saturation, and increasing simulated bit-length improves the ability to memorize random labels on synthetic trees. The authors report that these observations confirm the predicted dependence of VC on parameter count, WL-distinguishable count, and bit precision [2301.11039].

A plausible implication is that, in neural architectures, VC-tendency is not a binary bounded-versus-unbounded statement but a scaling law. Capacity tracks parameter count, depth, discretization resolution, and WL distinguishability in quantitatively different ways, with logarithmic dependence on $r$ and $u$ but linear dependence on $W$ and $L$ up to logarithmic factors [2410.15800] [2301.11039].

## 5. VC density and dp-rank in model theory

In model theory, VC-tendency concerns the asymptotic growth of shatter counts as the number of parameters increases. For every complete NIP theory $T$ and every $n\ge 1$,
$$
dpR(n)\le dens(n)\le dpR(n)+1.
$$
Here $dens(n)$ is the supremum of VC densities of formulas with $n$ parameter variables, and $dpR(n)$ is the maximum depth of an ICT pattern in $n$ variables. Consequently, strong dependence is equivalent to finite VC density [1108.4398].

The lower bound $dpR(n)\le dens(n)$ is obtained by showing that an ICT-pattern of depth $\kappa$ forces $\pi_\phi(A)\ge N^\kappa$ on suitable finite sets. The upper bound $dens(n)\le dpR(n)+1$ is derived through a combinatorial alternative: if a formula has VC-density strictly greater than $\kappa+1$, then one can extract an ICT-pattern of depth $\kappa+1$. This is the point at which combinatorial growth translates into model-theoretic independence [1108.4398].

Several standard theories illustrate the tendency. In o-minimal theories one has $dpR(n)=n$, hence $dens(n)=n$ or $n+1$, and more careful geometry yields $dens(n)=n$. In algebraically closed fields, stability gives $dpR(n)=0$ for all $n$, so $dens(n)\le 1$. In $P$-minimal or $p$-adic settings, one has $dpR(1)=1$, hence $dens(1)=1$, and more generally $dpR(n)=n$, $dens(n)=n$ [1108.4398].

The broader pattern is that $dens(n)$ grows linearly in $n$ up to an additive constant of $1$ in natural examples. The main unresolved issue recorded there is whether the upper bound can be sharpened to
$$
dens(n)\le dpR(n),
$$
which would imply integrality of VC-density and exact coincidence with dp-rank. No counter-example is known, but the general proof requires the additional “$+1$” slack [1108.4398].

## 6. Algorithmic thresholds, limitations, and open problems

For Hitting Set, low VC-dimension yields a sharp but limited tractability picture. If either the primal or dual VC-dimension is $1$, then Hitting Set is solvable in polynomial time. Once the VC-dimension rises to $2$, even with dual VC-dimension also $2$, the parameterized problem becomes W[1]-hard, and under ETH there is no algorithm of time $f(k)\cdot |X|^{o(k/\log k)}$ [1512.00481].

The same work shows that the raw VC-dimension can be too coarse, and introduces a finer threshold via $(\alpha,\beta)$-systems. Hitting Set on any $(3,5)$-system is solvable in polynomial time, while Hitting Set on $(3,6)$-systems is NP-hard. Thus, for $\alpha=3$, there is a sharp threshold:
- $\beta\le 5\Rightarrow P$,
- $\beta\ge 6\Rightarrow$ NP-hard [1512.00481].

These results guard against an overly strong reading of VC-tendency. Bounded VC-dimension often predicts improved behavior, but it does not uniformly imply easy optimization, fixed-parameter tractability, or bounded structural width. Three examples from the cited literature make this explicit. First, bounded VC-dimension does not imply bounded twin-width; it implies sub-linear twin-width growth [2606.21640]. Second, finite VC-dimension does not imply constant-factor approximability for identifying codes; interval graphs and $C_4$-free bipartite graphs already diverge at VC-dimension $2$ [1407.5833]. Third, low VC-dimension does not imply parameterized tractability for Hitting Set; W[1]-hardness already appears at VC-dimension $2$ with dual VC-dimension $2$ [1512.00481].

Several open problems remain central. For twin-width, it is unknown whether the exponent $1-1/(2k+1)$ is optimal for general VC-dimension $k$, and even for interval graphs there is a gap between the $O(n^{1/2})$ upper bound and the $\Omega(n^{1/4})$ lower bound. For $k=1$, VC-dimension-$1$ graphs are disjoint unions of cliques and have twin-width $0$; for $k=2$, the true exponent may lie strictly between $1/4$ and $1/2$ [2606.21640]. For model theory, the status of the “$+1$” in $dens(n)\le dpR(n)+1$ remains open [1108.4398]. For Hitting Set, open questions include whether the problem is FPT on $(3,6)$-systems and whether one can fully characterize the classical and parameterized complexity for general $(\alpha,\beta)$-systems when $\alpha\ge 3$ [1512.00481].

Taken together, these works show that VC-tendency is best understood as a family of threshold principles. The precise threshold variable may be VC-dimension, VC density, dual VC-dimension, WL color complexity, bit-length, or a refined shatter parameter, but the recurring pattern is that combinatorial shattering complexity governs transitions in width, code size, learnability, expressivity, and algorithmic hardness [2606.21640] [1407.5833] [2410.15800] [2301.11039] [1108.4398] [1512.00481].

Source: https://www.emergentmind.com/topics/vc-tendency