Level-2 Inference Overview

• Level-2 inference is a process that elevates analysis from base-level elements, such as text spans or individual actions, to higher-order units like subjects or sentences.
• It is applied across domains including text anonymization, document relation extraction, neural computation, and population statistics to capture latent structures.
• This hierarchical inference refines traditional methods by addressing unit shifts, enabling improved privacy metrics and more efficient computational architectures.

Level-2 inference is a field-dependent term for an inferential operation that is one layer above a base representation, observation, or decision rule. In recent arXiv literature it denotes subject-level PII inference in text anonymization, sentence-level inference in document-level relation extraction, algorithmic-level inference by nonlinear message-passing in neural computation, Neymanian or second-level population inference in statistics, inference of agents’ estimates of others’ objectives in dynamic games, and two-level computational schemes such as 2-level SLOPE and asynchronous two-level sparse attention (Oh et al., 23 Apr 2026, Tang et al., 2020, Raju et al., 2023, Dasgupta et al., 2012, Allefeld et al., 2015, Khan et al., 5 Aug 2025, Bu et al., 12 Jul 2025, Hu et al., 9 Apr 2026).

1. Cross-disciplinary meanings

Across the cited literature, “Level-2” does not name a single universal formalism. It marks a second inferential layer relative to a first layer that is treated as insufficient: text spans versus subjects, entity signals versus sentence signals, first-level subject estimates versus population inference, true objectives versus agents’ beliefs about one another, or single-stage sparsification versus two-stage selection.

Domain	Level-2 meaning	Representative paper
Text anonymization	Subject-level PII inference	(Oh et al., 23 Apr 2026)
Document-level relation extraction	Sentence-level inference	(Tang et al., 2020)
Neural computation	Algorithmic-level message-passing inference	(Raju et al., 2023)
Factorial and neuroimaging statistics	Neymanian or second-level population inference	(Dasgupta et al., 2012, Allefeld et al., 2015)
Dynamic games and inductive learning	Inference about others’ objectives or about rankings under novel cases	(Khan et al., 5 Aug 2025, O'Callaghan, 2019)
High-dimensional and systems inference	2-level penalty or two-level sparse attention	(Bu et al., 12 Jul 2025, Hu et al., 9 Apr 2026)

A plausible implication is that “Level-2” functions less as a domain-invariant object than as a recurring design pattern: it relocates the target of inference to a higher-order unit that better captures the phenomenon of interest.

2. Subject-level inference in text anonymization

Oh et al. define Level-2 inference in text anonymization at the level of the individual person rather than the text span. A “subject” is any individual person who can be identified by name, role, or other cues in a document, and multi-person texts typically mention $N \ge 1$ subjects. Their adversary is an LLM that identifies all subjects in the anonymized text and, for each subject $i$, attempts to infer a fixed set of PII categories comprising 15 fields: 5 CODE types and 10 NON-CODE types. The adversary is allowed inference rather than access to any external internal memory of the original text; it reasons over context left by the anonymizer (Oh et al., 23 Apr 2026).

The paper introduces two protection metrics. Let $O_i$ be the number of ground-truth PII values for subject $i$ in the original, and let $A_i$ be the number of PII values the adversary successfully infers about $i$ from the anonymized text, each scored $0.0/0.5/1.0$. The Collective Protection Rate is

$$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$

and the Individual Protection Rate is

$$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$

They also report $1-\mathrm{AAC}$ for the target subject only, to compare with prior single-subject evaluations (Oh et al., 23 Apr 2026).

The SPIA benchmark shifts the unit of evaluation from text spans to individuals and comprises 675 total documents: 144 ECHR legal judgments from TAB and 531 synthetic online posts from PANORAMA. It contains 1,712 subjects and 7,040 annotated PII values across 15 categories, with averages of 2.54 subjects per document and 4.11 PIIs per subject. The annotation pipeline has two stages: subject identification and per-subject PII inference. Each PII is labeled with Hardness 1–5 and Certainty 1–5; 85.7% of PIIs have Certainty $i$0. Human inter-annotator agreement is reported as subject matching $i$1 and PII inference $i$2 (Oh et al., 23 Apr 2026).

The empirical result is a span-versus-inference gap. Even when over 90% of PII spans are masked, subject-level inference protection drops as low as 33%. The paper’s example is TAB Longformer on legal texts, which achieves $i$3 but $i$4, implying that two-thirds of PIIs remain inferable by context. Across all 19 method × backbone configurations, gaps range from $i$5 to $i$6 in CPR versus ER. The multi-subject setting is also decisive: target-subject-focused anonymization leaves non-target subjects substantially more exposed than the target subject. Adversarial Anonymization yields the strongest CPR/IPR, up to approximately $i$7; DeID-GPT combines high utility, up to $i$8, with competitive inference protection; DP-Prompt has the lowest CPR, approximately $i$9, indicating that paraphrasing alone is insufficient (Oh et al., 23 Apr 2026).

3. Hierarchical and algorithmic Level-2 inference

In document-level relation extraction, Tang et al.’s Hierarchical Inference Network uses three granularities: entity level, sentence level, and document level. Level-2 inference is the sentence-level stage. Each sentence is encoded, and then “matched” against the entity-level inference vector. If $O_i$0 is the sentence embedding and $O_i$1 the entity-level inference vector, the sentence-level inference representation is

$O_i$2

This Level-2 representation is then aggregated by another BiLSTM plus attention to obtain the document-level vector. On DocRED, HIN-GloVe reports $O_i$3 (Test Ign F1 / F1), and HIN-BERT reports $O_i$4. The ablation study shows that removing the Level-2 sentence inference module drops F1 by approximately $O_i$5 points, while removing the hierarchical aggregation drops approximately $O_i$6 points (Tang et al., 2020).

In the neural-computation framework of “Inferring Inference,” Level 2 is the algorithmic layer between a normative latent-variable model and mechanistic neural recordings. The normative model posits latent causes $O_i$7, sensory inputs $O_i$8, and

$O_i$9

Algorithmic-level inference is then implemented as nonlinear message-passing on a graph-structured model of the world. Node states $i$0 are updated by canonical message functions $i$1, permutation-invariant aggregation, and a nonlinear state update. The inverse problem observes neural population responses $i$2 with a linear encoding

$i$3

and recovers the embedding $i$4, the internal graph $i$5, and the canonical message-parameters $i$6 by maximizing the observed-data likelihood with Particle EM, followed by greedy pruning. The recovered computational properties are statistically distinguishable up to a joint linear change of coordinates and a global coupling-scaling symmetry (Raju et al., 2023).

These two uses share a precise hierarchical logic. In HIN, Level-2 inference lifts entity-level signals to sentence-conditioned relational evidence. In “Inferring Inference,” Level 2 lifts a latent generative model to an explicit algorithmic inference mechanism that can itself be inferred from data. This suggests that one major role of Level-2 inference is to make intermediate computational structure observable and testable.

4. Statistical meanings: Neymanian, second-level, and hierarchical variational inference

In causal inference for $i$7 factorial designs, Dasgupta, Pillai and Rubin use “Level-2” to denote Neyman’s repeated-sampling inference. The framework defines unit-level potential outcomes $i$8 for each treatment combination $i$9, finite-population average factorial effects

$A_i$0

and unbiased estimators

$A_i$1

Under complete randomization, the sampling variance is

$A_i$2

Because $A_i$3 cannot be estimated without further assumptions, the Neyman variance estimator is conservative. The same framework also supports Fisher’s sharp-null randomization tests by imputing missing potential outcomes under $A_i$4 and re-randomizing under the original assignment mechanism (Dasgupta et al., 2012).

In information-based neuroimaging, Allefeld, Görgen and Haynes argue that a standard second-level $A_i$5-test on classification accuracy fails for valid population inference. The decisive constraint is that true accuracy $A_i$6 can never be below chance level $A_i$7. Under that constraint, the null hypothesis $A_i$8 collapses to the global null

$A_i$9

so rejecting a one-sample $i$0-test on $i$1 only allows the conclusion that some subjects have above-chance accuracy. The paper therefore proposes prevalence inference with the minimum statistic

$i$2

testing $i$3 for the population proportion $i$4 of subjects with above-chance effect. Under the prevalence null,

$i$5

The empirical example with $i$6 subjects reports that about 27% of voxels reject the global null at $i$7, whereas the standard second-level $i$8-test on accuracies rejects only about 14% of voxels at FWE $i$9 (Allefeld et al., 2015).

In variational inference for two-level linear mixed models, Nolan and Wand use “level-2” for models with group random effects. For groups $0.0/0.5/1.0$0,

$0.0/0.5/1.0$1

A mean-field variational approximation factors the posterior as

$0.0/0.5/1.0$2

with $0.0/0.5/1.0$3 multivariate normal, $0.0/0.5/1.0$4 inverse-$0.0/0.5/1.0$5, and $0.0/0.5/1.0$6 inverse-Wishart. The same model can be handled by variational message passing on a factor graph. The computational contribution is streamlined sparse linear algebra: naïve MFVB scales as $0.0/0.5/1.0$7, whereas two-level sparse least-squares and QR-based block computations reduce the cost to $0.0/0.5/1.0$8, and in balanced notation to $0.0/0.5/1.0$9 (Nolan et al., 2019).

Taken together, these statistical uses show three distinct meanings of Level-2 inference: assignment-based repeated-sampling inference, second-level population inference across subjects, and variational inference in models with two-level random effects.

5. Higher-order social, strategic, and inductive inference

Ostapov’s natural-language understanding system uses an object-oriented semantic model in which persons, things, events, places, and actions are represented by predicate frames. The Level-2 machinery addresses five families of problems: hypothesis checking, determination of agents and circumstances for non-typical actions, planning actions, cause determination, and state assessment of persons. Two core algorithms are used. Deduction performs direct logical matching; plausible reasoning is invoked when deduction fails and is grounded in social-psychology laws. Plausibility is written as a function

$$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$0

and combined along a reasoning chain as a product of law weights. The worked example contrasts a direct deduction for “Petrov shot the girl at 20:00 on Nov 7 in 9th Street” with a rejected hypothesis for “Ivanov” once path, motive, and tool checks force the combined plausibility to zero (Ostapov, 2012).

In inverse dynamic games, Level-2 inference denotes the problem “What does each agent believe about all agents’ objectives?” Agent $$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$1 holds its own estimate

$$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$2

where $$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$3 is $$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$4’s true objective parameter and $$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$5 are $$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$6’s beliefs about others. The observer’s inverse problem minimizes a trajectory-matching loss subject to the requirement that, for all $$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$7, $$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$8 is an LGNE of $$\mathrm{CPR} = 1 - \frac{\sum_{i=1}^N A_i}{\sum_{i=1}^N O_i},$$9 and the executed controls equal each agent’s own component. The paper proves that the Level-2 inverse-game loss is non-convex even in linear-quadratic games, and that Level-2 strictly generalizes Level-1 inference, which is recovered by constraining $$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$0. In the synthetic urban driving experiment, Level-2 inference uncovers each agent’s mistaken estimate of the other’s target lane, explains deadlock and subsequent behavior, and yields $$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$1 lower prediction loss than Level-1 (Khan et al., 5 Aug 2025).

O’Callaghan’s “second-order inductive inference” introduces a free hypothetical case $$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$2 and studies how rankings over eventualities adapt when a novel case type arrives. Under axioms A0–A3, a diversity axiom A4 or A4′, and P-Prudence, rankings admit an additive-separable numerical representation

$$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$3

The representation is unique up to scale and additive constant. Prudence is equivalent, in the finite-case setting, to a Jacobi identity

$$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$4

which functions as a robust test that novel cases will not induce intransitivity or dogmatic revision. The paper gives applications to startup success or failure, fake news, and robustly arbitrage-free yield curves (O'Callaghan, 2019).

A plausible implication is that these frameworks use Level-2 inference to move from surface evidence to latent intentional or counterfactual structure: motives behind actions, agents’ theories of one another, or rankings of rankings under novel experience.

6. Two-level computational architectures

In high-dimensional linear regression, 2-level SLOPE is a restricted form of SLOPE in which the monotone penalty sequence takes exactly two distinct values:

$$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$5

with hyperparameters $$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$6 and $$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$7. The estimator solves

$$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$8

Under AMP assumptions, the paper proves a sharp, tight characterization of the TPP–FDP trade-off:

$$\mathrm{IPR} = \frac{1}{N}\sum_{i=1}^N \left(1 - \frac{A_i}{O_i}\right).$$9

with equality attainable along the curve. Relative to LASSO, 2-level SLOPE introduces adaptivity with only three hyperparameters, breaks the Donoho–Tanner limit, and in the reported empirical settings achieves 5–30% lower MSE than LASSO, especially under high noise or correlated $1-\mathrm{AAC}$0 (Bu et al., 12 Jul 2025).

In long-context LLM inference, AsyncTLS implements Level-2 inference by combining a coarse block-level filtering stage with a fine token-level selection stage. If the KV cache is partitioned into blocks $1-\mathrm{AAC}$1 of size $1-\mathrm{AAC}$2, block scores are computed from max/min pooled key summaries and a grouped-query head, the top-$1-\mathrm{AAC}$3 blocks are retained, and token-level selection inside those blocks yields a final top-$1-\mathrm{AAC}$4 set for exact attention. Ignoring head and dimension factors, the cost is

$1-\mathrm{AAC}$5

contrasting with $1-\mathrm{AAC}$6 for full attention. AsyncTLS then couples this hierarchy to an asynchronous offloading engine: fine selection at step $1-\mathrm{AAC}$7 is performed over $1-\mathrm{AAC}$8 while new coarse blocks $1-\mathrm{AAC}$9 are prefetched, exploiting temporal locality so that

$i$00

rather than their sum. On Qwen3 and GLM-4.7-Flash across GQA and MLA architectures, the abstract reports accuracy comparable to full attention with $i$01–$i$02 operator speedups and $i$03–$i$04 end-to-end throughput improvements on $i$05–$i$06 contexts (Hu et al., 9 Apr 2026).

These two examples use “two-level” in an architectural sense rather than in a population or epistemic sense. The common feature is a constrained hierarchy that preserves much of the benefit of a richer object—general SLOPE or full attention—while reducing search or runtime complexity.

7. Recurring themes and limitations

Several papers use Level-2 inference to expose failures of a Level-1 proxy. In text anonymization, span-based recall is a poor proxy for true privacy because real adversaries perform contextual inference across redacted text. In information-based imaging, a standard second-level $i$07-test on accuracies does not support valid population inference because the effective null is the global null. In inverse games, Level-1 inference assumes homogeneous knowledge of objectives and therefore misses heterogeneous internal models (Oh et al., 23 Apr 2026, Allefeld et al., 2015, Khan et al., 5 Aug 2025).

A second recurring theme is unit shift. The inferential target moves from text spans to subjects, from entities to sentences and then documents, from first-level subject estimates to population prevalence, from observed actions to beliefs about others’ objectives, or from all penalty coordinates to a two-level penalty law. This suggests that Level-2 inference often appears when the scientifically relevant object is not directly observable at the base layer.

A third theme is hierarchy with constrained symmetry. In HIN, sentence-level inference is shared across sentences after semantic matching. In “Inferring Inference,” canonical message-functions are shared across edges, and the recovered computation is identifiable up to global transformation. In 2-level SLOPE and AsyncTLS, a small number of shared higher-level controls substitutes for a much larger unconstrained space (Tang et al., 2020, Raju et al., 2023, Bu et al., 12 Jul 2025, Hu et al., 9 Apr 2026).

The main misconception to avoid is that “Level-2 inference” names a single doctrine. Across the literature it is instead a family resemblance term for second-layer reasoning, estimation, or computation. What unifies the usages is not a common formal definition, but a common methodological move: inference is redirected to a higher-order object when first-order observables, summaries, or architectures are inadequate.