Neural Concept Verifier (NCV)

• Neural Concept Verifier (NCV) is a framework that integrates Prover-Verifier Games with structured concept encodings to enable interpretable, nonlinear classification in high-dimensional settings.
• It employs dual agents, Merlin and Morgana, that selectively reveal a minimal, human-interpretable subset of concepts through a game-theoretic mask selection to justify predictions.
• Empirical evaluations on datasets like CIFAR-100 and ImageNet-1k demonstrate NCV’s superior accuracy and robustness, providing quantifiable guarantees of completeness and soundness.

The Neural Concept Verifier (NCV) is a unified framework that integrates Prover-Verifier Games (PVGs) with structured concept encodings to enable interpretable, nonlinear classification in high-dimensional domains. NCV addresses the limitations of conventional Concept Bottleneck Models (CBMs), which typically rely on low-capacity linear decision boundaries, and extends the PVG paradigm to settings involving complex visual evidence and large-scale concept vocabularies. By leveraging recent advances in minimally supervised concept discovery, NCV achieves verifiable, interpretable predictions while maintaining or exceeding state-of-the-art accuracy on challenging datasets (Turan et al., 10 Jul 2025).

1. System Architecture and Components

NCV decomposes the classification process $f: \mathcal{X}\to\mathcal{Y}$ into three interacting modules:

• Concept Extractor ($g: \mathcal{X}\to\mathbb{R}^C$): Transforms high-dimensional inputs (e.g., images) into dense vectors of $C$ concept activations. Two instantiations are supported:
  • CLIP-based similarity: For each text concept $t_i$, set $$c_i = \langle e_{\rm img}(x), e_{\rm text}(t_i) \rangle$$ using CLIP encoders.
  • Unsupervised slot-based (NCB): Slot-attention encoders produce binary concept indicators representing disentangled attributes such as shape or color.
• Prover Agents ($M$, $\widehat{M}: \mathbb{R}^C\to\{0,1\}^C$): Merlin ($M$) and Morgana ($\widehat{M}$) produce sparse binary masks (budgeted by $m$) that select which concepts are revealed to the verifier.
• Verifier (Arthur, $g: \mathcal{X}\to\mathbb{R}^C$0): Receives only the masked concept vector and outputs a prediction. Constructed as a nonlinear predictor, using either a multilayer perceptron or Set Transformer depending on the concept encoding.

Given $g: \mathcal{X}\to\mathbb{R}^C$1: $g: \mathcal{X}\to\mathbb{R}^C$2 Only $g: \mathcal{X}\to\mathbb{R}^C$3 is provided to Arthur during inference. This three-agent PVG ensures decisions are succinctly justified by a small, interpretable concept subset.

2. Prover-Verifier Game Mechanics

NCV formalizes classification as a strategic min–max game:

• Mask Selection: Both Merlin and Morgana are neural networks (Set Transformer or MLP) that assign scalar logits to each concept. After applying $g: \mathcal{X}\to\mathbb{R}^C$4, resulting in exactly $g: \mathcal{X}\to\mathbb{R}^C$5 selected concepts:

$g: \mathcal{X}\to\mathbb{R}^C$6

where $g: \mathcal{X}\to\mathbb{R}^C$7 (similarly for $g: \mathcal{X}\to\mathbb{R}^C$8).

• Objectives:
  • Merlin minimizes the cross-entropy loss of Arthur’s prediction:

  $g: \mathcal{X}\to\mathbb{R}^C$9 - Morgana maximizes the same loss (misleading Arthur):

  $C$0 - Arthur’s loss interpolates between Merlin and Morgana, controlled by $C$1:

  $C$2

• Optimization: Training alternates between (1) updating Merlin and Morgana to optimize their objectives, and (2) updating Arthur to minimize $C$3, using the Adam optimizer (learning rates of $C$4-$C$5 for NCB, $C$6-$C$7 for CLIP-based encoders, batch size 256–512).

3. Theoretical Guarantees and Verifiability

• Verifiability Metrics:

  • Completeness (Accuracy): $C$8
  • Soundness (Robustness): Probability that Arthur resists misclassification or issues a “reject” when presented with $C$9.
• Information-Theoretic Bound: NCV inherits bounds from Merlin–Arthur Classifier theory [Wäldchen et al. 2024]:

$t_i$0

This certifies that the selected concept subset $t_i$1 retains quantifiable information about the task label via Fano’s inequality and the data-processing theorem.

4. Empirical Evaluation and Comparative Analysis

NCV is evaluated on a diverse array of datasets:

• CLEVR-Hans3/7 (synthetic, with ground-truth concept labels),
• CIFAR-100 and ImageNet-1k (high-dimensional natural images with CLIP-derived concept vocabularies).

A comparative summary is given below:

Dataset	Feature Space	Completeness (±σ)	Soundness (±σ)
CLEVR-Hans3	pixel	97.87 (0.24)	n/a
	pixel-MAC	96.59 (0.72)	99.99 (0.01)
	CBM (NCB)	95.44 (0.08)	n/a
	NCV (ours)	98.92 (0.32)	100.00 (0.00)
CIFAR-100	CLIP-Sim	83.32 (0.28)	99.99 (0.01)
ImageNet	CLIP-Sim	67.04 (0.16)	99.94 (0.02)

NCV consistently outperforms both CBM and pixel-based PVG baselines in high-dimensional and logically complex settings. Mask-size ablations show that completeness increases with larger $t_i$2, while soundness remains above 99.8%. Set Transformer-based selection improves performance when tight mask budgets are enforced.

5. Interpretability, Shortcut Mitigation, and Qualitative Behavior

The PVG structure enforces that the classifier’s decisions are grounded on a small human-interpretable subset of concepts, facilitating post-hoc explanations. Empirical analysis on CLEVR-Hans3 finds that, unlike Pixel-MAC approaches (which mask large contiguous regions), NCV selects specific, semantically relevant concepts (e.g., “small metal cube”), aligning closely with generative causal rules. On confounded datasets (e.g., CLEVR-Hans7), training with limited clean samples reveals that NCV’s validation–test performance gap decreases more rapidly than both linear and nonlinear CBMs, suggesting improved resistance to shortcut exploitation and spurious correlations.

6. Limitations and Open Challenges

NCV’s success is contingent on the quality of the underlying concept extractor $t_i$3. Noisy or semantically entangled encodings can impair both accuracy and interpretability. The three-agent min–max training increases optimization complexity and may cause instability. Use of large pretrained models (such as CLIP) propagates inherited biases (e.g., selection of misaligned concepts), and the expert-alignment between selected concepts and human-grounded explanations remains imperfect (Turan et al., 10 Jul 2025). Ongoing research focuses on developing robust, semantically aligned concept discovery, stabilizing the adversarial dynamics, and establishing stronger human interpretability guarantees.

7. Relation to Other Concept Verification Frameworks

While NCV structurally enforces verifiability through concept-space PVGs, alternative approaches such as the Select–Hypothesize–Verify (SIEVE) protocol (Ji et al., 26 Mar 2026) address the problem of grounding neuron-level semantic interpretations. SIEVE iteratively selects neurons based on activation discriminability, generates language-based concept hypotheses using CLIP-style alignment, and verifies hypothesized concepts using diffusion-based image synthesis and network re-activation. SIEVE demonstrably increases the mean activation rate of verified concepts by nearly 1.5× compared to previous methods, establishing that verification in high-dimensional settings is tractable even at the level of individual neurons. This suggests the breadth of the "concept verifier" paradigm extends from structured model-level games (as in NCV) to neuron-level hypothesis validation, highlighting a trend toward rigorous, testable interpretability in modern neural architectures (Turan et al., 10 Jul 2025, Ji et al., 26 Mar 2026).