Hierarchical Conformal Classification

• Hierarchical Conformal Classification (HCC) is an extension of conformal prediction that uses known taxonomic relationships to produce both leaf and internal node predictions.
• It computes node scores by aggregating descendant probabilities and uses calibrated nonconformity measures to ensure valid marginal coverage.
• HCC is applied in image, audio, and text classification to reduce prediction set sizes while maintaining interpretability and statistical reliability.

Hierarchical Conformal Classification (HCC) is a principled extension of classical conformal prediction for classification tasks, designed to take advantage of known hierarchical relationships among output labels. Unlike traditional conformal methods, which output "flat" sets of possible leaf-level labels, HCC produces prediction sets that may include both leaf and internal nodes in the class hierarchy—often modeled as a tree or a directed acyclic graph (DAG). This allows prediction sets to be simultaneously compact, more interpretable, and semantically meaningful, while offering rigorous statistical coverage guarantees in the presence of taxonomic structure. The HCC framework has been developed and analyzed in numerous settings, including multi-class, multi-label, and general structured prediction tasks (Hengst et al., 18 Aug 2025, Mortier et al., 31 Jan 2025, Norrenbrock et al., 25 Nov 2025, Zhang et al., 2024, Tyagi et al., 2024).

1. Class Hierarchies and Notation

HCC operates in settings where the set of possible labels $\mathcal{Y}$ is endowed with a hierarchy, typically represented as a tree or a more general DAG $(\mathcal{V}, E)$. Leaf nodes correspond to specific atomic classes, while internal nodes encode meaningful coarser groupings (e.g., in WordNet/ImageNet, leaves are fine-grained visual categories, and internal nodes are semantic hypernyms). Any subset of nodes $S \subseteq \mathcal{V}$ defines a "leaf cover" $\mathrm{Desc}(S)$, the set of all leaves reachable below nodes in $S$.

In many HCC algorithms, a prediction set is allowed to be any collection of nodes whose leaf cover contains the true class, enabling a trade-off between specificity (smaller leaf cover) and compactness (fewer nodes in the prediction set) (Hengst et al., 18 Aug 2025, Zhang et al., 2024, Mortier et al., 31 Jan 2025).

2. Core Methodology

The foundational procedure for HCC generalizes split conformal prediction to the hierarchical setting. The major steps are:

• Modeling and scoring: A base probabilistic classifier (e.g., neural network softmax outputs in $\mathbb{R}^{|\mathcal{Y}|}$) is used to assign probabilities to all leaves. Internal node scores are computed by summing probabilities over all descendant leaves:

$$\hat g(x)_v = \sum_{w \in \mathrm{Desc}(\{v\})} \hat f(x)_w$$

for $x$ an input and $v\in\mathcal{V}$ (Hengst et al., 18 Aug 2025, Zhang et al., 2024).

• Nonconformity scores: For a given candidate set $S\subseteq\mathcal{V}$ and calibration pair $(\mathcal{V}, E)$0, the conformity score is the highest node score along the true-ancestor path in $(\mathcal{V}, E)$1:

$(\mathcal{V}, E)$2

with $(\mathcal{V}, E)$3 (Hengst et al., 18 Aug 2025).

• Calibration: On a calibration set of size $(\mathcal{V}, E)$4, compute conformity scores $(\mathcal{V}, E)$5. The conformal threshold $(\mathcal{V}, E)$6 is typically taken as the $(\mathcal{V}, E)$7-quantile (Hengst et al., 18 Aug 2025, Zhang et al., 2024).
• Set construction: At test time, form a prediction set by including all nodes $(\mathcal{V}, E)$8 with $(\mathcal{V}, E)$9 or, equivalently, those whose empirical conformal $S \subseteq \mathcal{V}$0-values exceed $S \subseteq \mathcal{V}$1:

$S \subseteq \mathcal{V}$2

(Hengst et al., 18 Aug 2025).

• Optimization: The prediction set is selected as a solution of

$S \subseteq \mathcal{V}$3

subject to guaranteeing (empirical) coverage at level $S \subseteq \mathcal{V}$4 (Hengst et al., 18 Aug 2025).

This procedure efficiently computes hierarchical prediction sets with guaranteed marginal coverage.

3. Statistical Guarantees

HCC inherits the finite-sample, distribution-free marginal coverage properties of conformal prediction, extended to the leaf cover of the selected hierarchical set. Under the exchangeability assumption for calibration and test data, the following holds (Hengst et al., 18 Aug 2025, Zhang et al., 2024, Tyagi et al., 2024):

• For any fixed candidate set $S \subseteq \mathcal{V}$5 (or node collection), the split-conformal procedure ensures

$S \subseteq \mathcal{V}$6

• More generally, if set selection involves searching over multiple candidate NOL-covers (non-overlapping leaf covers), a Bonferroni correction or other multiple-testing adjustment can maintain the marginal coverage guarantee (Hengst et al., 18 Aug 2025).
• Family-wise error rate (FWER) control is achieved in tree-based multi-label settings via hierarchical testing with significance allocation across layers, either by fixed Bonferroni partitioning or adaptive tuning (Tyagi et al., 2024).
• Similar coverage guarantees hold for hierarchical prediction sets constructed via integer programming or dynamic programming in tree-structured hierarchies (Zhang et al., 2024, Mortier et al., 31 Jan 2025).

4. Algorithmic and Computational Aspects

The computational pipeline of HCC includes the following considerations:

• Candidate set search: The space of all hierarchical node collections is combinatorially large ($S \subseteq \mathcal{V}$7), but optimal sets can be restricted to non-overlapping, ancestor-independent NOL-covers, reducing computational cost to $S \subseteq \mathcal{V}$8 for trees of depth $S \subseteq \mathcal{V}$9 (Hengst et al., 18 Aug 2025).
• Dynamic programming: In tree-structured hierarchies, the set selection optimization can be solved using dynamic programming in $\mathrm{Desc}(S)$0 time, with $\mathrm{Desc}(S)$1 an upper bound on the number of nodes in the solution (Zhang et al., 2024, Mortier et al., 31 Jan 2025).
• Calibration cost: For $\mathrm{Desc}(S)$2 candidate thresholds and $\mathrm{Desc}(S)$3 calibration points, the total cost is $\mathrm{Desc}(S)$4 for generic integer programming, which reduces to $\mathrm{Desc}(S)$5 in trees (Zhang et al., 2024).
• Empirical scalability: In practice, $\mathrm{Desc}(S)$6 is kept small (e.g., $\mathrm{Desc}(S)$7), and the search is tractable in ImageNet-sized (1,000-class) hierarchies (Hengst et al., 18 Aug 2025, Norrenbrock et al., 25 Nov 2025).

5. Applications and Empirical Performance

HCC has been extensively validated in multi-class image, audio, and text classification:

• ImageNet-1K: On ImageNet-1K with a WordNet hierarchy, HCC achieves coverage $\mathrm{Desc}(S)$8 and compresses prediction sets by 30–60% compared to flat conformal methods. With $\mathrm{Desc}(S)$9, average set size for HCC is 2–4 coarse nodes (covering $S$025 leaves) versus 20–30 flat labels (Hengst et al., 18 Aug 2025, Zhang et al., 2024, Norrenbrock et al., 25 Nov 2025).
• Audio/text domains: On text (DBpedia) and audio (GTZAN) hierarchies, HCC consistently attained desired coverage with more compact and semantically meaningful sets than baselines (Hengst et al., 18 Aug 2025).
• CHiQPM (interpretable models): HCC has been applied within globally interpretable neural architectures, such as CHiQPM, where prediction sets are intrinsically coherent subtrees in a learned feature hierarchy, with empirical coverage matching theoretical targets and manageable set sizes on CUB-200, CARS-196, and ImageNet (Norrenbrock et al., 25 Nov 2025).
• Multi-label: In multi-label contexts, HCC is operationalized as a tree of label sets with split-conformal $S$1-values tested hierarchically, yielding shorter prediction sets and valid FWER control compared to power-set or binary-relevance conformal baselines (Tyagi et al., 2024).
• Structured prediction: In domains such as structured output prediction (e.g., string prefixes, year intervals), HCC achieves hierarchical compression and interpretability, preserving 90% coverage with smaller prediction set size than flat conformal approaches (Zhang et al., 2024).

6. Practical Insights, Interpretability, and User Studies

HCC methods offer prediction sets that are interpretable both globally—as unions of taxonomically meaningful classes—and locally, as output subtrees that reflect model uncertainty. Key empirical and usability findings include:

• Annotator studies reveal that human annotators significantly prefer HCC's hierarchical prediction sets over flat sets, primarily for their conciseness and alignment with human taxonomy reasoning. In controlled studies, HCC was favored in 57%–71% of cases, with Fleiss's $S$2 indicating fair-to-moderate inter-rater agreement (Hengst et al., 18 Aug 2025).
• In CHiQPM's implementation, prediction sets are always connected subtrees in the learned class-feature hierarchy, contributing both interpretability and statistical coverage (Norrenbrock et al., 25 Nov 2025).
• Performance advantages are especially pronounced in settings with large label spaces—HCC reduces the cognitive load on end users by grouping ambiguity into coarser, semantically unified output nodes (Hengst et al., 18 Aug 2025, Zhang et al., 2024).
• In multi-label and structured prediction contexts, hierarchical calibration and testing allow robust control of error rates and prediction set size even as the combinatorial complexity of the label space increases (Tyagi et al., 2024, Zhang et al., 2024).

7. Extensions and Open Challenges

Several variants of HCC have been proposed to address practical demands:

• Representation complexity: Limiting the number of internal nodes in prediction sets (so-called representation complexity) allows trade-off between interpretability (fewer nodes, more general sets) and precision (smaller leaf covers) (Mortier et al., 31 Jan 2025).
• Dynamic granularity: Dynamic-programming-based approaches support adjusting the complexity bound per instance, enabling flexible per-sample specificity (Mortier et al., 31 Jan 2025).
• Multi-label and DAG extensions: HCC formulations extend naturally beyond tree hierarchies to DAGs and multi-label settings, using generalized notion of leaf cover, though computational cost rises in general DAGs (Zhang et al., 2024, Tyagi et al., 2024).
• Conditional guarantees and loss structures: Conditional coverage under hierarchy constraints and alternative loss metrics (e.g., tree distance) remain open areas for research (Mortier et al., 31 Jan 2025).
• Scalability: Efficient candidate set search and calibration remain practical challenges for extremely large or deeply nested hierarchies, though structural reductions (NOL-covers, dynamic pruning) significantly alleviate this (Hengst et al., 18 Aug 2025, Zhang et al., 2024).

A plausible implication is that future work will refine instance-level adaptive complexity, alternative validity criteria (e.g., conditional, per-layer), and loss-aware selection, expanding HCC's utility in complex, structured-output predictive modeling.