Hierarchical Label Structures

• Hierarchical label structures are defined as tree or DAG-based organizations where labels derive meaning from both their ancestors and descendants.
• They enhance multi-label and multi-class classification by incorporating taxonomic, compositional, and correlation information for improved accuracy.
• These structures support varied domains like text, image, and medical segmentation through global, local, and embedding-based modeling techniques.

A hierarchical label structure is an organization of class labels according to a tree or directed acyclic graph (DAG), where each label’s semantic or representational meaning is defined not only by itself but also by its ancestors and descendants in the hierarchy. These structures are foundational in modern multi-label and multi-class classification across diverse modalities, enabling more accurate, interpretable, and scalable learning when labels exhibit taxonomic, compositional, or correlation structure. This article synthesizes principal concepts, formal definitions, methods, and empirical findings on hierarchical label structures, with references to major methodologies and theoretical underpinnings reported in the research literature.

1. Mathematical Formulation of Hierarchical Label Structures

Given an input space $\mathcal{X}$ and a finite set of atomic labels $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$, a hierarchical structure is defined by a directed graph $\mathcal{H} = (\mathcal{L}, \mathcal{E})$, where each edge $(\ell_i \to \ell_j) \in \mathcal{E}$ designates $\ell_i$ as a parent of $\ell_j$. Hierarchies can be trees or, more generally, DAGs (to permit multi-parent nodes).

• Hierarchy Notation
  • $\mathrm{Par}(\ell_j)$: Set of parent labels for $\ell_j$
  • $\mathrm{Ch}(\ell_i)$: Set of child labels for $\ell_i$
  • $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$0, $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$1: (Excludes $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$2 itself) Ancestors and descendants
  • $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$3: Recursively, $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$4 if no parent, $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$5 otherwise
  • $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$6: Set of labels at depth $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$7

A hierarchical classifier is a function $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$8, where for each $\mathcal{L} = \{\ell_1, \dots, \ell_m\}$9, the predicted subset $\mathcal{H} = (\mathcal{L}, \mathcal{E})$0 is ideally ancestor-closed (i.e., if $\mathcal{H} = (\mathcal{L}, \mathcal{E})$1 then $\mathcal{H} = (\mathcal{L}, \mathcal{E})$2), enforcing hierarchy consistency (Liu et al., 2023).

This formalism applies across classification domains, including text (Liu et al., 2023, Kumar et al., 2024), images (Noor et al., 2022, Stoimchev et al., 12 Mar 2026), and medical segmentation (Koitka et al., 2024).

2. Modeling Paradigms and Learning Algorithms

2.1 Global, Local, Hybrid, and Embedding-based Methods

Global (Flat) Models

Flat models ignore the hierarchy at prediction time, producing a score vector $\mathcal{H} = (\mathcal{L}, \mathcal{E})$3, often with a binary cross-entropy objective. Hierarchy-aware regularizers can penalize prediction scores that violate ancestor-descendant order, e.g. $\mathcal{H} = (\mathcal{L}, \mathcal{E})$4 (Liu et al., 2023).

Local Classifiers (Top-Down or Level-wise)

Hierarchical decompositions train specialized classifiers for each node or tree level, often using a top-down approach: At each level, prediction is conditioned on the activation of ancestors, proceeding from root to leaves. For example, a cascade of binary classifiers predicts presence/absence of each child label only if the parent was previously predicted (Liu et al., 2023, Noor et al., 2022).

Embedding-based and Neural Architectures

Recent work embeds inputs and labels in a shared representation space (Euclidean, hyperbolic), scoring their compatibility via inner products or distances. One example models both document and label embeddings in $\mathcal{H} = (\mathcal{L}, \mathcal{E})$5, and uses scoring $\mathcal{H} = (\mathcal{L}, \mathcal{E})$6 plus hierarchy-aware regularization $\mathcal{H} = (\mathcal{L}, \mathcal{E})$7 (Liu et al., 2023). Hyperbolic label embeddings are particularly effective for tree-like hierarchies, enabling exponential branching of semantic structure (Chen et al., 2019, López et al., 2020, Chatterjee et al., 2021).

Probabilistic Factorizations

For segmentation or classification per voxel/pixel, hierarchical chain rules rigorously define the probability of assigning a label $\mathcal{H} = (\mathcal{L}, \mathcal{E})$8 as the product of conditional probabilities along the unique path from the root to $\mathcal{H} = (\mathcal{L}, \mathcal{E})$9. In the SALT framework, this is implemented as

$(\ell_i \to \ell_j) \in \mathcal{E}$0

with the local softmax computed over sibling nodes at each parent (Koitka et al., 2024).

2.2 Hierarchy Discovery and Structure Induction

Many datasets lack explicit hierarchies. Methods can induce a hierarchy by clustering class-conditional distributions (means, spectral embeddings, or similarity measures). This process recursively partitions labels or groups them via Gaussian mixtures, yielding a multi-level tree and improved learning efficiency (Helm et al., 2021).

Recent frameworks such as SEAL jointly optimize the classifier and a latent label hierarchy, leveraging a differentiable tree-Wasserstein distance on the metric induced by the hierarchy, coupled with standard classification loss (Tan et al., 2023).

3. Loss Functions and Hierarchy-Aware Training Objectives

3.1 Hierarchy-Aware Supervised Losses

• Cross-Entropy with Chain Rule: Probabilities for coarse-to-fine prediction chains are modeled as $(\ell_i \to \ell_j) \in \mathcal{E}$1, with column-normalized transition matrices for each level. LHT introduces explicit transition networks for probabilistic level transitions and confusion loss for regularization (Wang et al., 2021).
• Margin Losses Enforcing Hierarchy: ML-CapsNet applies a multi-level margin loss where mispredictions at finer levels are penalized only if their ancestors are also predicted, and weight scheduling shifts focus from coarse to fine as training proceeds (Noor et al., 2022).
• Hyperbolic Regularization: Hyperbolic embedding approaches minimize geodesic distances between parent and child, which naturally encodes exponential branching and tree similarity (Chen et al., 2019, López et al., 2020, Chatterjee et al., 2021).
• Contrastive and Alignment Losses: Hierarchical multi-label contrastive learning penalizes discrepancies at all hierarchy levels using information from lowest common ancestors; contrastive coaching ensures that representations reflect not just instance-level similarity but hierarchy-induced distances (Zhang et al., 2022, Kumar et al., 2024).

3.2 Post-hoc Consistency and Inference

At prediction time, hierarchy-respecting inference is typically enforced by projecting predictions onto the hierarchy-closure (ensuring ancestor-completion), by topological decoding, or by applying per-level decision rules conditioned on parent predictions.

4. Evaluation Metrics Specific to Hierarchical Label Structures

Standard flat metrics are inadequate for hierarchical tasks. Hierarchy-aware evaluation accounts for ancestor-descendant overlap, partial correctness, and structural distances:

• Hierarchical Precision, Recall, F₁:

  $(\ell_i \to \ell_j) \in \mathcal{E}$2

  $(\ell_i \to \ell_j) \in \mathcal{E}$3

  (Liu et al., 2023)

• Distance-based Loss: Average tree distance between predicted and true leaves, $(\ell_i \to \ell_j) \in \mathcal{E}$4 (Liu et al., 2023).
• Average Hierarchical F₁, Tree-Induced Error, and Lowest Common Ancestor Metrics: Adjusted for multi-structure setups, with baseline comparison against flat and single-structure hierarchical models (Li et al., 2021).
• Early-Call AUC (eAUC): For ranking tasks, eAUC weights precision at high-confidence (early) calls, optimized via the HierLPR algorithm (Ho et al., 2018).

5. Empirical Findings, Application Domains, and Benchmarks

Hierarchical label structures consistently deliver accuracy, interpretability, and efficiency gains across domains:

• Text Classification: Hierarchical multi-label classification on datasets such as SciHTC (ACM CCS), PubMed, WOS, RCV1, NYTimes, and WIPO-α achieves superior Macro-F₁ and recall on rare and deep labels with global optimization, hyperbolic structures, or reinforcement learning-based assignment (Liu et al., 2023, Kumar et al., 2024, Mao et al., 2019).
• Image Recognition and Segmentation: Explicit embedding of hierarchical labels in CNNs, Vision Transformers, and Capsule Networks (with level-specific loss or class-tokens) improves both retrieval and classification at granular levels, with additional benefits in the low-label regime through self-supervised augmentation (Noor et al., 2022, Zhang et al., 2015, Stoimchev et al., 12 Mar 2026, Koitka et al., 2024).
• Unlabeled or Incomplete Hierarchies: Methods exist for the unsupervised or semi-supervised discovery of label hierarchies, typically via clustering or optimal transport alignment, often matching or exceeding the accuracy of pre-specified trees (Tan et al., 2023, Helm et al., 2021).
• Medical Imaging: Anatomical hierarchies modeled via per-node softmax chains (as in SALT) deliver consistent and interpretable multi-organ segmentation at clinical runtimes, outperforming non-hierarchical approaches (Koitka et al., 2024).
• Remote Sensing and Extreme Classification: Hierarchy-aware frameworks show strong performance on large-scale, sparse-label tasks, with computational advantages from partitioned or grouped approaches (Ubaru et al., 2020, Stoimchev et al., 12 Mar 2026).

Empirical ablations across most modalities reveal that embedding the entire label hierarchy, not just flat or per-level structure, yields robust, transferable representations and hierarchical error containment.

6. Current Challenges and Open Directions

Despite extensive progress, hierarchical label modeling presents persistent challenges:

• Label Imbalance and Data Scarcity: Most real hierarchies are power-law distributed in label frequency; adaptive sampling, meta-learning, and curriculum-based weighting address long-tail and zero-shot leaf nodes (Liu et al., 2023).
• Error Propagation and Deep Level Obfuscation: Mistakes at higher levels can block access to correct fine-level labels; solutions include global optimization, soft transitions (LHT), and reinforcement learning (Wang et al., 2021, Mao et al., 2019).
• Dynamic, Multi-Parent, and Evolving Hierarchies: Many problem domains require DAGs, not trees, and periodic expansion; architectures that generalize tree structures (e.g., GCNs on DAGs, graph self-attention) and continual learning schemes are active research areas (Koitka et al., 2024, Stoimchev et al., 12 Mar 2026).
• Unified and Multi-Structure Supervision: Fusing multiple complementary hierarchies (e.g., visual and semantic) in a single framework remains complex but is tractable with multi-task loss designs and shared feature extractors (Li et al., 2021).
• Evaluation and Interpretability: Hierarchical attention and per-level visualizations support fine-grained error diagnosis and model introspection, but standardized metrics and diagnostics remain underdeveloped (Zhang et al., 2020).

7. Summary Table: Representative Datasets with Hierarchical Label Structures

| Dataset | |$(\ell_i \to \ell_j) \in \mathcal{E}$5| | Depth ($(\ell_i \to \ell_j) \in \mathcal{E}$6) | Avg. Branching ($(\ell_i \to \ell_j) \in \mathcal{E}$7) | Domain | |-----------------|:-------------:|:-----------:|:--------------------:|-----------------------| | SciHTC (ACM CCS)| 1,233 | ~5 | 3.0 | Scientific text | | PubMed | 17,693 | 15 | 3.5 | Biomedical abstracts | | NYTimes | 2,318 | 10 | 4.0 | News articles | | WIPO-$(\ell_i \to \ell_j) \in \mathcal{E}$8 | 5,229 | 4 | 12.0 | Patent documents | | CIFAR-100 | 100 | 2 | 5 | Images | | SAROS (SALT) | 113 | 4 | ~3 | Medical CT seg. | | MLRSNet | 46,717 | 4* | - | Remote sensing |

*Depths for MLRSNet and others are approximate; some datasets feature DAG structure.

8. Concluding Perspective

Hierarchical label structures encode inductive biases and compositional information essential to scalable, interpretable, and high-performing classification systems. Modern research leverages explicit tree/DAG modeling, neural and hyperbolic embeddings, multi-task fusions, and algorithmic innovations in label discovery and evaluation. Persistent challenges—arising from dataset biases, architectural design, and hierarchy evolution—are active frontiers underscoring the centrality of label hierarchies across natural language, vision, and multidomain learning (Liu et al., 2023, Zhang et al., 2015, Koitka et al., 2024, Wang et al., 2021, Stoimchev et al., 12 Mar 2026).