Hierarchical Violation Rate (HVR)

• Hierarchical Violation Rate (HVR) is a metric that measures parent-child prediction inconsistencies in tree-structured label hierarchies.
• It is computed by comparing hard predicted labels across adjacent levels using nearest-prototype classification to detect violations.
• Empirical evaluations on datasets like CIFAR-100 and FGVC-Aircraft show that lower HVR values indicate improved semantic consistency and adherence to hierarchy.

Searching arXiv for the cited papers and any directly relevant hierarchical-consistency metrics in HMC. Hierarchical Violation Rate (HVR) is a dataset-level metric introduced for hierarchical multi-label classification (HMC) to quantify whether predicted labels remain structurally consistent with a tree-structured label hierarchy, rather than merely accurate at individual levels (Jiang et al., 19 Aug 2025). In the formulation used for HCAL, a sample receives one label per level, and a violation occurs when a predicted child label is paired with a predicted higher-level label that is not that child’s parent in the hierarchy. HVR is therefore a consistency metric: it evaluates parent-child prediction conflicts across adjacent levels, complements per-level accuracy, and is computed from hard predicted labels after nearest-prototype classification.

1. Formal definition and notation

The metric is introduced with the statement that HVR “evaluates the proportion of predictions that violate parent-child constraints in the label hierarchy” (Jiang et al., 19 Aug 2025). The paper defines the prediction for sample $i$ as

$$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$

where $\hat y_i^k$ is the predicted label at level $k$. The level-specific label sets are defined as

$$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$

The HVR formula is given as

$$HVR=\frac{1}{n \cdot R} \sum_{i=1}^{n} \sum_{k=2}^{m} \sum_{c \in \mathcal{C}^{k}} \mathbb{I}\left(\hat{y}_{i}^{k-1}=c \wedge \hat{y}_{i}^{k} \neq p(c)\right),$$

with normalization

$R=\sum_{k=1}^{m-1}\left|\mathcal{C}^{k}\right|.$

In this notation, $p(c)$ denotes the parent label of $c$ at level $k$, $$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$0 is the number of samples, $$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$1 is the number of hierarchy levels, $$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$2 is the set of labels at level $$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$3, and $$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$4 is described as the total number of parent-child relationships across all levels. Because the definition is written in terms of $$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$5, HVR uses discrete predicted labels rather than logits, probabilities, or thresholded scores.

2. Operational semantics in hierarchical multi-label classification

The intended HMC setting is a tree-structured label hierarchy in which labels are organized by levels and each sample receives one label per level (Jiang et al., 19 Aug 2025). The semantic constraint is that if a child label is predicted at level $$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$6, then the prediction at the next higher level $$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$7 should be that child’s parent. Standard per-level predictions can therefore be individually plausible while jointly contradictory.

Conceptually, the indicator

$$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$8

equals $$\hat{\mathbf{y}}_{i}=\left(\hat{y}_{i}^{1}, \hat{y}_{i}^{2}, \ldots, \hat{y}_{i}^{m}\right),$$9 exactly when the predicted child-parent pair is inconsistent, and $\hat y_i^k$0 otherwise. Since each sample has one predicted label at each level, at most one $\hat y_i^k$1 can satisfy $\hat y_i^k$2 for a fixed $\hat y_i^k$3. The numerator thus counts total parent-child inconsistencies across all samples and adjacent level transitions, and the final value is normalized by $\hat y_i^k$4.

A simple two-level illustration makes the definition concrete. Let level 1 contain $\hat y_i^k$5, let level 2 contain $\hat y_i^k$6, and let $\hat y_i^k$7, $\hat y_i^k$8, and $\hat y_i^k$9. If a sample is predicted as $k$0 and $k$1, the indicator is $k$2 because the predicted higher-level label is incompatible with the predicted child. If instead $k$3 and $k$4, the contribution is $k$5. For $k$6 samples with $k$7, the paper’s normalization gives $k$8, so $k$9 sample-level inconsistencies yield $$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$0.

3. What HVR measures, and what it does not

HVR is designed to capture hierarchical consistency or structural adherence, not ordinary classification correctness (Jiang et al., 19 Aug 2025). The paper explicitly argues that per-level accuracy does not encode whether multi-level predictions obey the hierarchy. A model can therefore achieve high accuracy at individual levels while still producing semantically impossible cross-level predictions.

This distinction is reflected in the joint use of $$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$1 and HVR. $$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$2 measures whether the level-$$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$3 label is correct, whereas HVR measures whether the combination of predicted labels across levels respects parent-child constraints. The metric is presented as evaluating “hierarchical consistency and generalization” and as measuring adherence of predictions to hierarchical label structures. In that role, HVR is not a replacement for accuracy; it is an additional diagnostic for semantic consistency.

A low HVR does not imply that predictions are correct. A hierarchy-consistent prediction can still be wrong at one or more levels. Conversely, strong per-level accuracy does not guarantee consistency. The metric is therefore most informative when read together with level-wise accuracy, especially in multi-task learning settings where different hierarchy levels are optimized partly independently.

4. Experimental use in HCAL

Within HCAL, HVR is one of the principal evaluation tools and is reported alongside per-level accuracy across CIFAR-100, FGVC-Aircraft, and Classroom (Jiang et al., 19 Aug 2025). The compared methods are MMoE, DMT, CAGrad, IMTL, Nash-MTL, CMT, and HCAL. On CIFAR-100, the reported HVR values are 0.0975 for MMoE, 0.1149 for DMT, 0.1059 for CAGrad, 0.1132 for IMTL, 0.1098 for Nash-MTL, 0.0601 for CMT, and 0.0582 for HCAL. On FGVC-Aircraft, the reported values are 0.1602, 0.2772, 0.1116, 0.1176, 0.1137, 0.1299, and 0.1012, respectively. On Classroom, the reported values are 0.0160, 0.0171, 0.0188, 0.0163, 0.0179, 0.0135, and 0.0114, respectively. The authors emphasize that HCAL achieves the lowest HVR on all three datasets and interpret this as evidence of improved semantic consistency and better adherence to hierarchical constraints.

The paper makes similar use of HVR in the ablation study on FGVC-Aircraft. The ablations examine MTL, Feature Aggregation (FA), Prototype Perturbation (PP), and Adaptive Weight Adjustment (AWA), and the full HCAL model reaches an HVR of 0.0327. Replacing MTL with single-task learning causes what the authors describe as a 19.7-fold worsening in HVR; disabling feature aggregation, prototype perturbation, or adaptive weighting also increases HVR sharply. In the experimental narrative, HVR is thus not an auxiliary metric but a primary instrument for arguing that each module contributes to preserving hierarchical consistency.

Several implementation details determine how HVR is instantiated in these experiments. Predictions are hard labels obtained by comparing extracted features to prototypes via cosine similarity and then performing nearest-prototype classification. CIFAR-100 and Classroom use two levels, whereas FGVC-Aircraft uses the last three levels of a four-level hierarchy. The hierarchy is encoded bottom-up, which is directly relevant because HVR checks child-to-parent compatibility between adjacent levels.

5. Scope, assumptions, and caveats

As defined, HVR checks only adjacent parent-child consistency; it does not directly verify arbitrary ancestor-descendant constraints across multiple hops (Jiang et al., 19 Aug 2025). In deeper hierarchies, a model may satisfy every adjacent relation while the metric does not separately score longer-range structural properties. The notation $$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$4 also assumes that each child has a unique parent, which fits the paper’s tree-structured hierarchy and a single-parent setting rather than a general DAG with multiple parents.

The metric is computed from hard predictions only. Because HVR is evaluated after nearest-prototype classification, it ignores confidence: a marginal inconsistency and a highly confident inconsistency are treated identically. This makes the metric purely output-structural.

The normalization is also somewhat unusual. The numerator counts instance-level violations over adjacent predicted labels, while the denominator uses

$$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$5

which is the total number of lower-level labels across levels rather than, in any obvious way, the number of per-instance adjacent-level checks. The intended meaning remains clear—a dataset-level normalized inconsistency rate—but the scaling is not simply the fraction of violated adjacent checks per sample. The paper should therefore be read according to its exact definition, while recognizing that the denominator is mathematically unconventional.

The authors connect low HVR to semantic consistency, generalization, and robustness. Strictly speaking, however, HVR itself measures output-level consistency rather than robustness directly. A plausible implication is that consistent reductions in HVR indicate that the learned representations preserve categorical structure more reliably under the tested conditions, but the metric itself remains a structural inconsistency measure.

6. Related metrics and terminological ambiguity

The HCAL paper defines only one formal HVR metric and does not introduce alternative HVR variants (Jiang et al., 19 Aug 2025). The only related quantitative measures used alongside it are per-level accuracies $$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$6. No additional consistency-specific metrics such as ancestor precision, path accuracy, exact-match consistency, or edge-wise consistency are introduced there.

A separate source of potential confusion is that the acronym “HVR” does not always denote Hierarchical Violation Rate. In "Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models," HVR refers to hierarchical visual recognition, not to a violation-count metric, and the paper does not explicitly define or report Hierarchical Violation Rate (He et al., 28 Feb 2026). Instead, it evaluates hierarchical consistency using Hierarchical Consistent Accuracy (HCA), Point-Overlap Ratio (POR), Strict Point-Overlap Ratio (S-POR), Top Overlap Ratio (TOR), and leaf-level accuracy $$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$7.

That distinction is technically important. In the TARA setting, HCA measures strict full-path correctness, POR measures the average fraction of correct levels, S-POR measures the longest contiguous correct segment, and TOR measures adjacent-level pair correctness (He et al., 28 Feb 2026). The paper notes that if one wished to derive an HVR-like quantity from those metrics, one might consider complements such as $$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$8 or $$\mathcal{C}^{k}=\bigcup_{i=1}^{n}\left\{y_{i}^{k}\right\}.$$9, but it never defines such complements. HVR in the HCAL sense is therefore a specific dataset-level measure of parent-child inconsistency under hard predictions, not a generic label for all hierarchy-aware evaluation.