Coarse-to-Fine Distillation Objective

• Coarse-to-fine distillation objective is a training pattern that progressively refines supervisory signals from broad class-level cues to localized, fine-grained information.
• It employs either multiterm objectives or sequential pipelines to integrate both high-level guidance and detailed refinements, addressing issues like granularity mismatches and weak supervision.
• Various implementations—ranging from prototype alignment and attention diversity to logit grouping and graph-based refinement—demonstrate its effectiveness in improving model precision across tasks.

Coarse-to-fine distillation objective denotes a family of training objectives in which knowledge transfer is organized across progressively finer semantic, structural, or representational levels rather than enforced in a single flat matching step. In the supplied literature, the coarse signal may be a class label, a bag label, a document-level score, or an initial pose estimate, while the fine signal may be a prototype-aligned representation, an instance pseudo-label, a finer class group, a limb-structure constraint, or a passage- or sentence-level relevance distribution (Ma et al., 26 Mar 2026, Wu et al., 4 Feb 2025, Li et al., 30 May 2025, Ji et al., 15 Aug 2025, Shenaj et al., 2022, Zhou et al., 2024). The resulting objectives are heterogeneous in form, but they share a common design principle: preserve coarse supervision while adding mechanisms that expose localized, low-confidence, or structurally constrained information that would otherwise be suppressed.

1. General formulation

Across the supplied works, coarse-to-fine distillation is not a single canonical loss but a design pattern. Some methods use a unified multiterm objective, in which coarse supervision is retained and finer constraints are added as auxiliary terms. Others use a sequential pipeline, with a first stage producing a coarse estimate and a second stage refining it under new supervision. Still others use self-distillation, where the same network converts coarse labels into finer pseudo-labels, rather than relying on an external teacher (Ma et al., 26 Mar 2026, Wu et al., 4 Feb 2025, Ji et al., 15 Aug 2025).

A useful summary is that coarse-to-fine objectives differ mainly in the space in which refinement occurs. In some cases the refinement is over class groups; in others it is over instances, attention maps, graph-structured outputs, or cross-modal alignments. This suggests that “coarse-to-fine” is best understood as a supervision schedule over granularity, not as a commitment to one particular loss family.

Method	Coarse signal	Fine signal
FD$^2$	class-level supervision	class prototypes and attention diversity
LadderMIL	bag label $Y$	pseudo instance label $Y'$
PCD	all-class logit distillation	stage-wise class groups and reverse refinement
CCDA	parent coarse class posterior	child fine-class probabilities
Two-stage pose KD	initial distilled pose	progressive GCN refinement outputs
Speech-image retrieval	SIC coarse alignment	SIM fine-grained matching

In this family, the teacher–student relation also varies. PCD uses a conventional teacher and student at the logits level (Li et al., 30 May 2025). CCDA uses the previous-stage segmentation model as teacher and the current finer-label model as student (Shenaj et al., 2022). LadderMIL instead uses one network for both roles, turning bag-level attention into instance supervision (Wu et al., 4 Feb 2025). FD$^2$ uses a teacher equipped with Counterfactual Attention Learning to produce prototypes and attention maps that become optimization targets during dataset distillation (Ma et al., 26 Mar 2026).

2. Prototype-, attention-, and instance-level refinement

FD$^2$ formulates coarse-to-fine distillation explicitly for fine-grained dataset distillation. The method starts from class-level supervision but adds two terms intended to preserve localized discriminative cues and diversify same-class distilled samples. The fine-grained characteristic constraint is

$$\mathcal{L}_{F}(\tilde{x}_{y,i})= \beta\,\ell_2(z_{y,i},c_y)+ (1-\beta)\left(1-\mathbb{E}_{k\neq y}\big[\ell_2(z_{y,i},c_k)\big]\right), \qquad \beta\in[0,1],$$

and the similarity constraint is

$$\mathcal{L}_{S}(\tilde{x}_{y,i})= 1-\mathbb{E}_{j<i}\big[\ell_2(A_{y,i},A_{y,j})\big],\qquad 1<i\le N_S.$$

The final optimization combines the underlying distillation objective, class supervision, prototype alignment and separation, and attention diversity, with $\lambda$ controlling the trade-off between $\mathcal{L}_F$ and $\mathcal{L}_S$ (Ma et al., 26 Mar 2026). The paper’s interpretation is precise: $Y$0 aligns each distilled image with its class prototype while repelling other prototypes, and $Y$1 prevents same-class synthetic samples from collapsing onto the same discriminative region.

The same paper makes the coarse-to-fine transition operational through Counterfactual Attention Learning during pretraining. CAL learns attention maps that localize discriminative regions and maintains a running class prototype via momentum updates. This gives the later distillation stage a fine-grained target space that is more specific than ordinary class-label supervision. The reported empirical pattern is that the method is especially effective on fine-grained datasets, whereas gains on general datasets are less consistent and depend more on student capacity (Ma et al., 26 Mar 2026).

LadderMIL provides a different formulation. Here the coarse supervision is the bag label $Y$2, and the fine supervision is a pseudo-instance label generated by the model’s own attention ranking. The framework selects the top-$Y$3 instances from each bag, initializes $Y$4, increases it by $Y$5 if bag-level performance does not improve for 3 consecutive epochs, and stops at $Y$6. Selected instances inherit the bag label, yielding an instance-level branch trained jointly with the bag-level branch (Wu et al., 4 Feb 2025). The objective is deliberately simple:

$Y$7

The distinctive feature is that coarse-to-fine distillation does not mean matching hidden features or teacher logits; it means converting weak bag supervision into instance supervision by attention-guided self-annotation (Wu et al., 4 Feb 2025).

These two formulations show that a coarse-to-fine distillation objective can target either representational geometry or supervision granularity. FD$Y$8 refines class-level distillation into prototype-space compactness, inter-class separation, and same-class diversity. LadderMIL refines bag-level MIL into instance-level classification. This suggests that the crucial variable is not whether the loss is called “distillation,” but whether the objective exposes finer discriminative structure than the original training signal.

3. Logit grouping and semantic hierarchy

Progressive Class-level Distillation formulates coarse-to-fine knowledge transfer directly in logit space. Its starting diagnosis is that standard logit distillation is dominated by high-confidence classes, which suppresses low-probability classes that still carry discriminative information. PCD therefore ranks classes by teacher–student discrepancy,

$Y$9

divides the ranked classes into stages, and applies bidirectional stage-wise distillation (Li et al., 30 May 2025). The stage sizes are

$Y'$0

so Fine-to-Coarse Learning starts from smaller, focused class subsets and expands toward all classes, while Coarse-to-Fine Learning reverses that schedule. Group-wise KL terms are weighted by

$Y'$1

and the intended overall objective is

$Y'$2

PCD is therefore distinctive in two ways: the coarse-to-fine refinement is explicit in the class partition schedule, and the final training uses both a forward progressive pass and a reverse refinement pass (Li et al., 30 May 2025).

CCDA applies a hierarchy-aware coarse-to-fine distillation objective to continual semantic segmentation under domain shift. The class taxonomy is refined over time: a coarse class at step $Y'$3 is split into a set of finer classes at step $Y'$4. The full loss is

$Y'$5

The key term is the coarse-class distillation constraint

$Y'$6

which constrains the predictions of the fine classes to resemble their parent coarse class, while unchanged fine classes are handled by a standard distillation term $Y'$7 (Shenaj et al., 2022). The method also introduces a coarse-to-fine weight initialization rule that copies parent weights to child classes and adjusts biases by $Y'$8, so that the child probabilities are not unfairly overconfident at initialization.

The reported ablations make the semantic role of this objective unusually clear. On GTA5 $Y'$9 Cityscapes, MiB achieves $^2$0 mIoU, SKDC $^2$1, and CCDA $^2$2; on GTA5 $^2$3 IDD, MiB achieves $^2$4, SKDC $^2$5, and CCDA $^2$6 (Shenaj et al., 2022). In the paper’s own characterization, the coarse-to-fine distillation is not merely output preservation but a semantic conservation law: the probability mass assigned to a parent class must be explained by its children in the next stage.

Taken together, PCD and CCDA show two distinct label-space versions of coarse-to-fine distillation. PCD refines a flat class simplex into staged discrepancy-based groups, whereas CCDA refines an explicit semantic hierarchy. Both are motivated by the insufficiency of one-shot all-class matching.

4. Structure-aware and graph-based refinement

The two-stage human pose distillation framework illustrates a coarse-to-fine objective in a structured output space. In the first stage, a smaller SimCC-based student is trained with feature distillation, pose distillation, and the native SimCC KL loss. The pose distillation term is

$^2$7

where the human joints structure loss compares the lengths of corresponding skeleton edges in teacher and student predictions. Distillation weights are decayed by

$^2$8

and the full stage-1 loss is

$^2$9

with $^2$0 and $^2$1 (Ji et al., 15 Aug 2025).

The second stage freezes the stage-1 student and trains an Image-Guided Progressive Graph Convolutional Network on the initial pose and multi-resolution image-guided joint features. The graph is the human skeleton $^2$2, and progressive supervision is applied at three graph outputs:

$^2$3

$^2$4

with $^2$5, $^2$6, and $^2$7 (Ji et al., 15 Aug 2025). The paper explicitly states that the method is two-stage rather than a single unified end-to-end loss: first coarse distillation, then fine pose refinement distillation.

This formulation is important because it broadens the notion of coarse-to-fine distillation beyond class probabilities. The coarse stage transfers high-level semantic knowledge and skeleton-aware structure; the fine stage performs graph-based correction using low- to high-resolution image context. The resulting objective is therefore not just “teacher imitation,” but teacher-guided progressive refinement of a structured estimate.

5. Cross-modal and retrieval-oriented extensions

In speech-image retrieval, the coarse-to-fine objective is realized as joint optimization of coarse embedding alignment and fine-grained pairwise matching. Speech-image contrastive learning aligns speech and image CLS embeddings by cross-entropy in both directions, while speech-image matching uses a multimodal encoder to perform binary matched-versus-unmatched classification (Zhou et al., 2024). Because the supervision is noisy, the paper adds momentum distillation: a momentum teacher produces soft targets, and the distillation-augmented SIC loss is

$^2$8

The full objective sums this contrastive-with-distillation term and the SIM loss (Zhou et al., 2024). Here the “coarse” level is global speech-image semantic alignment, and the “fine” level is multimodal interaction-based verification. The paper reports gains of +4.2% mean R@1 on both Flickr8k and SpokenCOCO relative to the previous best method, and attributes robustness in part to momentum distillation under noisy supervision (Zhou et al., 2024).

A closely related, though differently named, formulation appears in Fine-Grained Distillation for Long Document Retrieval. The method keeps inference-time dense retrieval unchanged but introduces multi-granular training-time supervision. Document-level contrastive learning is retained,

$^2$9

and finer passage- and sentence-level relevance distributions are matched by

$$\mathcal{L}_{F}(\tilde{x}_{y,i})= \beta\,\ell_2(z_{y,i},c_y)+ (1-\beta)\left(1-\mathbb{E}_{k\neq y}\big[\ell_2(z_{y,i},c_k)\big]\right), \qquad \beta\in[0,1],$$0

The final objective is

$$\mathcal{L}_{F}(\tilde{x}_{y,i})= \beta\,\ell_2(z_{y,i},c_y)+ (1-\beta)\left(1-\mathbb{E}_{k\neq y}\big[\ell_2(z_{y,i},c_k)\big]\right), \qquad \beta\in[0,1],$$1

Although this work is framed as fine-grained distillation rather than coarse-to-fine distillation, the training logic is analogous: a coarse document-level objective is supplemented with finer aligned supervision to correct a granularity mismatch (Zhou et al., 2022).

These cross-modal and retrieval-oriented formulations show that coarse-to-fine distillation need not be tied to classification. It can also mediate between efficient inference constraints and finer training-time supervision, particularly when the student is forced to compress heterogeneous or partially aligned inputs.

6. Scope, non-equivalence, and limitations

Not every method described as coarse-to-fine defines a coarse-to-fine distillation objective in the strong sense. FunnelRAG is mainly a coarse-to-fine progressive retrieval architecture: clustered documents are retrieved first, documents are pre-ranked next, and passages are post-ranked last. The paper does include a limited local-to-global signal transfer from post-ranking to pre-ranking, where passage-level post-ranking scores are aggregated to document scores and used to construct positives and negatives for a BPR loss,

$$\mathcal{L}_{F}(\tilde{x}_{y,i})= \beta\,\ell_2(z_{y,i},c_y)+ (1-\beta)\left(1-\mathbb{E}_{k\neq y}\big[\ell_2(z_{y,i},c_k)\big]\right), \qquad \beta\in[0,1],$$2

but it explicitly states that this is not a full hierarchical end-to-end distillation across all stages (Zhao et al., 2024). In that case, “coarse-to-fine” refers primarily to retrieval granularity, quantity, and model capacity rather than to a unified distillation objective.

The supplied material also includes cases where an objective cannot be reconstructed from the text at hand. For CCF-Net, the provided content does not contain the architecture, coarse versus fine localization modules, explicit loss functions, weighting between losses, or ablation studies, so a paper-faithful coarse-to-fine objective cannot be extracted from the supplied text (Chen et al., 2022). The same problem holds for HumanDiffusion: the supplied document is described as a rebuttal/template document containing references but not the method sections, equations, or losses for the Stylized Memory Retrieval and Multi-scale Cross-modality Alignment modules (Zhang et al., 2022).

A recurring misconception is therefore that any coarse-to-fine architecture automatically implies a coarse-to-fine distillation objective. The supplied works show otherwise. Some papers define explicit objective-level transitions from coarse supervision to fine supervision, such as FD$$\mathcal{L}_{F}(\tilde{x}_{y,i})= \beta\,\ell_2(z_{y,i},c_y)+ (1-\beta)\left(1-\mathbb{E}_{k\neq y}\big[\ell_2(z_{y,i},c_k)\big]\right), \qquad \beta\in[0,1],$$3, LadderMIL, PCD, CCDA, and the two-stage pose framework (Ma et al., 26 Mar 2026, Wu et al., 4 Feb 2025, Li et al., 30 May 2025, Shenaj et al., 2022, Ji et al., 15 Aug 2025). Others mainly define a coarse-to-fine pipeline and only a limited or indirect training-time transfer, as in FunnelRAG (Zhao et al., 2024). Still others use the label “coarse-to-fine” in title or abstract, but the supplied text is insufficient to expose the objective at all (Chen et al., 2022, Zhang et al., 2022).

Taken as a research pattern, the coarse-to-fine distillation objective is most clearly characterized by three properties. First, it preserves the coarse task rather than discarding it. Second, it introduces finer supervisory structure that is absent from the base objective, whether through prototypes, instances, stage-wise class groups, semantic hierarchies, graph outputs, or multimodal matching. Third, it is typically motivated by a failure mode of flat supervision: dominance of high-confidence classes, loss of localized cues, granularity mismatch, weak bag labels, or structural ambiguity. The supplied literature therefore supports a technical definition of coarse-to-fine distillation not as a single formula, but as a principled family of objectives for progressive knowledge transfer across granularity.