Explicit Fine-grained Alignment (EFA)

• Explicit Fine-grained Alignment (EFA) is a method that decomposes global alignment into inspectable local correspondences between substructures for improved task precision.
• It leverages explicit modules such as local similarity matrices, patch-token assignments, and criterion-conditioned supervision to complement implicit relational reasoning.
• EFA enhances performance in diverse applications like text-to-image retrieval and document understanding by providing fine-grained, actionable alignment signals.

Explicit fine-grained alignment (EFA) denotes a class of alignment strategies that expose and optimize local, structured, or criterion-level correspondences instead of relying only on global similarity or implicit fusion. In its most literal usage, FMFA for text-to-image person retrieval introduces an Explicit Fine-grained Alignment module to complement implicit relational reasoning, strengthen explicit cross-modal interactions, sparsify the similarity matrix, and use a hard coding method for local alignment (Yin et al., 17 Sep 2025). Across adjacent work, the same design principle appears in patch-level image-text matching, region-description contrastive alignment, token-level response supervision, and criterion-conditioned reward modeling. This suggests that EFA is best understood not as a single algorithmic template, but as a family of methods that make the alignment target itself more inspectable, decomposed, and operational (Wang et al., 2022).

1. Conceptual scope and defining properties

The central distinction in EFA-oriented work is between global alignment and fine-grained alignment. Global alignment places a whole image, document, 3D object, response, or trajectory near its paired text or preference target in a shared space. Fine-grained alignment instead operates on substructures: text tokens and image patches, region descriptions and visual regions, support and query feature positions, object crops and coordinates, or rubric criteria and response comparisons. In FMFA, this distinction is explicit: the framework combines a global matching branch with two complementary fine-grained mechanisms, namely implicit relational reasoning and the proposed EFA module (Yin et al., 17 Sep 2025).

A second defining property is explicitness. Some methods compute directly inspectable local correspondences. AETNet’s patch-level image-text alignment, for example, uses a rule-based OCR-layout assignment to map text tokens to image patches and then maximizes cosine similarity between each patch feature and the aggregated text feature for that patch (Wang et al., 2022). TOAN performs explicit query-conditioned support-to-query alignment by building a dense similarity matrix between all query and support spatial positions and using it to recompose support features in the query’s spatial frame (Huang et al., 2020). In 3DAlign-DAER, the explicit alignment variable is a token-to-point attention matrix $A_{\text{initial}} \in \mathbb{R}^{T \times N}$, later refined by a Dynamic Attention Policy through Monte Carlo tree search (Fan et al., 17 Nov 2025).

A third property is that EFA often preserves or introduces an intermediate alignment object. Depending on the domain, that object may be a similarity matrix, a patch-to-region assignment, a set of selected object crops, a criterion-conditioned reward, or a set of token-level edit weights. In this sense, explicitness is not identical to strong supervision. Some EFA-style methods are fully annotation-grounded, while others are weakly supervised but still expose an alignment structure that can be inspected or optimized directly (Liu et al., 11 Nov 2025).

2. Why EFA is needed

The motivating critique is consistent across multimodal retrieval, grounding, navigation, and language-model alignment: coarse objectives often hide the information that actually decides correctness. In text-to-image person retrieval, subtle cues such as clothing color, carried objects, hairstyle, and body-part details matter, yet prior methods based on implicit attention or relational reasoning may “yield only the final multimodal representation, without revealing the details of the aggregation process” (Yin et al., 17 Sep 2025). In document understanding, dense word-level, patch-level, and structural correspondences exist, but standard downstream fine-tuning of models such as LayoutLMv3 uses only task supervision unless explicit alignment objectives are added (Wang et al., 2022).

In grounding-oriented large vision-LLMs, coarse image-text alignment is linked to hallucination and factual error. Lyrics attributes better resistance to visual hallucinations to semantic-aware visual objects introduced by its Visual Refiner, which supplies image tagging, object detection, and semantic segmentation cues and exposes tags, boxes, and region-conditioned prompts as structured grounding signals (Lu et al., 2023). TinyGroundingGPT makes a similar diagnosis: text-coordinate alignment alone is incomplete because object images contain information richer than texts or coordinates, so the model explicitly aligns object texts, coordinates, and object crops (Wang et al., 2024).

The same critique extends beyond vision. FIGA argues that response-level preference signals tell a model only that an answer is good or bad, not which tokens or phrases are responsible for quality, and therefore proposes token-level positive and negative supervision derived from revisions (Guo et al., 2023). ProMedical argues that scalar or binary preference signals are especially inadequate in medicine because safety, factuality, completeness, instruction following, and communication quality are not reducible to one latent dimension; its solution is a rubric-based, criterion-conditioned reward framework (Geng et al., 9 Apr 2026). In both cases, the underlying claim is the same: alignment fails when the supervision collapses structured quality into a single undifferentiated signal.

3. Main technical patterns

One recurring EFA pattern is the use of an explicit local correspondence matrix. FAIR, a CLIP unsupervised adaptation method, computes crop-to-class interactions

$\widetilde{\Theta}_{ij}=\mathrm{sim}(f_i, Z_j^*)$

between localized visual crops $f_i$ and learnable Class Description Anchors $Z_j^*$, and then defines the Learned Alignment Score

$$\psi_{\mathrm{FAIR}}(x,y)=\sum_{i=1}^{N}\widetilde{w_i}\,\widetilde{\Theta}_{ij}\big|_{j=\mathcal{C}(y)}\,\mathbb{I}_{\{i\in\mathcal{I}_k\}},$$

where only top-$k$ relevant crops contribute (Ali et al., 13 Jul 2025). This is explicit local evidence aggregation rather than a single global CLIP cosine score.

A second pattern is region-level contrastive supervision. GeoAlignCLIP introduces Region-Phrase Alignment with

$$\mathcal{L}_{\mathrm{RPA}}=-\frac{1}{2K}\sum_{k=1}^{K}\Bigg[ \log\frac{\exp(\mathrm{sim}(v_r^k,t_r^k)/\tau)}{\sum_{l=1}^{K}\exp(\mathrm{sim}(v_r^k,t_r^l)/\tau)} + \log\frac{\exp(\mathrm{sim}(t_r^k,v_r^k)/\tau)}{\sum_{l=1}^{K}\exp(\mathrm{sim}(t_r^k,v_r^l)/\tau)} \Bigg],$$

explicitly pairing region embeddings $v_r^k$ with region-description embeddings $t_r^k$ (Yang et al., 10 Mar 2026). The paper augments this with Hard-Negative Alignment and multi-view consistency, making local alignment both explicit and discriminative.

A third pattern is factorized compositional alignment. In decomposed open-vocabulary segmentation, prompts are manually separated into one concept token sequence and multiple attribute token sequences, each attribute has its own cross-modal gating branch, and composition is enforced by multiplicative AND-style aggregation: $$\mathbf{G}_{\mathrm{total}}=\prod_{k=1}^{K}\mathbf{G}_{a_k}, \qquad \mathbf{F}_{\mathrm{out}}=\mathbf{Q}_c \odot \mathbf{G}_{\mathrm{total}}.$$ At the score level, the same conjunction is implemented in log space (Wang et al., 15 May 2026). This is EFA in a particularly strong sense because semantic factors are separated structurally before alignment and recombined by an explicit rule.

A fourth pattern is criterion-conditioned or token-conditioned optimization. FIGA assigns positive weights to added or substituted revised tokens and negative weights to deleted or substituted original tokens, then optimizes

$\widetilde{\Theta}_{ij}=\mathrm{sim}(f_i, Z_j^*)$0

thereby turning revision differences into explicit local alignment signals (Guo et al., 2023). ProMedical uses criterion-conditioned preference estimation

$\widetilde{\Theta}_{ij}=\mathrm{sim}(f_i, Z_j^*)$1

and trains a reward model with

$\widetilde{\Theta}_{ij}=\mathrm{sim}(f_i, Z_j^*)$2

which is explicit not at the token-region level but at the level of named clinical criteria (Geng et al., 9 Apr 2026).

4. Domain-specific realizations

The most direct use of the term appears in text-to-image person retrieval. FMFA treats “full-mode fine-grained alignment” as the combination of implicit relational reasoning from IRRA and explicit fine-grained alignment, with the latter designed to verify local correspondence rather than only trust a fused multimodal representation (Yin et al., 17 Sep 2025). In documents, AETNet inserts modality-specific encoders before multimodal fusion and adds patch-level image-text alignment through OCR-layout-based patch-text assignments, making downstream tuning itself alignment-enriched (Wang et al., 2022).

In open-world and grounded vision-language systems, EFA often appears as structured object grounding. Lyrics injects tags, boxes, and segmentation-aware object features into a BLIP-2-style stack and uses grounded captioning, referring expression comprehension, and referential dialogue templates to supervise object-level correspondence (Lu et al., 2023). TinyGroundingGPT makes the alignment triadic rather than merely phrase-to-box: object text, object coordinates, and object image crops are all paired through explicit instruction data, and a three-stage training process first aligns single-object representations and then multi-object global grounding (Wang et al., 2024).

In remote sensing and fine-grained recognition, EFA spans several granularities. GeoAlignCLIP uses region-level descriptions, hard negatives, and consistency constraints to move CLIP-style training beyond global image-caption alignment (Yang et al., 10 Mar 2026). OFA-Net applies explicit geometric feature alignment through rotated proposals and AlignConv, aligning whole-object features to object orientation and boundaries for fine-grained satellite imagery recognition (Ming et al., 2021). TOAN adapts support features to query features through a dense query-support similarity matrix, making few-shot fine-grained comparison explicitly correspondence-based rather than purely metric-based (Huang et al., 2020).

The concept also extends into medical and navigational settings. In medical image synthesis, fine-grained alignment is implemented by matching report tokens to a predefined visual codebook of image patches and using selected key patches as visual clues for generation (Chen et al., 2024). In vision-language navigation, FCA-NIG creates a dual-level annotation structure: sub-instructions are aligned with sub-trajectories, and entities extracted from those sub-instructions are aligned with GLIP-detected landmarks, yielding the FCA-R2R dataset (Cui et al., 10 Jun 2025). In language-model alignment, PsyMem uses 26 quantitative psychological indicators as an explicit role profile and adds explicit memory control, while ProMedical uses physician-derived rubrics and a tripartite schema of Proficiency, Excellence, and Safety Veto to disentangle high-stakes clinical criteria (Cheng et al., 19 May 2025).

5. Supervision regimes and empirical evidence

The literature shows that EFA can improve performance under very different supervision budgets. Some methods emphasize that they require no additional supervision beyond existing labels. FMFA states that its explicit fine-grained alignment and A-SDM improve matching “without requiring additional supervision” and achieves state-of-the-art performance among global matching methods on three public datasets (Yin et al., 17 Sep 2025). The decomposed open-vocabulary segmentation framework likewise states that explicit prompt decomposition does not require extra supervision beyond standard category-attribute annotations and improves unseen AP on UBC from 4.0 to 7.1, with AND-Eff rising from 0.533 to 0.959 (Wang et al., 15 May 2026).

When explicit local supervision is available, the gains are often strongest on the most localized alignment term. In AETNet, adding the architecture with supervised objective only yields 89.77 / 96.30 on FUNSD/CORD, but adding patch-level image-text alignment gives 90.69 / 96.78, and combining all alignment objectives reaches 91.55 / 97.04; the paper identifies PITA as the strongest single component (Wang et al., 2022). GeoAlignCLIP reports that adding Region-Phrase Alignment alone raises RRSIS-HR Acc@1 from 25.48 to 33.28, and the full model with RPA, VIC, HNA, and HTC reaches 42.28 (Yang et al., 10 Mar 2026).

Object-centric grounding data also shows direct benefits. TinyGroundingGPT reports that adding its additional multi-scale fine-grained grounding dataset improves the 3B model from 87.35 / 78.89 / 83.25 to 88.50 / 80.56 / 83.69 on RefCOCO, RefCOCO+, and RefCOCOg, and that adding global grounding data improves general benchmarks such as GQA from 61.7 to 63.3 and MMB from 63.1 to 66.4 (Wang et al., 2024). FAIR reports that removing LAS drops average accuracy on its six-dataset ablation from 75.93 to 68.89, and the full method improves over prior state of the art by 2.78% across 13 fine-grained datasets (Ali et al., 13 Jul 2025). FG-CLIP 2 reports large bilingual gains on explicitly fine-grained tasks, including FG-OVD hard performance of 52.3 with ViT-B/16 versus 24.9 for SigLIP 2, and COCO box classification Top-1 of 74.9 versus 53.4 for SigLIP 2 (Xie et al., 13 Oct 2025).

In language-model alignment, explicit decompositions also outperform holistic baselines. FIGA reports an average score of 34.9 versus 32.7 for DPO and 31.1 for SFT, with a particularly large TruthfulQA gain to 42.0 (Guo et al., 2023). ProMedical reports that ProMedical-RM-guided GRPO improves overall accuracy by 22.3% and safety compliance by 21.7%, and its ablations show that removing explicit criteria drops pairwise accuracy from 88.50 to 83.15, while removing safety data sharply degrades safety performance (Geng et al., 9 Apr 2026).

6. Limits, variants, and boundary cases

The literature also shows that EFA is not a single, fully standardized object. Some papers use the term directly, while others are better described as EFA-like or EFA-adjacent. REFA, for example, explicitly controls output length and EOS behavior and structures preferences into positive and negative response sets, but the paper itself does not present this as a full multi-attribute EFA framework; its relation to EFA is therefore narrower and focused on explicit control of one response attribute rather than broad semantic factorization (Gupta et al., 2024). Similarly, PsyMem is EFA-like because it replaces vague persona descriptions with 26 structured psychological indicators and explicit memory alignment, but its explicit variables are role and memory controls rather than cross-modal token-region correspondences (Cheng et al., 19 May 2025).

Explicitness itself also varies. GRM for image-text retrieval is explicit in the sense that it exposes token salience gates, patch-to-region assignment weights, and token-patch similarity matrices, but it does not use ground-truth region-word correspondences; it is therefore “explicit-but-weakly supervised” rather than fully supervised local matching (Liu et al., 11 Nov 2025). GeoAlignCLIP aligns regions with region descriptions rather than tokens with words, so its granularity is region-description level rather than dense token-patch level (Yang et al., 10 Mar 2026). OFA-Net aligns whole-object features to object orientation and boundaries, but it does not model part correspondences or semantic landmark alignment (Ming et al., 2021).

A second boundary concerns whether explicitness comes from the architecture, the data, or the objective. TinyGroundingGPT and FCA-NIG are largely data-centric: they construct training instances where local multimodal correspondences are the task itself (Wang et al., 2024, Cui et al., 10 Jun 2025). AETNet and FAIR are objective-centric, adding explicit patch-level or crop-level scoring terms (Wang et al., 2022, Ali et al., 13 Jul 2025). FG-CLIP 2 combines both strategies through region-text supervision, long-caption data, and auxiliary discriminative losses (Xie et al., 13 Oct 2025). This suggests that EFA is best viewed as a design space spanning supervision, representation, and optimization rather than a single methodological layer.

Finally, several papers point to unresolved issues. FG-CLIP 2 notes the need to handle even longer textual inputs and explicitly model relational structures among objects (Xie et al., 13 Oct 2025). TinyGroundingGPT’s data synthesis depends on external models such as GroundingDINO and GPT-4V, and the paper notes filtering requirements and ambiguities in dataset accounting (Wang et al., 2024). Medical image synthesis via prompt-to-codebook patch alignment provides explicit token-patch retrieval, but the paper does not formalize the similarity computation or isolate the alignment mechanism with dedicated ablations (Chen et al., 2024). These limitations indicate that EFA has matured into a recognizable research direction, but not yet into a single canonical formulation.