---
title: Dual Granularity Prompting (DGP)
url: https://www.emergentmind.com/topics/dual-granularity-prompting-dgp
type: topic
---

# Dual Granularity Prompting (DGP)

Dual Granularity Prompting (DGP) denotes a prompting strategy that combines two complementary levels of abstraction instead of relying on a single prompt granularity. In the most explicit formulation among the works considered here, DGSPNet for infrared small target detection uses **coarse-grained textual priors** together with **fine-grained personalized semantic descriptions** generated from the image itself, so that stable domain context and image-specific semantics are supplied in a single prompt system [2511.19306]. Closely related formulations preserve fine-grained target-node text while compressing neighborhood evidence into coarse summaries for fraud detection, combine global context and local appearance prompts in image restoration, and couple semantic-level abstraction with token-level pruning in chain-of-thought compression [2507.21653][2504.17825][2601.20467]. Multi-granularity prompting for dialogue topic shift detection extends the same principle to more than two levels—label, topic block, and turn/utterance—showing that the underlying idea is broader than any single architecture [2305.14006].

## 1. Conceptual basis and motivation

In DGSPNet, dual granularity is introduced because existing language-guided infrared small target detection methods are hindered by **annotation overhead** and **description discrepancy / semantic mismatch**. Prior approaches often need manually written text descriptions or costly VLM-generated captions, while infrared-specific descriptions are difficult to scale and can mischaracterize weak thermal signatures or clutter. The proposed remedy is to combine **coarse-grained textual priors**—described as easy, stable, and generic semantic context—with **fine-grained personalized semantic descriptions** that are image-specific and dynamically generated from the image itself [2511.19306].

A similar motivation appears in graph-enhanced fraud detection, where text-only graph prompting can produce extremely long prompts because multi-hop neighborhoods grow exponentially with hop count and each neighbor may carry dense textual information. There the dual-granularity solution is to keep **fine-grained text for the target node** while compressing **neighbor information into coarse-grained summaries**, thereby mitigating information overload and attention dilution [2507.21653].

These formulations indicate a common design pattern. A single prompt granularity is either too generic, too verbose, too expensive to obtain, or too brittle. This suggests that DGP is best understood as a mechanism for balancing a stable global prior against adaptive local evidence, with the precise meaning of “coarse” and “fine” determined by the task.

## 2. Dual-granularity semantic prompting in infrared small target detection

In DGSPNet, the coarse layer consists of predefined textual priors such as “infrared image,” “small target,” and scene context including “sky,” “ground,” “ocean,” and “suburb.” The paper gives the template example, “A photo of an infrared image, with targets in the sky/ground/ocean background.” These prompts are template-based, annotation-free, and encoded by a frozen text encoder into semantic embeddings
$$
f^{\text{text}} \in \mathbb{R}^{l \times d_t},
$$
where \(l\) is the text sequence length and \(d_t\) is the text hidden dimension [2511.19306].

The fine layer consists of learnable semantic tokens generated from the image through visual-to-textual mapping:
$$
\{s_i^*\}_{i=1}^{n_t} \in \mathbb{R}^{1 \times d}.
$$
These tokens are not manually labeled. Instead, they are learned from visual features and inserted into the text template to form an image-specific description. With \(n=2\), the template is “A photo of an infrared image, with \(s_1^*\) in \(s_2^*\) background.” In the authors’ formulation, the coarse prompt supplies stable semantic structure, while the fine-grained tokens fill in image-specific semantics [2511.19306].

This prompt construction is central to the method’s claim that language guidance can be used without manual annotation requirements at deployment. The prompt structure is fixed and template-based, but its fine semantic content is produced end-to-end from the input image itself.

## 3. Visual-to-textual mapping and text-guided feature modulation

The visual-to-textual pathway begins with the first three encoder layers, which produce multi-level features \(\{f^{(i)}\}_{i=1}^{3}\). These are described as low-to-middle-level cues such as edges, textures, local brightness, and spatial detail. They are downsampled and fused by depthwise convolutions and concatenation:
$$
f^{\text{img}}=\text{Conv}_{1\times1}\left(\text{Concat}\left(\text{DWConv}(f^{(1)},f^{(2)},f^{(3)})\right)\right).
$$
The fused feature is reshaped into sequence form and fed to an inversion net in which a set of learnable tokens \(T\) serves as the query in multi-head attention. The resulting learned tokens are projected into text space and inserted into the prompt template, yielding image-specific implicit semantic prompts [2511.19306].

The resulting text embeddings guide the visual pipeline through two attention modules. **Text-Guide Channel Attention (TGCA)** operates in the re-encoder on high-level features. It combines channel context from image features with text-derived channel descriptors:
$$
w^\text{ch} = \sigma \left(\text{Fn}\left(\text{Concat}\left(\text{GAP}\left(f^\text{mid}\right), \text{MLP}\left(f^\text{text}\right)\right)\right)\right),
$$
$$
f^\text{out} = f^\text{mid}+f^\text{mid}\cdot w^\text{ch}.
$$
The interpretation given in the summary is channel-wise semantic filtering: channels relevant to target semantics are amplified, irrelevant or noisy channels are suppressed, and residual addition preserves original information.

**Text-Guide Spatial Attention (TGSA)** operates in the decoder after upsampling and skip fusion. It uses the \([eot]\) token \(f^{[eot]}_\text{text}\) to compute a spatial compatibility score in a shared latent space:
$$
{w}^{(i)} = \text{Softmax}\left(\frac{\langle{v}^{(i)},\,{t}^{(i)}\rangle}{\sqrt{c}}\right) \in\mathbb{R}^{h_i\times w_i\times 1}.
$$
The refined decoder feature is then
$$
{f}_{d}^{(i-1)} = \text{Conv}_{3\times 3}(f_\text{fused}^{(i)} + {w}^{(i)} \cdot f_\text{fused}^{(i)}).
$$
This yields a full semantic loop in which low-to-middle-level features generate prompt tokens, high-level features are refined by TGCA, and multi-scale decoder features are spatially refined by TGSA [2511.19306].

## 4. Optimization, inference, and empirical behavior

DGSPNet uses a reconstruction pretraining stage to learn the inversion net. In that stage, the original decoder is replaced with a reconstruction decoder, the model reconstructs the input infrared image, and the inversion net is optimized with a contrastive objective:
$$
\mathcal{L} = \mathcal{L}_{\text{Contra}}\left(\text{sg}[f^{(5)}], f_\text{text}^{[eot]} \right)+ \mathcal{L}_{\text{MSE}}(\bm{out}, \bm{input}).
$$
After pretraining, inversion net weights are frozen and the full network is trained for segmentation with
$$
\mathcal{L} = \lambda_1 \mathcal{L}_{\text{BCE}}(\bm{m}, \bm{\hat{m}}) + \lambda_2 \mathcal{L}_{\text{SoftIoU}}(\bm{m}, \bm{\hat{m}}).
$$
During inference, the model uses the text prompts internally, requires no manual annotation, and runs the same dual-granularity mechanism end-to-end [2511.19306].

The reported benchmark results are: **IRSTD-1K** with IoU 70.87, \(P_d\) 93.26, and \(F_a\) 14.23; **NUDT-SIRST** with IoU 96.13, \(P_d\) 99.15, and \(F_a\) 0.32; and **NUAA-SIRST** with IoU 80.32, \(P_d\) 97.33, and \(F_a\) 9.26. The method is described as achieving state-of-the-art performance on three benchmark datasets. Ablation on prompt-driven modules shows that TGCA alone improves performance, TGSA and cross-attention help, and the full combination of **TGCA + Cross-Attention + TGSA** is best. On IRSTD-1K, IoU rises from 65.31 in the baseline to 70.87 in the full model.

The token-ablation study is especially diagnostic for DGP itself. On IRSTD-1K, 0 tokens gives IoU 66.41, 1 token gives 69.29, and **2 tokens gives 70.87**, while 3 or 4 tokens reduce stability or performance. The best prompt form is “A photo of an infrared image, with \(s_1^*\) in \(s_2^*\) background.” The authors interpret this as evidence that one token is better than none, two tokens best capture the target/background relation, and too many tokens introduce semantic ambiguity [2511.19306].

## 5. Cross-domain variants and related formulations

The dual-granularity idea has appeared in multiple technically distinct settings:

| Setting | Granularity design | Representative formulation |
|---|---|---|
| Infrared small target detection [2511.19306] | Coarse-grained textual priors + fine-grained personalized semantic descriptions | Template-based prompts, inversion net, TGCA, TGSA |
| Fraud detection with graph-enhanced LLMs [2507.21653] | Fine-grained target-node text + coarse-grained neighbor summaries | Node-level summarization, metapath summarization, numerical mean aggregation |
| Image restoration with diffusion transformers [2504.17825] | Global context prompts + local appearance prompts, fused with text prompts | Dual prompting control branch with CLIP image encoders and T5 |
| Dialogue topic shift detection [2305.14006] | Label + topic block + turn/utterance | Multi-granularity prompt-based framework with shared T5 decoder |
| Chain-of-thought compression [2601.20467] | Semantic-level compression + token-level pruning | HRA, LPD, DAG in a budget-conditioned reasoning pipeline |

The cross-domain record shows that “granularity” is not tied to a single modality. In fraud detection, it distinguishes target-node detail from neighborhood summaries; in image restoration, it distinguishes global context from local appearance; in dialogue, it distinguishes label semantics from topic-block and turn semantics; and in reasoning compression, it distinguishes semantic abstraction from token-level pruning. This suggests that DGP is a transferable design principle for organizing complementary evidence rather than a fixed prompt template.

At the same time, the domains differ in what is preserved and what is compressed. Fraud detection explicitly argues for task-agnostic textual summarization and statistical aggregation of numerical features, while CtrlCoT argues that generic token pruning is insufficient unless it preserves numbers, operators, and connective logic. The commonality is not the content of the prompt, but the decision to separate semantically different scales and optimize them jointly.

## 6. Scope, misconceptions, and acronym ambiguity

A common misconception is that DGP is merely “using text” in a downstream model. The infrared formulation explicitly states that it is not just “using text” in a generic sense, but a prompt system in which coarse prompts provide stable, annotation-free domain priors, fine-grained tokens are learned from the image itself via visual-to-text inversion, and text guidance is applied twice: for **channel selection** through TGCA and for **spatial localization** through TGSA [2511.19306].

Another misconception is that DGP must be textual in a narrow sense. In image restoration, the dual prompting module supplies **global context prompts** and **local appearance prompts** as extra conditional control, alongside textual prompts; in CtrlCoT, the two granularities are **semantic abstraction** and **token-level pruning**, not scene descriptions or captions [2504.17825][2601.20467]. This suggests that the defining feature of DGP is complementary granularity, not a particular token source.

The acronym itself is also overloaded in arXiv usage. In gravitational theory, “DGP” refers to **Dvali–Gabadadze–Porrati** gravity rather than prompt design, as in work on cascading DGP and on adding a second DGP brane [1212.0561][1908.01227]. In machine learning contexts, therefore, “Dual Granularity Prompting” is best identified from the surrounding task domain and formulation rather than from the acronym alone.

Source: https://www.emergentmind.com/topics/dual-granularity-prompting-dgp