Papers
Topics
Authors
Recent
Search
2000 character limit reached

NVQG: Negative Visual Query Guidance

Updated 4 July 2026
  • Negative Visual Query Guidance (NVQG) is a technique that integrates explicit negative constraints into visual model processing to suppress undesired content.
  • It leverages interventions in feature and attention spaces within diffusion models, style prompting, and retrieval to finely control output quality.
  • NVQG methods offer practical benefits such as enhanced diversity, improved retrieval accuracy, and reduced similarity to undesired content, as shown by specific performance metrics.

Negative Visual Query Guidance (NVQG) denotes a family of mechanisms that steer visual models away from unwanted content by injecting explicitly negative constraints into generation, retrieval, or alignment. In recent work, the term has been used for image-based adversarial guidance during diffusion inference, query-swapped negative branches for style prompting, dynamically regenerated negative prompts from intermediate denoising states, representation steering for negated retrieval queries, hard-negative supervision in egocentric localization, and conditional semantic comparison for medical vision-LLMs. The unifying objective is explicit absence control: the system is not only asked what to produce or retrieve, but also what to exclude (Singh et al., 2024, Chang et al., 30 Oct 2025, Jeong et al., 8 Oct 2025, Ko et al., 28 May 2026, Sammani et al., 20 Mar 2026, Kim et al., 30 Jan 2026, Xu et al., 2022).

1. Conceptual scope and representative formulations

NVQG is not a single algorithm. Rather, it is a recurring design pattern in which a negative query modifies the model at the locus where visual semantics are assembled: transformer features, attention outputs, denoising scores, similarity geometry, or candidate comparisons. In diffusion models, this usually means altering the reverse trajectory at inference time; in retrieval and alignment, it means changing ranking or discrimination so that negated constraints become first-class decision variables.

Setting Negative signal Representative formulation
Visual-reference diffusion guidance Reference-image features Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}
Style prompting Query-swapped leakage branch ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})
Few-step negative prompting Flipped negative values OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N
Orthogonal concept suppression Orthogonal negative attention component O=OposαNO' = O_{\text{pos}} - \alpha N_\perp
Dynamic negative prompting Time-varying negative branch ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})
Evidence-calibrated removal Presence-constrained projection ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t

A persistent theme is that text-only negation is often too coarse. Several papers explicitly argue that negative prompts alone struggle with complex visual concepts, copyrighted characters, or precise exclusions, which motivates moving the negative signal into feature space or attention space rather than leaving it in prompt text alone (Singh et al., 2024, Guo et al., 11 Aug 2025, Ko et al., 28 May 2026).

2. Feature-space repulsion from visual references

A direct diffusion instantiation of NVQG is "Negative Token Merging" (NegToMe), which performs adversarial guidance using visual features from a reference image or from other images in the same batch. NegToMe is inserted between the attention and MLP branches inside transformer blocks and operates on the attention output tensor ORB×N×DO \in \mathbb{R}^{B \times N \times D}. For each source token, it computes cosine similarity against reference tokens, selects the best-matching token if the maximum similarity exceeds a threshold τ\tau, and applies linear extrapolation to repel the source away from the matched target: Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}. Tokens without a sufficiently strong match are left unchanged. The same framework supports masked matching through a bias term log(M~ref+ϵ)\log(\tilde{M}_{\text{ref}}+\epsilon) when only part of the reference is undesired (Singh et al., 2024).

This mechanism is explicitly training-free, uses only marginally higher inference time, and is compatible with both latent-diffusion UNets and MM-DiT architectures such as Flux. The paper reports that it adds less than ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})0 inference time, can be implemented in a few lines of code, and works even for models that do not natively support negative prompts. Its applications include batch-wise diversity enhancement and copyrighted-content mitigation. In the latter setting, masked NegToMe reduces visual similarity to copyrighted content by ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})1 while maintaining text-image alignment and quality; in the former, it decreases DreamSim, increases Entropy, and preserves or improves FID and IS across SDXL and Flux (Singh et al., 2024).

The significance of this line of work is that the negative query is no longer an external text string but a visual exemplar encoded at the same layer and timestep as the current latent state. NVQG here is therefore literal visual-query guidance: the model is repelled from a specific reference in its own internal feature geometry.

3. Attention-space NVQG in style prompting and few-step generation

In style-conditioned generation, NVQG has been used to suppress content leakage from visual style prompts. StyleKeeper extends classifier-free guidance to self-attention swapping: the main branch keeps the query from the text-conditioned generation process while injecting keys and values from the style branch, and the negative branch intentionally simulates leakage by swapping in queries from the visual style branch. The operational score is

ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})2

The method is applied only to self-attention in late upblocks, with DDIM sampling for 50 steps and ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})3. For real-image style prompts, the style latent is obtained by stochastic encoding and optional AdaIN-based color calibration over a selected timestep window. Quantitatively, StyleKeeper reports Gram-based style similarity of ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})4 versus ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})5 for StyleAligned, ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})6 for IP-Adapter, ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})7 for DB-LoRA, and ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})8 for StyleDrop, and a user-study preference of ϵ^(xt)=(1+w)ϵ¨θ(xt,Qtext,KVvisual)wϵθ(xt,)wϵ¨θ(xt,Qvisual)\hat{\epsilon}(x_t) = (1+w)\,\ddot{\epsilon}_\theta(x_t, Q_{\text{text}}, KV_{\text{visual}}) - w\,\epsilon_\theta(x_t,\emptyset) - w'\,\ddot{\epsilon}_\theta(x_t, Q_{\text{visual}})9 against OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N0, OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N1, OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N2, and OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N3 respectively (Jeong et al., 8 Oct 2025).

For few-step image and video generation, Value Sign Flip (VSF) implements NVQG by flipping the sign of attention values associated with negative tokens while preserving the keys. With text tokens partitioned into positive and negative subsets, the output becomes

OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N4

In MMDiT architectures, the negative prompt is duplicated into OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N5 and OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N6, masks isolate the intended image-to-negative path, and an optional bias OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N7 is added to the logits for OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N8. The design remains single-pass per step, unlike CFG-style two-pass subtraction. On SD-3.5 Large Turbo, the reported scores are Positive OVSF=APVPαANVNO_{\text{VSF}} = A_P V_P - \alpha A_N V_N9, Negative O=OposαNO' = O_{\text{pos}} - \alpha N_\perp0, and Quality O=OposαNO' = O_{\text{pos}} - \alpha N_\perp1, compared with NAG at O=OposαNO' = O_{\text{pos}} - \alpha N_\perp2, NASA at O=OposαNO' = O_{\text{pos}} - \alpha N_\perp3, and no negative guidance at O=OposαNO' = O_{\text{pos}} - \alpha N_\perp4; the paper also notes runtime of O=OposαNO' = O_{\text{pos}} - \alpha N_\perp5s for VSF versus O=OposαNO' = O_{\text{pos}} - \alpha N_\perp6s baseline on A100 at O=OposαNO' = O_{\text{pos}} - \alpha N_\perp7 with 8 steps (Guo et al., 11 Aug 2025).

A third attention-space variant, Orthogonal Negative Guidance, operates in the image-to-text attention outputs of MM-DiT blocks. For each image token and head, it computes positive and negative attention outputs, projects the negative vector onto the positive one, and subtracts only the orthogonal component:

O=OposαNO' = O_{\text{pos}} - \alpha N_\perp8

The image-side features are shared between positive and negative branches for spatial alignment. The reported defaults are O=OposαNO' = O_{\text{pos}} - \alpha N_\perp9, ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})0, ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})1 for FLUX-dev and ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})2, ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})3, ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})4 for FLUX-schnell. On the combined DCS-Bench evaluation, the method reaches ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})5 suppression, Alignment ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})6, and Quality ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})7 on FLUX-dev, and ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})8, ϵ^t=ϵuncond+ωpos(ϵposϵuncond)ωneg(t)(ϵnegϵuncond)\hat{\epsilon}_t = \epsilon_{\text{uncond}} + \omega_{\text{pos}}(\epsilon_{\text{pos}}-\epsilon_{\text{uncond}}) - \omega_{\text{neg}}(t)(\epsilon_{\text{neg}}-\epsilon_{\text{uncond}})9, and ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t0 on FLUX-schnell; human preference is ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t1 versus ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t2 for Qwen-Image-Edit and ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t3 for NAG (Ko et al., 28 May 2026).

4. Dynamic and evidence-calibrated negative prompting

A separate NVQG line keeps the negative signal textual but regenerates or reweights it using intermediate visual evidence. Dynamic VLM-guided negative prompting runs a standard CFG-based latent diffusion model and, at query steps ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t4 out of 50, decodes an intermediate clean image estimate ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t5, queries Qwen2.5-VL-7B-Instruct for a concise negative prompt, encodes the returned text, and uses a three-branch score

ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t6

The reported system uses Stable Diffusion v1.4, DPM-Solver++ with 50 steps, and ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t7. At ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t8, dynamic NVQG attains ASR/TR of ϵtCRG=ϵθ(xt,t,c)λ0wtCRGΔϵt\epsilon_t^{\text{CRG}} = \epsilon_\theta(x_t,t,c) - \lambda_0 w_t^{\text{CRG}} \Delta\epsilon_t9 on Ring-a-Bell-16, ORB×N×DO \in \mathbb{R}^{B \times N \times D}0 on P4D, and ORB×N×DO \in \mathbb{R}^{B \times N \times D}1 on Unlearn-Diff, with CLIP approximately ORB×N×DO \in \mathbb{R}^{B \times N \times D}2 and FID approximately ORB×N×DO \in \mathbb{R}^{B \times N \times D}3. The trade-off is latency: about ORB×N×DO \in \mathbb{R}^{B \times N \times D}4s per image versus about ORB×N×DO \in \mathbb{R}^{B \times N \times D}5s for static negatives and about ORB×N×DO \in \mathbb{R}^{B \times N \times D}6s with no negatives (Chang et al., 30 Oct 2025).

Concept Removal Guidance (CRG) replaces fixed negative weights with an evidence-calibrated projection. It estimates concept presence from the model’s own noise predictions via a step-wise presence signal ORB×N×DO \in \mathbb{R}^{B \times N \times D}7, then solves a minimum-perturbation problem that enforces a threshold ORB×N×DO \in \mathbb{R}^{B \times N \times D}8 on the predicted presence of the unwanted concept. When suppression is triggered, the update is

ORB×N×DO \in \mathbb{R}^{B \times N \times D}9

with τ\tau0. On SD v1.4 with 50-step DDPM, inference time rises from τ\tau1s for the base model to τ\tau2s for CRG. Across six red-teaming suites for nudity removal, the reported Attack Success Rate is as low as τ\tau3–τ\tau4 depending on the suite, while COCO-1k benign fidelity reaches CLIP approximately τ\tau5 and FID approximately τ\tau6 (Choi et al., 29 Jun 2026).

These two approaches share the same principle: the negative guidance strength should depend on what the evolving image already contains, not only on a static negative list supplied before denoising begins.

5. NVQG beyond generation: retrieval, localization, and vision-language alignment

Outside diffusion sampling, NVQG has been used to make negated queries operational in ranking and discrimination. In egocentric Visual Query 2D Localization, the baseline detector exhibited high false positives on background frames because training used only clean, stable, labeled frames while evaluation contained noisy, blurry, or unlabeled background frames. The proposed solution explicitly treated proposals from background frames, cross-video samples, and low-IoU proposals as negatives, applied hard-negative mining, and maintained a positive-negative ratio of τ\tau7. This reduced training from about 15 days to less than 24 hours, improved stAP to τ\tau8, which is τ\tau9 higher than the baseline, and obtained first place on the public leaderboard (Xu et al., 2022).

In CLIP-like retrieval, negation has been reformulated as a geometric intervention problem. A study on negation in joint vision-language embeddings constructs 4,000 matched affirmative–negated COCO caption pairs, shows that logistic regression on EOS hidden states reaches at least Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}0 test accuracy at intermediate layers for ViT-B/32, ViT-B/16, and ViT-L/14, and uses the resulting classifier weights as a negation direction. Test-time steering is applied as

Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}1

With best Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}2, steering reaches Top-1/Avg.5/Top-5 scores of Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}3 on SimpleNeg under Qwen-3-VL-32B judging, surpassing the CLIP baseline at Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}4, and improves Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}5 on the shifted N-COCO benchmark to Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}6 versus Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}7 for CLIP (Sammani et al., 20 Mar 2026).

In medical vision-LLMs, Bi-MCQ reformulates alignment as conditional semantic comparison rather than global InfoNCE-style similarity maximization. It jointly trains Image-to-Text and Text-to-Image multiple-choice tasks over affirmative, negative, and mixed prompts, using direction-specific cross-attention modules and MLP scorers. The inference procedure computes paired scores for “There is [disease].” and “There is no [disease].” and converts them into a Positive–Negative Combined probability via a 2-way softmax. On ChestXray14, Open-I, CheXpert, and PadChest, the reported negation gains reach up to Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}8 AUC over CARZero zero-shot performance, PNC improves by up to Omerge=(1+αt)OsrcαtOtargetO_{\text{merge}} = (1+\alpha_t) O_{\text{src}} - \alpha_t O_{\text{target}}9, and the affirmative-negative AUC gap is reduced by an average of log(M~ref+ϵ)\log(\tilde{M}_{\text{ref}}+\epsilon)0 relative to InfoNCE-based fine-tuning (Kim et al., 30 Jan 2026).

Across these settings, NVQG is not limited to synthesis. It also denotes methods that force the model to adjudicate absence claims, reject confounders, and distinguish semantically plausible but logically incorrect matches.

6. Evaluation regimes, failure modes, and recurrent misconceptions

A central misconception in this literature is that negation can be handled by appending “not,” “no,” or “without” to a prompt. Multiple papers reject that premise in different ways. Visual-reference guidance argues that text-only negative prompts are coarse and often insufficient for complex visual concepts or precise copyrighted elements; few-step guidance argues that conventional negative prompting in distilled models frequently yields mixtures of positive and negative images; negation-aware retrieval argues that cosine ranking does not test logical correctness; and medical alignment work argues that InfoNCE treats negation as a minor linguistic variation rather than a meaning-inverting operator (Singh et al., 2024, Guo et al., 11 Aug 2025, Sammani et al., 20 Mar 2026, Kim et al., 30 Jan 2026).

Evaluation is correspondingly heterogeneous. Generation papers report suppression and fidelity using FID, IS, DreamSim, CLIPScore, VQAScore, Entropy, DINO, Gram-matrix similarity, LPIPS, ASR, and Toxic Rate; retrieval and alignment papers use Recall@K, Top-1/Avg.5/Top-5 under MLLM-as-a-judge, stAP25 and tAP25, and Positive/Negative/PNC AUC. One recurrent methodological point is that retrieval-only metrics can obscure whether negation is actually understood, which motivates semantic-first gating and judge-based verification in the CLIP negation work (Chang et al., 30 Oct 2025, Jeong et al., 8 Oct 2025, Ko et al., 28 May 2026, Xu et al., 2022, Sammani et al., 20 Mar 2026).

Failure modes are also consistent across formulations. Excessive repulsion coefficients or weak similarity thresholds can distort features and degrade fidelity in NegToMe; over-strong negative guidance can introduce artifacts or semantic drift in dynamic VLM prompting; large log(M~ref+ϵ)\log(\tilde{M}_{\text{ref}}+\epsilon)1 can oversuppress style richness in StyleKeeper; high log(M~ref+ϵ)\log(\tilde{M}_{\text{ref}}+\epsilon)2 can remove adjacent or entangled structures in VSF and orthogonal guidance; single-direction steering in CLIP remains limited for double negation, quantifiers, and ambiguous scope; and batch-level candidate construction in Bi-MCQ can be sensitive to prompt design and class imbalance (Singh et al., 2024, Chang et al., 30 Oct 2025, Jeong et al., 8 Oct 2025, Guo et al., 11 Aug 2025, Ko et al., 28 May 2026, Sammani et al., 20 Mar 2026, Kim et al., 30 Jan 2026).

A plausible implication is that NVQG is becoming a general control principle rather than a narrow prompting trick. The surveyed methods place negative information where the model actually resolves visual semantics: token correspondence, attention aggregation, denoising evidence, candidate competition, or embedding geometry. Their differences lie less in objective—explicit concept suppression—and more in where the negative query enters the computation and how aggressively it is allowed to perturb the positive trajectory.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Negative Visual Query Guidance (NVQG).