--- title: Soft Query Refinement (SQR) url: https://www.emergentmind.com/topics/soft-query-refinement-sqr type: topic --- # Soft Query Refinement (SQR) Soft Query Refinement (SQR) is a query-refinement mechanism introduced in the agentic video retrieval framework "VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinement" [2607.00446]. In that setting, SQR refines search query tokens in a continuous latent space rather than rewriting queries in discrete text space, so that a system can iteratively retrieve videos from a large corpus, verify whether the retrieved video matches the query, and, once a match is found, perform query-conditioned temporal grounding within the video. SQR is therefore not a generic synonym for query rewriting; in its canonical formulation, it is a latent-space refinement procedure coupled to a dense retriever and trained by retrieval and downstream task signals. ## 1. Problem setting and motivation VideoSearch-R1 studies **Video Corpus Moment Retrieval (VCMR)**, which decomposes into two linked stages: **inter-video reasoning / large-scale video corpus retrieval (VR)** and **intra-video reasoning / temporal grounding** [2607.00446]. Given a textual query \(q\) and a large corpus \(\mathcal{V}\), the system must first retrieve the matching video \(v \in \mathcal{V}\), and then predict the start and end timestamps \((s,e)\) of the query-relevant moment in that video. Because VCMR requires success in both stages, a retrieval error makes temporal grounding automatically wrong. The paper’s central motivation is that most prior pipelines treat retrieval as a fixed preprocessing step. A typical pipeline uses a video retrieval model to produce a top-\(K\) candidate list and then passes one or a few top videos to a grounding or captioning model operating only inside the selected video. This produces two failure modes: retrieval errors cannot be corrected by downstream reasoning, and query refinement—when present—is usually done by verbose discrete text rewriting that is not directly optimized for the retriever’s embedding space [2607.00446]. SQR is the mechanism that closes this loop. At iteration \(t\), the system holds a query representation \(q_t\), retrieves a top-1 video \(v_t\), verifies whether the result is a `match` or `not match`, and, if necessary, refines the query before the next retrieval round. This converts retrieval from a one-shot preprocessing stage into a multi-turn, self-correcting procedure. The paper explicitly frames this as iterative video retrieval and reasoning rather than a retrieval stage followed by a separate intra-video model. ## 2. Formal definition of SQR In VideoSearch-R1, the dense video search engine uses an embedding function \(f\), and retrieval at turn \(t\) is defined by $$ v_t = \mathcal{R}(q_t) = \arg\max_{v \in \mathcal{V}} f(q_t)^\top f(v). $$ If the controller predicts `not match`, it performs SQR by autoregressively generating a fixed number of latent tokens $$ q_t^{\text{soft}} \in \mathbb{R}^{N \times D}, $$ where \(N\) is the number of soft query tokens and \(D\) is the hidden size of the LLM [2607.00446]. These are not vocabulary tokens. Instead, “the hidden state corresponding to the previously generated token is projected through a linear layer and used directly as the input embedding for the next token,” so the soft-token generation step is $$ h_n = \mathrm{Transformer}(h_{` token triggers latent-token generation, and a linear projection maps decoder hidden states to soft query tokens. Each turn follows a rigid template. The controller receives the system prompt, the user query text, and retrieved video frames. If the video matches, it outputs `...`, `matched`, timestamps in `` and ``, and then ``. If the video does not match, it outputs `...`, `not matched`, and ``. After ``, the model switches from ordinary text decoding to latent-token generation. Those soft tokens are stored as embeddings and passed to the retriever as part of the next query sequence. The multi-turn loop continues until a `match` is found and temporal grounding is produced or until the turn limit \(T\) is reached. In the main experiments, \(T=2\), while the analysis reports that gains saturate around \(T=3\) [2607.00446]. The paper therefore defines SQR not as an isolated embedding trick, but as one operator in a broader verification–refinement–retrieval–grounding loop. ## 4. Supervision, rewards, and policy optimization Training proceeds in two stages: supervised fine-tuning (SFT) and reinforcement learning with **Group Relative Policy Optimization (GRPO)** [2607.00446]. In SFT, the losses are $$ \mathcal{L}_{\text{verif}} = -\log P(r, y^{\text{ret}} \mid q, v), \qquad \mathcal{L}_{\text{time}} = -\log P(y^{\text{time}} \mid q,v,r,y^{\text{ret}}), $$ and the total SFT objective is $$ \mathcal{L}_{\text{SFT}} = \mathcal{L}_{\text{verif}} + \mathcal{L}_{\text{ret}} + \mathbbm{1}_{y^{\text{ret}}=\texttt{`match'}}(\mathcal{L}_{\text{time}}). $$ The implementation uses 2K steps per dataset, 2K samples with 1K positive and 1K negative pairs for CoT, and AdamW with learning rate \(2 \times 10^{-5}\). The retriever remains frozen. Visual input uses up to 64 frames sampled at 1 FPS, for 4096 visual tokens. The number of soft tokens is fixed at \(N=8\). After SFT, GRPO optimizes the full reasoning policy. The paper defines four reward components: a format reward, a verification reward, a retrieval reward, and a temporal grounding reward. The retrieval reward is tied directly to the contrastive SQR loss, $$ R^{\text{ret}} = \exp(-\mathcal{L}_{\text{ret}}), $$ and the temporal grounding reward is the temporal IoU between predicted and ground-truth spans. The total sample reward is $$ R_i = R_i^{\text{format}} + R_i^{\text{verif}} + R_i^{\text{ret}} + \mathbbm{1}_{y^{\text{ret}}=\texttt{`match'}}(R_i^{\text{time}}). $$ GRPO then uses group-wise normalized advantages, $$ A_i = \frac{R_i - \mathrm{mean}(\{R_j\})}{\mathrm{std}(\{R_j\})}, $$ with rollout group size \(G=8\). The RL stage uses AdamW with learning rate \(5 \times 10^{-7}\), weight decay \(0.01\), max grad norm \(1.0\), decoding temperature \(1.0\), and KL coefficient \(\beta = 0.01\). A central property of this design is that SQR is trained by trajectory-level signals derived from retrieval and downstream grounding performance, not only by language modeling. ## 5. Empirical behavior and ablation evidence The evaluation uses the VERIFIED-VCMR benchmarks **ActivityNet-FIG**, **Charades-FIG**, and **DiDeMo-FIG**, with VR metrics \(R@1\), \(R@5\), \(R@10\), \(R@100\), VCMR metrics IoU/R@1 at thresholds \(0.3\), \(0.5\), and \(0.7\), and verification accuracy (VER) [2607.00446]. The paper reports state-of-the-art performance across the three datasets. The clearest direct comparison between SQR and text-level refinement is on ActivityNet-FIG. Relative to the zero-shot Qwen3-VL-2B baseline, both HQR and SQR improve end-to-end performance, but SQR is stronger on retrieval and uses fewer refinement tokens. For VCMR, SQR reaches 33.8 at 0.3/R@1, 22.3 at 0.5/R@1, and 12.3 at 0.7/R@1. For VR, it reaches 61.1 at R@1, 81.7 at R@5, 88.5 at R@10, and 98.4 at R@100. Verification accuracy is 83.3. The average number of generated refinement tokens per episode is 8.0 for SQR, versus 26.8 for HQR [2607.00446]. The paper’s interpretation is that HQR’s verbose rewritten queries can introduce semantic noise, whereas SQR applies minimal targeted corrections directly in the embedding space. The ablations clarify the role of optimization. On DiDeMo-FIG, the zero-shot baseline obtains VCMR 22.0 / 10.6 / 4.0 at IoU thresholds 0.3 / 0.5 / 0.7, VER 62.8, and VR R@1 54.8. Stage 1 only, with SFT, improves VR R@1 to 57.4 and VER to 66.0, but VCMR 0.3/R@1 falls to 20.4. Stage 1 plus Stage 2, with SFT + RL, raises VCMR to 33.3 / 30.2 / 19.7, VER to 74.6, and VR R@1 to 59.0 [2607.00446]. This shows that supervised InfoNCE and verification training are insufficient by themselves to align retrieval and temporal grounding; RL is necessary for end-task optimization. Reward ablations support the same point. On DiDeMo-FIG, RL with only \(R^{\text{ret}}\) raises VR R@1 from 57.4 to 58.0 but reduces VCMR 0.3/R@1 from 20.4 to 18.6. Adding \(R^{\text{verif}}\) raises VER to 75.0 and VR R@1 to 59.7, but VCMR remains 19.3 at 0.3/R@1. Only the full reward \(R^{\text{ret}} + R^{\text{verif}} + R^{\text{time}}\) yields the reported 33.3 / 30.2 / 19.7 VCMR performance [2607.00446]. The paper therefore treats temporal grounding reward as essential rather than auxiliary. The analysis of capacity and iteration is consistent with the formulation. As the number of soft tokens \(N\) increases, R@1 over refined samples monotonically increases, and gains from additional turns improve from \(T=1\) to \(T=2\) before saturating around \(T=3\). A qualitative example involving a woman brushing hair reports that progressively adding 1–8 soft tokens improves the rank of the ground-truth video until it reaches rank 1, with the refinement encoding increasingly specific attributes such as “blonde hair,” “being combed by a person,” and “light blue wall” [2607.00446]. ## 6. Related formulations, misconceptions, and limitations A common misconception is to equate SQR with any form of query rewriting. In VideoSearch-R1, “soft” refers specifically to refinement in the retriever’s continuous embedding space rather than rewriting in discrete text space [2607.00446]. Related work shows that the broader idea of soft refinement appears in several other forms, but those mechanisms are not identical. In database systems, "Data-Aware Socratic Guidance" embeds interactive clarification as a first-class operator and asks one targeted clarification only when the expected value of clarification exceeds the cost of dialogue [2508.05061]. In knowledge-graph query answering, NQR applies soft entity constraints by additively reranking answer scores from incremental positive and negative preference examples while regularizing against the base score distribution with a KL term [2508.13663]. In video moment retrieval within a single given video, MRNet’s query refinement module builds word-, phrase-, and sentence-wise features through Conv1D, FFN, and pooling rather than corpus-level retrieval refinement [2501.10692]. RL-driven prompt refinement for general LLM inputs operates in discrete text space with PPO and a joint quality-safety reward rather than in retriever embedding space [2407.01461]. Search agents such as SmartSearch selectively refine low-quality intermediate queries using process rewards and regeneration of downstream reasoning, again through explicit text queries rather than latent token concatenation [2601.04888]. The limitations of SQR in VideoSearch-R1 are largely those of a latent retriever-coupled refinement policy. The mechanism depends on the base retriever’s embedding space; if the video search engine is weak or biased, SQR can only partially compensate [2607.00446]. The system can still fail when the initial retrieval is severely off-target, because verification may misjudge the result or the refinement may move in the wrong direction. Reward ablations also expose a trade-off between retrieval and temporal localization: adding \(R^{\text{time}}\) improves VCMR but can slightly reduce VR or VER stability. Another limitation is interpretability: unlike HQR, soft tokens are not human-readable. Finally, the evaluation focuses on short multi-turn interaction, with main experiments at \(T=2\) and saturation around \(T=3\), so longer refinement chains remain underexplored. The acronym itself is also overloaded. In a distinct line of work on online experimentation, "SQR" denotes "Speed, Quality, Risk" rather than Soft Query Refinement [1801.08532]. Within retrieval and reasoning, however, the term most precisely denotes the continuous latent-space query refinement mechanism introduced in VideoSearch-R1 [2607.00446]. Source: https://www.emergentmind.com/topics/soft-query-refinement-sqr