MCoT-RE: Chain-of-Thought Re-Ranking for CIR

Updated 6 July 2026

The paper introduces a training-free zero-shot framework that preserves reference image context while incorporating modification text for composed image retrieval.
MCoT-RE employs a multi-step reasoning prompt to generate two captions—modification-focused and integration-focused—capturing both change and preservation aspects.
The framework uses a two-stage retrieval and re-ranking process with fused embeddings, achieving up to 8.58% improvement in R@1 on CIR benchmarks.

MCoT-RE, introduced in "MCoT-RE: Multi-Faceted Chain-of-Thought and Re-Ranking for Training-Free Zero-Shot Composed Image Retrieval" (Park et al., 17 Jul 2025), is a training-free zero-shot framework for composed image retrieval (CIR). In CIR, a composed query consists of a reference image $I_R$ and a modification text $T_M$ , and the system must retrieve a target image $I_T$ from a gallery $G=\{I_j\}_{j=1}^N$ such that the requested visual changes are reflected while essential visual attributes of the reference image are preserved. MCoT-RE addresses two limitations identified in prior training-free CIR: early information loss in sequential VLM–LLM pipelines, and context neglect in direct single-pass MLLM captioning. Its central design is to prompt an MLLM to generate two complementary target captions and then use those captions, together with the reference image, in a two-stage retrieval-and-re-ranking pipeline (Park et al., 17 Jul 2025).

1. Task formulation and motivation

Composed image retrieval is defined by a reference image $I_R$ , a modification text $T_M$ , and a gallery $G$ . The retrieval objective is not merely to apply the textually specified change. The target image should both reflect the visual changes described in $T_M$ and preserve essential visual attributes of $I_R$ , including background, style, and non-mentioned attributes. Formally, CIR aims to rank $G$ so that $T_M$ 0 appears at the top (Park et al., 17 Jul 2025).

MCoT-RE is explicitly positioned in the training-free zero-shot regime. “Zero-shot” means that no training or fine-tuning is performed on CIR-specific labeled triplets $T_M$ 1. “Training-free” means that only frozen pre-trained models are used; no additional pre-training, including pseudo-token training or language-only CIR pre-training, is required. This setting is motivated by the expense of supervised CIR triplets and by the desire to deploy on new domains without retraining (Park et al., 17 Jul 2025).

The framework is motivated by two specific shortcomings in earlier training-free approaches. In sequential VLM–LLM pipelines, a VLM first captions the reference image without seeing the modification text, and an LLM then combines that caption with $T_M$ 2. The paper identifies early information loss, limited cross-modal reasoning, and error propagation in this design. In direct MLLM captioning, the model jointly receives $T_M$ 3 and $T_M$ 4, but tends to over-focus on explicit text modifications and underuse contextual visual information such as background, style, layout, and secondary objects. MCoT-RE is designed to resolve both failure modes within a single training-free pipeline (Park et al., 17 Jul 2025).

2. Multi-faceted Chain-of-Thought

The “MCoT” component of MCoT-RE is a structured prompt that guides an MLLM $T_M$ 5 through a four-step reasoning process over the reference image, the modification text, and their interaction. The model outputs two captions in a single MLLM call: $T_M$ 6 Here $T_M$ 7 is the modification-focused caption, and $T_M$ 8 is the integration-focused caption (Park et al., 17 Jul 2025).

The four reasoning steps are explicit. Step 1 is understanding the reference image, including primary objects, attributes, background elements, composition, spatial arrangement, and style. Step 2 is interpreting the modification request by parsing which objects or attributes are changed and what type of change is requested. Step 3 is reasoning about visual changes, including which elements must change, which must remain invariant, and what side effects may be implied. Step 4 is anticipating the target image and generating two captions with distinct roles (Park et al., 17 Jul 2025).

The distinction between the two captions is central. The modification-focused caption explicitly describes changed attributes and newly requested content, and intentionally restricts itself to the change. The integration-focused caption describes the full anticipated target image, combining explicit modifications with preserved visual context from the reference image, including background, style, ambiance, and spatial relationships. The paper’s prompt template asks the MLLM to reason step-by-step, identify changed versus unchanged attributes, and output both captions with those designated roles (Park et al., 17 Jul 2025).

This dual-caption design operationalizes a two-part notion of user intent: what should change and what should remain. The paper’s conceptual discussion states that user intent in CIR contains both “change” and “preserve” aspects. MCoT-RE encodes that distinction directly into the prompt rather than attempting to compress both aspects into a single target caption (Park et al., 17 Jul 2025).

3. Two-stage retrieval and multi-grained re-ranking

After caption generation, MCoT-RE uses a CLIP-like VLM with image encoder $T_M$ 9 and text encoder $I_T$ 0. Retrieval proceeds in two stages. Stage 1 uses only the modification-focused caption: $I_T$ 1 and for each gallery image $I_T$ 2,

$I_T$ 3

The top- $I_T$ 4 images under this similarity define the candidate set $I_T$ 5. The typical values are $I_T$ 6 for FashionIQ and $I_T$ 7 for CIRR (Park et al., 17 Jul 2025).

The rationale for this first stage is explicit: the modification-focused caption is most aligned with the instruction itself, so filtering with $I_T$ 8 ensures that candidate images strongly reflect the requested change even if finer contextual alignment is not yet resolved. In the paper’s terminology, Stage 1 secures “hard alignment with explicit modifications” (Park et al., 17 Jul 2025).

Stage 2 re-ranks the candidate set using multi-grained fusion of three signals: the modification-focused text embedding $I_T$ 9, the integration-focused text embedding $G=\{I_j\}_{j=1}^N$ 0, and the visual reference embedding $G=\{I_j\}_{j=1}^N$ 1. The combined query vector is

$G=\{I_j\}_{j=1}^N$ 2

with $G=\{I_j\}_{j=1}^N$ 3, $G=\{I_j\}_{j=1}^N$ 4, and $G=\{I_j\}_{j=1}^N$ 5 in the reported experiments. Candidate images are then re-ranked by

$G=\{I_j\}_{j=1}^N$ 6

The final ranking is obtained by sorting candidates in descending similarity (Park et al., 17 Jul 2025).

The fused representation is described as multi-grained because it combines a textual change-only grain, a textual full-context grain, and a low-level visual grain. The dominant weight on $G=\{I_j\}_{j=1}^N$ 7 indicates that the integration-focused caption is the primary signal in final ranking, while the modification-focused caption and raw reference embedding remain small but non-zero anchors. The paper reports that removing filtering or re-ranking causes substantial performance drops, and therefore presents both stages as necessary rather than interchangeable (Park et al., 17 Jul 2025).

Representation	Source	Function
$G=\{I_j\}_{j=1}^N$ 8	modification-focused caption	global filtering
$G=\{I_j\}_{j=1}^N$ 9	integration-focused caption	context-sensitive re-ranking
$I_R$ 0	reference image embedding	visual anchoring

4. Pre-trained components, data, and evaluation protocol

MCoT-RE uses only frozen pre-trained components. The MLLM $I_R$ 1 is Gemini 1.5, used via prompt-chaining to generate $I_R$ 2 and $I_R$ 3 at inference time. The VLM backbone is CLIP-based, with ViT-L/14 and ViT-G/14 variants. The image encoder is $I_R$ 4, the text encoder is $I_R$ 5, and all components remain frozen throughout evaluation (Park et al., 17 Jul 2025).

The framework evaluates on two standard CIR benchmarks. FashionIQ is a fashion-domain dataset with three categories—Shirt, Dress, and Toptee—and consists of 46k training images and 15k validation/test images. Because the method is training-free, results are reported on the validation split. CIRR contains more than 36k real-life images and emphasizes natural, complex scenes in which modifications may be subtle and under-specified; evaluation is performed on the official test split (Park et al., 17 Jul 2025).

The reported metrics follow benchmark conventions. On FashionIQ, Recall@10 and Recall@50 are reported per category. On CIRR, the paper reports Recall@K as well as subset recall metrics $I_R$ 6, $I_R$ 7, and $I_R$ 8, which evaluate performance on subsets of visually similar images and therefore probe fine-grained discrimination (Park et al., 17 Jul 2025).

An important implementation property is that hyperparameters are chosen once and reused. The same prompt structure is used across datasets, the fusion weights are fixed at $I_R$ 9 and $T_M$ 0, and the candidate counts are fixed at $T_M$ 1 for FashionIQ and $T_M$ 2 for CIRR. This is why the paper characterizes the framework as plug-and-play on top of general-purpose MLLMs and CLIP models (Park et al., 17 Jul 2025).

5. Empirical performance and qualitative behavior

On FashionIQ with the ViT-L/14 backbone, MCoT-RE improves R@10 over OSrCIR in all three categories: Shirt from 33.17 to 36.60, Dress from 29.70 to 35.94, and Toptee from 36.92 to 43.55. With ViT-G/14, the corresponding gains are from 38.65 to 42.35 on Shirt, from 33.02 to 34.51 on Dress, and from 41.04 to 45.74 on Toptee. The paper summarizes these results as up to 6.24–6.63% improvement in R@10 over previous training-free state of the art on FashionIQ (Park et al., 17 Jul 2025).

On CIRR, with ViT-L/14, MCoT-RE reaches R@1 $T_M$ 3, R@5 $T_M$ 4, R@10 $T_M$ 5, and $T_M$ 6, outperforming training-free baselines such as WeiMoCIR and OSrCIR. With ViT-G/14, it reports R@1 $T_M$ 7, R@5 $T_M$ 8, R@10 $T_M$ 9, and $G$ 0. The headline comparison is up to 8.58% improvement in R@1 over training-free baselines on CIRR (Park et al., 17 Jul 2025).

The paper also notes that MCoT-RE is competitive with zero-shot methods that require additional pre-training steps, including Pic2Word, LinCIR, CompoDiff, and SEARLE, although its principal comparison target is strictly training-free methods (Park et al., 17 Jul 2025).

Qualitative examples clarify the intended behavior. For the query “Mirror the image,” MCoT-RE retrieves a mirrored version of the reference scene while maintaining other contextual elements. For the query “Remove one vending machine and place two blue trash cans next to it,” the modification-focused caption encodes the removal and addition, the integration-focused caption anchors these changes in the original background, and the retrieved image preserves the contextual backdrop while reflecting the requested edit. The paper interprets these examples as evidence that MCoT-RE identifies both what changes and what remains (Park et al., 17 Jul 2025).

The reported failure modes are equally specific. When modifications are underspecified or ambiguous, the MLLM may over-rely on context and make errors. The paper also states that using only the integration caption leads to large performance drops, which is presented as evidence that explicit modification alignment is indispensable (Park et al., 17 Jul 2025).

6. Position within the broader chain-of-thought and re-ranking literature

The acronym “MCoT” is not unique. In the broader literature, "From Perception to Reasoning: Deep Thinking Empowers Multimodal LLMs" (Zhu et al., 17 Nov 2025) uses MCoT to denote Multimodal Chain-of-Thought and treats prompting, search, self-consistency, verification, and re-ranking as major inference-time mechanisms. A different work, "Markov Chain of Thought for Efficient Mathematical Reasoning" (Yang et al., 2024), uses MCoT to denote Markov Chain of Thought. In MCoT-RE, by contrast, the term refers specifically to multi-faceted Chain-of-Thought in which two complementary textual facets are generated for CIR (Park et al., 17 Jul 2025).

Relative to other multi-chain methods, MCoT-RE uses a narrowly targeted form of reasoning diversification. "Answering Questions by Meta-Reasoning over Multiple Chains of Thought" (Yoran et al., 2023) generates multiple chains and then uses a separate meta-reasoner to analyze them jointly. "RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts" (Wu et al., 25 Feb 2025) ranks CoTs in RAG via preference learning over per-document CoTs. MCoT-RE differs in that both captions are generated in a single MLLM call, and re-ranking is performed not by a second LLM judge but by CLIP-style similarity over a fused query representation (Park et al., 17 Jul 2025).

Within image retrieval more specifically, "Chain-of-Thought Re-ranking for Image Retrieval Tasks" (Wu et al., 18 Sep 2025) proposes listwise MLLM re-ranking with query deconstruction and per-candidate evaluation across text-to-image retrieval, CIR, and chat-based image retrieval. MCoT-RE instead uses the MLLM before ranking to synthesize two query-side captions, and then relies on frozen encoders for candidate filtering and re-ranking. This distinction places it closer to query reformulation than to candidate-side listwise judgment (Park et al., 17 Jul 2025).

The primary conceptual contribution of MCoT-RE within this landscape is therefore not generic multi-path search, multi-agent critique, or listwise candidate comparison. It is a task-specific factorization of CIR intent into a modification-focused facet and an integration-focused facet, followed by two-stage retrieval that first enforces directive alignment and then restores contextual fidelity (Park et al., 17 Jul 2025).

7. Limitations, misconceptions, and prospective developments

A common misconception is to treat MCoT-RE as a supervised or lightly adapted CIR model. The paper states the opposite: it is zero-shot, training-free, and uses only frozen pre-trained models. FashionIQ and CIRR are used for evaluation, not for training, and no finetuning or additional pre-training is introduced (Park et al., 17 Jul 2025).

A second misconception is to equate the re-ranking stage with direct MLLM ranking. In MCoT-RE, re-ranking is performed through cosine similarity in a CLIP-like embedding space, using a fused vector constructed from $G$ 1, $G$ 2, and $G$ 3. The MLLM participates in reasoning and caption generation, not in candidate-side listwise ranking (Park et al., 17 Jul 2025).

The paper’s explicit future direction is hierarchical attribute reasoning. The proposed extension decomposes visual attributes into low-level factors such as color and texture, mid-level factors such as pattern and shape, and high-level factors such as style and composition, then designs CoT prompts that reason separately at these levels and integrate them afterward. This would extend the present two-facet formulation into a finer-grained hierarchy (Park et al., 17 Jul 2025).

A broader caution emerges from adjacent reranking research. "Rethinking Reasoning in Document Ranking: Why Chain-of-Thought Falls Short" (Lu et al., 10 Oct 2025) reports that reasoning-augmented document rerankers can underperform direct rerankers because of calibration failures, positive-class bias, and overthinking. This does not contradict the results reported for MCoT-RE, which are positive on CIR benchmarks, but it suggests that explicit reasoning is not universally beneficial for reranking. A plausible implication is that MCoT-RE benefits from assigning free-form reasoning to query construction while keeping final ranking in a frozen embedding space, thereby avoiding some failure modes associated with direct generative rerankers (Lu et al., 10 Oct 2025).

In that sense, MCoT-RE occupies a specific and technically constrained point in the design space. It does not claim that more reasoning is always better, nor that a single MLLM caption can resolve CIR intent. Its contribution is the structured use of two complementary captions and a two-stage retrieval pipeline that balances modification fidelity with preservation of reference-image context (Park et al., 17 Jul 2025).