M-FashionIQ: Multi-Mod CIR Benchmark

• The paper introduces M-FashionIQ, a single-turn CIR benchmark that replaces brief modification texts with detailed multi-modification instructions.
• It leverages a multimodal large language model to generate rich queries that capture multiple entities, clauses, and fine-grained garment attributes.
• Enhanced annotation and quality control lead to improved retrieval metrics, addressing limitations like insufficient entity coverage and clause misalignment.

Searching arXiv for papers directly relevant to M-FashionIQ and adjacent benchmarks. arXiv search: M-FashionIQ composed image retrieval FashionIQ multi-modification M-FashionIQ is an instruction-rich, fashion-domain multi-modification benchmark for composed image retrieval (CIR) introduced in the TEMA framework to make CIR closer to realistic user behavior (Li et al., 23 Apr 2026). It is constructed from the original FashionIQ dataset by keeping the same reference–target image pairs and retrieval task formulation while replacing each short modification sentence with a much richer Multi-Modification Text (MMT) that describes several simultaneous changes to the reference image (Li et al., 23 Apr 2026). Within the broader FashionIQ lineage, M-FashionIQ therefore occupies a specific position: it is a single-turn extension of FashionIQ designed to stress multi-clause, multi-entity, and fine-grained textual modification, rather than a conversational or multi-turn benchmark (Wu et al., 2019).

1. Definition and nomenclature

In TEMA, M-FashionIQ denotes the fashion-domain multi-modification extension of FashionIQ (Li et al., 23 Apr 2026). The underlying CIR triplets are preserved, but the modification text is expanded into MMT so that the query can express multiple modification clauses, multiple affected entities, and richer attribute or relation descriptions. Formally, the paper defines a CIR triplet as

$\mathcal{T}=\{(x_r,t_m,x_t)_n\}_{n=1}^N,$

where $x_r$ is the reference image, $t_m$ is the multi-modification text, and $x_t$ is the target image; the learning objective is to make the composed query embedding $\mathcal{F}(x_r,t_m)$ close to $\mathcal{F}(x_t)$ (Li et al., 23 Apr 2026).

M-FashionIQ is derived from FashionIQ, whose original purpose was interactive fashion image retrieval using a reference product image and natural-language feedback describing how the desired item differs from the current candidate (Wu et al., 2019). FashionIQ provides human-generated relative captions, product descriptions or metadata, and derived visual attribute labels across three categories: Dresses, Shirts, and Tops/Tees (Wu et al., 2019). M-FashionIQ inherits the same image universe and retrieval structure, but changes the textual side of the benchmark.

A recurrent source of confusion is nomenclature. In the TEMA line of work, M-FashionIQ means a multi-modification single-turn CIR benchmark (Li et al., 23 Apr 2026). In contrast, CIRCLED discusses an earlier Multi-turn FashionIQ benchmark, which is a distinct resource for multi-turn composed image retrieval sessions and is criticized for being built by simply concatenating single-turn FashionIQ queries (Takeda et al., 26 May 2026). The two datasets address different failure modes and should not be conflated.

Benchmark	Core formulation	Distinguishing property
FashionIQ	Single-turn CIR	Human-generated relative captions plus metadata and attributes
M-FashionIQ	Single-turn multi-modification CIR	Multi-Modification Text replacing short modification text
Multi-turn FashionIQ	Multi-turn CIR	Sessions built by concatenating single-turn FashionIQ queries

2. Motivation: from simple modification text to instruction-rich queries

The central motivation for M-FashionIQ is the claim that standard FashionIQ-style CIR is too simple for practical use (Li et al., 23 Apr 2026). TEMA identifies two limitations in prevailing setups. The first is Insufficient Entity Coverage: when multiple objects or attributes in the image should be modified, the annotation tends to focus on only the most salient regions and omit other entities, yielding weak supervision for omitted parts. The second is Clause-Entity Misalignment: real retrieval requests may contain multiple clauses constraining the same entity, or one clause affecting multiple entities, while the original annotations do not express these fine-grained one-to-many relationships (Li et al., 23 Apr 2026).

FashionIQ itself already showed that relative natural-language feedback is richer than fixed attribute vocabularies. Its caption analysis reported that about 69.5% of sampled captions are composite attribute phrases and 30.5% are single-attribute descriptions, with additional use of negation and comparative language; the semantic breakdown includes 49% direct reference, 32% comparison, and 19% direct comparison (Wu et al., 2019). M-FashionIQ extends that logic further: instead of a short cue, the query becomes an instruction-intensive text intended to articulate several simultaneous edits (Li et al., 23 Apr 2026).

The paper characterizes the new benchmark as “instruction-rich” and explicitly states that the goal is not simply to improve benchmark scores, but to create a setting closer to real-world retrieval scenarios (Li et al., 23 Apr 2026). This suggests that M-FashionIQ should be understood less as a cosmetic reannotation and more as a stress test for compositional language grounding in CIR.

3. Construction pipeline and annotation control

M-FashionIQ is created by starting from FashionIQ triplets and using a Multimodal LLM, specifically Llama 3.2 in the main description, to generate candidate MMTs from the reference–target image pair while conditioning on the original short modification text (Li et al., 23 Apr 2026). The prompts are designed to remain faithful to the original modification intent while expanding it into a more detailed instruction-like description. For FashionIQ, prompt design is tailored to clothing-specific properties such as shape and color, because desired modifications often involve garment components including straps, sleeves, neckline, skirt, or belt (Li et al., 23 Apr 2026).

After MLLM generation, the pipeline applies a hallucination check using GPT-4o to remove obvious invented content (Li et al., 23 Apr 2026). This is followed by a hybrid quality check combining human and machine effort. Ten research assistants review and revise the MMTs without looking at the images first, checking Consistency, Accuracy, Diversity, and Quality (Li et al., 23 Apr 2026). A final Content Filter removes text that describes the target image in isolation rather than describing a change relative to the reference image, because such text would violate the referential nature of CIR (Li et al., 23 Apr 2026).

The construction logic is significant because M-FashionIQ does not alter the original image pairs or evaluation protocol; it expands only the textual query (Li et al., 23 Apr 2026). A practical implication is direct comparability with FashionIQ at the image level, while substantially increasing the linguistic burden placed on the retrieval model.

4. Data characteristics and benchmark scale

FashionIQ contains 77,684 images across three categories—Dresses, Shirts, and TopsTees in the TEMA description—and TEMA treats these as three independent datasets following the original protocol (Li et al., 23 Apr 2026). In the appendix, the paper reports processing a total of 24,016 queries in FashionIQ to build M-FashionIQ (Li et al., 23 Apr 2026). The resulting multi-modification split includes approximately 18K triplets for training and about 6K triplets for testing, with roughly 46K images for training and 15K for testing (Li et al., 23 Apr 2026).

The textual expansion is substantial. The original FashionIQ modification text is reported to have a minimum length of 3 tokens, a maximum of 37, and an average of 24.7, whereas M-FashionIQ has a minimum of 25 tokens, a maximum of 327, and an average of 152.7 (Li et al., 23 Apr 2026). The paper further states that M-FashionIQ contains all three modification attributes of interest—CR (Complex Relations), ME (Multiple Entities), and FG (Fine-Grained)—whereas the original FashionIQ does not exhibit them in the same comprehensive way (Li et al., 23 Apr 2026).

A notable claim is that M-FashionIQ reduces false negatives (Li et al., 23 Apr 2026). The argument is that original CIR annotations can miss valid target-like images because short text fails to express all relevant changes; after expansion into MMT, more visually similar candidates are correctly recognized as satisfying the query. This connects to a broader issue later emphasized by QuRe: standard CIR training on FashionIQ often treats the annotated target as the only positive and all other in-batch images as negatives, even though some of those negatives may be genuinely relevant (Kwak et al., 16 Jul 2025). M-FashionIQ addresses this at the annotation level by making more of the intended constraints explicit.

5. Role in evaluation and reported retrieval performance

M-FashionIQ is one of the two main benchmarks used to evaluate TEMA under the multi-modification CIR setting, together with M-CIRR (Li et al., 23 Apr 2026). The evaluation follows the FashionIQ spirit and uses recall at rank $K$ (R@K) as the primary metric, with R@10, R@50, and category-wise averages across Dresses, Shirts, and TopsTees (Li et al., 23 Apr 2026). The comparison includes traditional model-based methods such as TIRG, CLVC-Net, FashionViL, and MGUR, as well as VLP-based methods including FashionSAP, BLIP4CIR, BLIP4CIR+Bi, and Candidate (Li et al., 23 Apr 2026).

The paper reports that BLIP is used as the backbone for several comparisons because CLIP’s text encoder has token-length limits that make it unsuitable for the long MMTs (Li et al., 23 Apr 2026). This is a technically important benchmark property: the query length distribution is not merely longer than FashionIQ’s, but long enough to affect backbone selection and model feasibility.

On M-FashionIQ, TEMA achieves 45.74 R@10 / 69.48 R@50 on Dresses, 50.35 / 71.26 on Shirts, and 55.67 / 75.52 on TopsTees, with an overall average of 50.59 R@10 / 72.09 R@50 (Li et al., 23 Apr 2026). The strongest prior baseline in the table, Candidate, reaches 47.38 / 66.71 on the averaged M-FashionIQ metrics, so TEMA improves over it by roughly +3.21 R@10 and +5.38 R@50 on the average (Li et al., 23 Apr 2026).

The ablation study links benchmark difficulty to model design. Removing the PA module reduces the average M-FashionIQ score to 47.80 / 69.83, while removing the EM module reduces it to 45.41 R@10 / 68.18 R@50 (Li et al., 23 Apr 2026). The paper interprets this as evidence that M-FashionIQ genuinely requires explicit entity coverage and clause aggregation, rather than merely tolerating longer descriptions.

6. Relationship to adjacent benchmarks and open issues

M-FashionIQ belongs to a broader trajectory of FashionIQ-derived benchmarks, but it addresses a different problem from multi-turn datasets. CIRCLED describes the earlier Multi-turn FashionIQ benchmark as the only publicly available multi-turn CIR dataset before CIRCLED and criticizes it for being built by simply concatenating single-turn FashionIQ queries, without guarantees that intermediate turns steadily move toward the final ground-truth image (Takeda et al., 26 May 2026). CIRCLED then introduces consistency and diversity criteria for multi-turn sessions, including $\varepsilon$-consistency and $\tau$-diversity, to remedy those weaknesses (Takeda et al., 26 May 2026). By contrast, M-FashionIQ does not attempt to model dialogue history; it intensifies the complexity of a single CIR turn (Li et al., 23 Apr 2026).

This distinction addresses a common misconception. M-FashionIQ is not a conversational benchmark, and the TEMA paper explicitly acknowledges that the resulting benchmark is still single-turn, while interactive multi-turn retrieval remains an open problem (Li et al., 23 Apr 2026). Conversely, Multi-turn FashionIQ and CIRCLED are concerned with how retrieval should evolve over turns, not with expanding each turn into a long instruction (Takeda et al., 26 May 2026).

Another important contextual issue is evaluation. QuRe argues that standard FashionIQ evaluation is target-centric and may not reflect user satisfaction because multiple images can satisfy the query even when only one is labeled as the target (Kwak et al., 16 Jul 2025). M-FashionIQ’s richer descriptions plausibly mitigate part of that mismatch by making more constraints explicit, but the benchmark remains within the conventional CIR paradigm of retrieving the target image from a reference image plus modification text (Li et al., 23 Apr 2026). A plausible implication is that M-FashionIQ complements, rather than replaces, efforts aimed at human-preference alignment and relevance-sensitive training.

Overall, M-FashionIQ is best understood as a FashionIQ-derived benchmark that operationalizes a specific claim about CIR realism: retrieval systems should be evaluated not only on simple relative captions, but also on long, instruction-rich multi-modification text that covers multiple entities, multiple clauses, and fine-grained garment changes (Li et al., 23 Apr 2026). Its principal contribution is therefore methodological as much as empirical: it turns linguistic under-specification in FashionIQ-style CIR into an explicit benchmark variable, while remaining compatible with the original FashionIQ retrieval paradigm (Wu et al., 2019).