---
title: 'M-FashionIQ: Multi-Mod CIR Benchmark'
url: https://www.emergentmind.com/topics/m-fashioniq
type: topic
---

# M-FashionIQ: Multi-Mod CIR Benchmark

Searching arXiv for recent papers directly relevant to M-FashionIQ and adjacent benchmarks.
arXiv search: M-FashionIQ composed image retrieval FashionIQ multi-modification
M-FashionIQ is an instruction-rich, fashion-domain multi-modification benchmark for composed image retrieval (CIR) introduced in the TEMA framework to make CIR closer to realistic user behavior [2604.21806]. It is constructed from the original FashionIQ dataset by keeping the same reference–target image pairs and retrieval task formulation while replacing each short modification sentence with a much richer Multi-Modification Text (MMT) that describes several simultaneous changes to the reference image [2604.21806]. Within the broader FashionIQ lineage, M-FashionIQ therefore occupies a specific position: it is a single-turn extension of FashionIQ designed to stress multi-clause, multi-entity, and fine-grained textual modification, rather than a conversational or multi-turn benchmark [1905.12794].

## 1. Definition and nomenclature

In TEMA, M-FashionIQ denotes the fashion-domain multi-modification extension of FashionIQ [2604.21806]. The underlying CIR triplets are preserved, but the modification text is expanded into MMT so that the query can express multiple modification clauses, multiple affected entities, and richer attribute or relation descriptions. Formally, the paper defines a CIR triplet as
$$
\mathcal{T}=\{(x_r,t_m,x_t)_n\}_{n=1}^N,
$$
where $x_r$ is the reference image, $t_m$ is the multi-modification text, and $x_t$ is the target image; the learning objective is to make the composed query embedding $\mathcal{F}(x_r,t_m)$ close to $\mathcal{F}(x_t)$ [2604.21806].

M-FashionIQ is derived from FashionIQ, whose original purpose was interactive fashion image retrieval using a reference product image and natural-language feedback describing how the desired item differs from the current candidate [1905.12794]. FashionIQ provides human-generated relative captions, product descriptions or metadata, and derived visual attribute labels across three categories: Dresses, Shirts, and Tops/Tees [1905.12794]. M-FashionIQ inherits the same image universe and retrieval structure, but changes the textual side of the benchmark.

A recurrent source of confusion is nomenclature. In the TEMA line of work, M-FashionIQ means a multi-modification single-turn CIR benchmark [2604.21806]. In contrast, CIRCLED discusses an earlier Multi-turn FashionIQ benchmark, which is a distinct resource for multi-turn composed image retrieval sessions and is criticized for being built by simply concatenating single-turn FashionIQ queries [2605.26734]. The two datasets address different failure modes and should not be conflated.

| Benchmark | Core formulation | Distinguishing property |
|---|---|---|
| FashionIQ | Single-turn CIR | Human-generated relative captions plus metadata and attributes |
| M-FashionIQ | Single-turn multi-modification CIR | Multi-Modification Text replacing short modification text |
| Multi-turn FashionIQ | Multi-turn CIR | Sessions built by concatenating single-turn FashionIQ queries |

## 2. Motivation: from simple modification text to instruction-rich queries

The central motivation for M-FashionIQ is the claim that standard FashionIQ-style CIR is too simple for practical use [2604.21806]. TEMA identifies two limitations in prevailing setups. The first is **Insufficient Entity Coverage**: when multiple objects or attributes in the image should be modified, the annotation tends to focus on only the most salient regions and omit other entities, yielding weak supervision for omitted parts. The second is **Clause-Entity Misalignment**: real retrieval requests may contain multiple clauses constraining the same entity, or one clause affecting multiple entities, while the original annotations do not express these fine-grained one-to-many relationships [2604.21806].

FashionIQ itself already showed that relative natural-language feedback is richer than fixed attribute vocabularies. Its caption analysis reported that about **69.5%** of sampled captions are composite attribute phrases and **30.5%** are single-attribute descriptions, with additional use of negation and comparative language; the semantic breakdown includes **49% direct reference**, **32% comparison**, and **19% direct comparison** [1905.12794]. M-FashionIQ extends that logic further: instead of a short cue, the query becomes an instruction-intensive text intended to articulate several simultaneous edits [2604.21806].

The paper characterizes the new benchmark as “instruction-rich” and explicitly states that the goal is not simply to improve benchmark scores, but to create a setting closer to real-world retrieval scenarios [2604.21806]. This suggests that M-FashionIQ should be understood less as a cosmetic reannotation and more as a stress test for compositional language grounding in CIR.

## 3. Construction pipeline and annotation control

M-FashionIQ is created by starting from FashionIQ triplets and using a Multimodal Large Language Model, specifically **Llama 3.2** in the main description, to generate candidate MMTs from the reference–target image pair while conditioning on the original short modification text [2604.21806]. The prompts are designed to remain faithful to the original modification intent while expanding it into a more detailed instruction-like description. For FashionIQ, prompt design is tailored to clothing-specific properties such as shape and color, because desired modifications often involve garment components including straps, sleeves, neckline, skirt, or belt [2604.21806].

After MLLM generation, the pipeline applies a **hallucination check** using **GPT-4o** to remove obvious invented content [2604.21806]. This is followed by a **hybrid quality check** combining human and machine effort. Ten research assistants review and revise the MMTs without looking at the images first, checking **Consistency, Accuracy, Diversity, and Quality** [2604.21806]. A final **Content Filter** removes text that describes the target image in isolation rather than describing a change relative to the reference image, because such text would violate the referential nature of CIR [2604.21806].

The construction logic is significant because M-FashionIQ does not alter the original image pairs or evaluation protocol; it expands only the textual query [2604.21806]. A practical implication is direct comparability with FashionIQ at the image level, while substantially increasing the linguistic burden placed on the retrieval model.

## 4. Data characteristics and benchmark scale

FashionIQ contains **77,684 images** across three categories—**Dresses, Shirts, and TopsTees** in the TEMA description—and TEMA treats these as three independent datasets following the original protocol [2604.21806]. In the appendix, the paper reports processing a total of **24,016 queries** in FashionIQ to build M-FashionIQ [2604.21806]. The resulting multi-modification split includes approximately **18K triplets for training and about 6K triplets for testing**, with roughly **46K images for training and 15K for testing** [2604.21806].

The textual expansion is substantial. The original FashionIQ modification text is reported to have a minimum length of **3 tokens**, a maximum of **37**, and an average of **24.7**, whereas M-FashionIQ has a minimum of **25 tokens**, a maximum of **327**, and an average of **152.7** [2604.21806]. The paper further states that M-FashionIQ contains all three modification attributes of interest—**CR (Complex Relations), ME (Multiple Entities), and FG (Fine-Grained)**—whereas the original FashionIQ does not exhibit them in the same comprehensive way [2604.21806].

A notable claim is that M-FashionIQ reduces **false negatives** [2604.21806]. The argument is that original CIR annotations can miss valid target-like images because short text fails to express all relevant changes; after expansion into MMT, more visually similar candidates are correctly recognized as satisfying the query. This connects to a broader issue later emphasized by QuRe: standard CIR training on FashionIQ often treats the annotated target as the only positive and all other in-batch images as negatives, even though some of those negatives may be genuinely relevant [2507.12416]. M-FashionIQ addresses this at the annotation level by making more of the intended constraints explicit.

## 5. Role in evaluation and reported retrieval performance

M-FashionIQ is one of the two main benchmarks used to evaluate TEMA under the multi-modification CIR setting, together with M-CIRR [2604.21806]. The evaluation follows the FashionIQ spirit and uses **recall at rank $K$ (R@K)** as the primary metric, with **R@10**, **R@50**, and category-wise averages across Dresses, Shirts, and TopsTees [2604.21806]. The comparison includes traditional model-based methods such as **TIRG**, **CLVC-Net**, **FashionViL**, and **MGUR**, as well as VLP-based methods including **FashionSAP**, **BLIP4CIR**, **BLIP4CIR+Bi**, and **Candidate** [2604.21806].

The paper reports that BLIP is used as the backbone for several comparisons because CLIP’s text encoder has token-length limits that make it unsuitable for the long MMTs [2604.21806]. This is a technically important benchmark property: the query length distribution is not merely longer than FashionIQ’s, but long enough to affect backbone selection and model feasibility.

On M-FashionIQ, TEMA achieves **45.74 R@10 / 69.48 R@50** on Dresses, **50.35 / 71.26** on Shirts, and **55.67 / 75.52** on TopsTees, with an overall average of **50.59 R@10 / 72.09 R@50** [2604.21806]. The strongest prior baseline in the table, **Candidate**, reaches **47.38 / 66.71** on the averaged M-FashionIQ metrics, so TEMA improves over it by roughly **+3.21 R@10** and **+5.38 R@50** on the average [2604.21806].

The ablation study links benchmark difficulty to model design. Removing the **PA** module reduces the average M-FashionIQ score to **47.80 / 69.83**, while removing the **EM** module reduces it to **45.41 R@10 / 68.18 R@50** [2604.21806]. The paper interprets this as evidence that M-FashionIQ genuinely requires explicit entity coverage and clause aggregation, rather than merely tolerating longer descriptions.

## 6. Relationship to adjacent benchmarks and open issues

M-FashionIQ belongs to a broader trajectory of FashionIQ-derived benchmarks, but it addresses a different problem from multi-turn datasets. CIRCLED describes the earlier Multi-turn FashionIQ benchmark as the only publicly available multi-turn CIR dataset before CIRCLED and criticizes it for being built by simply concatenating single-turn FashionIQ queries, without guarantees that intermediate turns steadily move toward the final ground-truth image [2605.26734]. CIRCLED then introduces consistency and diversity criteria for multi-turn sessions, including $\varepsilon$-consistency and $\tau$-diversity, to remedy those weaknesses [2605.26734]. By contrast, M-FashionIQ does not attempt to model dialogue history; it intensifies the complexity of a single CIR turn [2604.21806].

This distinction addresses a common misconception. M-FashionIQ is not a conversational benchmark, and the TEMA paper explicitly acknowledges that the resulting benchmark is still **single-turn**, while interactive multi-turn retrieval remains an open problem [2604.21806]. Conversely, Multi-turn FashionIQ and CIRCLED are concerned with how retrieval should evolve over turns, not with expanding each turn into a long instruction [2605.26734].

Another important contextual issue is evaluation. QuRe argues that standard FashionIQ evaluation is target-centric and may not reflect user satisfaction because multiple images can satisfy the query even when only one is labeled as the target [2507.12416]. M-FashionIQ’s richer descriptions plausibly mitigate part of that mismatch by making more constraints explicit, but the benchmark remains within the conventional CIR paradigm of retrieving the target image from a reference image plus modification text [2604.21806]. A plausible implication is that M-FashionIQ complements, rather than replaces, efforts aimed at human-preference alignment and relevance-sensitive training.

Overall, M-FashionIQ is best understood as a FashionIQ-derived benchmark that operationalizes a specific claim about CIR realism: retrieval systems should be evaluated not only on simple relative captions, but also on long, instruction-rich multi-modification text that covers multiple entities, multiple clauses, and fine-grained garment changes [2604.21806]. Its principal contribution is therefore methodological as much as empirical: it turns linguistic under-specification in FashionIQ-style CIR into an explicit benchmark variable, while remaining compatible with the original FashionIQ retrieval paradigm [1905.12794].

Source: https://www.emergentmind.com/topics/m-fashioniq