Synonym-Enriched Visual Corpus (SEViC)

• SEViC is a visual-text corpus that augments image annotations with multiple synonyms and definitions per category to improve dense visual grounding.
• It employs a three-stage process using LLMs and CLIP-based ranking to enrich canonical labels from COCO2017 and LVIS with lexical variations.
• Empirical results show that integrating synonyms and definitions yields up to a +2.6 point improvement in novel-class AP, reducing grounding inconsistency.

Searching arXiv for the cited papers and related context. The Synonym-Enriched Visual Corpus (SEViC) denotes a visual-text corpus construction paradigm in which image annotations are augmented with semantically equivalent expressions and textual definitions, and, in the specific formulation introduced with SynCLIP, a concrete corpus built on COCO2017 train and the LVIS vocabulary to support synonym-coherent language-image pretraining for open-vocabulary dense perception (Xie et al., 13 Jul 2026). In that instantiation, SEViC extends category supervision beyond canonical labels by attaching multiple synonyms and at least one definition to each category, thereby targeting synonym-induced grounding inconsistency, namely the phenomenon that semantically equivalent expressions can produce different spatial attention patterns in CLIP-based open-vocabulary dense perception systems (Xie et al., 13 Jul 2026). A distinct earlier use of the name appears in a construction guide adapted from automatic visual concept discovery, where SEViC is framed as a corpus assembled from parallel text and visual corpora by filtering visually discriminative terms and clustering them into visual concepts (Sun et al., 2015). Taken together, these two usages situate SEViC at the intersection of multimodal representation learning, lexical variation modeling, and visually grounded concept induction.

1. Definition and scope

In the SynCLIP formulation, SEViC is built on the COCO2017 train split, comprising 118,287 images, together with the LVIS vocabulary (Xie et al., 13 Jul 2026). COCO contributes 80 coarse categories with extensive “in-context” images, while LVIS contributes 1,152 finer-grained, long-tailed categories defined over the same images; jointly these produce 1,232 distinct object categories, and two meta-categories, “object” and “background,” are further inserted to obtain a total of 1,234 categories (Xie et al., 13 Jul 2026). Each image remains annotated exactly as in the original COCO/LVIS releases, but the category schema is enriched by textual metadata consisting of a definition string and an array of synonyms for each category (Xie et al., 13 Jul 2026).

In this sense, SEViC is not merely a synonym lexicon. It is a corpus-level alignment resource in which image-level category annotations are paired with lexical variants and category definitions so that dense visual grounding can be trained against multiple surface forms rather than a single canonical noun (Xie et al., 13 Jul 2026). This suggests that the central design objective is invariance to linguistic paraphrase at the level of spatial attention, not only improved text classification or retrieval.

A second, earlier construction scheme derives from “Automatic Concept Discovery from Parallel Text and Visual Corpora” (Sun et al., 2015). There, a SEViC-like resource is assembled from image-caption pairs by mining candidate terms, filtering them according to visual discriminative power, grouping them via visual and semantic similarity, and storing clustered “visual concepts” with pooled images and classifier metadata (Sun et al., 2015). In that usage, synonym enrichment emerges from concept grouping rather than from ontology-driven category augmentation. The two formulations are methodologically different, but both treat lexical multiplicity as a resource for better visual grounding.

2. Corpus construction in SynCLIP

The SynCLIP version of SEViC follows a three-stage textual augmentation procedure (Xie et al., 13 Jul 2026). First, category names are collected together with any human-written synonyms or definitions already provided by LVIS. Second, a LLM, specifically DeepSeek, is prompted to generate for each category a concise definition when LVIS lacks one and a list of up to 10 semantically equivalent synonyms or noun phrases (Xie et al., 13 Jul 2026). Third, ChatGPT is used for consistency validation to filter out generated synonyms that are ambiguous, polysemous in other contexts, or inconsistent with the intended visual class (Xie et al., 13 Jul 2026).

The reported corpus statistics are specific. SEViC contains 1,234 categories, 11,558 total enriched expressions, and 118,287 images; the annotation types are category labels as in COCO/LVIS plus, for each category, a definition string and an array of synonyms (Xie et al., 13 Jul 2026). The corpus provides up to 10 synonyms per category, with an average of approximately 8–9, and at least 1 definition per category (Xie et al., 13 Jul 2026). The same source states an average of approximately 9.37 synonyms plus definitions per category (Xie et al., 13 Jul 2026).

The construction process is deliberately lightweight in its dependence on image-side modification: the images are unchanged from COCO2017 train, while the augmentation occurs in the category-text layer (Xie et al., 13 Jul 2026). A plausible implication is that the resource can be integrated into pretraining pipelines without re-annotating bounding boxes or masks, since the corpus enriches label semantics rather than geometric supervision.

3. Synonym ranking, validation, and representation

Before the final human-in-the-loop large-language-model check, candidate synonyms undergo an initial automatic ranking by CLIP text-text similarity (Xie et al., 13 Jul 2026). Given the original category embedding $t_{0}$ and a candidate synonym embedding $t_{i}$, the similarity score and normalized weight are defined as

$$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$

where $\tau$ is a temperature with default value $\tau=0.07$ (Xie et al., 13 Jul 2026). The top-$K$ highest-weight synonyms are then retained, with $K \le 10$ (Xie et al., 13 Jul 2026). The stated rationale is to ensure that retained candidates are close in CLIP’s semantic space (Xie et al., 13 Jul 2026).

This ranking mechanism is important because SEViC does not treat synonym generation as purely generative. The generated set is constrained by a pretrained multimodal embedding space before downstream validation (Xie et al., 13 Jul 2026). That design couples lexical augmentation to the representational geometry of CLIP, which is also the encoder family used later in pretraining. This suggests an internal consistency principle: synonym candidates are selected in the same semantic space in which synonym robustness is later optimized.

The data format makes the representational intent explicit. The corpus stores per-category definitions and synonym arrays in sevic_categories.json, and per-image category linkages in sevic_image_index.json (Xie et al., 13 Jul 2026). Example entries include canonical names such as “boat” and “giraffe,” each accompanied by a definition and a list of synonym strings (Xie et al., 13 Jul 2026). Because each image record reproduces category-specific definitions and synonyms, the image index directly supports mini-batch retrieval of lexical variants during pretraining (Xie et al., 13 Jul 2026).

Component	File	Stored content
Category metadata	sevic_categories.json	Definition and synonyms per category
Image linkage	sevic_image_index.json	File name and category entries per image
Images	images/	COCO2017 train images

4. Integration into SynCLIP pretraining

Within SynCLIP, SEViC is used at the level of text-conditioned dense pretraining (Xie et al., 13 Jul 2026). For each training image $I$, the system retrieves its base label set $T_{\rm label}$, such as “dog,” together with the enriched expression set $$T_{\rm sem} = \{\text{synonyms} \cup \text{definition}\}$$ (Xie et al., 13 Jul 2026). Both sets are encoded with the frozen CLIP text encoder $t_{i}$0 to obtain

$t_{i}$1

where $t_{i}$2 is the number of expressions (Xie et al., 13 Jul 2026). Dense image tokens are then computed from student and teacher variants, yielding $t_{i}$3 (Xie et al., 13 Jul 2026).

The Semantic-consistent Spatial Attention alignment module, abbreviated SSA, defines pairwise spatial attention maps as

$t_{i}$4

The semantic alignment loss is

$t_{i}$5

This loss explicitly minimizes discrepancies between attention maps from canonical labels and enriched expressions (Xie et al., 13 Jul 2026). In effect, SEViC supplies the alternate linguistic views that make this alignment objective possible.

The Spatial Attention Refinement module, abbreviated SAR, further uses definitions and synonyms to select the top-$t_{i}$6 most attended tokens from $t_{i}$7, derive their spatial correlation via a frozen vision foundation model, and aggregate them into refined attention according to

$t_{i}$8

with optimal $t_{i}$9 (Xie et al., 13 Jul 2026). Here the enriched expressions are not passive metadata; they serve as active queries that affect token selection and refinement. In each mini-batch, one enriched expression per object instance is sampled randomly, cycling through synonyms and definition, and no special curriculum beyond uniform sampling is used (Xie et al., 13 Jul 2026).

5. Empirical effects on grounding robustness

The empirical role of SEViC is reported through ablation and robustness analyses using an F-ViT+DeCLIP backbone on OV-COCO AP$$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$0 novel classes (Xie et al., 13 Jul 2026). In the ablation setting, the base model without SEViC achieves 41.0, adding SSA with synonyms only gives 42.3, adding SSA with definitions only gives 42.1, and adding SSA with both synonyms and definitions yields 43.6 (Xie et al., 13 Jul 2026). The reported interpretation is that jointly using synonyms and definitions provides a +2.6 point gain on novel classes (Xie et al., 13 Jul 2026).

A robustness experiment substitutes synonyms at evaluation time and compares the degradation in OV-COCO AP$$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$1 on novel classes (Xie et al., 13 Jul 2026). For F-ViT+DeCLIP, performance changes from 41.1 to 31.5, a drop of 9.6 points; for F-ViT+SynCLIP, performance changes from 43.6 to 39.2, a drop of 4.4 points (Xie et al., 13 Jul 2026). The smaller drop is presented as evidence that SEViC reduces grounding inconsistency under lexical variation (Xie et al., 13 Jul 2026).

The source summary characterizes the empirical effect as yielding +2–3 points on novel-class AP and reducing synonym-induced performance drops by over 50% (Xie et al., 13 Jul 2026). Since the detailed tables are reported only for the conditions above, that broader phrasing should be read as a compact synthesis rather than as an additional benchmark breakdown. The evidence nonetheless supports a narrow conclusion: in the reported SynCLIP configuration, enriching categories with synonyms and definitions materially improves novel-class robustness to wording changes (Xie et al., 13 Jul 2026).

Setting	OV-COCO AP$$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$2 nov.
Base (no SEViC)	41.0
+ SSA (synonyms only)	42.3
+ SSA (definitions only)	42.1
+ SSA (synonyms + definitions)	43.6

Model	Synonym substitution result
F-ViT+DeCLIP	41.1 $$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$3 31.5
F-ViT+SynCLIP	43.6 $$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$4 39.2

6. Relation to automatic concept discovery

The earlier framework in “Automatic Concept Discovery from Parallel Text and Visual Corpora” provides a different route to a synonym-enriched visual resource (Sun et al., 2015). Instead of starting from a fixed ontology such as COCO plus LVIS, it begins by parsing captions, extracting unigrams tagged as NOUN, VERB, ADJ, ADV together with dependency bigrams, and discarding terms occurring fewer than $$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$5 times in the corpus (Sun et al., 2015). For a term $$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$6, the associated positive image set is defined as $$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$7, with negatives sampled from the remaining images (Sun et al., 2015).

Each term is then scored by visual discriminative power. A 2-fold cross-validated linear SVM $$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$8 is trained on image features $$s_{i} \;=\;\frac{t_{0}\,\cdot\,t_{i}} {\lVert t_{0}\rVert\;\lVert t_{i}\rVert} \quad\text{and}\quad w_{i} \;=\;\frac{\exp(s_{i}/\tau)}{\sum_{j}\exp(s_{j}/\tau)},$$9, for example fc6 features of a pre-trained CNN, and cross-validated Average Precision is computed as

$\tau$0

Terms with $\tau$1, with $\tau$2 exemplified as 0.15, are discarded (Sun et al., 2015). Surviving terms are then compared by visual similarity and semantic similarity. Semantic similarity uses skip-gram word embeddings with cosine similarity

$\tau$3

Visual and semantic affinities are fused element-wise as

$\tau$4

and spectral clustering on the normalized Laplacian yields $\tau$5 concept groups (Sun et al., 2015). Each cluster then functions as a “visual concept” whose term set can be interpreted as a synonym-like or semantically related group grounded by pooled images (Sun et al., 2015).

This older pipeline differs substantially from the SynCLIP version of SEViC. The concept-discovery approach is bottom-up, caption-derived, classifier-filtered, and cluster-based; the SynCLIP approach is top-down, ontology-based, LLM-enriched, and category-centric (Sun et al., 2015, Xie et al., 13 Jul 2026). Still, both attach significance to the same core problem: words that are semantically close should ideally map to coherent visual entities. A plausible implication is that the two lines of work represent complementary regimes—one optimized for discovering emergent concepts from weak text-image parallelism, the other for stabilizing supervision in dense open-vocabulary perception.

7. Limitations, misconceptions, and extensions

Several limitations are stated explicitly for the SynCLIP formulation of SEViC (Xie et al., 13 Jul 2026). The quality of large-language-model output directly affects the corpus; hallucinated or overly broad synonyms harm consistency, and rigorous human- or LLM-in-the-loop validation is essential (Xie et al., 13 Jul 2026). Very fine-grained distinctions, such as between species of birds, may require substantially more domain-expert supervision (Xie et al., 13 Jul 2026). These caveats indicate that lexical enrichment is not intrinsically beneficial; its utility depends on semantic precision relative to the visual class taxonomy.

A common misconception would be to treat SEViC as merely adding more text strings to labels. In the SynCLIP setting, the corpus is tied to explicit mechanisms—SSA and SAR—that operate on spatial attention consistency and refinement (Xie et al., 13 Jul 2026). Without such objectives, synonym augmentation alone would not necessarily enforce coherent grounding. Conversely, the reported ablations show that definitions alone or synonyms alone improve performance, but the combination improves it more, which indicates that the benefit is not reducible to simple lexical redundancy (Xie et al., 13 Jul 2026).

The documented best practices for adaptation to new domains are concrete. For medical imaging, the recommendation is first to collect a domain ontology such as UMLS, then prompt an LLM for medically precise synonyms and lay definitions, while always validating automatically via a second LLM check or a small set of in-domain experts (Xie et al., 13 Jul 2026). Domain-specific text corpora may also be incorporated to bias synonym generation toward terms used by practitioners (Xie et al., 13 Jul 2026). Suggested enhancements include dynamically weighting synonyms during pretraining by their CLIP-similarity score $\tau$6, expanding definitions to include visual attributes such as color and texture in a structured schema followed by subregion alignment via multi-token queries, and introducing a curriculum that first trains on coarse categories and progressively adds finer-grained synonyms (Xie et al., 13 Jul 2026).

These proposed extensions remain suggestions rather than reported results (Xie et al., 13 Jul 2026). They nevertheless clarify the conceptual horizon of SEViC: it is a mechanism for structured lexical diversification under visual grounding constraints. In that respect, SEViC occupies a broader methodological space that includes ontology augmentation, synonym calibration in embedding space, dense cross-modal attention alignment, and concept discovery from parallel corpora (Xie et al., 13 Jul 2026, Sun et al., 2015).