Multi-Granularity Implicit Text (MIT)

• Multi-Granularity Implicit Text (MIT) is a framework where textual signals are distributed across multiple structural levels and require joint interpretation to resolve ambiguity.
• It employs hierarchical decomposition with cross-level interactions to fuse implicit supervision from objects, parts, words, and sequences, enhancing segmentation, detection, and tracking tasks.
• Empirical evaluations on benchmarks like MMR, DAT, and MG-MotionLLM demonstrate that MIT improves performance metrics by leveraging structured granularity in both visual and language modalities.

Searching arXiv for the cited works to ground the article in the current literature. [arXiv search] query: (Jang et al., 18 Mar 2025) Multi-Granularity Implicit Text (MIT) denotes a family of modeling problems in which the relevant textual signal is distributed across multiple levels of structure and is not fully explicit at any single level. In the cited literature, MIT is used either directly or as an interpretive label for settings where meaning must be inferred jointly across granularities such as object and part, word and line and paragraph and page, global sequence and local segment, or document and sentence and word. In several of these works, the specific term does not appear in the original paper; instead, it is used to characterize architectures and datasets that combine implicit textual supervision, hierarchical structure, and cross-level reasoning (Jang et al., 18 Mar 2025, Wan et al., 2024, Wu et al., 3 Apr 2025).

1. Conceptual scope and defining properties

The central property of MIT is that textual intent is not reducible to a single explicit label or span. In reasoning segmentation, an instruction such as “turn on the TV” can imply multiple valid objects and multiple valid parts, so the correct output is a set of masks rather than a single target. In unified text detection, “implicit” refers to textual structure that is hard to localize with purely local cues, including arbitrarily curved text, partially occluded text, low-contrast or noisy regions, small or blurred instances, and complex layouts with overlapping and nested blocks. In motion-language modeling, the relevant supervision spans global motion-level captions, segment-level temporal descriptions, and local body-part-level descriptions, together with weak or derived signals such as time boundaries and special tokens. In weakly supervised text classification, latent class evidence emerges only when document-, sentence-, and word-level cues are jointly modeled rather than treated independently (Jang et al., 18 Mar 2025, Wan et al., 2024, Wu et al., 3 Apr 2025, Kargupta et al., 2023).

A recurrent distinction in this literature is between explicit text and implicit text. Explicit text directly specifies the target or label. Implicit text instead constrains interpretation through attributes, relations, functions, temporal boundaries, structural priors, or weak textual surrogates. MGP-STR describes this as “implicit linguistic integration,” because it injects linguistic knowledge through a multi-granularity output space of characters, BPE subwords, and WordPiece subwords while using no independent LLM (Da et al., 2023). DTLLM-VLT uses “implicit” to describe target grounding through attributes, relations, and spatio-temporal cues rather than direct coordinates or target-identity shortcuts (Li et al., 2024).

This suggests that MIT is best understood not as a single task definition but as a recurring design pattern: preserve multiple representational levels, expose their interactions, and treat ambiguity as structured supervision rather than annotation noise.

2. Representation hierarchies and learning mechanisms

A common MIT mechanism is explicit hierarchical decomposition coupled with learned cross-level interaction. In MMR, the task input is an image $I \in \mathbb{R}^{H\times W\times 3}$ and an implicit instruction text $t$, and the output is a set of segmentation masks $M=\{M_k\}$ for all valid targets, including objects and parts. The proposed M$^2$SA framework is trained end-to-end with a text loss and a mask loss, using $L = L_{txt} + L_{mask}$ and $$L_{mask} = \lambda_{bce} L_{bce} + \lambda_{dice} L_{dice}$$, with $\lambda_{bce} = 0.5$ and $\lambda_{dice} = 2.0$. Multiple special [SEG] tokens align textual response generation with multi-target mask decoding, while early local feature fusion injects boundary-sensitive information for part-level delineation (Jang et al., 18 Mar 2025).

DAT formalizes cross-level interaction in a different way. It jointly models word, line, paragraph, and page with grouped object queries and a masked global self-attention module over the concatenated query tensor. The interaction factor $\mathcal{I}$ controls which cross-granularity interactions are allowed: $\mathcal{I}=1$ permits adjacent interactions, $t$0 additionally enables word↔paragraph and line↔page, and $t$1 is fully connected. The best setting in ablations is $t$2, indicating that structured adjacent masking supplies a useful inductive bias while reducing noise (Wan et al., 2024).

MG-MotionLLM places different granularities in a unified token space. The T5 vocabulary is extended to $t$3, where $t$4 is the text vocabulary, $t$5 indexes motion VQ tokens, and $t$6 contains special tokens such as <Motion Tokens>, </Motion Tokens>, <SEP>, and <Motionless>. Alignment is learned through prompt templates spanning global captioning, detailed motion scripts, temporal boundary localization, and fine-grained synthesis. The overall objective is $t$7 (Wu et al., 3 Apr 2025).

MEGClass and MahNN show analogous mechanisms in NLP. MahNN applies syntactical attention at the symbolic level and semantical attention at the latent dimension level, then feeds the resulting multi-channel tensor into a CNN. MEGClass estimates a document class distribution from sentence discriminativeness, contextualizes sentences with multi-head self-attention, and optimizes a distribution-weighted contrastive loss so that a document representation aligns with multiple plausible class prototypes instead of collapsing prematurely to one label (Liu et al., 2020, Kargupta et al., 2023).

3. Vision-language grounding, segmentation, and counting

The most explicit MIT formulation in vision-language grounding appears in MMR, a benchmark for multi-target and multi-granularity reasoning segmentation. MMR contains 194,398 implicit question–answer pairs over 57,643 images, with train 154,127, val 8,194, and test 32,077. It inherits 75 object categories and 445 part categories from PACO-LVIS after filtering and selection, preserves instance-level identities and explicit object–part linkage, and packages each sample with the global caption, question, answer, and list of target references paired with masks. On average there are 1.8 targets per answer; the maximum is 16. The test set is partitioned into object-only, part-only, and mixed subsets, and evaluation uses gIoU and cIoU. On MMR test object-only, M$t$8SA-Llama2-13B reaches gIoU 42.3 and cIoU 57.6; on part-only, 13.6 and 27.2; on mixed, 31.6 and 47.6. Ablations show that multiple [SEG] tokens are necessary for multi-target decoding and that early local feature fusion is especially beneficial for part-only evaluation (Jang et al., 18 Mar 2025).

MIT also appears in open-world counting when the ambiguity lies in the granularity of “what to count.” “Count Anything at Any Granularity” redefines open-world counting as multi-grained counting with five explicit levels: identity-level, attribute-level, category-level, instance-type level, and concept-level. Each object is represented by $t$9, and a query defines a target set $M=\{M_k\}$0 and an optional distractor set $M=\{M_k\}$1. KubriCount, the resulting dataset, contains 110,507 images, approximately 7.3M annotated instances, 157 categories across 16 super-categories, and 198,702 queries. HieraCount uses text and visual exemplars as complementary specifications, with a loss $M=\{M_k\}$2 and Hungarian matching. On KubriCount, HieraCount achieves overall MAE 4.67 and RMSE 11.07, while Level 4 remains the hardest, with 8.37 and 17.14. On PairTally, it reaches MAE 36.27 under positive-only prompting, ahead of CountGD and CountGD++ (Liu et al., 11 May 2026).

These two lines of work share a precise MIT property: correct grounding requires the model to represent multiple admissible interpretations while preserving the level at which the interpretation is valid. In MMR this is object versus part and single versus multiple targets; in HieraCount it is category versus type versus attribute versus concept. In both cases, explicit granularity is used to convert underspecified language into a verifiable set-membership problem.

4. Text-centered computer vision: detection, recognition, and tracking

DAT extends MIT to unified text detection. It treats words, lines, paragraphs, and pages as jointly predicted levels in a single end-to-end system built on a Swin Transformer Large backbone, a DINO-style decoder with grouped queries, across-granularity interactive attention, and a prompt-based segmentation module. Its segmentation branch uses $M=\{M_k\}$3. DAT-DET has about 228.29M parameters and about 394 GFLOPs; DAT with segmentation has about 284.65M and about 474 GFLOPs. Despite unified multi-task coverage, testing FPS 3.57 is reported as competitive with single-task baselines. On ICDAR2015, DAT-DET achieves $M=\{M_k\}$4, $M=\{M_k\}$5, $M=\{M_k\}$6; on Total-Text, DAT-SEG reaches $M=\{M_k\}$7, $M=\{M_k\}$8, $M=\{M_k\}$9; on CTW1500, DAT-SEG reaches $^2$0; on DIW, DAT-SEG reaches mIoU 98.65 (Wan et al., 2024).

MGP-STR treats MIT as implicit linguistic integration in scene text recognition. A ViT backbone with an Adaptive Addressing and Aggregation module feeds three parallel heads: character-level, BPE, and WordPiece. Fusion is done either by a Confidence-based Fusion Strategy or a Learnable Fusion Strategy. MGP-STR_LFS achieves an average 94.05% on standard scene text recognition benchmarks and 96.40% when trained with real data. The paper reports that MGP-STR_Vision reaches average 92.73%, MGP-STR_CFS reaches 93.35%, and MGP-STR_LFS reaches 94.05%. The design injects linguistic structure through output granularity rather than a separate LM (Da et al., 2023).

DTLLM-VLT applies MIT to visual-language tracking through two controlled axes: semantic extent and semantic density. It generates four kinds of descriptions for each video—initial concise, initial detailed, dense concise, and dense detailed—using SAM for target masking and Osprey for masked-region description generation. Dense updates are generated every 100 frames, motivated by a 4-second short-term memory proxy at 25 FPS. Across OTB99_Lang, LaSOT, TNL2K, and MGIT generation statistics, the framework produces 7,238 initial descriptions in total and 128.4K dense descriptions, for a total word count of about 1.9M with about 14.8K unique words. In direct testing with MMTrack, OTB99_Lang improves from official AUC 69.0 to 70.6 with initial concise text, while MGIT improves from 73.5 to 74.2 with dense concise text. After retraining, OTB99_Lang reaches 71.3 AUC with dense concise, and MGIT reaches 74.4 with dense detailed (Li et al., 2024).

Across these three systems, MIT serves different immediate objectives—structural localization, robust recognition, and semantic guidance for tracking—but the shared mechanism is stable: multiple granularities are not post-hoc annotations but active computational variables.

5. Temporal and multimodal MIT: motion and audio-visual segmentation

MG-MotionLLM defines a multi-granularity motion-language setting in which global motion-level text, segment-level temporal text, and local body-part-level text are trained together. HumanML3D contributes 14,616 motion sequences with 44,970 sequence-level captions, while FineMotion contributes 420,968 snippet BPM descriptions, of which about 5% are manually annotated and the rest are automatically derived. The model uses HumanML3D’s 263-D motion features and a VQ-VAE tokenizer, then trains a unified T5 on 28 tasks during Granularity-Synergy Pre-training. Representative HumanML3D results for the base model are Top-3 R-Precision = 0.802, FID = 0.303, and Diversity = 9.960 for text-to-motion; Top-1 R-Precision = 0.592, MM-Dist = 2.581, and BERTScore = 36.7 for motion-to-text; and sequence-level BLEU@4 = 66.44, ROUGE = 65.5, and BERTScore = 52.3 for motion-to-detailed text on FineMotion (Wu et al., 3 Apr 2025).

“Implicit Counterfactual Learning for Audio-Visual Segmentation” extends MIT to a modality-shared semantic bridge for AVS. Here MIT is a set of latent, language-like embeddings retrieved at video-, segment-, and frame-level from VideoCLIP, CLIP, and CLAP, then fused into a factual text representation. Semantic Counterfactual learning generates orthogonal latent counterfactuals, while Collaborative Distribution-Aware Contrastive Learning aligns visual, audio, and MIT distributions. Training uses MIT, SC, and CDCL, but inference introduces no runtime overhead because MIT and SC are not used at test time. On the PVT-v2 backbone, the method reports S4 J&F 90.07, M3 J&F 69.89, and AVSS J&F 48.16, improving over the cited second-best results by 1.77%, 4.69%, and 4.06%, respectively. Ablations show that segment-level MIT contributes the largest incremental gain among the three granularities (Zha et al., 28 Jul 2025).

These works emphasize that MIT is not confined to visible textual strings. It can include latent “textual” embeddings, time tokens, auto-generated detailed scripts, and special control tokens. This suggests that, in multimodal sequence modeling, the “text” in MIT is often a semantic interface rather than a human-readable sentence.

6. NLP formulations, evaluation regimes, and recurring limitations

In NLP, MIT has been instantiated both in supervised architectures and in extremely weak supervision. MahNN combines a Bi-LSTM, syntactical attention over tokens, semantical attention over latent dimensions, a multi-channel tensor, and a CNN classifier. On sentence-level classification benchmarks, MahNN-3 reports 82.57 on MR, 93.75 on Subj, and 89.75 on MPQA, while remaining close to strong baselines on SST-1 and SST-2. The paper identifies computational cost from the $^2$1 association matrices in syntactical attention and notes that implicit semantics requiring world knowledge, figurative language, or cross-sentence coreference remain challenging (Liu et al., 2020).

MEGClass addresses MIT under extremely weak supervision using only class surface names. It jointly models documents, sentences, and words, estimates a soft document class distribution from sentence discriminativeness, contextualizes sentence representations with multi-head self-attention, and iteratively updates class prototypes from the most confident documents. Across seven benchmarks, it reports 81.72/80.63 on 20News, 93.06/91.93 on NYT-Loc, 89.24/71.06 on NYT-Fine, and 66.37/64.24 on 20News-Fine in Micro/Macro-F1. The paper notes smaller benefits on short texts, sensitivity to label name ambiguity, and the difficulty of fine-grained overlapping classes (Kargupta et al., 2023).

No single evaluation protocol unifies MIT across domains. Reasoning segmentation uses gIoU and cIoU; unified text detection uses precision, recall, F-score, mAP, and mIoU; motion-LLMs use R-Precision, FID, MM-Dist, Diversity, BLEU, ROUGE, and BERTScore; counting uses MAE and RMSE; visual-language tracking uses AUC, precision, and normalized precision; AVS uses Jaccard index, F-score, and mean J&F (Jang et al., 18 Mar 2025, Wan et al., 2024, Wu et al., 3 Apr 2025, Liu et al., 11 May 2026, Li et al., 2024, Zha et al., 28 Jul 2025). This suggests that MIT is presently a cross-domain methodological motif rather than a standardized benchmark family.

The limitations are likewise domain-specific but conceptually aligned. MMR omits humans and human parts because PACO-LVIS lacks those classes. DAT notes computational cost, incomplete labels across multilingual granularities, and failure cases with extremely small or blurred text. MG-MotionLLM identifies noise in auto-generated detailed scripts, limitations of the 263-D motion representation for fingers and face, and latency from sequence-to-sequence generation. MGP-STR notes challenges for very long heavily curved strings, artistic fonts, pure digit sequences, and English-centric vocabularies. HieraCount finds Level 4 within-category type discrimination hardest. ICF for AVS notes sensitivity to temporal partitioning and the cost of orthogonalization and diffusion. DTLLM-VLT observes that current trackers underutilize long detailed text without retraining (Jang et al., 18 Mar 2025, Wan et al., 2024, Wu et al., 3 Apr 2025, Da et al., 2023, Liu et al., 11 May 2026, Zha et al., 28 Jul 2025, Li et al., 2024).

Taken together, these works indicate that MIT is a general strategy for representing ambiguity, hierarchy, and weakly localized semantics. Its recurring thesis is that meaning emerges more reliably when multiple granularities are modeled jointly, whether the units are parts and objects, words and paragraphs, clips and snippets, or class names and documents.