TSF: Target-Specific Tracked Features

Updated 5 July 2026

TSF is defined as referent-specific visual features extracted from tracked object regions and injected into LLM inputs to ensure temporally aligned cues.
Its integration in video MLLMs addresses issues like spatial drift and identity switches by maintaining referential consistency across frames.
SPARROW’s TSF mechanism employs tracked crops, K-means compression, and a dual-prompt design to enhance both geometric localization and temporal stability.

Target-Specific Tracked Features (TSF) denotes, in its most explicit recent formulation, a set of referent-specific visual features extracted from tracked object regions across a video and injected into a multimodal LLM so that the model receives temporally aligned cues about the same referred object instance across time (Alansari et al., 12 Mar 2026). The term is not used uniformly across arXiv literature, however. In some works it refers to target-adaptive tracking representations, in others to target-specific but non-tracked feature learning, and in still others the acronym “TSF” names an unrelated method family such as Time Series Forest (Ji et al., 2024). A rigorous account therefore has to distinguish the SPARROW definition from broader target-conditioned tracking mechanisms and from purely terminological overlaps.

1. Definition and terminological scope

In the SPARROW framework, TSF is defined as referent-specific visual features extracted from tracked object regions across a video and injected into the multimodal input during training. Its stated purpose is to provide “temporally aligned, referent-specific cues during training” and thereby enable “identity persistence without explicit tracking at inference” (Alansari et al., 12 Mar 2026). This definition is narrower than generic feature adaptation: TSF is not any target-aware descriptor, but a specific mechanism built from tracked crops, compacted into tokens, and appended to the language-model input.

The broader literature represented here shows that the phrase “target-specific” often survives while “tracked features” changes meaning. Some tracking papers instantiate TSF-like behavior without using the exact acronym, by dynamically selecting discriminative channels, storing per-track appearance attributes, or constructing template-conditioned fine-grained target descriptors. Other papers use “TSF” for unrelated constructs and must not be conflated with SPARROW’s notion.

Context	Meaning	Core representation
SPARROW	Target-Specific Tracked Features	Tracked crop features projected as LLM tokens
Visual tracking papers	Target-adaptive tracked representation	Channel subsets, candidate tuples, template memories, or per-track attribute banks
Non-tracking papers	Different use of “TSF”	Time-series forests, polynomial features, or sparse signal-strength fields

This heterogeneity matters because the same initials can denote different objects. A technically precise usage therefore treats SPARROW TSF as the canonical definition of “Target-Specific Tracked Features,” while viewing adjacent tracking papers as related feature-design paradigms rather than identical formulations.

2. TSF in pixel-grounded video MLLMs

SPARROW introduces TSF to address a specific failure mode of pixel-grounded video MLLMs: temporal referential inconsistency. The paper argues that prior systems such as VideoGLaMM, UniPixel, and GLUS often rely on a static segmentation token, typically [SEG], which conveys semantics but lacks temporal context. As a result, models can segment the correct object in easy frames yet fail to preserve identity when motion, occlusion, reappearance, or distractor similarity becomes significant (Alansari et al., 12 Mar 2026).

The failures are named explicitly. Spatial drift denotes gradual movement of the predicted region away from the intended referent. Identity switches occur when the model jumps from the correct instance to a similar distractor. Ambiguous re-identification arises when a target disappears and later reappears, but the model no longer knows which instance is the intended one. Unstable initialization refers to poor first-frame grounding that propagates downstream. SPARROW’s central distinction is between semantic grounding and referential consistency: a static [SEG] token indicates what category or phrase to ground, whereas TSF is intended to preserve which exact evolving instance is being discussed over the sequence.

SPARROW couples TSF with a separate dual-prompt design. TSF is responsible for temporal referential stability; the [BOX] + [SEG] pair handles spatial precision and initialization stability. The [BOX] token provides a coarse geometric prior, while [SEG] refines semantic mask grounding. The paper’s ablation on Ref-DAVIS17 shows that these mechanisms are complementary rather than redundant: baseline performance is 69.5 with no TSF and BOX off, 72.4 with train-only TSF and BOX off, 72.5 with BOX on and no TSF, 76.8 with train-only TSF and BOX on, and 77.7 with train+inference TSF and BOX on (Alansari et al., 12 Mar 2026). The decomposition supports the interpretation that TSF primarily stabilizes identity over time, whereas BOX chiefly improves geometric localization.

3. Construction and integration of TSF tokens

SPARROW constructs TSF from tracked crops of the referred target. Given a textual query $Q$ , an offline pseudo-supervision pipeline first detects the referent with GroundingDINO, then propagates it through the clip with CLDTracker, crops the tracked regions, encodes the crops with the spatial encoder $\mathcal{F}_g$ , compresses them by K-means, projects them into the LLM embedding space through the spatial V $\to$ L adapter $\mathcal{W}_g$ , and appends the resulting tokens as $Z_{\text{TSF}}$ (Alansari et al., 12 Mar 2026).

The paper formalizes the multimodal input as

$\mathcal{I} = [\,Q;\,Z_g;\,Z_h;\,Z_{\text{TSF}}\,].$

Here $Q$ is the tokenized query, $Z_g$ the projected spatial visual features, $Z_h$ the projected temporal visual features, and $Z_{\text{TSF}}$ the projected target-specific tracked tokens. TSF is therefore injected at the LLM input level rather than into SAM2, an external memory bank, or the [BOX]/[SEG] decoder prompts.

Compression is intentionally aggressive. The tracked candidate boxes $\mathcal{F}_g$ 0 are clustered in a joint visual–spatial feature space, and the sample nearest each centroid is retained to form $\mathcal{F}_g$ 1 with $\mathcal{F}_g$ 2. In practice SPARROW sets $\mathcal{F}_g$ 3, so TSF adds 4 extra tokens per target. The appendix reports that several projection paths were tested—no TSF, temporal-only, dual-path concatenation, dual-path summation, and spatial-only—and that spatial-only projection performs best, consistent with TSF’s intended role as an object-centric appearance and identity cue rather than a global temporal descriptor (Alansari et al., 12 Mar 2026).

Training uses TSF primarily as a training-time injection mechanism. Stage 1, termed Target-Specific Information Injection, updates the V $\mathcal{F}_g$ 4L adapters $\mathcal{F}_g$ 5, the L $\mathcal{F}_g$ 6V SEG adapter $\mathcal{F}_g$ 7, and LoRA parameters inside the LLM, while keeping the visual backbone and pixel decoders frozen. The total loss is

$\mathcal{F}_g$ 8

Notably, no standalone TSF-specific temporal consistency loss is introduced; TSF changes the supervision context by augmenting the input with temporally aligned referent tokens.

4. Empirical behavior, dataset support, and operational limits

SPARROW supports TSF with a curated referential video corpus built from HC-STVG, VID-Sentence, A2D Sentences, LaSOT, MeViS, GOT-10k, and Ref-SAV, standardized into $\mathcal{F}_g$ 9 tuples. The final corpus contains 30,646 video sequences and 45,231 question-answer pairs, and TSF tokens are precomputed offline for the model (Alansari et al., 12 Mar 2026).

The clearest empirical isolation comes from Ref-DAVIS17. With BOX off, performance rises from 69.5 without TSF to 72.4 with train-only TSF and 75.3 with train+inference TSF. With BOX on, the corresponding values are 72.5, 76.8, and 77.7. The appendix also reports robustness under corrupted pseudo-supervision: 76.8 with clean TSF, 76.0 with box jitter of $\to$ 0, 75.4 with 10% ID-switch injection, 76.0 with 30% trajectory dropout, versus a 72.5 no-TSF baseline when BOX is on. These numbers indicate that even noisy TSF supervision remains above the no-TSF baseline.

At the system level, the full SPARROW method—attributed jointly to TSF pretraining and dual-prompt design—improves up to +8.9 $\to$ 1 on MeViS, up to +7.3 $\to$ 2 on Ref-DAVIS17, about +5 mIoU on VidSTG, and up to +5.4 CLAIR on VideoGCG. Practical costs are also stated explicitly. Offline TSF generation over all 30,646 clips is estimated at 51.9 hours on 1×A100. Stage 1 training costs 12 h on 8×A100, and full SPARROW adds +32 GPU-hours over baseline. Default inference omits TSF and therefore avoids detector/tracker overhead, but optional test-time TSF incurs latency

$\to$ 3

which is about 18.1 s for $\to$ 4 targets and $\to$ 5 frames, or about 20 ms/target/frame (Alansari et al., 12 Mar 2026).

These details delimit the method’s scope. TSF materially improves referential stability, yet its quality depends on pseudo-trajectories from GroundingDINO and CLDTracker, and optional inference-time use introduces nontrivial runtime. The paper also leaves some implementation details unspecified, including the exact TSF token embedding dimensionality and the exact definition of the joint visual–spatial clustering feature.

5. TSF as a broader tracking design pattern

Outside SPARROW, several tracking papers instantiate target-specific tracked representations without formalizing them as TSF tokens. In S-KeepTrack, designed for the TSFMO benchmark of small and fast moving objects, the target representation is a candidate tuple combining appearance, position, and confidence. KeepTrack uses $\to$ 6, while S-KeepTrack adds a low-level branch $\to$ 7 and fuses high-level and low-level assignment matrices by $\to$ 8. The reported best setting is $\to$ 9, and the ablation shows that low-level cues contribute more than high-level cues for these tiny, fast targets (Zhang et al., 2022). This establishes a TSF-like principle: when appearance evidence is weak, tracked features must encode candidate-local appearance, confidence, and position rather than appearance alone.

DFST provides a different formulation. It extends the Adaptive Color Names tracker by dynamically re-ranking features at each frame using foreground pixels from the target box and background pixels from the immediate surround. The tracker operates over 10 Color Names channels plus grayscale/luminance information, and selects up to 8 channels after ranking them with a supervised adaptation of Infinite Feature Selection. In this setting, “target-specific” means that the chosen channel subset is the one that best discriminates the current target from its local background in the current frame (Roffo et al., 2016).

In thermal infrared tracking, DCFG constructs a template-conditioned feature representation by dividing template channels into groups, suppressing dominant regions, and enforcing inter-group diversity. The paper reports best performance at $\mathcal{W}_g$ 0 groups with suppression coefficient $\mathcal{W}_g$ 1, and uses the diversity regularizer

$\mathcal{W}_g$ 2

to push feature groups toward distinct discriminative target regions (Xiong et al., 19 Apr 2025). A related infrared design, TSF-SiamMU, decomposes the template into shallow and deep components and updates it through a multi-template residual rule,

$\mathcal{W}_g$ 3

thereby combining initial-template stability with current and accumulated appearance adaptation (Yan et al., 2023).

In online multi-object tracking, FeatureSORT makes target-specific tracked features explicit as per-track memories. Each trajectory stores a ReID embedding stack $\mathcal{W}_g$ 4, clothes-color stack $\mathcal{W}_g$ 5, clothes-style stack $\mathcal{W}_g$ 6, and direction state $\mathcal{W}_g$ 7, and matches new detections through a combined distance in which motion, edge, color, and style are weighted by $\mathcal{W}_g$ 8, while IoU and direction act as gates (Hashempoor et al., 2024). This usage is closer to classical target memory than to SPARROW’s token injection, but the conceptual link is direct: the tracked object is represented by persistent, target-specific cues rather than by geometry alone.

Several arXiv papers use the initials “TSF” in ways that are technically adjacent or completely unrelated to SPARROW’s definition. Slim-TSF is explicitly a Sliding Window Multivariate Time Series Forest for solar-flare forecasting, built from interval statistics such as mean, standard deviation, and slope, followed by pooled max, min, and mean features. Its target is the temporal evolution of solar active regions over 12-hour observation segments for predicting whether an M/X-class flare will occur in the next 24 hours; this is not Target-Specific Tracked Features in the SPARROW sense (Ji et al., 2024).

Tensor Machines learn a small number of explicit target-specific polynomial features for supervised prediction. The learned features are products of linear forms,

$\mathcal{W}_g$ 9

optimized directly from labels rather than sampled as target-agnostic random features (Yang et al., 2015). These features are target-specific, but not tracked.

CoDT uses a target-specific branch in open-set cross-domain 3D action recognition, where target-domain skeleton sequences are encoded by $Z_{\text{TSF}}$ 0 and $Z_{\text{TSF}}$ 1, learned through contrastive learning, online clustering, and collaborative pairwise consistency with a domain-shared branch (Liu et al., 2022). Again, the term refers to target-domain specialization rather than tracked object identity.

Two further cases are closer to tracking but still differ materially from SPARROW TSF. A topological motion-discrimination method classifies small targets at range from persistent-homology features of normalized sub-tracks, treating motion dynamics rather than appearance as the discriminative signal (Emerson et al., 2021). The TSSG model, by contrast, tracks target signal strengths on a spatial grid via sparse Kalman filtering, so the state is a sparse field of signal-strength coefficients rather than an identity-bound per-target descriptor (Farahmand et al., 2011).

This suggests a useful editorial distinction. TSF proper can be reserved for the SPARROW mechanism: tracked referent crops converted into multimodal tokens for video MLLMs. TSF-like methods then describe a broader family of target-adaptive tracking representations: dynamic channel selection, candidate-level appearance–position features, template-conditioned fine-grained descriptors, multi-template infrared representations, per-track attribute banks, topological motion signatures, or sparse target-induced feature fields. Such a distinction preserves the precision of the SPARROW definition while acknowledging the larger design space of target-specific feature tracking (Alansari et al., 12 Mar 2026).