PIZA: Progressive Iterative Zooming Adapter

• The paper introduces PIZA as a parameter-efficient module that transforms a static grounding model into a multi-step search process for small-object REC.
• It leverages a compact zooming-step embedding to condition PEFT components like CoOp, LoRA, and Adapter+, enabling progressive coarse-to-fine localization in challenging driving scenes.
• Empirical results demonstrate that PIZA, with minimal additional parameters, significantly boosts high-IoU accuracy compared to single-shot and sliding-window baselines.

Searching arXiv for the specified papers to ground the article in the cited sources. Progressive-Iterative Zooming Adapter (PIZA) is an adapter module for parameter-efficient fine-tuning in referring expression comprehension (REC) that enables models to progressively zoom in and localize small objects. It is introduced together with the small object REC (SOREC) dataset for driving scenarios, and is formulated as a mechanism that converts a static, single-shot grounding model into a search-based model conditioned on both language and the history of previous zoom boxes (Goto et al., 4 Oct 2025). In the reported instantiation, PIZA is applied to GroundingDINO and operates by encoding the sequence of past bounding boxes into a compact zooming-step embedding, injecting that embedding into parameter-efficient adaptation components such as CoOp prompts, LoRA bottlenecks, or Adapter+ layers while keeping the base model mostly frozen (Goto et al., 4 Oct 2025). The term also admits a broader conceptual connection to training-free iterative adapter frameworks for generative modeling, especially insofar as both rely on progressive refinement and adapter-style conditioning, but PIZA itself is presented for small-object REC rather than story visualization (Mao et al., 2024).

1. Problem setting and definition

Referring expression comprehension takes an image $\bm{x} \in \mathbb{R}^{W \times H \times C}$ and a natural-language referring expression $\bm{t}$, and predicts a bounding box $\hat{\bm{b}}$ for the described object:

$$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$

where $(x_0, y_0)$ and $(x_1, y_1)$ are top-left and bottom-right coordinates (Goto et al., 4 Oct 2025). REC models must align the semantics of $\bm{t}$ with visual content and localize a unique object. Classical REC benchmarks such as RefCOCO/+/g mostly involve medium or large objects in COCO images, and modern vision-language detectors like GroundingDINO reach more than $90\%$ Acc on RefCOCO, so the standard-sized REC problem is described as essentially solved (Goto et al., 4 Oct 2025).

PIZA is motivated by a different regime: extremely small targets in high-resolution driving scenes. In SOREC, source images are high-resolution street scenes with average resolution $3407 \times 2470$, and target objects include traffic signs, lights, and distant vehicles (Goto et al., 4 Oct 2025). The dataset description states that each bounding box typically occupies approximately $0.05\%$ of the entire image area, and all target bounding boxes occupy less than $\bm{t}$0 of the image area (Goto et al., 4 Oct 2025). Formally, the small-object regime is characterized by a ground-truth box $\bm{t}$1 such that

$\bm{t}$2

with area ratio $\bm{t}$3 and often around $\bm{t}$4 (Goto et al., 4 Oct 2025).

The central definition of PIZA is an extension of a base grounding model $\bm{t}$5 into an autoregressive zooming model:

$\bm{t}$6

where $\bm{t}$7 is the history of previously predicted boxes (Goto et al., 4 Oct 2025). The process begins from the full-image box $\bm{t}$8 and proceeds by repeated cropping and refinement until a learned end-of-search criterion terminates the sequence. This makes PIZA both progressive, in the sense of coarse-to-fine cropping, and iterative, in the sense that each prediction conditions on prior zoom steps (Goto et al., 4 Oct 2025).

2. Motivation: small-object REC in driving scenes

The difficulty of small-object REC is attributed to four factors. First, resolution and feature sparsity arise because downsampling large images for transformer vision backbones such as Swin-T severely reduces the effective resolution of tiny objects; such objects may collapse to one or two patch tokens or be lost entirely (Goto et al., 4 Oct 2025). Second, there is a mismatch between pre-training and fine-tuning: GroundingDINO is pre-trained on O365, GoldG, GRIT, V3Det, and related corpora with much larger object-size distributions, whereas SOREC contains very small targets and long descriptions, with average expression length $\bm{t}$9 words versus $\hat{\bm{b}}$0 in RefCOCO (Goto et al., 4 Oct 2025). Third, driving scenes are cluttered and contain many similar objects, so descriptions frequently rely on relations such as “small blue sign on the right side of the road, below the speed limit sign,” which are difficult to resolve when the model only has a coarse global view (Goto et al., 4 Oct 2025). Fourth, computational constraints make naive sliding windows and dense tiling unattractive; these baselines are experimentally reported as both slower and less accurate than PIZA (Goto et al., 4 Oct 2025).

Within this setting, PIZA is positioned not as a new backbone but as an adapter-based mechanism that makes an existing model actively zoom into promising regions (Goto et al., 4 Oct 2025). The base model parameters remain mostly frozen, while the system trains only the PIZA module and the selected parameter-efficient fine-tuning (PEFT) components: CoOp prompts, LoRA matrices, Adapter+ layers, and LayerNorms depending on the variant (Goto et al., 4 Oct 2025). The reported best configuration, PIZA-Adapter+, uses only $\hat{\bm{b}}$1M trainable parameters on top of a $\hat{\bm{b}}$2M-parameter GroundingDINO backbone (Goto et al., 4 Oct 2025).

A plausible implication is that PIZA addresses the small-object regime primarily by altering the inference geometry rather than by expanding the visual backbone. Rather than forcing a single pass to resolve a target occupying approximately $\hat{\bm{b}}$3 of an image, the method transforms localization into a short search process whose average ground-truth length on SOREC is $\hat{\bm{b}}$4 steps (Goto et al., 4 Oct 2025).

3. Architecture and zooming-step representation

The base architecture in the main experiments is GroundingDINO in the MM-GroundingDINO implementation. It has five components: a BERT text encoder, a Swin-T image encoder producing multi-scale feature maps, a feature enhancer with text-to-image and image-to-text cross-attention together with self-attention and feed-forward networks, a language-guided query selection module, and a cross-modality decoder that refines queries into grounded boxes and class logits (Goto et al., 4 Oct 2025). PEFT methods are applied in the feature enhancer and decoder (Goto et al., 4 Oct 2025).

PIZA adds an auxiliary module that maps the sequence of zoom boxes $\hat{\bm{b}}$5 to a zooming-step embedding $\hat{\bm{b}}$6 (Goto et al., 4 Oct 2025). For each box $\hat{\bm{b}}$7, a $\hat{\bm{b}}$8-dimensional low-level feature vector $\hat{\bm{b}}$9 is computed from normalized area, relative size to the previous step, normalized width and height, and normalized center coordinates:

$$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$0

$$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$1

$$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$2

$$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$3

These vectors form a sequence $$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$4 (Goto et al., 4 Oct 2025).

The sequence is then processed by learnable Fourier embeddings, followed by a transformer encoder over $$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$5, and average pooling over positions to obtain a single embedding $$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$6 (Goto et al., 4 Oct 2025). The main text uses $$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$7, and the parameter count for this module is reported as $$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$8M (Goto et al., 4 Oct 2025). Two auxiliary heads are trained from the pooled embedding: an EOS head for binary classification,

$$\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),$$9

and a progress head for scalar regression,

$(x_0, y_0)$0

representing normalized progress along the search trajectory (Goto et al., 4 Oct 2025). At inference time, the EOS head is used to stop the zoom loop (Goto et al., 4 Oct 2025).

This design makes the search history explicit. Rather than conditioning on cropped pixels alone, the model receives a compact representation of how much zooming has already occurred, how the crop geometry has evolved, and where the current crop lies within the original image. This suggests that PIZA operationalizes “zoom state” as a first-class conditioning signal.

4. Progressive-iterative search process and training formulation

The inference process begins with

$(x_0, y_0)$1

so the initial crop is the full image (Goto et al., 4 Oct 2025). At step $(x_0, y_0)$2, low-level features $(x_0, y_0)$3 are computed from the history of boxes, the PIZA module produces $(x_0, y_0)$4, EOS probability $(x_0, y_0)$5, and progress estimate $(x_0, y_0)$6, the model predicts the next box $(x_0, y_0)$7 in global coordinates, and the corresponding crop $(x_0, y_0)$8 is extracted (Goto et al., 4 Oct 2025). If EOS is predicted, or if a maximum step count is reached, the process stops; on SOREC, two to three steps are described as sufficient in practice (Goto et al., 4 Oct 2025).

The paper formalizes the search sequence as

$(x_0, y_0)$9

with $(x_1, y_1)$0 and $(x_1, y_1)$1 covering $(x_1, y_1)$2 for $(x_1, y_1)$3 (Goto et al., 4 Oct 2025). Training does not rely solely on the original dataset $(x_1, y_1)$4 of triples $(x_1, y_1)$5. Instead, PIZA constructs an extended dataset $(x_1, y_1)$6 containing search processes,

$(x_1, y_1)$7

where $(x_1, y_1)$8, $(x_1, y_1)$9, $\bm{t}$0, $\bm{t}$1, and $\bm{t}$2 (Goto et al., 4 Oct 2025). The labels are

$\bm{t}$3

Intermediate boxes $\bm{t}$4 are generated so that their area ratios match the distribution $\bm{t}$5 estimated from pre-training datasets such as O365 and GoldG (Goto et al., 4 Oct 2025). For a target with area ratio $\bm{t}$6, the method samples $\bm{t}$7, chooses the number of zoom steps

$\bm{t}$8

and defines zoom factors using an exponentially weighted formulation:

$\bm{t}$9

Box areas are then given by

$90\%$0

with aspect ratio interpolation

$90\%$1

followed by

$90\%$2

Centers are aligned to the ground-truth object center subject to image-bound constraints (Goto et al., 4 Oct 2025).

Training samples a random intermediate step $90\%$3 and optimizes the base GroundingDINO losses together with the EOS and progress losses (Goto et al., 4 Oct 2025). The PIZA-augmented loss is

$90\%$4

where $90\%$5 includes localization and contrastive components, $90\%$6 is binary cross-entropy, and $90\%$7 is mean squared error (Goto et al., 4 Oct 2025).

5. PEFT variants and integration into GroundingDINO

PIZA is described as PEFT-agnostic and is instantiated in four forms: PIZA-CoOp, PIZA-LoRA, PIZA-Adapter+, and PIZA-VPT, with the main experiments emphasizing PIZA-Adapter+ as the strongest variant (Goto et al., 4 Oct 2025). In all cases, the role of PIZA is to make the chosen adaptation mechanism zoom-aware by injecting the zooming-step embedding $90\%$8.

PIZA-CoOp augments text-side prompts. Baseline CoOp prepends learnable embeddings $90\%$9 to text tokens:

$3407 \times 2470$0

PIZA-CoOp further inserts the zoom embedding as additional prompt tokens:

$3407 \times 2470$1

where $3407 \times 2470$2 is a stack of $3407 \times 2470$3 linear layers and experiments use $3407 \times 2470$4 (Goto et al., 4 Oct 2025). The trainable parameters comprise the prompt embeddings, the linear layers $3407 \times 2470$5, and LayerNorms in the text encoder, for a total of approximately $3407 \times 2470$6M parameters (Goto et al., 4 Oct 2025).

PIZA-LoRA modifies attention projections. Standard LoRA changes a linear map $3407 \times 2470$7 to $3407 \times 2470$8, so for input $3407 \times 2470$9 the output is

$0.05\%$0

PIZA-LoRA adds a zoom-conditioned term in the LoRA bottleneck:

$0.05\%$1

with $0.05\%$2, $0.05\%$3, and $0.05\%$4 (Goto et al., 4 Oct 2025). It is applied to text-to-image and image-to-text cross-attention modules in the feature enhancer and to self-attention modules in both feature enhancer and decoder. The main rank is $0.05\%$5, and total trainable parameters are $0.05\%$6M for PIZA-LoRA versus $0.05\%$7M for vanilla LoRA (Goto et al., 4 Oct 2025).

PIZA-Adapter+ conditions bottleneck adapters in the visual transformer stream. Adapter layers are placed after self-attention and feed-forward blocks in the feature enhancer, with bottleneck dimension $0.05\%$8 and standard form

$0.05\%$9

PIZA-Adapter+ introduces two conditioning designs (Goto et al., 4 Oct 2025). Type A, used when data is small, uses the progress value $\bm{t}$00 passed through a time embedding module with Fourier features and an MLP, adding the resulting conditioning vector to the output of a channel-wise scaling layer in the adapter (Goto et al., 4 Oct 2025). Type B, used when data is larger, uses the full embedding $\bm{t}$01 and drops the progress head (Goto et al., 4 Oct 2025). In both cases, the adapter hidden representation is modulated by $\bm{t}$02 or $\bm{t}$03, making the block step-aware (Goto et al., 4 Oct 2025). Total trainable parameters are $\bm{t}$04M, compared with $\bm{t}$05M for Adapter+ without PIZA (Goto et al., 4 Oct 2025).

The following table summarizes the reported PEFT variants.

Variant	Injection mechanism	Trainable parameters
PIZA-CoOp	Zoom embedding inserted as additional prompt tokens	~0.9M
PIZA-LoRA	Adds $\bm{t}$06 in LoRA bottleneck for attention projections	1.5M
PIZA-Adapter+	Conditions post-attention and post-FFN adapters with $\bm{t}$07 or $\bm{t}$08	3.5M
PIZA-VPT	Visual prompt tuning variant used in appendix	not specified here

In integration terms, PIZA-CoOp acts on the text stream, PIZA-LoRA on visual and cross-modal attention projections, and PIZA-Adapter+ mainly on the visual stream of the feature enhancer (Goto et al., 4 Oct 2025). The detector head of GroundingDINO remains responsible for predicting the next zoom box (Goto et al., 4 Oct 2025).

6. Empirical performance, ablations, and comparisons

SOREC uses the metrics $\bm{t}$09, $\bm{t}$10, and $\bm{t}$11, where $\bm{t}$12 is mean accuracy over IoU thresholds from $\bm{t}$13 to $\bm{t}$14 in increments of $\bm{t}$15 (Goto et al., 4 Oct 2025). The dataset split comprises Train-S with $\bm{t}$16k expressions, Train-L with $\bm{t}$17, Validation with $\bm{t}$18, Test-A with $\bm{t}$19, and Test-B with $\bm{t}$20 (Goto et al., 4 Oct 2025).

On SOREC with GroundingDINO and limited data (Train-S), zero-shot GroundingDINO attains validation performance of mAcc $\bm{t}$21, $\bm{t}$22 $\bm{t}$23, and $\bm{t}$24 $\bm{t}$25 (Goto et al., 4 Oct 2025). Full fine-tuning of the $\bm{t}$26M-parameter model reaches validation performance $\bm{t}$27, while Adapter+ without PIZA reaches $\bm{t}$28 (Goto et al., 4 Oct 2025). PIZA-Adapter+ with only $\bm{t}$29M trainable parameters yields validation $\bm{t}$30, Test-A $\bm{t}$31, and Test-B $\bm{t}$32 (Goto et al., 4 Oct 2025). With Train-L, PIZA-Adapter+ further improves to validation $\bm{t}$33, Test-A $\bm{t}$34, and Test-B $\bm{t}$35 (Goto et al., 4 Oct 2025).

The most emphasized comparison is at high IoU. Relative to full fine-tuning under Train-S, PIZA-Adapter+ is slightly lower on validation mAcc but higher on $\bm{t}$36 and substantially higher on $\bm{t}$37: $\bm{t}$38 versus $\bm{t}$39 (Goto et al., 4 Oct 2025). The paper explicitly notes that this is “exactly where small-object precision matters” (Goto et al., 4 Oct 2025).

Across other PEFT schemes, adding PIZA also improves performance. On validation under Train-S, LoRA reaches $\bm{t}$40, while PIZA-LoRA reaches $\bm{t}$41 (Goto et al., 4 Oct 2025). CoOp reaches $\bm{t}$42, while PIZA-CoOp reaches $\bm{t}$43 (Goto et al., 4 Oct 2025).

Ablation studies attribute the gains to the PIZA mechanism itself. Removing the PIZA module reverts performance to baseline Adapter+, LoRA, or CoOp numbers (Goto et al., 4 Oct 2025). For Adapter+, the comparison is especially large: validation $\bm{t}$44 with PIZA versus $\bm{t}$45 without (Goto et al., 4 Oct 2025). A further ablation, “w/o embedding insertion,” retains EOS and progress supervision but does not inject $\bm{t}$46 into the adapters; performance drops slightly relative to full PIZA, indicating that conditioning feature computation on zoom embedding contributes beyond auxiliary supervision alone (Goto et al., 4 Oct 2025).

The trajectory design also matters. For PIZA-Adapter+ on Train-S, the full distribution-based generation method with average $\bm{t}$47 steps gives validation $\bm{t}$48 (Goto et al., 4 Oct 2025). Enforcing one step gives $\bm{t}$49, enforcing two steps gives $\bm{t}$50, and enforcing three steps gives $\bm{t}$51 (Goto et al., 4 Oct 2025). This indicates that multi-step zooming is important and that the automatically derived search-length distribution is slightly better than fixed-step schemes (Goto et al., 4 Oct 2025).

Comparison to sliding windows and tile grids is central to the method’s positioning. Sliding-window baselines use windows of size $\bm{t}$52 and strides $\bm{t}$53, and although they improve over vanilla GroundingDINO, they are significantly lower in accuracy than PIZA at comparable or higher compute (Goto et al., 4 Oct 2025). The best sliding window is reported as $\bm{t}$54 slower and uses $\bm{t}$55 more trainable parameters (Goto et al., 4 Oct 2025). Tile-grid baselines also lag behind PIZA, and the paper notes that first-step windows are effectively huge, around $\bm{t}$56, which is $\bm{t}$57 more area than typical PIZA first crops around $\bm{t}$58 (Goto et al., 4 Oct 2025).

PIZA is also evaluated on large multimodal models. With PIZA-LoRA at LoRA rank $\bm{t}$59, validation mAcc improves from $\bm{t}$60 to $\bm{t}$61 on Qwen2-VL-7B, from $\bm{t}$62 to $\bm{t}$63 on InternVL2.5-8B, and from $\bm{t}$64 to $\bm{t}$65 on LLaVA-NeXT-7B (Goto et al., 4 Oct 2025). On RefCOCO, where targets are standard-sized, PIZA-Adapter+ yields $\bm{t}$66 about $\bm{t}$67 compared with $\bm{t}$68 for Adapter+ and $\bm{t}$69 for full fine-tuning, while zero-shot is about $\bm{t}$70 (Goto et al., 4 Oct 2025). The paper interprets this as evidence that PIZA does not hurt performance on normal-sized objects because EOS is predicted after one step, effectively reducing the model to a single-shot detector (Goto et al., 4 Oct 2025).

7. Related concepts, limitations, and broader significance

PIZA is specifically proposed for small-object REC, but its design has an explicit conceptual affinity with iterative adapter frameworks developed in other domains. The most direct comparison in the provided material is with Story-Adapter, a training-free framework for long story visualization that iteratively regenerates each frame by conditioning on all images from the previous iteration through a Global Reference Cross-Attention module (Mao et al., 2024). Story-Adapter embodies progressive refinement, iterative conditioning, and adapter-style integration into a pre-trained diffusion model without retraining (Mao et al., 2024). The connection is conceptual rather than task-level: Story-Adapter conditions image generation on prior global story embeddings, whereas PIZA conditions localization on the history of prior zoom boxes (Mao et al., 2024). This suggests a broader pattern in which adapter modules are used to encode search or refinement state and inject it into frozen foundation models.

Several misconceptions can be clarified from the reported formulation. PIZA is not a new backbone, and it does not replace GroundingDINO’s detector head or cross-modal decoder (Goto et al., 4 Oct 2025). It is also not a sliding-window heuristic, since the next crop is predicted by a learned vision-LLM conditioned on the zoom trajectory rather than enumerated exhaustively (Goto et al., 4 Oct 2025). Nor is it purely an auxiliary stopping mechanism: the ablations show that injecting the zoom embedding into PEFT modules materially improves performance (Goto et al., 4 Oct 2025).

The reported limitations are also specific. PIZA still struggles under heavy occlusion or when targets are extremely close to many similar instances (Goto et al., 4 Oct 2025). Its inference requires two to three passes through the model, so although it is much more efficient than sliding windows, it remains costlier than a single forward pass (Goto et al., 4 Oct 2025). The method also depends on strong pre-trained detectors or grounders, and the trajectory-construction procedure uses matching to $\bm{t}$71, exponential weighting, and hand-tuned hyperparameters $\bm{t}$72, which the paper describes as effective but somewhat complex and heuristic (Goto et al., 4 Oct 2025).

The broader significance of PIZA lies in showing that parameter-efficient adaptation can be used to alter the operational behavior of a frozen vision-LLM from one-shot prediction to guided multi-step search. The evidence for generalization includes improvements not only for GroundingDINO but also for multiple large multimodal models, together with negligible degradation on standard REC benchmarks (Goto et al., 4 Oct 2025). The authors identify extensions beyond driving, including aerial or satellite imagery, surveillance, medical imaging, video data, and video architectures, as well as more adaptive zooming strategies and end-to-end learned zoom policies (Goto et al., 4 Oct 2025). A plausible implication is that PIZA’s main contribution is methodological: it frames extreme-scale localization as a short autoregressive search process that can be layered onto existing grounding architectures through compact conditioning modules rather than through wholesale retraining.