---
title: 'PIZA: Progressive Iterative Zooming Adapter'
url: https://www.emergentmind.com/topics/progressive-iterative-zooming-adapter-piza
type: topic
---

# PIZA: Progressive Iterative Zooming Adapter

Searching arXiv for the specified papers to ground the article in the cited sources.
Progressive-Iterative Zooming Adapter (PIZA) is an adapter module for parameter-efficient fine-tuning in referring expression comprehension (REC) that enables models to progressively zoom in and localize small objects. It is introduced together with the small object REC (SOREC) dataset for driving scenarios, and is formulated as a mechanism that converts a static, single-shot grounding model into a search-based model conditioned on both language and the history of previous zoom boxes [2510.03701]. In the reported instantiation, PIZA is applied to GroundingDINO and operates by encoding the sequence of past bounding boxes into a compact zooming-step embedding, injecting that embedding into parameter-efficient adaptation components such as CoOp prompts, LoRA bottlenecks, or Adapter+ layers while keeping the base model mostly frozen [2510.03701]. The term also admits a broader conceptual connection to training-free iterative adapter frameworks for generative modeling, especially insofar as both rely on progressive refinement and adapter-style conditioning, but PIZA itself is presented for small-object REC rather than story visualization [2410.06244].

## 1. Problem setting and definition

Referring expression comprehension takes an image $\bm{x} \in \mathbb{R}^{W \times H \times C}$ and a natural-language referring expression $\bm{t}$, and predicts a bounding box $\hat{\bm{b}}$ for the described object:
$$
\hat{\bm{b}} = F(\bm{x}, \bm{t}), \quad \bm{b} = (x_0, y_0, x_1, y_1),
$$
where $(x_0, y_0)$ and $(x_1, y_1)$ are top-left and bottom-right coordinates [2510.03701]. REC models must align the semantics of $\bm{t}$ with visual content and localize a unique object. Classical REC benchmarks such as RefCOCO/+/g mostly involve medium or large objects in COCO images, and modern vision-language detectors like GroundingDINO reach more than $90\%$ Acc on RefCOCO, so the standard-sized REC problem is described as essentially solved [2510.03701].

PIZA is motivated by a different regime: extremely small targets in high-resolution driving scenes. In SOREC, source images are high-resolution street scenes with average resolution $3407 \times 2470$, and target objects include traffic signs, lights, and distant vehicles [2510.03701]. The dataset description states that each bounding box typically occupies approximately $0.05\%$ of the entire image area, and all target bounding boxes occupy less than $1\%$ of the image area [2510.03701]. Formally, the small-object regime is characterized by a ground-truth box $\bm{b}^*$ such that
$$
|\bm{b}^*| \ll WH, \quad |\bm{b}| = (x_1 - x_0)(y_1 - y_0),
$$
with area ratio $r^* = |\bm{b}^*|/(WH) \in (0, 0.01)$ and often around $5\times 10^{-4}$ [2510.03701].

The central definition of PIZA is an extension of a base grounding model $F$ into an autoregressive zooming model:
$$
\hat{\bm{b}}_{i+1} = F_{\text{-piza}}(\bm{x}_i, \bm{t}, \bm{b}_{0:i}),
$$
where $\bm{b}_{0:i} = (\bm{b}_0, \dots, \bm{b}_i)$ is the history of previously predicted boxes [2510.03701]. The process begins from the full-image box $\bm{b}_0 = (0,0,W,H)$ and proceeds by repeated cropping and refinement until a learned end-of-search criterion terminates the sequence. This makes PIZA both progressive, in the sense of coarse-to-fine cropping, and iterative, in the sense that each prediction conditions on prior zoom steps [2510.03701].

## 2. Motivation: small-object REC in driving scenes

The difficulty of small-object REC is attributed to four factors. First, resolution and feature sparsity arise because downsampling large images for transformer vision backbones such as Swin-T severely reduces the effective resolution of tiny objects; such objects may collapse to one or two patch tokens or be lost entirely [2510.03701]. Second, there is a mismatch between pre-training and fine-tuning: GroundingDINO is pre-trained on O365, GoldG, GRIT, V3Det, and related corpora with much larger object-size distributions, whereas SOREC contains very small targets and long descriptions, with average expression length $25.5$ words versus $3.52$ in RefCOCO [2510.03701]. Third, driving scenes are cluttered and contain many similar objects, so descriptions frequently rely on relations such as “small blue sign on the right side of the road, below the speed limit sign,” which are difficult to resolve when the model only has a coarse global view [2510.03701]. Fourth, computational constraints make naive sliding windows and dense tiling unattractive; these baselines are experimentally reported as both slower and less accurate than PIZA [2510.03701].

Within this setting, PIZA is positioned not as a new backbone but as an adapter-based mechanism that makes an existing model actively zoom into promising regions [2510.03701]. The base model parameters remain mostly frozen, while the system trains only the PIZA module and the selected parameter-efficient fine-tuning (PEFT) components: CoOp prompts, LoRA matrices, Adapter+ layers, and LayerNorms depending on the variant [2510.03701]. The reported best configuration, PIZA-Adapter+, uses only $3.5$M trainable parameters on top of a $173$M-parameter GroundingDINO backbone [2510.03701].

A plausible implication is that PIZA addresses the small-object regime primarily by altering the inference geometry rather than by expanding the visual backbone. Rather than forcing a single pass to resolve a target occupying approximately $0.05\%$ of an image, the method transforms localization into a short search process whose average ground-truth length on SOREC is $2.11$ steps [2510.03701].

## 3. Architecture and zooming-step representation

The base architecture in the main experiments is GroundingDINO in the MM-GroundingDINO implementation. It has five components: a BERT text encoder, a Swin-T image encoder producing multi-scale feature maps, a feature enhancer with text-to-image and image-to-text cross-attention together with self-attention and feed-forward networks, a language-guided query selection module, and a cross-modality decoder that refines queries into grounded boxes and class logits [2510.03701]. PEFT methods are applied in the feature enhancer and decoder [2510.03701].

PIZA adds an auxiliary module that maps the sequence of zoom boxes $\bm{b}_{0:i}$ to a zooming-step embedding $\bm{h}\in\mathbb{R}^d$ [2510.03701]. For each box $\bm{b}_j = (x_0^{(j)}, y_0^{(j)}, x_1^{(j)}, y_1^{(j)})$, a $6$-dimensional low-level feature vector $\bm{l}_j$ is computed from normalized area, relative size to the previous step, normalized width and height, and normalized center coordinates:
$$
s_j = |\bm{b}_j|/|\bm{b}_0| = \frac{(x^{(j)}_1 - x^{(j)}_0)(y^{(j)}_1 - y^{(j)}_0)}{WH},
$$
$$
r_0 = 1,\quad r_j = \frac{|\bm{b}_j|}{|\bm{b}_{j-1}|},
$$
$$
w_j = \frac{x^{(j)}_1 - x^{(j)}_0}{W},\quad
h_j = \frac{y^{(j)}_1 - y^{(j)}_0}{H},
$$
$$
\bar{x}_j = \frac{x^{(j)}_0 + x^{(j)}_1}{2W},\quad
\bar{y}_j = \frac{y^{(j)}_0 + y^{(j)}_1}{2H}.
$$
These vectors form a sequence $\bm{l}_{0:i} \in \mathbb{R}^{(i+1)\times 6}$ [2510.03701].

The sequence is then processed by learnable Fourier embeddings, followed by a transformer encoder over $\bm{l}_{0:i}$, and average pooling over positions to obtain a single embedding $\bm{h}\in\mathbb{R}^d$ [2510.03701]. The main text uses $d=16$, and the parameter count for this module is reported as $0.27$M [2510.03701]. Two auxiliary heads are trained from the pooled embedding: an EOS head for binary classification,
$$
\hat{y}_{i+1} \in \{\text{[CONT]}, \text{[EOS]}\},
$$
and a progress head for scalar regression,
$$
\hat{z}_{i+1} \in [0,1],
$$
representing normalized progress along the search trajectory [2510.03701]. At inference time, the EOS head is used to stop the zoom loop [2510.03701].

This design makes the search history explicit. Rather than conditioning on cropped pixels alone, the model receives a compact representation of how much zooming has already occurred, how the crop geometry has evolved, and where the current crop lies within the original image. This suggests that PIZA operationalizes “zoom state” as a first-class conditioning signal.

## 4. Progressive-iterative search process and training formulation

The inference process begins with
$$
\bm{b}_0 = (0,0,W,H),
$$
so the initial crop is the full image [2510.03701]. At step $i$, low-level features $\bm{l}_{0:i}$ are computed from the history of boxes, the PIZA module produces $\bm{h}_i$, EOS probability $\hat{y}_{i+1}$, and progress estimate $\hat{z}_{i+1}$, the model predicts the next box $\hat{\bm{b}}_{i+1}$ in global coordinates, and the corresponding crop $\bm{x}_{i+1}$ is extracted [2510.03701]. If EOS is predicted, or if a maximum step count is reached, the process stops; on SOREC, two to three steps are described as sufficient in practice [2510.03701].

The paper formalizes the search sequence as
$$
P = (\bm{b}_0, \bm{b}_1, \dots, \bm{b}_T),
$$
with $\bm{b}_T \approx \bm{b}^*$ and $\bm{b}_i$ covering $\bm{b}_j$ for $i<j$ [2510.03701]. Training does not rely solely on the original dataset $\mathcal{D}$ of triples $(\bm{x},\bm{t},\bm{b}^*)$. Instead, PIZA constructs an extended dataset $\mathcal{E}$ containing search processes,
$$
(\bm{x}, \bm{t}, P^*, \bm{y}^*, \bm{z}^*),
$$
where $P^*=(\bm{b}_0^*,\dots,\bm{b}_{T^*}^*)$, $\bm{b}_0^*=(0,0,W,H)$, $\bm{b}_{T^*}^*=\bm{b}^*$, $y_j^*\in\{\text{CONT},\text{EOS}\}$, and $z_j^*=j/T^*$ [2510.03701]. The labels are
$$
y_j^* =
\begin{cases}
\text{[CONT]} & 0 \le j < T^* \\
\text{[EOS]} & j = T^*
\end{cases},
\quad
z_j^* = \frac{j}{T^*}.
$$

Intermediate boxes $\bm{b}_j^*$ are generated so that their area ratios match the distribution $p(r)$ estimated from pre-training datasets such as O365 and GoldG [2510.03701]. For a target with area ratio $r^*=|\bm{b}^*|/(WH)$, the method samples $r_k\sim p(r)$, chooses the number of zoom steps
$$
T^* = \arg\min_T \left(\frac{1}{r^* \prod_{k=1}^{T} r_k} - 1 \right),
$$
and defines zoom factors using an exponentially weighted formulation:
$$
z_j^* =
\left(
\frac{1}{r^*\prod_{k=1}^{T^*} r_k^{\omega_k}}
\right)^{\frac{1}{T^*}} r_j^{-1},
\quad
\omega_k = \frac{\lambda_1 e^{-\lambda_2 k}}{\sum_{k'=1}^{T^*} \lambda_1 e^{-\lambda_2 k'}}.
$$
Box areas are then given by
$$
S_j^* = \left( \prod_{k=1}^{T^*-j} z_{T^*-k} \right) |\bm{b}^*|,
$$
with aspect ratio interpolation
$$
a_j = \frac{W}{H} - \left(\frac{W}{H} - 1\right)\frac{WH - S_j^*}{WH - S_{T^*-1}^*},
$$
followed by
$$
w_j^* = a_j \sqrt{S_j^*},\quad
h_j^* = a_j^{-1} \sqrt{S_j^*}.
$$
Centers are aligned to the ground-truth object center subject to image-bound constraints [2510.03701].

Training samples a random intermediate step $i\in\{1,\dots,T^*-1\}$ and optimizes the base GroundingDINO losses together with the EOS and progress losses [2510.03701]. The PIZA-augmented loss is
$$
\mathcal{L} = \mathcal{L}_{\text{GDINO}} + \lambda_{\text{EOS}}\mathcal{L}_{\text{EOS}} + \lambda_{\text{prog}}\mathcal{L}_{\text{prog}},
$$
where $\mathcal{L}_{\text{GDINO}}$ includes localization and contrastive components, $\mathcal{L}_{\text{EOS}}$ is binary cross-entropy, and $\mathcal{L}_{\text{prog}}$ is mean squared error [2510.03701].

## 5. PEFT variants and integration into GroundingDINO

PIZA is described as PEFT-agnostic and is instantiated in four forms: PIZA-CoOp, PIZA-LoRA, PIZA-Adapter+, and PIZA-VPT, with the main experiments emphasizing PIZA-Adapter+ as the strongest variant [2510.03701]. In all cases, the role of PIZA is to make the chosen adaptation mechanism zoom-aware by injecting the zooming-step embedding $\bm{h}$.

PIZA-CoOp augments text-side prompts. Baseline CoOp prepends learnable embeddings $\bm{e}$ to text tokens:
$$
G(\bm{x}, \bm{t}) = F(\bm{x}, [\bm{e}, \bm{t}]).
$$
PIZA-CoOp further inserts the zoom embedding as additional prompt tokens:
$$
G_{\text{-piza}}(\bm{x}, \bm{t}) = F(\bm{x}, [\bm{e}, H(\bm{h}), \bm{t}]),
$$
where $H$ is a stack of $L$ linear layers and experiments use $L=8$ [2510.03701]. The trainable parameters comprise the prompt embeddings, the linear layers $H$, and LayerNorms in the text encoder, for a total of approximately $0.9$M parameters [2510.03701].

PIZA-LoRA modifies attention projections. Standard LoRA changes a linear map $W$ to $W+BA$, so for input $\bm{x}$ the output is
$$
W\bm{x} + BA\bm{x}.
$$
PIZA-LoRA adds a zoom-conditioned term in the LoRA bottleneck:
$$
W\bm{x} + BA\bm{x} + BC\bm{h},
$$
with $A\in\mathbb{R}^{r\times d_{\text{in}}}$, $B\in\mathbb{R}^{d_{\text{out}}\times r}$, and $C\in\mathbb{R}^{r\times d}$ [2510.03701]. It is applied to text-to-image and image-to-text cross-attention modules in the feature enhancer and to self-attention modules in both feature enhancer and decoder. The main rank is $r=16$, and total trainable parameters are $1.5$M for PIZA-LoRA versus $1.3$M for vanilla LoRA [2510.03701].

PIZA-Adapter+ conditions bottleneck adapters in the visual transformer stream. Adapter layers are placed after self-attention and feed-forward blocks in the feature enhancer, with bottleneck dimension $d_{\text{bottleneck}}=256$ and standard form
$$
\text{Adapter}(\bm{h}_{\text{feat}})
= \bm{h}_{\text{feat}} + W_{\text{up}}\,\sigma(W_{\text{down}}\,\bm{h}_{\text{feat}}).
$$
PIZA-Adapter+ introduces two conditioning designs [2510.03701]. Type A, used when data is small, uses the progress value $\hat{z}\in[0,1]$ passed through a time embedding module with Fourier features and an MLP, adding the resulting conditioning vector to the output of a channel-wise scaling layer in the adapter [2510.03701]. Type B, used when data is larger, uses the full embedding $\bm{h}$ and drops the progress head [2510.03701]. In both cases, the adapter hidden representation is modulated by $\bm{h}$ or $\hat{z}$, making the block step-aware [2510.03701]. Total trainable parameters are $3.5$M, compared with $3.3$M for Adapter+ without PIZA [2510.03701].

The following table summarizes the reported PEFT variants.

| Variant | Injection mechanism | Trainable parameters |
|---|---|---:|
| PIZA-CoOp | Zoom embedding inserted as additional prompt tokens | ~0.9M |
| PIZA-LoRA | Adds $BC\bm{h}$ in LoRA bottleneck for attention projections | 1.5M |
| PIZA-Adapter+ | Conditions post-attention and post-FFN adapters with $\bm{h}$ or $\hat{z}$ | 3.5M |
| PIZA-VPT | Visual prompt tuning variant used in appendix | not specified here |

In integration terms, PIZA-CoOp acts on the text stream, PIZA-LoRA on visual and cross-modal attention projections, and PIZA-Adapter+ mainly on the visual stream of the feature enhancer [2510.03701]. The detector head of GroundingDINO remains responsible for predicting the next zoom box [2510.03701].

## 6. Empirical performance, ablations, and comparisons

SOREC uses the metrics $\text{mAcc}$, $\text{Acc}_{50}$, and $\text{Acc}_{75}$, where $\text{mAcc}$ is mean accuracy over IoU thresholds from $0.50$ to $0.95$ in increments of $0.05$ [2510.03701]. The dataset split comprises Train-S with $10$k expressions, Train-L with $61{,}369$, Validation with $10{,}712$, Test-A with $10{,}815$, and Test-B with $17{,}104$ [2510.03701].

On SOREC with GroundingDINO and limited data (Train-S), zero-shot GroundingDINO attains validation performance of mAcc $4.0$, $\text{Acc}_{50}$ $6.4$, and $\text{Acc}_{75}$ $1.8$ [2510.03701]. Full fine-tuning of the $173$M-parameter model reaches validation performance $58.6 / 51.7 / 30.2$, while Adapter+ without PIZA reaches $55.4 / 48.1 / 24.8$ [2510.03701]. PIZA-Adapter+ with only $3.5$M trainable parameters yields validation $56.8 / 53.5 / 41.8$, Test-A $61.6 / 59.6 / 50.1$, and Test-B $49.5 / 45.9 / 34.1$ [2510.03701]. With Train-L, PIZA-Adapter+ further improves to validation $66.5 / 60.6 / 42.9$, Test-A $69.9 / 66.2 / 51.7$, and Test-B $58.6 / 52.2 / 33.6$ [2510.03701].

The most emphasized comparison is at high IoU. Relative to full fine-tuning under Train-S, PIZA-Adapter+ is slightly lower on validation mAcc but higher on $\text{Acc}_{50}$ and substantially higher on $\text{Acc}_{75}$: $41.8$ versus $30.2$ [2510.03701]. The paper explicitly notes that this is “exactly where small-object precision matters” [2510.03701].

Across other PEFT schemes, adding PIZA also improves performance. On validation under Train-S, LoRA reaches $44.3 / 38.5 / 21.8$, while PIZA-LoRA reaches $48.0 / 44.7 / 34.9$ [2510.03701]. CoOp reaches $41.5 / 36.1 / 20.4$, while PIZA-CoOp reaches $42.2 / 39.1 / 29.7$ [2510.03701].

Ablation studies attribute the gains to the PIZA mechanism itself. Removing the PIZA module reverts performance to baseline Adapter+, LoRA, or CoOp numbers [2510.03701]. For Adapter+, the comparison is especially large: validation $56.8 / 53.5 / 41.8$ with PIZA versus $55.4 / 48.1 / 24.8$ without [2510.03701]. A further ablation, “w/o embedding insertion,” retains EOS and progress supervision but does not inject $\bm{h}$ into the adapters; performance drops slightly relative to full PIZA, indicating that conditioning feature computation on zoom embedding contributes beyond auxiliary supervision alone [2510.03701].

The trajectory design also matters. For PIZA-Adapter+ on Train-S, the full distribution-based generation method with average $2.11$ steps gives validation $56.8 / 53.5 / 41.8$ [2510.03701]. Enforcing one step gives $54.6 / 47.5 / 25.5$, enforcing two steps gives $56.9 / 53.1 / 40.8$, and enforcing three steps gives $54.5 / 50.3 / 39.0$ [2510.03701]. This indicates that multi-step zooming is important and that the automatically derived search-length distribution is slightly better than fixed-step schemes [2510.03701].

Comparison to sliding windows and tile grids is central to the method’s positioning. Sliding-window baselines use windows of size $\{500,750,1000,2000\}$ and strides $S\in\{W,W/2\}$, and although they improve over vanilla GroundingDINO, they are significantly lower in accuracy than PIZA at comparable or higher compute [2510.03701]. The best sliding window is reported as $7.3\times$ slower and uses $49.4\times$ more trainable parameters [2510.03701]. Tile-grid baselines also lag behind PIZA, and the paper notes that first-step windows are effectively huge, around $3407\times 2470$, which is $36\times$ more area than typical PIZA first crops around $500\times 500$ [2510.03701].

PIZA is also evaluated on large multimodal models. With PIZA-LoRA at LoRA rank $128$, validation mAcc improves from $3.8$ to $27.9$ on Qwen2-VL-7B, from $0.2$ to $20.7$ on InternVL2.5-8B, and from $0.7$ to $10.8$ on LLaVA-NeXT-7B [2510.03701]. On RefCOCO, where targets are standard-sized, PIZA-Adapter+ yields $\text{Acc}_{50}$ about $87.4$ compared with $86.9$ for Adapter+ and $89.2$ for full fine-tuning, while zero-shot is about $50.4$ [2510.03701]. The paper interprets this as evidence that PIZA does not hurt performance on normal-sized objects because EOS is predicted after one step, effectively reducing the model to a single-shot detector [2510.03701].

## 7. Related concepts, limitations, and broader significance

PIZA is specifically proposed for small-object REC, but its design has an explicit conceptual affinity with iterative adapter frameworks developed in other domains. The most direct comparison in the provided material is with Story-Adapter, a training-free framework for long story visualization that iteratively regenerates each frame by conditioning on all images from the previous iteration through a Global Reference Cross-Attention module [2410.06244]. Story-Adapter embodies progressive refinement, iterative conditioning, and adapter-style integration into a pre-trained diffusion model without retraining [2410.06244]. The connection is conceptual rather than task-level: Story-Adapter conditions image generation on prior global story embeddings, whereas PIZA conditions localization on the history of prior zoom boxes [2410.06244]. This suggests a broader pattern in which adapter modules are used to encode search or refinement state and inject it into frozen foundation models.

Several misconceptions can be clarified from the reported formulation. PIZA is not a new backbone, and it does not replace GroundingDINO’s detector head or cross-modal decoder [2510.03701]. It is also not a sliding-window heuristic, since the next crop is predicted by a learned vision-language model conditioned on the zoom trajectory rather than enumerated exhaustively [2510.03701]. Nor is it purely an auxiliary stopping mechanism: the ablations show that injecting the zoom embedding into PEFT modules materially improves performance [2510.03701].

The reported limitations are also specific. PIZA still struggles under heavy occlusion or when targets are extremely close to many similar instances [2510.03701]. Its inference requires two to three passes through the model, so although it is much more efficient than sliding windows, it remains costlier than a single forward pass [2510.03701]. The method also depends on strong pre-trained detectors or grounders, and the trajectory-construction procedure uses matching to $p(r)$, exponential weighting, and hand-tuned hyperparameters $\lambda_1,\lambda_2$, which the paper describes as effective but somewhat complex and heuristic [2510.03701].

The broader significance of PIZA lies in showing that parameter-efficient adaptation can be used to alter the operational behavior of a frozen vision-language model from one-shot prediction to guided multi-step search. The evidence for generalization includes improvements not only for GroundingDINO but also for multiple large multimodal models, together with negligible degradation on standard REC benchmarks [2510.03701]. The authors identify extensions beyond driving, including aerial or satellite imagery, surveillance, medical imaging, video data, and video architectures, as well as more adaptive zooming strategies and end-to-end learned zoom policies [2510.03701]. A plausible implication is that PIZA’s main contribution is methodological: it frames extreme-scale localization as a short autoregressive search process that can be layered onto existing grounding architectures through compact conditioning modules rather than through wholesale retraining.

Source: https://www.emergentmind.com/topics/progressive-iterative-zooming-adapter-piza