InstaDA: Dual-Agent Augmentation & Privacy

• InstaDA is a framework with two distinct constructs: a training-free dual-agent augmentation system for instance segmentation and a privacy-preserving method via geometric mixup.
• The dual-agent system leverages a Text-Agent for generating diverse prompts and an Image-Agent for realigning data, resulting in AP improvements (e.g., +4.0 in box AP).
• In its privacy variant, InstaDA augments InstaHide with geometric transformations, though it remains vulnerable to advanced reconstruction attacks like fusion-denoising.

In the papers represented here, “InstaDA” refers to two distinct constructs. In instance segmentation, InstaDA is a training-free Dual-Agent system for augmenting datasets through a Text-Agent and an Image-Agent, with the explicit goal of increasing diversity while realigning synthetic data to the original training distribution (Hou et al., 3 Sep 2025). In privacy-preserving training, InstaDA denotes InstaHide with Data Augmentation, a variant of InstaHide in which private images are geometrically augmented before mixup and sign-flip encryption (Luo et al., 2021). The two usages are unrelated in objective and methodology: one is an augmentation pipeline for detection and segmentation, the other a protection mechanism whose security is then analyzed and attacked.

1. Terminological scope and neighboring usages

The 2025 instance-segmentation usage defines InstaDA as a Dual-Agent augmentation system operating through two independent and automated workflows whose outputs are merged via Copy-Paste into standard detectors such as CenterNet2 + Swin-L in Detectron2 (Hou et al., 3 Sep 2025). The 2021 privacy usage defines InstaDA as InstaHide augmented with random geometric transformations, producing ciphertext images and mixed labels from private and public components (Luo et al., 2021).

Adjacent literature contains related instance-level ideas without using the same name as the primary framework. “Dynamic Instance Domain Adaptation” (DIDA) treats each training example as its own “fine domain” and replaces explicit domain-level alignment with instance-adaptive residual calibration (Deng et al., 2022). A separate exposition of IMED presents it as realizing an instance-level UDA method akin to “InstaDA,” but the framework itself is named “Instance-aware Model Ensemble With Distillation” (Wu et al., 2022). This suggests that, in current usage, “InstaDA” is not a stable field-wide term but rather a context-dependent label tied to specific papers.

2. InstaDA as a dual-agent augmentation system for instance segmentation

In modern computer vision, instance segmentation is described as one of the most annotation-hungry tasks because every object instance in every image must be outlined at the pixel level, and large vocabularies such as LVIS’s 1,203 categories exacerbate severe class imbalance (Hou et al., 3 Sep 2025). The motivating example given is that “rare” LVIS classes may appear $228\times$ less often than “frequent” ones. Prior diffusion-plus-Copy-Paste systems such as X-Paste and DiverGen are reported to improve box AP by $+3$–$4$, but two limitations are emphasized: prompt engineering often relies on shallow “diversify prompts” instructions to LLMs, and the existing training set is largely ignored during generation.

InstaDA addresses these two gaps with a Dual-Agent system. The Text-Agent (T-Agent) is focused on maximizing visual diversity through deep LLM–diffusion collaboration. The Image-Agent (I-Agent) is designed to enrich and realign the generated data to the original training distribution. The stated division of labor is explicit: T-Agent systematically engineers prompts to cover the “space” of visual diversity—color, pose, background, and lighting—whereas I-Agent uses each training image as a source of structure, semantic cues, and latent initialization, ensuring that generated instances remain well aligned with the original data distribution.

The system is training-free and its two agents run as fully automated, independent pipelines. Their outputs are finally merged via Copy-Paste into the downstream instance-segmentation training workflow. A plausible implication is that the framework treats augmentation not as a single generator but as a coordination problem between prompt-space exploration and image-conditioned distribution correction.

3. T-Agent and I-Agent workflows

The T-Agent generates category-conditioned crops annotated with pseudo-masks and captions. Its first stage is a two-stage LLM collaboration: DeepSeek-R1 extracts a set of visual “elements” $\{e_1,\dots,e_K\}$ that influence diversity, and DeepSeek-V3 composes prompts by sampling combinations of these elements. For category $c$, the initial prompt is

$$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$

Generation is then performed with Flux and SD3.5 using LoRA reduction to 8 sampling steps. Each generated image is decoded to obtain a bounding box and mask via BiRefNet, with TensorRT acceleration reported from $4.4\,\text{s}\to0.11\,\text{s}$ (Hou et al., 3 Sep 2025).

The distinctive component is the Prompt Rethink feedback loop. For a generated image $I^{(t)}$, InstaDA computes a CLIP dual-similarity score consisting of text consistency with the prompt and image consistency with real images of class $c$:

$$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$

If either score falls below its threshold, the prompt is marked suboptimal and refined either by simplification or by re-sampling elements. Only images passing both CLIP thresholds are retained, and their crops are pasted onto real backgrounds via Copy-Paste.

The I-Agent is explicitly complementary. Rather than prioritizing diversity, it conditions on each real training image $+3$0 to preserve fidelity to the original distribution. Its inputs include $+3$1, edge maps $+3$2 and $+3$3, a fused control

$+3$4

the latent initialization $+3$5, and an adaptive denoising strength $+3$6. Diffusion then produces

$+3$7

Because masks drift after generation, InstaDA applies SAM with the original bounding box as spatial prompt to obtain a new mask $+3$8. Filtration is then performed proportionally within class. For each mask $+3$9, the percentile is

$4$0

and for image $4$1,

$4$2

All $4$3 are sorted and only the top $4$4 are kept. The retained I-Agent instances form a “source pool,” which is used together with the T-Agent’s synthetic pool in the final Copy-Paste stage. Qualitative figures are reported to show richer style, pose, and lighting variations from T-Agent and subtle yet semantically faithful transformations by I-Agent.

4. Empirical performance and ablations of the augmentation InstaDA

The evaluation is conducted on LVIS 1.0 with $4$5 train and $4$6 validation images spanning $4$7 categories partitioned into rare, common, and frequent subsets (Hou et al., 3 Sep 2025). The baseline is CenterNet2 + Swin-L, trained for $4$8 iterations with batch $4$9, learning rate $\{e_1,\dots,e_K\}$0, and $\{e_1,\dots,e_K\}$1. Reported metrics are $\{e_1,\dots,e_K\}$2, $\{e_1,\dots,e_K\}$3, and category-frequency-stratified $\{e_1,\dots,e_K\}$4 and $\{e_1,\dots,e_K\}$5.

Method	$\{e_1,\dots,e_K\}$6	$\{e_1,\dots,e_K\}$7
CenterNet2	47.5	42.3
X-Paste	50.9	45.4
DiverGen	51.2	45.5
InstaDA	51.5	45.6

Relative to the baseline, InstaDA improves box AP by $\{e_1,\dots,e_K\}$8 and mask AP by $\{e_1,\dots,e_K\}$9 (Hou et al., 3 Sep 2025). Relative to DiverGen, it improves box AP by $c$0 and mask AP by $c$1. The class-specific numbers further show $c$2, $c$3, $c$4, $c$5, $c$6, and $c$7. The reported gains over DiverGen include $c$8 in box AP on common categories, $c$9 in mask AP on common categories, and $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$0 in mask AP on frequent categories.

The ablation results are used to separate the contributions of diversity, filtration, and distribution realignment. With $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$1 synthetic samples versus DiverGen’s $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$2, the T-Agent still yields $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$3 box AP and $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$4 mask AP, and this is interpreted as evidence that visual diversity is more valuable than sheer volume. Without T-Agent, increasing synthetic data from $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$5 leads to overfitting and AP degradation, whereas with T-Agent diversity AP rises steadily, for example by $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$6 box AP at $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$7, before plateauing.

Additional ablations isolate the other components. Integrating I-Agent atop standard Copy-Paste adds $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$8 box AP and $$P^{(0)} = \text{“a photo of a single \{c\} with [}e_i, e_j, \dots\text{].”}$$9 mask AP. Removing Prompt Rethink lowers $4.4\,\text{s}\to0.11\,\text{s}$0 from $4.4\,\text{s}\to0.11\,\text{s}$1 and $4.4\,\text{s}\to0.11\,\text{s}$2 from $4.4\,\text{s}\to0.11\,\text{s}$3. CLIP dual-similarity outperforms single-thresholded $4.4\,\text{s}\to0.11\,\text{s}$4 or $4.4\,\text{s}\to0.11\,\text{s}$5, yielding $4.4\,\text{s}\to0.11\,\text{s}$6 versus $4.4\,\text{s}\to0.11\,\text{s}$7 or $4.4\,\text{s}\to0.11\,\text{s}$8. In the segmentation stage, BiRefNet yields $4.4\,\text{s}\to0.11\,\text{s}$9 versus SAM-bg’s $I^{(t)}$0, especially on complex backgrounds. The conclusion drawn in the paper is that deep LLM–diffusion prompt planning, feedback-driven prompt revision, and training-image conditioning jointly address diversity and domain fidelity with no additional training overhead.

5. InstaDA as InstaHide with Data Augmentation

In the privacy literature, InstaDA is defined as InstaHide augmented with geometric transformations prior to encryption (Luo et al., 2021). Let the private image set be $I^{(t)}$1, with each $I^{(t)}$2 and one-hot label $I^{(t)}$3. For each $I^{(t)}$4, one generates $I^{(t)}$5 random geometric augmentations $I^{(t)}$6 and sets $I^{(t)}$7. A transformation $I^{(t)}$8 is called an $I^{(t)}$9-augmentation if every pixel can move by at most $c$0 of the image width or height.

All augmented images are stacked into

$c$1

and then shuffled to remove index structure. For each shuffled $c$2, one samples a second augmented private image $c$3, selects $c$4 public images $c$5, draws nonnegative mixup weights $c$6 summing to one, and samples a sign-flip mask $c$7. The published ciphertext is

$c$8

or, in compact form,

$c$9

The released label is the corresponding mixture of the two private labels,

$$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$0

The intended effect of the augmentation is to make Carlini et al.’s original reconstruction attack fail. However, the 2021 study asks whether InstaHide with data augmentation is secure and answers negatively by proposing a fusion-denoising attack. This places InstaDA in a different conceptual role from the 2025 augmentation framework: here it is itself the object of attack rather than a tool for data generation.

6. Fusion-denoising attack and security analysis

The threat model assumes that the adversary sees all ciphertexts $$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$1 and labels $$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$2, and knows $$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$3 and the augmentation family, but not the private images or sign masks (Luo et al., 2021). The attack proceeds in three stages: remove sign flips by taking $$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$4, cluster the resulting images into groups that share the same private source, and reconstruct a private image from each cluster using a Fusion–Denoising Network (FDN).

The clustering module is a comparative network. It takes a pair $$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$5 and outputs a similarity score $$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$6. Architecturally it is a Siamese network with two multi-resolution towers: a high-resolution branch that crops the center $$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$7 patch of each $$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$8 image, and a low-resolution branch that downsamples the full $$s_{\text{text}} = \mathrm{CLIP}_{\text{text}}(P^{(t)}, \text{caption of } I^{(t)}), \qquad s_{\text{img}} = \mathrm{CLIP}_{\text{image}}(I^{(t)}, \text{real images of class } c).$$9 image to $+3$00. Each branch passes through a small ResNet block; the outputs are concatenated and mapped through a sigmoid dense layer. Training uses binary cross-entropy,

$+3$01

After training, a weighted graph with edges $+3$02 is built and dense clusters are extracted by spectral or clique-finding methods.

The FDN then reconstructs the private image from a cluster $+3$03. Each image is first re-weighted by its private mixup coefficient, $+3$04. A noise-variance correction is then applied: letting $+3$05, define $+3$06 and $+3$07. An image-relaxing block performs stride-2 convolution followed by stride-2 transposed convolution, which is described as a lightweight registration module that implicitly aligns patches across augmentations. Fusion is either choose-max or average, with “max” reported to work better when $+3$08 and “average” when $+3$09. The denoising backbone is a Residual Non-Local Attention Network with loss

$+3$10

The experiments use CIFAR-10, CIFAR-100, STL-10, and CelebA, all resized to $+3$11, with $+3$12 private+public images per cipher unless otherwise stated and $+3$13 augmentations per private image. The main metric is SSIM with window size $+3$14, and PSNR trends are stated to be similar. On CIFAR-100, averaged over $+3$15 test images, the full FDN substantially exceeds both Carlini et al.’s original attack (CA) and the variant CA-CN that replaces clustering with the comparative network. For varying cluster size $+3$16, the reported SSIM at $+3$17 is $+3$18 for FDN, $+3$19 for CA-CN, and $+3$20 for CA. Under varying augmentation level $+3$21 at $+3$22, FDN retains SSIM $+3$23 at $+3$24 and $+3$25 at $+3$26, compared with $+3$27 for CA-CN and $+3$28 for CA.

The security conclusion is direct: data augmentation alone does not prevent reconstruction, and InstaDA provides no meaningful extra defense beyond baseline InstaHide (Luo et al., 2021). The reason given is that feature-space registration, multi-cipher fusion, and variance-aware reweighting allow the attacker to amplify the true signal and suppress noise. Suggested future defenses include stronger nonlinear per-pixel encryption than sign flips, per-instance random feature dropout or projection into a secret subspace before mixup, provable differential-privacy mechanisms, and hybrid cryptographic MPC/HE training.