AutoMIA: Automated MIA & Mirror Art

• AutoMIA is an automated framework for membership inference attack discovery and mirror illusion art design using agent-based closed-loop optimization.
• It eliminates manual feature engineering by employing iterative exploration and exploitation agents to navigate high-dimensional search spaces with interpretable outputs.
• The system integrates LLM-driven strategy generation for MIA and differentiable voxel-based optimization for art, achieving superior performance and rapid convergence.

AutoMIA refers to several automated, agentic computational frameworks for (1) membership inference attacks (MIA) in machine learning and (2) the inverse design of mirror illusion art. Across both domains, AutoMIA is characterized by the removal of manual engineering through agent-driven closed-loop optimization, enabling exploration and refinement of high-dimensional search spaces with interpretable program outputs. The key variants—AutoMIA for automated MIA design and AutoMIA for mirror illusion art—rely on distinct algorithmic pipelines, with the former utilizing LLM-driven strategy generation and the latter leveraging differentiable voxel-based optimization with custom regularization mechanisms.

1. Automated Membership Inference Attack Discovery

AutoMIA, as introduced in "Automated Membership Inference Attacks: Discovering MIA Signal Computations using LLM Agents" (Tran et al., 19 Mar 2026) and further refined in "AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration" (Liu et al., 1 Apr 2026), automates the core signal-design step in membership inference attacks (MIAs). MIAs aim to determine whether a given data point $x$ was used in the training set of a victim model $M_\theta$. The challenge is to construct a scoring function $f(o,x)$ (where $o=M_\theta(x)$) that discriminates between “member” and “non-member” status.

AutoMIA formalizes the search for optimal signal functions as

$$f^* = \arg\max_{f \in \mathcal{F}} \mathcal{J}(f, D_{\text{MIA}_{\text{train}}}),$$

where $\mathcal{J}$ is an empirical attack utility metric (e.g., AUC). The agentic architecture leverages LLMs to explore, implement, and evaluate a dynamic library of scoring strategies, eliminating manual feature engineering and facilitating rapid adaptation to diverse data modalities, models, and threat surfaces.

2. Agent-Based Architecture and Optimization Workflow

AutoMIA employs a hybrid agentic loop, with modules that include:

• Exploration/Exploitation Agents: The Explorer agent generates novel, high-level scoring concepts while the Exploiter agent refines and recombines previously top-performing strategies.
• Programmer and Executor: Translates signal designs into executable code (Python), executes them on target model outputs, and records empirical performance.
• Analyzer and Guidance: Analyzes run results, supports feedback-driven categorization (strong/mid/weak), and issues natural language guidance to bias future exploration.

The iterative optimization alternates between exploration and exploitation, evaluates candidate programs according to measured attack utility (AUC, TPR@FPR), and updates a shared results database. Contextual “sliding window” selection ensures both high-performing and poor strategies influence the agent's credit assignment for strategy refinement. The pseudocode for the main loop is:

$$\begin{aligned} &\textbf{Input:}\;B,\,C,\,DB\ &\textbf{Initialize: seed design from baseline, run experiment, store in }DB\ &\quad\text{for iteration }i=1\,\ldots\,B:\ &\quad\quad\text{if }i\bmod 3=0\ \text{then }d\leftarrow\mathrm{Explore}(DB,C)\ &\quad\quad\text{else }d\leftarrow\mathrm{Exploit}(DB,C)\ &\quad\quad c\leftarrow\mathrm{Programmer}(d,C)\ &\quad\quad r\leftarrow\mathrm{Executor}(c,C)\ &\quad\quad\text{if }r.\mathrm{status}=ok\ \text{then }r.\mathrm{analysis}\leftarrow\mathrm{Analyzer}(r,d),\,DB\leftarrow DB\cup\{(d,c,r)\} \end{aligned}$$

This evolutionary mechanism is complemented by empirical scoring and feedback. Novelty is judged via LLM comparisons to past solutions; performance uplift is prioritized in parent selection ($$P(d_{\text{parent}} = s_i) \propto |\mathrm{AUC}(s_i) - 0.5|$$).

3. Mathematical Formulation of Candidate Membership Signals

AutoMIA systematically traverses high-dimensional families of functional forms, including statistics on model outputs and data that surpass fixed heuristics. Examples include:

• Geometric Edit-Distance Signal (for black-box LLMs): Aggregates consistency of generations with respect to each other and a ground truth via geometric mean of median normalized edit distances.

$$f = \mathrm{clamp}\left(\sqrt{S_1 S_2},\,0,\,1\right); \quad S_1 = 1 - \mathrm{median}_i\left(\widehat{d}(g_i, r)\right),\; S_2 = 1 - \mathrm{median}_{i<j}\left(\widehat{d}(g_i, g_j)\right)$$

• Rare-Trigram Aggregation (for black-box LLMs): Sums $\log 1/[\mathrm{freq}(\tau)\cdot r(\tau)]$ across observed trigrams $M_\theta$0, enhancing sensitivity to rare, memorized text.
• Rank-Stability (for vision-LLMs): Measures stability of token rankings under input perturbations, based on average pairwise normalized inversion distance.

$M_\theta$1

• Positionally-Decayed Log-Ratio Variance (for vision-LLMs): Position-weighted variance of log-ratios, capturing overconfident, sharply peaked output distributions.

Each candidate function is implemented as a code snippet evaluated on real model outputs, and high-performing examples are empirically shown to outperform known human-crafted heuristics.

4. Empirical Benchmarks and Results

AutoMIA has been evaluated on diverse threat settings and datasets:

• Threat settings: Black-box LLM API queries (only text output); gray-box VLMs (partial logit access).
• Models and Data: Pythia 1.4B, OPT 7B (language); DALL·E, LLaVA, MiniGPT-4 (vision-language); datasets include ArXiv abstracts, GitHub code, PubMed, DALL·E/Flickr image–text pairs.
• Baselines: Hallinan et al. n-gram overlap, Li et al. Rényi entropy, OpenEvolve program evolution, classical perplexity/log-gap metrics.

Quantitative results indicate systematic improvements:

Dataset/Method	Best Handcrafted	OpenEvolve	AutoMIA
ArXiv (LLM, black-box)	0.547	0.593	0.687 (+0.094)
GitHub (LLM, black-box)	0.664	0.719	0.750 (+0.031)
Image logits (VLM)	0.594	0.612	0.752 (+0.140)

AutoMIA consistently achieves absolute AUC improvements up to 0.18 over the best human-crafted and code-evolution baselines. Empirical convergence is observed within ∼15 agent episodes; guidance mechanisms and contrasting context windows are necessary for optimal performance (Tran et al., 19 Mar 2026, Liu et al., 1 Apr 2026).

5. Strengths, Limitations, and Future Work in MIA

AutoMIA's principal strengths are its complete automation of feature engineering, its adaptability across model architectures and data domains, and the interpretability of discovered signal formulas. Ablation studies confirm the necessity of agentic feedback and top/bottom strategy contrast for rapid progression.

Limitations include computational resource demands (multiple LLM calls per round), lack of formal convergence guarantees (improvement is empirical), and potential vulnerability to adversarial misuse (dual-use concerns). AutoMIA currently automates only signal-design, with query strategy still manual.

Future directions entail end-to-end agentic MIA (jointly learning query and signal strategies), scalable distributed execution, extending to privacy and unlearning verification, and benchmarking across additional modalities (audio, video, structured code) and model classes (Tran et al., 19 Mar 2026, Liu et al., 1 Apr 2026).

6. AutoMIA for Mirror Illusion Art: Differentiable Inverse Design

A separate AutoMIA pipeline, distinguished from the MIA context, automates mirror illusion art creation—a 3D computational design problem where a single physical object yields two target appearances (as seen directly and as reflected in a mirror) (Zhu et al., 2 Jul 2026). The goal is automatic generation of a printable, smooth 3D object matching both front-view and mirror projection images.

The core representation is a differentiable, dense voxel grid $M_\theta$2 with per-voxel position, opacity, and RGB color. Rendering employs PyTorch3D to composite color along each pixel ray.

The total loss over shape and color for both views is:

$M_\theta$3

where $M_\theta$4 is cross-entropy on foreground masks, and $M_\theta$5 is per-pixel $M_\theta$6 loss.

To address instability and artifacts, AutoMIA for mirror illusion art introduces:

• Projection-Alignment Component (PAC) selection: Connected voxel components are retained only if projections onto both views match target masks, scored by combined IoU and silhouette overflow.
• Position-Weighted Adaptive (PWA) suppression: Shape loss is spatially reweighted to penalize voxels whose projections leak far outside the target mask, using distance-scaled weights per pixel.
• Internal Voxel Preservation (IVP): Enforces minimum density for internal (fully surrounded) voxels to ensure global object connectivity and prevent internal fractures.
• Shape-Color Decoupled (SCD) optimization: Three-stage loss scheduling, with initial shape-only phase, followed by joint shape-color optimization, and concluding with color-only refinement.

Optimization proceeds for 1,000 epochs, alternating random viewpoints near $M_\theta$7, $M_\theta$8. Every $M_\theta$9 steps, PAC pruning removes ‘bad’ components; the mesh is extracted post hoc and Taubin-smoothed.

Experimental results show automatic generation of smooth, artifact-free objects achieving high “Smooth Level” (0.989), low “Noise Level” (0.049), high “Shape Score” (0.931), and low “Color Score” (0.018), with mean design time ∼76s and 2.6 GB GPU memory using a single NVIDIA RTX 3090 (Zhu et al., 2 Jul 2026).

7. Impact and Generalizations

AutoMIA demonstrates that agent-based, closed-loop optimization—whether via program synthesis (MIA) or differentiable optimization (mirror art)—can outperform traditional, manually-crafted baselines in domains requiring search over complex, high-dimensional functional spaces. In the privacy context, AutoMIA delivers on automation, transferability, and interpretability, offering state-of-the-art signal discovery and adaptation to new threat models. For computational art and inverse graphics, the same underlying principles enable rapid, high-fidelity, and generalizable design of physical artifacts, drastically reducing manual intervention.

Ongoing work in both research lines seeks to extend AutoMIA to new application domains, support end-to-end agentic pipeline automation, and address engineering limitations in robustness, scaling, and adversarial control (Tran et al., 19 Mar 2026, Liu et al., 1 Apr 2026, Zhu et al., 2 Jul 2026).