--- title: Expectation over Transformation (EOT) Framework url: https://www.emergentmind.com/topics/expectation-over-transformation-eot-framework type: topic --- # Expectation over Transformation (EOT) Framework Expectation Over Transformation (EOT) is a formal framework for constructing and evaluating robust defenses against adversarial and distribution-shifted inputs in machine learning systems. The EOT methodology defines robustness with respect to a distribution over input transformations, ensuring that predictions or defenses account for the full range of possible input modifications, rather than any single realization. This framework has been pivotal in the design and evaluation of purification-based adversarial defenses, protective perturbation for privacy, and robust model training. EOT clarifies the ambiguities seen in prior stochastic defenses by establishing deterministic, expectation-based criteria and enabling principled gradient-based attacks. Extensions such as Expectation Over Learned distribution of Transformations (EOLT) replace the fixed sampling distribution in EOT with a learned, instance-adaptive policy, yielding demonstrable empirical gains in challenging applications. ## 1. Mathematical Foundation of EOT EOT is formulated around a deterministic classifier $f$, a stochastic transformation $T(x)$ defined on the input $x$, and the induced classifier $F(x) = \mathbb{E}_{t \sim T(x)}[f(t)]$, which averages the classifier’s output over the transformation distribution. In adversarial defense settings, $T(x)$ is frequently implemented as a stochastic pre-processing or purification operation—such as K-step Langevin dynamics using an energy-based model (EBM)—to remove adversarial artifacts while preserving class semantics [2005.13525]. EOT's key criterion is that prediction and robustness evaluation rely on $F(x)$ rather than on the stochastic composition $f \circ T$. In practice, $F(x)$ is approximated via Monte Carlo averaging, $\hat{F}_H(x) = (1/H) \sum_{h=1}^H f(\hat{x}_h)$, with $\hat{x}_h \sim T(x)$ sampled independently. For a given input $x$, the classifier’s output is then defined as $c(x) = \arg\max_{j}[F(x)_j]$. In protective perturbation contexts, such as robust watermarking against DeepFake face swapping, EOT formalizes the search for a perturbation $\delta^*$ satisfying $\|\delta\|_p \leq \epsilon$ by maximizing the expected loss over transformations, \[ \delta^* = \arg\max_{\|\delta\|_p \leq \epsilon} \mathbb{E}_{t \sim \mathcal{U}(S_p)}[L(F(t(x+\delta)), F(x))] \] where $t$ is sampled uniformly from a set of allowable transformations $S_p$, and $L$ quantifies disruption to the system of interest [2512.07228]. ## 2. EOT in Adversarial Defense EOT addresses core ambiguities prevalent in stochastic defense literature. Previous approaches often relied on low-sample approximations (e.g., applying $T$ a handful of times and using majority vote), making model predictions sensitive to sampling variance and open to attack. In EOT, robustness and accuracy are evaluated against $F(x)$, a deterministic expectation, ensuring that adaptive adversaries cannot exploit stochasticity for successful attacks. The framework leverages the long-run dynamics of the Langevin chain for purification, using an EBM with the following iterative update: \[ X_{k+1} = X_k - \frac{\tau^2}{2} \nabla_x U(X_k; \theta) + \tau Z_k,\quad Z_k \sim \mathcal{N}(0, I) \] For large $K$, the chain erases local adversarial structure (memoryless steady-state property), while metastability preserves global content, thereby removing perturbations without inducing class transitions [2005.13525]. The deterministic nature of $F(x)$ also obviates errors stemming from finite-sample randomness, a critical consideration for any reliable deployment. ## 3. Adaptive Attacks and EOT Attacks EOT provides a principled basis for constructing adaptive attacks, most notably through the “EOT attack” and the BPDA+EOT gradient methodology. The attacker’s objective is to maximize the loss $L(F(x), y)$ with respect to $x$, under a given perturbation constraint set $S$ (e.g., within an $\ell_p$-ball). Since $F(x)$ may be intractable, gradients are approximated as: \[ \Delta_{EOT}(x, y) = \nabla_x L(\hat{F}_{H_{\text{adv}}}(x), y) \] For non-differentiable or black-box $T(x)$, the Backward Pass Differentiable Approximation (BPDA) is combined with EOT, using the identity surrogate to propagate gradients. The update within Projected Gradient Descent (PGD) is then: \[ x_{i+1} = \mathrm{Proj}_S[x_i + \alpha \cdot g(x_i, y)] \] where $g(x, y)$ is the BPDA+EOT gradient estimate [2005.13525]. In the context of protective perturbations, the perturbation update is carried out by iteratively sampling a batch of transforms each PGD step and averaging the gradient, reflecting the expectation in the objective [2512.07228]. ## 4. Algorithmic Implementations and Practical Considerations ### Energy-Based Purification Defense - **EBM Training**: Utilizes long-run Langevin sampling (e.g., $K_{\mathrm{train}} = 100$ steps, $\tau = 0.0$2) with persistent pools and alternates optimization between Adam and SGD. - **EOT Defense**: - Applies a large ($H_{\mathrm{def}}=150$) sample average over $K_{\mathrm{def}}=1500$ Langevin-denoised samples per input. - Prediction $c_{\mathrm{def}}(x)$ is determined by the argmax over the averaged logits. - **BPDA+EOT Attack**: - Employs $H_{\mathrm{adv}}=15$ attack samples per step, verified by the defense sample ($H_{\mathrm{def}}$), typically with $N=50$ PGD steps, $\alpha=2/255$, and $\epsilon=8/255$ in $\ell_\infty$. ### Datasets and Architectures - Benchmarks: CIFAR-10, SVHN, CIFAR-100. - EBM: Lightweight ConvNet ($\sim$700K parameters). - Classifier: Wide ResNet 28-10 [2005.13525]. ### Protective Perturbation for DeepFake Face Swapping - **Standard EOT**: Samples transformations uniformly from a pool $S_p$, using PGD (e.g., $N=150$ iterations, $\alpha=0.01$, $\epsilon=0.05$ in $\ell_\infty$). - **Observed Limitations**: Uniform transformation sampling is suboptimal; some transformations (e.g., blur) are defensive bottlenecks, while others (e.g., hue) cause overfitting and degrade transferability [2512.07228]. ## 5. Extensions: Learning the Transformation Distribution (EOLT) Recent advances parameterize the transformation distribution itself via a policy network $\pi_\theta(t|x)$, introducing the Expectation Over Learned distribution of Transformation (EOLT) framework. EOLT replaces the uniform prior of EOT with an adaptive, instance-aware policy. The policy network is trained using REINFORCE-style updates, where rewards are based on achieved disruption (e.g., identity change under face swapping) measured over a set of validation transformations. EOLT adopts a bilevel approach: - **Inner Loop**: Solves for the perturbation $\delta^*(\theta)$ maximizing expected loss over the learned transformation distribution. - **Outer Loop**: Optimizes the policy parameters $\theta$ to maximize disruption across a held-out validation set while regularizing for diversity (e.g., with probability cap and entropy penalties). Backbones for $\pi_\theta$ include CNNs and vision transformers. Empirical results demonstrate that EOLT achieves a 26.2% reduction in identity similarity on FFHQ over standard EOT, with 20–30% gains in challenging categories such as blur and stylization. The policy rapidly learns to allocate more weight to transformations with strong cross-transform generalization (e.g., gaussian blur, uniform noise), culling toxic transforms (hue, solarize) [2512.07228]. ## 6. Empirical Results and Comparative Robustness EOT-based defenses using energy-based model purification yield natural and robust accuracies competitive with or exceeding those of adversarially trained models: | Dataset | EOT Natural | EOT Robust (BPDA+EOT) | PGD-AT Robust | |-------------|-------------|------------------------|---------------| | CIFAR-10 | 0.8412 | 0.5490 | 0.458 | | SVHN | 0.9223 | 0.6755 | 0.5039 | | CIFAR-100 | 0.5166 | 0.2610 | 0.2547 | On CIFAR-10, natural accuracy of f(x) is 0.9530 and robust accuracy under direct PGD is 0.0, but under EOT defense robust accuracy is 0.5490, outperforming standard PGD-based adversarial training (0.458). For DeepFake protection, EOLT reduces identity similarity by 26%, with comparable improvements under both seen and unseen transformations [2005.13525, 2512.07228]. A plausible implication is that the EOT and EOLT frameworks provide robust, principled foundations for reliable defense model evaluation and construction across vision and privacy-sensitive applications. ## 7. Impact, Limitations, and Future Directions EOT removes ambiguities inherent in ad-hoc stochastic defenses, supporting rigorous model evaluation and attack design. It underlies a broad class of modern purification and transformation-robust defenses. However, standard EOT's uniform sampling may be fundamentally limited in applications where transformation vulnerabilities are unevenly distributed. EOLT demonstrates that learning the transformation distribution—via adaptive, instance-specific policies—provides substantial gains and opens future research directions in policy structure, regularization, and co-design with generative models. Empirical evidence suggests EOT and EOLT can secure naturally trained models to robustness levels previously attainable only via adversarial training, and that adaptive transformation distribution learning is broadly applicable for enhanced protection in tasks such as anti-DeepFake perturbation. Extensions may focus on joint transformation-perturbation learning, multimodal settings, and theoretical characterization of optimal transformation policies [2005.13525, 2512.07228]. Source: https://www.emergentmind.com/topics/expectation-over-transformation-eot-framework