Self-Invoking Code Generation in GLRT Defense

• Self-Invoking Code Generation is a method where code dynamically generates and executes new code segments, enabling adaptive and automated enhancements in computational tasks.
• The GLRT framework applies a generalized likelihood ratio test to jointly estimate signal templates and adversarial perturbations, utilizing a double-sided ReLU for efficient coordinate-wise projection.
• Empirical analysis demonstrates that this approach achieves exponential error decay and outperforms traditional methods even under worst-case, noise-aware adversarial attacks.

Below is a self‐contained summary of the multi‐hypothesis GLRT framework as developed in “Generalized Likelihood Ratio Test for Adversarially Robust Hypothesis Testing” by Puranik, Madhow and Pedarsani. We assume familiarity with basic hypothesis‐testing notation but spell out all the relevant formulae and algorithmic steps needed to implement the defense.

1. Problem Formulation We observe a vector x∈ℝᵈ and wish to decide among M hypotheses H₀,…,H_{M–1} under an adversarial perturbation θ∈ℝᵈ satisfying ||θ||∞≤ε. We treat θ as a nuisance parameter.

a. M‐ary composite hypothesis testing with nuisance θ For each hypothesis Hᵢ, the attacker may add a perturbation θ∈Θ≜{θ:||θ||∞≤ε}. Thus the composite hypotheses are Hᵢ: X = Sᵢ + θ + N, θ∈Θ, i=0,…,M–1, where Sᵢ∈ℝᵈ is the known “signal template” or class‐mean under Hᵢ, and N∼𝒩(0,σ²I) is white Gaussian noise.

b. Joint likelihood and adversarial constraint The conditional (joint) density under Hᵢ and θ is p(x│Hᵢ,θ) = 1/( (2πσ²)^{d/2} ) * exp(−||x−Sᵢ−θ||²/(2σ²) ). The adversarial set is Θ = { θ∈ℝᵈ : ||θ||∞ ≤ ε }.

2. GLRT Decision Rule for Multi‐Hypothesis

a. GLRT statistic The Generalized Likelihood Ratio Test compares the maximum likelihood under each hypothesis (including the best‐fitting θ) to a reference. A convenient form for M‐ary testing is simply to pick the hypothesis whose maximized likelihood is largest. Equivalently, one can form ratios to a nominal reference hypothesis H_{i₀}, but that only shifts all log‐likelihoods by a common constant. Concretely, define for each i Λᵢ(x) = max_{θ∈Θ} p(x│Hᵢ,θ). Then the GLRT chooses the i that maximizes Λᵢ(x).

b. Final decision rule In practice we work with negative–log‐likelihood (quadratic cost) Cᵢ(x) = min_{θ:||θ||∞≤ε} || x − Sᵢ − θ ||² = || gε( x − Sᵢ ) ||², where gε(·) is the “double‐sided ReLU” defined coordinate‐wise by gε(u) = sign(u)·max(0, |u| − ε). The GLRT then is ĥ(x) = argmin_{i=0,…,M−1} Cᵢ(x). Equivalently, in likelihood form ĥ(x) = argmax_{i} max_{θ:||θ||∞≤ε} p(x│Hᵢ,θ).

3. Joint Estimation Algorithm To implement ĥ(x)=argminᵢCᵢ(x), we must for each i compute θ̂ᵢ = argmin_{θ:||θ||∞≤ε} ||x − Sᵢ − θ||². – In the Gaussian+ℓ∞ case this has the closed‐form solution (coordinate‐wise projection): θ̂ᵢ[j] = fε( x[j] − Sᵢ[j] ), where fε(u)=u−gε(u)=sign(u)·min(ε, |u|). – In a general model p(x│Hᵢ,θ), one can carry out a constrained gradient‐descent (or projected gradient) over θ∈Θ to find θ̂ᵢ, and then evaluate the likelihood. Finally choose i that gives the largest p(x│Hᵢ,θ̂ᵢ).

4. Theoretical Performance Analysis

a. Asymptotic error under worst‐case attack Focusing on the binary case (M=2) with symmetric S₀=+μ, S₁=−μ, the authors show via a coordinate‐wise CLT that under the worst‐case attack θ* = −ε·sign(μ) under H₀, +ε·sign(μ) under H₁, the GLRT’s error probability satisfies, for large dimension d, Pₑ ≈ Q( ( ∑{i=1}^d m_i ) / √(∑{i=1}^d ρ_i² ) ), where m_i, ρ_i² are the mean and variance of the per‐coordinate cost‐difference. In the high‐SNR limit (σ→0) this gives exponential decay Pₑ≲exp(−k·d). Moreover, this asymptotic exponent matches that of the minimax‐optimal classifier under the full‐budget attack.

b. Worst‐case attack characterization Even though GLRT is designed to estimate θ, one can show that for binary Gaussian models the same worst‐case perturbation θ* that defeats (minimax) robust linear classifiers also maximizes the GLRT’s error. In the multi‐class setting, a noise‐aware adversary with full knowledge of x and its noise component can test, for the true class j, whether any of the other hypotheses i≠j can be made more likely by choosing θ = −ε·sign( S_j − S_i ). If so, that θ* is a worst‐case attack. The authors give a simple procedure: for each “competitor” i compute C_i(x) under that θ; if any C_i<C_j, misclassification is forced.

5. Non‐Asymptotic and Simulation Results

a. Robustness–accuracy tradeoff in multi‐class simulations – GLRT vs. minimum‐distance vs. a “Pairwise Robust Linear” (PRL) benchmark: PRL runs all M(M–1)/2 binary minimax classifiers (one per pair) and declares class j only if it wins against all others. – Under noise‐agnostic attacks (adversary does not know N) and noise‐aware attacks (knows N), GLRT outperforms minimum‐distance by a large margin at high ε, and outperforms PRL for weaker attacks. See Fig. 10 of the paper for ternary examples. – Noise‐aware GLRT performance provides a lower‐bound on error (since that attack is strongest).

6. Heuristics for Noise‐Agnostic Attacks in High‐SNR

When the adversary does not know N but does know the true class j and S₀,…,S_{M−1}, it solves maximize_{||θ||∞≤ε} P[ĥ(X+θ+N)≠j]. In general M>2 this is intractable. At high SNR, however, one can show the error is dominated by the “nearest‐neighbor” hypothesis i* that is easiest to confuse with j.

a. Nearest‐Neighbor (NN) class for GLRT Define pairwise difference μ{jk} = (S_j−S_k)/2. Observation 5 (GLRT‐NN): i*(j) = argmin{k≠j} ∑{ℓ: |μ{jk}[ℓ]|≥ε} (|μ{jk}[ℓ]| − ε)². Then the suggested noise‐agnostic attack is θₐgₙ = −ε·sign( μ{j,i*(j)} ), which empirical studies show is close to optimal at high SNR.

b. Varying‐strength attack heuristic If the actual attack strength κ≤ε is weaker than the design budget, one replaces ε in the criterion by (ε+κ)/2 or uses the bound ∑{ℓ: |μ{jk}[ℓ]|≥(κ+ε)/2} (2|μ_{jk}[ℓ]| − κ − ε)² to pick the dominant competitor.

Implementation sketch (Gaussian case) 1. Pre‐compute templates S₀,…,S_{M−1}. 2. Upon receiving x, for each i: a. Compute residual r_i = x − Sᵢ. b. Compute θ̂ᵢ[j]=fε(r_i[j]) coordinate‐wise. c. Compute cost Cᵢ = ||gε(r_i)||² = ||r_i − θ̂ᵢ||². 3. Output ĥ = argminᵢ Cᵢ.

Summary * The GLRT defense treats the adversary’s perturbation as a nuisance parameter and jointly estimates (h,θ). * It admits a closed‐form implementation for Gaussian models with ℓ∞‐bounded attacks, via double‐sided ReLU nonlinearity. * The worst‐case attack for binary Gaussian GLRT coincides with the minimax‐optimal attack, and the GLRT’s asymptotic error exponent matches the minimax defense. * In the multi‐class setting, GLRT outperforms naïve minimum‐distance and pairwise‐linear defenses, and simple heuristics (nearest‐neighbor‐based) provide near‐optimal noise‐agnostic attacks in high SNR.