---
title: Dice Projected Gradient Descent (DPGD)
url: https://www.emergentmind.com/topics/dice-projected-gradient-descent-dpgd
type: topic
---

# Dice Projected Gradient Descent (DPGD)

Searching arXiv for the cited DPGD-related papers and terminology to ground the article in current records.
Dice Projected Gradient Descent (DPGD) is a polysemous term whose meaning depends on research context. In adversarial robustness, it denotes an attack-specific variant of $\ell_\infty$-projected gradient descent introduced within Dice Adversarial Robustness Distillation (DARD), where the PGD loss is split between correctly classified and misclassified samples and reweighted across iterations [2509.11525]. Earlier literature used the acronym DPGD for online gradient-free distributed projected gradient descent in time-varying multi-agent optimization [1903.07106] and for decentralized projected gradient descent in constrained decentralized optimization [2303.08412]. In low-rank optimization, the related expression “Dice-PGD” or projected-projected gradient descent refers to a two-projection scheme on the determinantal variety, combining a projection of $-\nabla f(X)$ onto the tangent cone with a projection back onto the feasible low-rank set [2201.03962]. The term therefore does not identify a single canonical algorithm across arXiv literature.

## 1. Terminology and scope

The strongest current match to the literal phrase “Dice Projected Gradient Descent” appears in the DARD framework, which introduces DPGD as “an adversarial example generalization method optimized for effective attack” [2509.11525]. In that usage, DPGD is a training-time adversarial attack designed for robustness distillation rather than a distributed optimization method.

At the same time, the acronym DPGD has an established earlier history in optimization. In “Randomized Gradient-Free Distributed Online Optimization via a Dynamic Regret Analysis,” DPGD stands for “online gradient-free distributed projected gradient descent” on a strongly connected directed graph, with Gaussian smoothing and dynamic regret guarantees [1903.07106]. In “On the convergence analysis of the decentralized projected gradient descent method,” the paper uses “DPG” in the text, but the method matches what is typically called decentralized projected gradient descent, with convergence to an $O(\sqrt{\alpha})$-neighborhood in general convex domains, an $O(\alpha)$-neighborhood in a half-space case, and exact convergence under diminishing step sizes [2303.08412]. In “Low-rank optimization methods based on projected-projected gradient descent that accumulate at Bouligand stationary points,” the phrase “Dice-PGD / DPGD” is used for a projected-projected scheme in which the “double projection” structure is central [2201.03962].

| Usage | Core meaning | Paper |
|---|---|---|
| Dice Projected Gradient Descent | $\ell_\infty$ PGD variant for adversarial example generation | [2509.11525] |
| Distributed Projected Gradient Descent | Online gradient-free distributed projected descent with regret analysis | [1903.07106] |
| Decentralized Projected Gradient Descent | Constrained decentralized optimization with projection | [2303.08412] |
| Dice-PGD / Projected-Projected Gradient Descent | Tangent-cone projection plus feasible-set projection for low-rank optimization | [2201.03962] |

A recurrent misconception is that DPGD is a universally standardized name. The literature instead shows a context-dependent acronym, with materially different update rules, objectives, and guarantees.

## 2. DPGD in adversarial robustness distillation

In the DARD framework, DPGD is described as an attack-specific variant of $\ell_\infty$-PGD designed to generate stronger and more “useful” adversarial examples for robustness distillation than standard PGD [2509.11525]. The motivation has two parts. First, standard PGD optimizes a cross-entropy loss uniformly over examples, whereas DPGD explicitly separates correctly predicted and misclassified samples. Second, adversarial examples are the main training data for the student model in DARD, so the quality and diversity of generated perturbations directly affect distillation.

The backbone remains standard $\ell_\infty$-bounded PGD. Given a natural input $\mathbf{x}_{\text{nat}}$ with label $\mathbf{y}$ and classifier $f$, the adversarial example is obtained from
$$
\max_{\|\delta\|_\infty \le \epsilon} \mathcal{L}\bigl(f(\mathbf{x}_{\text{nat}}+\delta), \mathbf{y}\bigr),
$$
with projected gradient ascent
$$
\mathbf{x}_{\text{adv}}^{t+1}
=
\Pi_{\mathcal{B}_\infty(\mathbf{x}_{\text{nat}},\epsilon)}
\Big(
\mathbf{x}_{\text{adv}}^{t}
+
\alpha \cdot \mathrm{sign}\big(
\nabla_{\mathbf{x}_{\text{adv}}^{t}}
\mathcal{L}(f(\mathbf{x}_{\text{adv}}^{t}), \mathbf{y})
\big)
\Big).
$$
DPGD modifies the loss inside this update.

The defining loss is
$$
\mathcal{L}\bigl(f(\mathbf{x}_{\text{adv}}), \mathbf{y}\bigr)
=
(1-\lambda)\,\mathcal{L}_{p_T}(\mathbf{x}_{\text{adv}}, \mathbf{y})
+
\lambda\,\mathcal{L}_{\rho_F}(\mathbf{x}_{\text{adv}}, \mathbf{y}),
$$
with a time-varying weight
$$
\lambda = \frac{t-1}{2T}, \qquad t \in \{1,\dots,T\}.
$$
Here, $\mathcal{L}_{p_T}$ is the loss on correctly classified examples and $\mathcal{L}_{\rho_F}$ is the loss on misclassified examples. Operationally, at early iterations $\lambda \approx 0$, so the loss is dominated by correctly predicted samples; as $t$ increases, the contribution of misclassified samples gradually grows. The paper describes the intended behavior as aggressively breaking initially correct examples early in the attack and later refining perturbations on already misclassified examples [2509.11525].

This correctness-based split can be written through the indicator
$$
m_i^{t} = \mathbf{1}\big[\arg\max f(\mathbf{x}_{\text{adv},i}^{t}) = y_i\big].
$$
With the partitions
$$
\mathcal{P}_T^t = \{i : m_i^t = 1\}, \qquad \mathcal{F}_\rho^t = \{i : m_i^t = 0\},
$$
the batch loss becomes
$$
\mathcal{L}^{t}
=
(1-\lambda_t)\,
\frac{1}{|\mathcal{P}_T^t|}
\sum_{i\in \mathcal{P}_T^t}
\ell(f(\mathbf{x}_{\text{adv},i}^{t}), y_i)
+
\lambda_t\,
\frac{1}{|\mathcal{F}_\rho^t|}
\sum_{i\in \mathcal{F}_\rho^t}
\ell(f(\mathbf{x}_{\text{adv},i}^{t}), y_i).
$$
The paper emphasizes that, in this classification adaptation, “Dice” does not mean the classical segmentation Dice coefficient per se; it refers to the explicit partition and reweighting of correctly and incorrectly classified subsets across iterations [2509.11525].

## 3. Role within DARD and training objective

Within DARD, DPGD is not merely an evaluation attack. It is used to generate the adversarial examples that drive both teacher adversarial pretraining and student robustness distillation [2509.11525]. The pipeline described in the paper has three stages. First, DPGD generates adversarial examples using the student model. Second, these DPGD adversarial examples are used to adversarially pretrain the teacher. Third, during student training, the student again generates adversarial examples via DPGD, while the teacher’s soft predictions on natural and adversarial inputs are averaged to form soft labels.

The teacher-side soft label combination is
$$
P(x)
=
(1-\lambda_{\text{comb}})\,P_{\text{nat}}(x)
+
\lambda_{\text{comb}}\,P_{\text{adv}}(x_{\text{adv}}),
$$
with $\lambda_{\text{comb}}$ chosen as $0.5$ in experiments. The student objective is given as
$$
\arg\min_{\theta_S}
\Bigl(
(1-\alpha)\,CE\big(S(x_{\text{adv}}), y\big)
+
\alpha \tau^2 KL\big(S(x), P(x)\big)
\Bigr),
$$
where $\theta_S$ are student parameters, $CE$ is cross-entropy on adversarial inputs, and $KL$ aligns student predictions on clean inputs with the teacher’s combined soft labels [2509.11525].

The best-performing DARD variant is described as using only adversarial examples for the $CE$ term, teacher soft labels averaged from both clean and adversarial inputs for the distillation term, and DPGD for generating adversarial examples. An alternative called TDARD jointly uses natural and adversarial inputs as training data, while PGDARD replaces DPGD with standard PGD. The reported ablation states that PGDARD underperforms full DARD, which the authors use as evidence that DPGD generates more effective training-time adversarial examples [2509.11525].

The implementation details specified in the experimental section are an $\ell_\infty$ threat model with $\epsilon = 8/255$, number of steps $T = 20$, step size $\alpha = 2/255$, and the dynamic schedule $\lambda_t = (t-1)/(2T)$. The text also mentions “dynamic step-size tuning and channel-wise gradient masking” in the contributions list, but does not provide explicit formulas or implementation details in the main text [2509.11525].

## 4. Empirical behavior and relation to standard attacks

Relative to standard PGD, DPGD keeps the same projected sign-gradient update but changes the optimization objective from a single uniform loss to a two-part loss with dynamic weighting [2509.11525]. Standard PGD treats every sample identically at every iteration; DPGD instead changes emphasis over time, first focusing on correctly classified examples and later increasing attention to already misclassified examples. The paper presents this as a mechanism for producing stronger and more informative adversarial examples for robustness distillation.

The reported robustness evaluations are conducted under standard attacks rather than DPGD itself. The paper lists FGSM, PGD, BIM, T-PGD, and AutoAttack as evaluation attacks, and tables report results under clean accuracy, PGD20, BIM, and AutoAttack [2509.11525]. For ResNet-18 on CIFAR-10, the reported clean accuracy is $81.33\%$ for SAT, $84.01\%$ for ARD, and $83.56\%$ for DARD; under PGD20 the reported accuracies are $49.68\%$, $50.71\%$, and $52.63\%$; under BIM they are $50.27\%$, $50.12\%$, and $53.07\%$; under AutoAttack they are $46.63\%$, $45.78\%$, and $47.75\%$, respectively [2509.11525]. For CIFAR-100 under PGD20, the reported accuracies are $23.79\%$ for SAT, $25.61\%$ for ARD, and $28.80\%$ for DARD [2509.11525].

The empirical claim attached to DPGD is therefore indirect but specific: training with DPGD inside DARD yields models that are more robust under external attacks than the same distillation framework using standard PGD. The paper does not isolate a standalone attack success rate comparison between DPGD and PGD; instead, the ablation is framed through downstream robustness and convergence behavior [2509.11525].

A second common misconception concerns the word “Dice.” In this adversarial setting, the paper does not define a literal Dice coefficient loss for classification. The “Dice” component is the correctness-based partition of the minibatch and the dynamic reweighting schedule applied to the two resulting subsets [2509.11525].

## 5. Earlier DPGD usage in distributed and decentralized optimization

Before the DARD formulation, DPGD primarily denoted distributed or decentralized projected gradient methods. In the online distributed optimization setting, the 2019 paper studies a group of agents whose local objective functions vary with time and are revealed only after each decision is executed [1903.07106]. No gradients are available, and each agent can only update from revealed scalar function values and neighbor information. The proposed algorithm is an online gradient-free distributed projected gradient descent method in which each agent approximates the gradient based on two point values and Gaussian smoothing.

For agent $i$, the core update is
$$
x_{t+1}^i
=
\Pi_\Omega\Bigg[
\sum_{j=1}^N [\mathbf{W}_r]_{ij} x_t^j + \delta y_t^i
-
\gamma_t g_{\mu^i}^t(x_t^i)
\Bigg],
$$
together with the auxiliary surplus update
$$
y_{t+1}^i
=
\sum_{j=1}^N [\mathbf{W}_c]_{ij} y_t^j
-
\sum_{j=1}^N [\mathbf{W}_r]_{ij} x_t^j
+
x_t^i
-
\delta y_t^i.
$$
Here $g_{\mu^i}^t(x_t^i)$ is the gradient-free oracle
$$
g_{\mu^i}^t(x)
=
\frac{f_i^t(x + \mu_t^i \xi_t^i) - f_i^t(x)}{\mu_t^i}\,\xi_t^i,
\qquad
\xi_t^i \sim \mathcal{N}(0,I_p),
$$
and the projection $\Pi_\Omega$ enforces the convex compact constraint set $\Omega$ [1903.07106]. The paper proves a dynamic regret bound as a function of minimizer path length, step size, and smoothing parameter, and shows that the dynamic regret is sublinear in $T$ if the minimizer path length also grows sublinearly. It also gives the rate
$$
\mathbf{E}[\mathbf{R}_i(T)] \le \mathcal{O}\big(\omega_T^{1/3} T^{2/3}\big)
$$
under the choice $\gamma_0 = \omega_T^{2/3}$, $\mu_0 = \omega_T^{1/3}$, $\alpha = 2/3$, and $\beta = 1/3$ [1903.07106].

In the constrained decentralized setting, the 2023 paper analyzes
$$
x_i(t+1)=\mathcal{P}_\Omega\Big[\sum_{j=1}^n w_{ij} x_j(t) -\alpha(t)\nabla f_i(x_i(t))\Big],
$$
with a symmetric doubly stochastic mixing matrix $W$, connected undirected graph, $L$-smooth local functions, and $\mu$-strongly convex aggregate objective [2303.08412]. Under constant sufficiently small step size, each $x_i(t)$ converges to an $O(\sqrt{\alpha})$-neighborhood of the optimal point in a general convex closed domain. If the domain is the half-space $\mathbb{R}^{d-1}\times\mathbb{R}_{+}$, with optimizer on the boundary and positive normal derivative, the neighborhood improves to $O(\alpha)$. For diminishing step sizes $\alpha(t)=v/(t+w)^p$, the method achieves exact convergence with $\|x_i(t)-x_*\| = O(t^{-p/2})$ for $p\in(0,1)$ and $O(t^{-1/2})$ for $p=1$ [2303.08412].

Neither of these papers uses “Dice” as the formal expansion of DPGD. Their relevance lies in showing that the acronym had a well-developed earlier meaning centered on projected gradient methods over networks.

## 6. Dice-PGD as projected-projected gradient descent in low-rank optimization

A different line of work uses “Dice-PGD / DPGD” for projected-projected gradient descent on the real determinantal variety
$$
R_{\le r}^{m \times n} := \{X \in \mathbb{R}^{m \times n} \mid \operatorname{rank} X \le r\},
$$
where the objective is to minimize a differentiable function with locally Lipschitz continuous gradient over matrices of upper-bounded rank [2201.03962]. The key geometric object is the tangent cone $T(R_{\le r}^{m \times n},X)$, and the associated stationarity measure is
$$
s_f(X) = \big\| \Pi_{T(R_{\le r}^{m \times n},X)}(-\nabla f(X)) \big\|.
$$
A point is stationary precisely when $s_f(X)=0$, which is equivalent to Bouligand stationarity.

The projected-projected step has two projections. First, $-\nabla f(X)$ is projected onto the tangent cone to obtain a feasible first-order descent direction
$$
G \in \Pi_{T(R_{\le r}^{m \times n},X)}(-\nabla f(X)).
$$
Second, after taking a step $X+\alpha G$, the method projects back onto $R_{\le r}^{m \times n}$,
$$
Y \in \Pi_{R_{\le r}^{m \times n}}(X+\alpha G),
$$
with backtracking line search enforcing the Armijo condition
$$
f(Y) \le f(X) - c\alpha s_f(X)^2.
$$
This double-projection structure is the source of the “projected-projected” terminology [2201.03962].

The paper then extends this basic step with rank reduction. Given the numerical rank $\operatorname{rank}_\Delta X_i$, it considers lower-rank projections $\hat{X}_i^j$ and applies one projected-projected step from each candidate, selecting the one with minimum objective value. The main theorem states that if the sequence is finite, its last element is stationary, and if it is infinite, each accumulation point is stationary, i.e., a zero of $s_f$ [2201.03962]. The paper frames this as an apocalypse-free first-order method whose accumulation points are Bouligand stationary.

In this literature, “Dice” is tied neither to distributed consensus nor to adversarial robustness. It denotes, informally, the double projection operation and, in the rank-reduced variant, the trial of multiple projected candidates across ranks [2201.03962].

## 7. Conceptual synthesis

Across these papers, DPGD designates three distinct projected-gradient constructions. In adversarial robustness, it is an $\ell_\infty$ attack with a dynamically reweighted two-part loss over correct and incorrect samples [2509.11525]. In distributed and decentralized optimization, it is a constrained multi-agent projected gradient method, gradient-free in the online directed-graph setting [1903.07106] and gradient-based in the strongly convex decentralized setting [2303.08412]. In low-rank optimization, it is a projected-projected method on a singular algebraic variety, analyzed through tangent cones and Bouligand stationarity [2201.03962].

The principal unifying feature is the presence of projection as a structural device: projection onto an $\ell_\infty$ perturbation set in adversarial attack generation, projection onto a feasible convex set in network optimization, and projection onto both tangent and feasible low-rank geometry in determinantal-variety optimization. Beyond that shared motif, the objectives, information models, convergence criteria, and meanings of “Dice” are not interchangeable.

Source: https://www.emergentmind.com/topics/dice-projected-gradient-descent-dpgd