Dual Averaging with EControl

• The paper introduces Dual Averaging with EControl, a novel method that integrates error compensation into the dual accumulation to mitigate compression errors in composite optimization.
• It shifts the handling of compression error from the proximal update to the dual space, preserving the additive structure necessary for convergence analysis.
• Experimental and theoretical results confirm that this method extends error-feedback techniques to convex composite settings, enhancing communication efficiency in distributed systems.

Searching arXiv for the specified paper and closely related dual averaging / EControl references. Dual Averaging with EControl is a communication-efficient optimization method for convex composite problems of the form

$\min_{x\in \mathbb{R}^d} F(x):=f(x)+\psi(x),$

designed for distributed optimization with compressed communication. Its defining idea is to combine dual averaging with EControl, a refined error-compensation mechanism, so that compression error is handled in the dual accumulation rather than inside a proximal-gradient update. In this formulation, the additive structure required by error-feedback methods is restored in dual space, which enables the first strong convergence analysis for error feedback in composite optimization (Gao et al., 3 Oct 2025).

1. Concept, terminology, and scope

In the relevant literature, Dual Averaging with EControl denotes a method for composite optimization with compressed communication in which the smooth part $f$ is convex and smooth, the composite term $\psi$ is convex and proximable, and communication error is managed through an auxiliary residual process attached to the dual variable rather than to the primal proximal step (Gao et al., 3 Oct 2025).

The composite term $\psi$ may encode a nonsmooth regularizer such as $\ell_1$, an indicator function of a convex constraint set, or other proximable structure. This places the method squarely in the composite convex setting rather than the unconstrained smooth setting that underlies the classical error-feedback literature (Gao et al., 3 Oct 2025).

EControl is described as a state-of-the-art variant of the error-feedback mechanism. Its role is to control the effect of compression error in a way compatible with dual averaging: the method forms a corrected signal, compresses it, and stores the uncommunicated remainder in memory. The distinctive feature is not merely residual storage, but the fact that this mechanism is attached to the dual accumulation variable, where additivity is natural (Gao et al., 3 Oct 2025).

The name is specific to the compressed composite-optimization setting, but nearby dual-averaging literature uses closely related ideas under different terminology. In decentralized composite optimization over stochastic communication networks, dynamic averaging consensus or tracking is used to control dual-estimation and consensus error, even though the term EControl is not used explicitly (Liu et al., 2021). This suggests that the central idea of “control” in these works is regulation of error in the dual or consensus process, rather than event-triggered control.

2. Problem formulation and the failure of standard error feedback

The basic optimization problem is

$\min_{x\in \mathbb{R}^d} F(x):=f(x)+\psi(x),$

with compressed communication used because transmitting full-precision vectors is expensive. Workers compute gradients or stochastic gradients of the smooth part $f$, but messages are passed through a compressor $\mathcal C(\cdot)$, and the resulting distortion must be controlled without breaking convergence in the presence of the composite term $\psi$ (Gao et al., 3 Oct 2025).

The key obstruction is that standard error feedback is analytically tied to the smooth unconstrained case. There, the update has a simple additive structure, and the compressed step plus residual correction can be represented through a virtual iterate. In the composite setting, however, a natural proximal-gradient step would take the form

$$x_{t+1}=\operatorname{prox}_{\eta_t\psi}\bigl(x_t-\eta_t\widetilde g_t\bigr),$$

and the nonlinear prox map destroys the additive bookkeeping used by classical error-feedback proofs (Gao et al., 3 Oct 2025).

The central incompatibility is therefore structural rather than cosmetic. Since

$f$0

cannot be decomposed in the way needed by the virtual-iterate argument, the standard error-feedback mechanism and its standard analysis do not extend cleanly to composite optimization. Dual Averaging with EControl resolves this by moving compression and residual correction to the dual space of dual averaging, where cumulative sums remain additive (Gao et al., 3 Oct 2025).

This shift also clarifies what the method is not. It is not a direct modification of proximal SGD with compressed gradients, and it is not a reformulation of standard error feedback around a prox step. Its premise is that composite structure should be handled by the dual-averaging reconstruction map, while compression error should be accumulated and compensated in dual form (Gao et al., 3 Oct 2025).

3. Exact dual averaging backbone and the EControl mechanism

The exact dual-averaging backbone forms a cumulative dual variable

$f$1

and then reconstructs the primal point through a proximal or mirror step of the form

$f$2

or, under an alternative normalization, through a Bregman-regularized equivalent involving $f$3. Here $f$4 is the accumulated dual variable, $f$5 is the cumulative weight factor, and $f$6 generates the mirror geometry (Gao et al., 3 Oct 2025).

Dual Averaging with EControl replaces exact dual accumulation by a compressed and compensated recursion: $f$7

$f$8

$f$9

$\psi$0

$\psi$1

The residual $\psi$2 stores exactly the part of the corrected signal that was lost by compression (Gao et al., 3 Oct 2025).

This architecture preserves the additive structure that standard error feedback requires, but relocates it to the appropriate object: the dual increment rather than the argument of a prox map. Compression is therefore applied to the corrected dual-space quantity, not to the final primal update. That separation is the method’s defining design choice (Gao et al., 3 Oct 2025).

Several special cases are immediate. If $\psi$3, the method reduces to an unconstrained smooth compressed method; if $\psi$4 is the identity compressor, it reduces to exact dual averaging; and if there is no compression error, the residual remains zero and the method becomes standard dual averaging (Gao et al., 3 Oct 2025).

At the implementation level, each worker communicates a compressed corrected dual increment rather than a raw gradient. The memory overhead is one additional model-sized buffer,

$\psi$5

and the extra computation is one vector addition to form the corrected message, one residual update after compression, and one dual-averaging primal solve (Gao et al., 3 Oct 2025).

4. Inexact dual averaging analysis and convergence claims

A major contribution of the method is a new analysis framework for inexact dual averaging. Rather than constructing a virtual primal iterate, the analysis models compressed communication directly as dual inexactness: $\psi$6 where $\psi$7 is the compression-induced distortion controlled by the EControl recursion (Gao et al., 3 Oct 2025).

The analytical strategy relies on the stability of the dual-averaging map

$\psi$8

Because the subproblem contains the mirror or prox regularization, perturbations in $\psi$9 can be translated into controlled perturbations of the corresponding primal point. The prox or mirror operator is therefore analyzed once as a stable map from dual space to primal space, instead of being forced into an additive virtual-iterate construction (Gao et al., 3 Oct 2025).

This yields a descent or regret inequality for exact dual averaging plus additive perturbation terms involving $\psi$0. EControl is then used to show that the cumulative discrepancy between the exact and compressed dual sums remains bounded in a useful way. The result is a strong convergence analysis for composite optimization with error feedback, which the paper identifies as the first such result (Gao et al., 3 Oct 2025).

The paper states four conclusions of central importance. First, classical error-feedback analysis does not extend to composite optimization because the composite update destroys the additive structure. Second, Dual Averaging with EControl restores analyzability and convergence by shifting compression inexactness into the cumulative dual recursion. Third, this extends error-feedback theory to convex composite problems. Fourth, when

$\psi$1

the theory recovers the best-known guarantees from the smooth unconstrained error-feedback literature (Gao et al., 3 Oct 2025).

The experimental section is described as complementing the theoretical findings. The intended empirical message is that the proposed method behaves well on composite problems under compressed communication, whereas standard error-feedback-style approaches are inadequate or unsupported in that setting (Gao et al., 3 Oct 2025).

5. Decentralized and networked antecedents of error-controlled dual averaging

Although Dual Averaging with EControl is defined in the compression setting, earlier decentralized work developed closely related error-control mechanisms in stochastic communication networks. In decentralized composite optimization over $\psi$2 agents, one such method maintains local dual estimates $\psi$3 and auxiliary variables $\psi$4 through the dynamic consensus recursions

$\psi$5

$\psi$6

followed by a local dual-averaging prox step for $\psi$7. The authors emphasize that this produces more accurate local estimates of the global dual variable and yields the first linearly convergent DDA-type decentralized algorithm, as well as the first algorithm with global linear convergence for decentralized composite optimization in stochastic networks (Liu et al., 2021).

In that line of work, the auxiliary recursion for $\psi$8 acts as a correction term controlling drift between local estimates and the network-wide average signal. The paper explicitly interprets this as control of dual estimation error and corresponding primal consensus error. This is close in substance to EControl, even though the terminology used is dynamic averaging consensus or tracking rather than error feedback (Liu et al., 2021).

A related precursor studies decentralized constrained convex optimization and tailors a second-order dynamic average consensus protocol to dual averaging: $\psi$9

$\ell_1$0

$\ell_1$1

That method is proved to attain $\ell_1$2 convergence for general convex and smooth problems, improving over the $\ell_1$3 behavior known for existing DDA methods at the time (Liu et al., 2020).

These decentralized results are not about compressed communication and do not use the EControl name. A plausible implication is that they provide the networked analogue of the same underlying principle: dual averaging can tolerate aggressive weighting or stronger rates only when the estimation dynamics of the global dual quantity are explicitly corrected.

6. Broader controlled variants of dual averaging

Dual Averaging with EControl also belongs to a broader family of methods in which some aspect of dual-averaging dynamics is explicitly controlled. One example is extended regularized dual averaging (XRDA), which introduces a control sequence governing the backward or proximal coefficient through

$\ell_1$4

In this framework, the backward step size used in regularized dual averaging can be kept bounded rather than growing without bound, while the method retains the same convergence rate as RDA for general convex objectives (Siegel et al., 2019).

Another adjacent framework is the unified mirror-descent/dual-averaging family in which the next dual representative need not equal the pure DA update $\ell_1$5, but may be chosen through controlled interpolation. A concrete instance is

$\ell_1$6

with $\ell_1$7 recovering dual averaging and $\ell_1$8 recovering mirror descent. This makes controlled modification of dual updates an explicit part of the algorithmic design space (Juditsky et al., 2019).

Taken together, these works indicate that “control” in dual-averaging research has several distinct meanings. In Dual Averaging with EControl, it refers to compression-residual compensation in the dual accumulation (Gao et al., 3 Oct 2025). In decentralized DDA, it refers to dynamic consensus or tracking of the global dual signal (Liu et al., 2021). In XRDA, it refers to explicit control of the backward or proximal step size (Siegel et al., 2019). In unified mirror-descent/dual-averaging schemes, it refers to controlled selection of the dual representative or interpolation between DA and MD (Juditsky et al., 2019).

The specific significance of Dual Averaging with EControl is therefore precise rather than generic. It does not rename all controlled dual-averaging methods. It identifies one previously unresolved setting—composite optimization with compressed communication—in which standard error feedback is structurally mismatched, and shows that relocating error compensation into the dual-averaging state restores a viable convergence theory (Gao et al., 3 Oct 2025).