Worker-Average Gap Covariance

• Worker-Average Gap Covariance is a measure of worker-level deviations centered by their average, capturing heterogeneity that informs both optimization geometry and aggregate inference.
• In Local SGD, it serves as a cheap Hessian-free estimator by tracking gap vectors whose spectral properties align with dominant curvature directions of the loss surface.
• In econometric and macroeconomic models, the covariance informs fixed effect regressions and heterogeneous-reset Phillips curves by quantifying sampling uncertainty and aggregate composition effects.

Across recent arXiv work, worker–average gap covariance denotes covariance objects built from worker-level quantities after centering by a worker average. In Local SGD, the worker–average gap is $\Delta_t^i := w_t^i-\bar w_t$ and its empirical covariance $C=\mathbb E[\Delta\,\Delta^\top]$ is used as a cheap Hessian-free estimator of the dominant subspace (Dimlioglu et al., 26 May 2026). In high-dimensional two-way fixed effect regression, the centered quantities are $\hat\alpha_i-\bar\alpha$, and their covariance is a closed-form function of the deterministic equivalent variance-covariance matrix (He et al., 7 Jan 2026). A related macroeconomic construction is the within-country covariance between a worker’s cost-push exposure and her reset frequency, which enters the heterogeneous-reset Phillips curve as the “reset-heterogeneity wedge” (Sun, 31 Mar 2026).

1. Domain-specific meanings and common structure

The recent literature uses related covariance constructions in distinct settings rather than a single field-independent definition. In each case, the object is built from worker-level heterogeneity relative to a worker average, and the covariance captures information that aggregation would otherwise suppress.

Setting	Centered worker object	Role
Local SGD	$\Delta_t^i = w_t^i-\bar w_t$	Empirical covariance $C$ estimates sharp dominant directions
Two-way fixed effects	$\hat\alpha_i-\bar\alpha$	Gap covariance is computed from $\mathfrak V_{\alpha,\lambda}$
Heterogeneous-reset Phillips curve	$\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$	Enters as the “reset-heterogeneity wedge”

In optimization, the covariance is geometric: it measures how workers drift away from the synchronized mean during local updates. In econometrics, it is inferential: it describes the variance-covariance structure of centered estimated worker fixed effects. In macroeconomics, it is compositional: it measures the covariance between a worker’s cost-push exposure and her wage-reset frequency. The common principle is that worker-level dispersion relative to an average can enter aggregate behavior at first order rather than as a negligible residual.

2. Local SGD formulation and covariance dynamics

In “Worker Disagreement Reveals Sharp Directions in Local SGD” (Dimlioglu et al., 26 May 2026), communication rounds are indexed by $t$, with worker parameters $w_t^i\in\mathbb R^D$ for $C=\mathbb E[\Delta\,\Delta^\top]$0, and $C=\mathbb E[\Delta\,\Delta^\top]$1 their average just before synchronization. The worker–average gap is defined by

$C=\mathbb E[\Delta\,\Delta^\top]$2

By construction, $C=\mathbb E[\Delta\,\Delta^\top]$3. Collecting gap vectors over rounds and over workers yields the empirical covariance

$C=\mathbb E[\Delta\,\Delta^\top]$4

where one averages over a buffer of $C=\mathbb E[\Delta\,\Delta^\top]$5 observed gaps at different rounds.

At the start of a communication round all workers are synchronized at $C=\mathbb E[\Delta\,\Delta^\top]$6. Each worker then takes $C=\mathbb E[\Delta\,\Delta^\top]$7 local SGD steps

$C=\mathbb E[\Delta\,\Delta^\top]$8

where $C=\mathbb E[\Delta\,\Delta^\top]$9 is the stochastic-gradient noise. Defining the within-round deviations

$\hat\alpha_i-\bar\alpha$0

and applying a Taylor-linearization of $\hat\alpha_i-\bar\alpha$1 around the round-average while neglecting higher-order remainders gives the one-step recurrence

$\hat\alpha_i-\bar\alpha$2

where $\hat\alpha_i-\bar\alpha$3 is the Hessian at the round-start and $\hat\alpha_i-\bar\alpha$4 is the centered noise.

Unrolling from $\hat\alpha_i-\bar\alpha$5 gives the final gap

$\hat\alpha_i-\bar\alpha$6

Taking covariance and using $\hat\alpha_i-\bar\alpha$7 yields

$\hat\alpha_i-\bar\alpha$8

This is the exact expression up to the linearization error. In the small–step-size limit,

$\hat\alpha_i-\bar\alpha$9

The paper’s central claim is that the worker-average gap covariance is shaped by stochastic-gradient noise and Hessian curvature. This is why worker disagreement is informative about local loss geometry rather than merely an implementation artifact of distributed training.

3. Spectral alignment, subspace estimation, and empirical behavior

The key spectral alignment result states that if $\Delta_t^i = w_t^i-\bar w_t$0 and $\Delta_t^i = w_t^i-\bar w_t$1 in the same eigenbasis, then $\Delta_t^i = w_t^i-\bar w_t$2 is diagonalized by the same $\Delta_t^i = w_t^i-\bar w_t$3 (Dimlioglu et al., 26 May 2026). For each Hessian eigenpair $\Delta_t^i = w_t^i-\bar w_t$4,

$\Delta_t^i = w_t^i-\bar w_t$5

where $\Delta_t^i = w_t^i-\bar w_t$6. Thus $\Delta_t^i = w_t^i-\bar w_t$7 is also an eigenvector of $\Delta_t^i = w_t^i-\bar w_t$8, with eigenvalue proportional to $\Delta_t^i = w_t^i-\bar w_t$9. Since gradient-noise variance typically scales as $C$0 with $C$1, the gap covariance puts the most weight on the largest $C$2. Hence the top-$C$3 eigenvectors of $C$4 coincide with the leading Hessian eigenspace.

The practical estimator is explicitly low-rank. One keeps a rolling buffer $C$5 of the most recent $C$6 gaps, forms the Gram matrix $C$7, eigendecomposes $C$8, and sets

$C$9

The columns of $\hat\alpha_i-\bar\alpha$0 are orthonormal and span the same subspace as the gaps. Estimating the top $\hat\alpha_i-\bar\alpha$1 gap-directions requires keeping the largest $\hat\alpha_i-\bar\alpha$2 eigenvalues in $\hat\alpha_i-\bar\alpha$3, at cost $\hat\alpha_i-\bar\alpha$4, which is feasible for $\hat\alpha_i-\bar\alpha$5. The reported hyperparameter regime is $\hat\alpha_i-\bar\alpha$6, $\hat\alpha_i-\bar\alpha$7, $\hat\alpha_i-\bar\alpha$8, and $\hat\alpha_i-\bar\alpha$9 in the usual stable regime.

Empirical evaluation is reported on an MLP on MNIST-5k, a ReLU CNN on CIFAR10-5k, and a 2-layer Transformer on SST2-5k. The metric is

$\mathfrak V_{\alpha,\lambda}$0

where $\mathfrak V_{\alpha,\lambda}$1 is the true dominant Hessian projector and $\mathfrak V_{\alpha,\lambda}$2 is the gap-subspace projector. The key findings are that even a modest buffer $\mathfrak V_{\alpha,\lambda}$3 recovers 70–80 % of the dominant component, $\mathfrak V_{\alpha,\lambda}$4 often exceeds 90 %, $\mathfrak V_{\alpha,\lambda}$5 steadily rises during training and plateaus above 0.8, and smaller $\mathfrak V_{\alpha,\lambda}$6 such as $\mathfrak V_{\alpha,\lambda}$7 yields even higher alignment for a given $\mathfrak V_{\alpha,\lambda}$8. The empirical conclusion is that worker disagreement in Local SGD is a cheap, Hessian-free proxy for the sharp, dominant eigendirections.

4. Macroeconomic covariance and the reset-heterogeneity wedge

In “The Inflation of Resetting Workers” (Sun, 31 Mar 2026), the standard wage Phillips curve aggregates away from which workers reset wages when. The paper argues that this aggregation omits a first-order term: the covariance between workers’ cost-push exposure and their reset frequency. Formally, letting $\mathfrak V_{\alpha,\lambda}$9 denote the cost-push exposure and $\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$0 the wage-reset frequency,

$\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$1

In the two-type expositional case with types $\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$2 and weights $\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$3, the within-country covariance is written as

$\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$4

where

$\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$5

is the “salient experienced inflation” of type $\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$6, and $\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$7 is her reset probability.

The object enters the aggregate wage Phillips curve by aggregating each type’s Calvo-wage Phillips curve,

$\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$8

The exposure term satisfies

$\mathrm{Cov}_{(i)}(\epsilon_i,f_i)$9

with $t$0 and $t$1. The resulting heterogeneous-reset Phillips curve is

$t$2

The final term, $t$3, is the “reset-heterogeneity wedge.”

The wedge is identically zero in the standard model because either $t$4 or the baskets $t$5 are identical across $t$6, so the covariance vanishes. Whenever high-exposure workers reset more often and essentials prices jump, $t$7 and the standard Phillips curve omits a first-order term. In this setting, worker-level covariance is not a second-order compositional detail; it directly shifts the aggregate inflation equation.

5. Sufficient statistics, quantitative implications, and policy design

The macroeconomic covariance effect is summarized by two sufficient statistics that are directly computable from micro data (Sun, 31 Mar 2026). The first is Reset-Weighted Experienced Inflation,

$t$8

interpreted as the average inflation rate faced by those workers who are actually renegotiating their wage this period. It replaces $t$9 in the wage-reset first-order condition.

The second is Marginal Wage Setter Inflation,

$w_t^i\in\mathbb R^D$0

where $w_t^i\in\mathbb R^D$1 is the “propagation weight” of the sector $w_t^i\in\mathbb R^D$2 employing type $w_t^i\in\mathbb R^D$3, combining labor share, input-output centrality, and price rigidity. MWSI is interpreted as the sector-weighted covariance between being at the reset margin and experiencing higher inflation. Proposition 4 shows that, to first order, the additional cumulative core inflation relative to the standard model is

$w_t^i\in\mathbb R^D$4

so MWSI is a one-number forecast of the omitted persistence.

Under the euro-area baseline calibration with a 40 percent peak imported-essentials shock, $w_t^i\in\mathbb R^D$5 percent on average across six countries, implying extra cumulative core inflation $w_t^i\in\mathbb R^D$6 pp$w_t^i\in\mathbb R^D$7quarters, approximately 7 percent of total. When monetary policy is delayed by five quarters, the same $w_t^i\in\mathbb R^D$8 raises the aggregate gap to 15.6 pp$w_t^i\in\mathbb R^D$9quarters, approximately 10.3 percent of total, and in a high-MWSI synthetic economy to 40.3 pp$C=\mathbb E[\Delta\,\Delta^\top]$00quarters, approximately 26.4 percent. The cumulative-wage response contains the term

$C=\mathbb E[\Delta\,\Delta^\top]$01

so $C=\mathbb E[\Delta\,\Delta^\top]$02 is the first-order composition correction.

The same-openness experiment isolates within-country composition. Two economies both with 28 percent import share and average $C=\mathbb E[\Delta\,\Delta^\top]$03 differ by Country A with $C=\mathbb E[\Delta\,\Delta^\top]$04 percent and cumulative core $C=\mathbb E[\Delta\,\Delta^\top]$05 pp$C=\mathbb E[\Delta\,\Delta^\top]$06q, versus Country B with $C=\mathbb E[\Delta\,\Delta^\top]$07 and cumulative core $C=\mathbb E[\Delta\,\Delta^\top]$08 pp$C=\mathbb E[\Delta\,\Delta^\top]$09q. The 6.6 percentage-point-quarters difference arises solely because low-income workers both spend more on essentials and reset wages more often. Out of sample, the model correctly predicts the persistence ranking across the UK, the US, and Japan.

Because the covariance wedge is a cross-sectional object the interest rate cannot eliminate, the paper states that a two-instrument mix is strictly better. A targeted subsidy to the bottom-quintile essentials price reduces $C=\mathbb E[\Delta\,\Delta^\top]$10 directly, and Proposition 6(ii) gives the closed-form optimal subsidy:

$C=\mathbb E[\Delta\,\Delta^\top]$11

Table 8 shows that combining moderate tightening with such an essentials subsidy reduces union-wide welfare loss by 32 percent relative to aggressive tightening alone. A plausible implication is that, in this framework, the relevant aggregate statistic for policy is not only average exposure but the covariance between exposure and reset probability.

6. Centered worker-effect covariances in high-dimensional fixed effect regression

In “Ridge Estimation of High Dimensional Two-Way Fixed Effect Regression” (He et al., 7 Jan 2026), the relevant worker-average gap covariance arises in the two-way fixed effect model

$C=\mathbb E[\Delta\,\Delta^\top]$12

where $C=\mathbb E[\Delta\,\Delta^\top]$13 and $C=\mathbb E[\Delta\,\Delta^\top]$14 are worker and firm fixed effects. With worker-incidence matrix $C=\mathbb E[\Delta\,\Delta^\top]$15 and firm-incidence matrix $C=\mathbb E[\Delta\,\Delta^\top]$16, the model is

$C=\mathbb E[\Delta\,\Delta^\top]$17

The ridge estimator with separate penalties $C=\mathbb E[\Delta\,\Delta^\top]$18 solves

$C=\mathbb E[\Delta\,\Delta^\top]$19

and in stacked form

$C=\mathbb E[\Delta\,\Delta^\top]$20

Under a sparse bipartite-graph model and penalties satisfying

$C=\mathbb E[\Delta\,\Delta^\top]$21

the bias and the variance-covariance matrix of the vector of estimated fixed effects converge to deterministic equivalents that depend only on the expected network. The regularized worker Laplacian is

$C=\mathbb E[\Delta\,\Delta^\top]$22

and the deterministic-equivalent variance-covariance matrix is

$C=\mathbb E[\Delta\,\Delta^\top]$23

Letting $C=\mathbb E[\Delta\,\Delta^\top]$24, the covariance of worker-average gaps is

$C=\mathbb E[\Delta\,\Delta^\top]$25

This is a closed-form in terms of the deterministic equivalent $C=\mathbb E[\Delta\,\Delta^\top]$26.

Practical computation proceeds by estimating or fixing the block-model parameters, building $C=\mathbb E[\Delta\,\Delta^\top]$27, forming the regularized expected Laplacian $C=\mathbb E[\Delta\,\Delta^\top]$28, inverting it numerically, and plugging into the deterministic-equivalent formulas for $C=\mathbb E[\Delta\,\Delta^\top]$29 and $C=\mathbb E[\Delta\,\Delta^\top]$30. Under the maintained asymptotic regime with $C=\mathbb E[\Delta\,\Delta^\top]$31, $C=\mathbb E[\Delta\,\Delta^\top]$32, sparse links, bounded expected degrees, and $C=\mathbb E[\Delta\,\Delta^\top]$33, the plug-in serves as a high-probability approximation to the true sampling bias-covariance of the worker fixed effects.

The three uses of worker-average gap covariance are therefore technically distinct but structurally related. In Local SGD it is a covariance of parameter deviations that reveals sharp dominant Hessian directions; in heterogeneous-reset macroeconomics it is a covariance of worker exposure and reset probability that survives aggregation as a first-order wedge; in high-dimensional fixed effect regression it is a covariance of centered estimated worker effects determined by deterministic-equivalent network objects. This suggests that the unifying theme is not a single formula but a recurring methodological claim: worker-level heterogeneity relative to an average can encode geometry, persistence, or sampling uncertainty that disappears in representative or fully aggregated descriptions.