---
title: Gradient-Bridged Co-Training
url: https://www.emergentmind.com/topics/gradient-bridged-co-training
type: topic
---

# Gradient-Bridged Co-Training

Gradient-Bridged Co-Training is an umbrella description for learning procedures in which models or tasks are coupled through gradient-mediated signals rather than only through shared parameters, pseudo-label exchange, or output-level consistency. In the papers most directly associated with this perspective, the “bridge” is implemented in several distinct ways: by explicitly maximizing cross-task loss reduction induced by another task’s gradient in multi-task learning, by routing pixel-wise photometric gradients according to an online disagreement distribution, by converting the outputs of a non-differentiable model into reinforcement-learning rewards for a differentiable model, or by adversarially reshaping gradients so that their origin becomes statistically indistinguishable [2303.05847] [2605.07945] [2604.16378] [1806.08028]. The common thread is that gradient flow itself becomes the object of coordination.

## 1. Conceptual scope and relation to classical co-training

In this usage, Gradient-Bridged Co-Training is not a single canonical algorithm. It is a family resemblance across methods that treat gradients, gradient-induced loss changes, or gradient exposure as the medium through which coupled learners influence one another. Some methods operate within ordinary shared-parameter multi-task learning, some within self-supervised monocular geometry, some across differentiable and non-differentiable model families, and some through auxiliary adversarial critics on gradient tensors [2303.05847] [2605.07945] [2604.16378] [1806.08028].

This broad usage differs from classical co-training in the Blum-Mitchell sense. The relevant papers explicitly distinguish their mechanisms from view-based pseudo-label exchange. CoGrad is described instead as “co-optimization through transfer-aware gradient coupling,” not classical semi-supervised co-training [2303.05847]. CoopNet similarly does not exchange pseudo-labels or features across branches; its interaction occurs through a disagreement statistic that controls where each branch receives gradients [2605.07945]. RCT is reciprocal in the sense of alternating bidirectional adaptation, but supervision is exchanged as embeddings and reward signals rather than as labels [2604.16378]. GREAT is closer to adversarial alignment or gradient-space distillation than to co-training, since the auxiliary model is a critic over gradient tensors rather than a peer predictor [1806.08028].

A useful organizing distinction is between four bridge types present in the literature:

| Method | Bridge variable | Coupling mechanism |
|---|---|---|
| CoGrad | $\Delta^k \bm{L}_{i \to j}$ and $\bm H_j \bm g_i$ | Transfer-aware gradient modification |
| CoopNet | $A(p)$ and central/tail quantiles | Pixel-wise gradient routing and weighting |
| RCT | $Q_\phi(x,a)$ | Reward-mediated policy-gradient update |
| GREAT | Gradient tensors | Adversarial gradient indistinguishability |

This suggests that the unifying object is not necessarily a raw gradient vector. In some cases the bridge is an explicit second-order derivative term, in others a distribution over disagreement values that determines where gradients are permitted, and in others a scalar evaluative signal that is converted into gradients only inside one component of the system.

## 2. Core mathematical patterns

A recurrent pattern is that one learner’s update is evaluated by the effect it has on another learner’s objective. CoGrad formalizes this most directly. For tasks $i$ and $j$, with shared parameters $\theta$, per-task loss $\bm L_t(\theta)$, and gradient $\bm g_t(\theta)=\nabla_\theta \bm L_t(\theta)$, it defines a virtual update
$$
\theta^{k+\tau_i}=\theta^k-\gamma_i \bm{g}_i(\theta^k)
$$
and then defines transfer from task $i$ to task $j$ as
$$
\Delta^k \bm{L}_{i \to j} = \bm{L}_j(\theta^k) - \bm{L}_j(\theta^{k+\tau_i}).
$$
A first-order Taylor expansion yields
$$
\Delta^k \bm{L}_{i \to j} \approx \gamma_i \bm{g}_i^T( \theta^k ) \bm{g}_j(\theta^k),
$$
so the gradient inner product is interpreted as a first-order surrogate for actual cross-task loss reduction rather than as an end in itself [2303.05847].

A second pattern is that the bridge may be constructed from output disagreement and then used to allocate gradients. In CoopNet, the depth-plus-pose branch and the optical-flow branch produce two reconstructions of a target frame, and their per-pixel reconstruction-error difference is
$$
A(p)=\rho\!\left(I_t(p),\hat I^{D,\alpha}_s(p)\right) -\rho\!\left(I_t(p),\hat I^F_s(p)\right).
$$
Central quantiles of the empirical distribution of $A$ define a region $V_\eta = [q_{-\eta},q_{\eta}]$, and that region determines where the rigid branch receives gradients, while the flow branch is trained on all pixels with larger weight on the tails [2605.07945]. Here the bridge is not a direct cross-gradient term; it is a shared statistic derived from the two branches’ competing explanations of the same data.

A third pattern is surrogate bridging across a non-differentiable boundary. In RCT, an LLM defines a stochastic policy $\pi_\theta(a \mid x)$ over binary actions, while a Random Forest supplies an action-conditioned evaluative score
$$
Q_\phi(x,a) = \begin{cases} p_\phi(y=1 \mid x_{\text{RF}}), & \text{if } a=1, \\ 1 - p_\phi(y=1 \mid x_{\text{RF}}), & \text{if } a=0. \end{cases}
$$
This enters the hybrid reward
$$
R(x,a,y) = \lambda R_{\text{task}(a,y)} + (1-\lambda) Q_\phi(x,a),
$$
with $R_{\text{task}(a,y)} = \mathbf{1}[a=y]$, and PPO converts that scalar signal into parameter updates for the LLM [2604.16378]. The RF never participates in backpropagation, but it still shapes the LLM’s gradient trajectory.

A fourth pattern is adversarial coupling in gradient space. GREAT assumes that gradient tensors contain task-, class-, or model-specific statistical information, and uses an auxiliary classifier to predict the origin of a gradient tensor while the main model is trained to make that prediction difficult [1806.08028]. The bridge is therefore an auxiliary min-max game over gradients themselves.

## 3. Transfer-aware coordination in multi-task learning: CoGrad

CoGrad addresses multi-task learning with tasks $\mathcal{T} = \{t\}_{t=1}^T$, shared parameters $\theta$, task-specific parameters $\phi_t$, and shared updates
$$
\theta^{k+1}=\theta^k-\eta \sum_{t \in \mathcal{T}}w_t\nabla_{\theta} \bm L_t(\mathcal{D}_t;{\theta}^k,{\phi}_t^k).
$$
The motivating application is recommendation and advertising, with tasks such as CTR, CVR, and page-view prediction sharing a backbone and having separate heads [2303.05847].

The paper’s main criticism of prior gradient methods is conceptual. PCGrad and GradVac modify gradient directions; GradNorm and MetaBalance homogenize magnitudes; MGDA and CAGrad treat the problem as a multi-objective tradeoff on direction or magnitude. CoGrad argues that such approaches focus on alignment itself rather than on the actual transfer effect. Because shared capacity contains both general/shared knowledge and task-specific knowledge, too much alignment can crowd out task-specific knowledge, whereas too much specialization can reduce cross-task generalization [2303.05847].

From the transfer quantity $\Delta^k \bm L_{i \to j}$, CoGrad derives
$$
\nabla_{\theta} \Delta^k \bm L_{i \to j} = \gamma_i \bm H_j(\theta^k) \bm g_i(\theta^k),
$$
where $\bm H_j(\theta^k)$ is the Hessian of task $j$’s loss with respect to shared parameters. The modified gradient for task $j$ becomes
$$
\widehat{\bm g}_j(\theta^k) = \bm g_j(\theta^k) - \gamma_i \bm H_j(\theta^k) \bm g_i(\theta^k),
$$
and in the general multi-task case
$$
\widehat{\bm g}_i(\theta^k) = \bm g_i(\theta^k) - \sum_{ j \neq i, j \in \mathcal{T} }\gamma_j \bm H_i(\theta^k) \bm g_j(\theta^k), \forall i \in \mathcal{T}.
$$
The shared update then uses the weighted aggregation of these modified gradients, while task-specific parameters are updated normally [2303.05847].

The practical version replaces explicit Hessian computation with
$$
\bm H_j(\theta^k) \bm g_i(\theta^k) = \lambda^k \bm g_j(\theta^k) \odot \bm g_j(\theta^k)\odot \bm g_i(\theta^k),
$$
with $\lambda^k = 1.0$, giving
$$
\widehat{\bm g}_i(\theta^k) = \bm g_i(\theta^k) - \sum_{ j \neq i, j \in \mathcal{T} }\gamma_j \bm g_i(\theta^k) \odot \bm g_i(\theta^k)\odot \bm g_j(\theta^k).
$$
The paper states that exact $\bm H_j g_i$ is too expensive in storage and computation, whereas the approximation makes CoGrad computationally efficient, simple to implement, and adds only negligible computation increase [2303.05847].

Empirically, CoGrad is evaluated on Ali-CCP and Ecomm, and in a 15-day online A/B test on a real advertising system. On Ecomm with Shared Bottom, CoGrad reports CTR GAUC $76.17$ versus STL $76.24$ and CVR GAUC $79.61$; on Ali-CCP with Shared Bottom, CTR AUC is $63.97$ and CVR AUC is $67.78$ [2303.05847]. In the online test, it yields CTR $+2.03\%$, CVR $+4.75\%$, CPC $-1.64\%$, and CPA $-5.23\%$ [2303.05847]. The paper also reports that PCGrad raises gradient similarity the most, whereas CoGrad achieves better overall performance with only a moderate increase in similarity, supporting the claim that maximizing alignment alone is not the right objective [2303.05847].

In the context of Gradient-Bridged Co-Training, the significance of CoGrad is that the bridge is explicitly defined as loss reduction from one task to another. The tasks are not merely prevented from conflicting; they are coupled through a differentiable transference objective.

## 4. Distribution-aware gradient routing in self-supervised geometry: CoopNet

CoopNet studies self-supervised monocular video learning with three networks: a depth network $D_\theta$, a pose network $T_\phi$, and an optical flow network $F_\psi$ [2605.07945]. The standard photometric loss is
$$
\rho(x,y)=\alpha \frac{1-\mathrm{SSIM}(x,y)}{2}+(1-\alpha)\lvert x-y\rvert,
$$
and reconstructions of a target frame $I_t$ are obtained either through rigid reprojection using depth and pose or through a dense flow field [2605.07945].

The paper’s central observation is that naive joint self-supervision is biased because the optical flow branch predicts an unconstrained $2$D displacement field directly, whereas the depth-plus-pose branch must satisfy projective geometry and camera-motion consistency. The flow branch is therefore described as intrinsically better at minimizing photometric error. A sign-based split such as assigning depth-plus-pose the pixels where $A(p)<0$,
$$
A(p)=\rho\!\left(I_t(p),\hat I^{D,\alpha}_s(p)\right) -\rho\!\left(I_t(p),\hat I^F_s(p)\right),
$$
creates competition for supervision rather than cooperation, because the stronger flow network tends to win more pixels and can starve the rigid branch [2605.07945].

CoopNet replaces winner-take-all routing with a quantile-based distribution model. Let $q_\eta$ denote the $(0.5+\eta)$-quantile of the density of $A$, and define
$$
V_\eta = [q_{-\eta},q_{\eta}].
$$
Then the depth-plus-pose branch receives gradients only from pixels in the central interval $V_\eta$, where the two branches approximately agree, while the flow branch is trained on all pixels with larger weight on the tails [2605.07945]. The split losses are
$$
L_\delta=\sum_{p\in P} w(p)\,\rho\!\left(I_t(p),\hat I^F_s(p)\right),
$$
$$
L_{\theta,\alpha}= \sum_{p\in A^{-1}(V_\eta)} \rho\!\left(I_t(p),\hat I^{D,\alpha}_s(p)\right),
$$
and
$$
L_{\text{CoopNet}} = L_{\theta,\alpha}+L_\delta.
$$
The quantiles are computed on the fly every epoch using the $P^2$ streaming quantile algorithm, and the neighborhood used in the current epoch is determined from the previous epoch’s quantile values [2605.07945].

A further regularization prior is defined from normalized vector-flow mismatch:
$$
\Delta_{\text{flow}(p)} = \frac{\left|F_{D,\alpha}(p)-F_s(p)\right|} {\left|F_{D,\alpha}(p)\right|^2+\left|F_s(p)\right|^2},
$$
and the refined rigid-valid set is
$$
V = V_\eta \cap V_\zeta, \qquad V_\zeta = V_{\text{flow},x}\cap V_{\text{flow},y}.
$$
This is intended to reduce contamination from moving objects in low-texture or homogeneous regions [2605.07945].

The full objective is
$$
L_{\text{final}} = L_{\text{CoopNet}} + \lambda_{gc}L_{gc} + \lambda_{fwd,bwd}L_{fwd,bwd} + \lambda_s L_s + \lambda_{ep}L_{ep} + \lambda_{var}L_{var},
$$
with $\lambda_{gc}=0.001$, $\lambda_{fwd,bwd}=0.001$, $\lambda_s=0.01$, $\lambda_{ep}=0.001$, and $\lambda_{var}=10^{-6}$ [2605.07945]. Training uses PyTorch, Adam, $\beta_1=0.99$, $\beta_2=0.999$, $30$ epochs, batch size $4$, learning rate $10^{-4}$ reduced to $10^{-5}$ after $20$ epochs, a burn-in of $5$ epochs during which depth-plus-pose are trained with Monodepth2, and hyperparameters $\eta = 0.15$ and $\zeta = 0.25$ [2605.07945].

The paper’s most relevant empirical claim is that the major gains come from $L_{\text{CoopNet}}$ itself and that the subsidiary loss benefits are marginal as compared to $L_{\text{CoopNet}}$ [2605.07945]. On KITTI $192\times$, example depth results include Monodepth2 with Abs Rel $0.115$, SGDepth with $0.117$, and CoopNet with $0.113$ [2605.07945]. Qualitative depth maps are reported to show better handling of thin structures, high-texture regions, and moving objects [2605.07945].

For Gradient-Bridged Co-Training, CoopNet illustrates a form of bridge that is indirect but still optimization-level: outputs of one branch affect where the other branch is allowed to receive gradients. This suggests a broader notion of gradient bridging in which the key operation is adaptive control over gradient exposure rather than direct gradient arithmetic.

## 5. Reward-mediated bridging across incompatible model families: Reciprocal Co-Training

RCT couples a differentiable ClinicalBERT classifier with a non-differentiable Random Forest classifier for binary prediction from tabular clinical or biomedical data [2604.16378]. The differentiable component consumes a deterministic textual serialization of each tabular record in a standardized patient-card format, while the RF continues to operate on structured variables [2604.16378].

The LLM is denoted $f_\theta$ and defines a stochastic binary policy
$$
\pi_\theta(a \mid x), \quad a \in \{0,1\},
$$
where $x$ is the textualized record [2604.16378]. The backbone is frozen; only LoRA adapters inserted into attention layers, the classification head, and the value head are updated during PPO training [2604.16378]. From the final hidden layer [CLS] token, the model produces
$$
h_\theta(x) \in \mathbb{R}^{768},
$$
which is reduced by PCA to $5$ principal components,
$$
\tilde{h}_\theta(x) = \text{PCA}(h_\theta(x)),
$$
and appended to the original RF input to form
$$
x_{\text{RF}} = [x_{\text{tab}}, \tilde{h}_\theta(x)].
$$
This is the forward transfer path from LLM to RF [2604.16378].

The feedback path runs in the opposite direction. The RF, denoted $g_\phi$, outputs a probability estimate $p_\phi(y = 1 \mid x_{\text{RF}})$ and evaluates each sampled LLM action through
$$
Q_\phi(x,a) = \begin{cases} p_\phi(y=1 \mid x_{\text{RF}}), & \text{if } a=1, \\ 1 - p_\phi(y=1 \mid x_{\text{RF}}), & \text{if } a=0. \end{cases}
$$
This enters the hybrid reward
$$
R(x,a,y) = \lambda R_{\text{task}(a,y)} + (1-\lambda) Q_\phi(x,a),
$$
with
$$
R_{\text{task}(a,y)} = \mathbf{1}[a=y].
$$
The LLM objective is
$$
J(\theta) = \mathbb{E}_{x,a} \left[ R(x,a,y) \right],
$$
and PPO uses the clipped surrogate
$$
\begin{aligned}
L_{\text{PPO}}(\theta) = \mathbb{E} \Big[ & \min \big( r_t(\theta) A_t, \operatorname{clip}(r_t(\theta), 1-\epsilon, 1+\epsilon) A_t \big) \Big] \\
& + c_1 L_{\text{value}} - c_2 H(\pi_\theta),
\end{aligned}
$$
where
$$
r_t(\theta) = \frac{\pi_\theta(a \mid x)} {\pi_{\theta_{\text{old}}}(a \mid x)}.
$$
The paper includes these PPO equations but does not specify how $A_t$ is computed, whether generalized advantage estimation is used, or the values of $\epsilon$, $c_1$, and $c_2$ [2604.16378].

Training is alternating and iterative:
$$
\theta^{(t+1)} \approx \arg\max_\theta \, J(\theta; \phi^{(t)}), \qquad
\phi^{(t+1)} = \arg\min_\phi \, L_{\text{RF}}(\phi; \theta^{(t+1)}).
$$
At each outer iteration, the RF is held fixed while PPO updates the LLM; then the updated LLM produces new embeddings and the RF is retrained on the augmented features [2604.16378]. Early stopping uses patience $=5$ on validation ROC-AUC [2604.16378].

On the MS dataset, the paper reports RF ROC-AUC $0.672 \to 0.721$ and LLM ROC-AUC $0.644 \to 0.700$, with RF PR-AUC $0.523 \to 0.607$ and LLM PR-AUC $0.523 \to 0.539$ [2604.16378]. On Breast Cancer, RF ROC-AUC is reported as $0.988 \to 0.999$ and LLM ROC-AUC as $0.761 \to 0.999$; on Diabetes, RF ROC-AUC is $0.826 \to 0.829$ and LLM ROC-AUC is $0.817 \to 0.826$ [2604.16378]. The ablation table further reports that iterative refinement improves both models on MS, with LLM ROC-AUC $0.700$ full versus $0.694$ single-pass and RF ROC-AUC $0.721$ full versus $0.705$ single-pass [2604.16378].

Within a Gradient-Bridged Co-Training perspective, RCT is important because it shows that the bridge need not be an actual derivative through the full system. The differentiable model receives a scalar evaluative signal from the non-differentiable model, and policy gradients convert that signal into an update. This is a surrogate, reward-mediated bridge rather than end-to-end backpropagation.

## 6. Adversarial alignment in gradient space: GREAT

GREAT, “GRadiEnt Adversarial Training,” starts from the premise that gradient tensors contain latent information about whatever tasks are being trained and can therefore be used as objects of supervision [1806.08028]. The backpropagation relation
$$
\delta^l=(w^{l+1})^T\delta^{l+1}\odot \sigma'(z^l),
$$
with
$$
\delta^L=\nabla_a C \odot \sigma'(z^L),
$$
is used to motivate the claim that a layer’s gradient depends on both the loss and succeeding weights [1806.08028]. The method introduces an auxiliary network that classifies the origin of a gradient tensor and a sign-reversed adversarial signal
$$
\varrho^{l+1}=-w^{l+1}\varrho^{l}\odot \sigma'(z^l),
$$
so that the main network is trained both for its primary task and to fool the auxiliary gradient classifier [1806.08028].

In adversarial robustness, GREAT aims to make gradients class-agnostic. The paper writes the intended condition as
$$
\nabla J(\theta,x_i,y_i)= \nabla J(\theta,x_i,\hat y_i) \quad,\quad \forall \hat y_i\neq y_i,\ \forall i\in N.
$$
The defense objective is
$$
\min_{\theta} \hat{J}(\theta,x,y) + \alpha \max_{\acute\theta} J(\acute{\theta}, \nabla \bar J(\theta,x,y), y),
$$
and GREAT is combined with GREACE, which modifies the backward gradient of cross-entropy as
$$
\nabla_a \hat C \mapsto \nabla_a C + \beta * \sigma(\acute a)\mathds{1}_{\hat y \neq y}.
$$
On CIFAR-10 under non-targeted FGSM, the reported accuracies are baseline $32.75$, adversarial training $56.88$, GREACE $77.90$, GREAT $47.51$, and GRE(AT+CE) $81.28$; under non-targeted iFGSM they are baseline $1.99$, adversarial training $16.73$, GREACE $72.40$, GREAT $15.45$, and GRE(AT+CE) $77.04$ [1806.08028]. The paper explicitly notes that GREAT alone is not enough against strong iterative attacks and that the large gains come from GREAT plus GREACE [1806.08028].

In knowledge distillation, the gradients compared are teacher and student input gradients,
$$
g_s = \nabla J(\theta,x,y), \qquad g_t = \nabla J(\tau,x,y),
$$
and the binary discriminator objective is
$$
(1-\alpha)\min_{\theta} J(\theta,x,y) + \alpha \min_{\theta}\max_{\omega} D(\theta,\omega,x,y),
$$
with
$$
D(\theta,\omega,x,y) = E_{t\sim \nabla T(x)} \log f(t,\omega) + E_{s\sim \nabla J(\theta,x,y)} \log(1-f(s,\omega)).
$$
On CIFAR-10 with CNN-5 student and ResNet-18 teacher, GREAT reports $85.72 / 66.55$ in dense and $5\%$ sparse regimes, compared with baseline $84.74 / 65.41$ and distillation $85.69 / 66.45$ [1806.08028]. On CIFAR-10 with ResNet-18 student and ResNeXt teacher, GREAT reports $93.43 / 67.80$, while distillation reports $93.65 / 67.69$ [1806.08028]. The paper emphasizes that GREAT is especially useful in the sparse-data regime and is less hyperparameter-sensitive than temperature-based distillation [1806.08028].

In multi-task learning, GREAT introduces Gradient Alignment Layers (GALs), one per task, inserted between a shared encoder and each task decoder, active only during the backward pass and dropped at inference [1806.08028]. The task-specific gradients with respect to the last shared encoder feature tensor are
$$
g_i^f \gets \nabla J_i(\omega_i, x, y_i),
$$
and GALs scale them elementwise as $g_i^f \gamma_i$ [1806.08028]. The multitask objective is written as
$$
\min_{\theta,\omega_{1,\cdots N}}\sum_i J_i(\theta,\omega_i,\gamma_i,x,y_i) + \max_{\acute\theta,\gamma_{i\cdots N}} \acute J(\acute\theta,\nabla J_i(\omega_i,x,y_i)\gamma_i,\acute y).
$$
On NYUv2, GREAT reports Depth RMSE $0.779$, Normal loss $0.167$, and Keypoint RMSE $0.382$, compared with Equal weighting $0.861/0.207/0.407$, Uncertainty weighting $0.796/0.192/0.389$, and GradNorm $0.810/0.169/0.377$ [1806.08028].

For Gradient-Bridged Co-Training, GREAT provides a methodological template in which gradients are treated as cross-model or cross-task messages and an auxiliary discriminator becomes the mechanism that regularizes those messages.

## 7. Comparative interpretation, misconceptions, and limitations

Across these papers, several distinct meanings of “bridge” appear. CoGrad uses explicit cross-task transfer and a cross-task Hessian-gradient product [2303.05847]. CoopNet uses a shared disagreement statistic and quantile-based masking [2605.07945]. RCT uses a black-box evaluator that returns scalar action scores and relies on PPO to propagate their effect into the LLM [2604.16378]. GREAT uses an auxiliary adversary over gradient tensors and gradient reversal [1806.08028]. A plausible implication is that Gradient-Bridged Co-Training is best understood as a design principle rather than a single optimization family.

A frequent misconception is to equate all such methods with gradient alignment. The surveyed papers repeatedly distinguish more specific objectives. CoGrad argues that maximizing cosine similarity alone can over-privilege general/shared knowledge and crowd out task-specific knowledge [2303.05847]. CoopNet is not GradNorm-style balancing of task losses by gradient magnitudes and not PCGrad-style conflict projection; it is a data-dependent partition of training signal [2605.07945]. RCT does not differentiate through the RF and therefore should not be described as end-to-end joint training; its bridge is policy-gradient mediated [2604.16378]. GREAT is not inherently cooperative in the ordinary sense, because its main criterion is adversarial indistinguishability in gradient space [1806.08028].

The assumptions and failure modes are correspondingly heterogeneous. CoGrad relies on local first-order Taylor approximations and a heuristic Hessian approximation $\bm H_j g_i \approx \bm g_j \odot \bm g_j \odot \bm g_i$ [2303.05847]. CoopNet assumes that rigid pixels cluster near the center of the disagreement distribution and that moving pixels populate the tails, while also acknowledging that photometric loss remains weak in homogeneous regions and introducing $\Delta_{\text{flow}}$ as a corrective prior [2605.07945]. RCT depends on meaningful scalar confidence estimates from the RF, stable alternating updates, and useful LLM embeddings for the RF feature space; the paper also notes RL instability, reward sensitivity, lack of theoretical convergence guarantees, and RF oscillations across iterations in the MS dataset [2604.16378]. GREAT depends on the hypothesis that gradients encode transferable information and, in robustness settings, can devolve into gradient obfuscation unless paired with a stronger primary loss such as GREACE [1806.08028].

Taken together, these works support a technically specific interpretation of Gradient-Bridged Co-Training: learning systems can be coupled through gradient-derived quantities even when they do not share the same architecture, hypothesis class, or optimization regime. The bridge may be direct, as in transfer-maximizing gradient modification; indirect, as in disagreement-controlled gradient routing; surrogate, as in reward-mediated policy gradients; or adversarial, as in gradient-space indistinguishability. What remains constant is that cooperation is formulated not only at the level of predictions or features, but at the level of how updates are generated, filtered, or redirected [2303.05847] [2605.07945] [2604.16378] [1806.08028].

Source: https://www.emergentmind.com/topics/gradient-bridged-co-training