Dynamic Self-Supervised Loss in Seismic Processing

Updated 8 July 2026

Dynamic self-supervised loss is an adaptive objective that updates its target, weighting, or regulation terms during training to enable automated amplitude matching in seismic multiple suppression.
It leverages a trainable scalar and uncertainty-weighted composite loss to automatically balance reconstruction and constraint terms, eliminating manual tuning.
This approach enhances model robustness and efficiency by adapting to evolving data structures, reducing training epochs, and preventing representation collapse.

Dynamic self-supervised loss, in the cited literature, denotes self-supervised objectives whose effective target, weighting, or regulating term is not fixed a priori but changes during optimization or is learned jointly with model parameters. In seismic processing, "Adaptive Self-Supervised Surface-Related Multiple Suppression" formulates this idea through a trainable scaling factor for injected multiples and a composite loss with homoscedastic uncertainty-based adaptive weighting, thereby eliminating manual amplitude tuning in an unlabeled setting (Song et al., 29 Apr 2026). Related SSL work uses dynamic weighting between feature-decorrelation and sample-uniformity losses, data-derived pseudo-whitening targets, and learnable pairwise semantic regulators, indicating that “dynamic” refers not to a single loss formula but to a family of adaptive objective designs (Kokilepersaud et al., 18 May 2025, Mohamadi et al., 2024, Song et al., 2024).

1. Problem setting and motivation

Effective suppression of surface-related multiples is essential to prevent imaging artifacts and erroneous structural interpretations. In the seismic setting, conventional approaches rely on accurate priors or subsurface model knowledge, while supervised learning methods require labeled data that are impractical to obtain for real seismic data. A recently proposed SSL framework addressed these limitations by integrating multi-dimensional convolution (MDC) for multiple generation with a two-stage training strategy, eliminating the need for both prior knowledge and labeled data. Its remaining limitation was the manual selection of a scaling factor to match the amplitudes between the MDC-generated multiples and the true multiples, which introduced subjectivity and limited practical applicability (Song et al., 29 Apr 2026).

The adaptive formulation replaces that manual choice with a learnable scalar and merges the earlier warm-up and IDR procedure into a unified single-stage training pipeline. This change is significant because the scaling mismatch is not peripheral: the one-shot MDC prediction shares the same spatial and temporal structure as the true multiples but may differ in amplitude or wavelet. The dynamic loss therefore addresses a concrete identifiability issue inside the self-supervised construction rather than adding a generic regularizer (Song et al., 29 Apr 2026).

A broader motivation appears in SSL research outside geophysics. AdaDim argues that fixed weighting between dimension-contrastive and sample-contrastive terms fails to reflect the observed dynamics of $H(R)$ and $I(R;Z)$ during training, while theoretical work on representation learning dynamics shows that naïve SSL objectives can lead to dimension collapse unless additional constraints couple the coordinates of the representation (Kokilepersaud et al., 18 May 2025, Esser et al., 2023). This suggests that dynamic SSL losses are often introduced when a static objective imposes the wrong invariance, the wrong trade-off, or the wrong geometry over the course of optimization.

The adaptive seismic method begins from raw shot gathers $d_{\rm raw}$ and generates predicted surface-related multiples through a single MDC operation,

$d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$

This one-shot MDC implements the free-surface feedback model. Each predicted multiple $d_{\rm m}$ shares the same spatial and temporal structure as the true multiples but may differ in amplitude or wavelet (Song et al., 29 Apr 2026).

The network $f_{\theta}$ is a modified 2D U-Net with five encoding blocks and five symmetric decoding blocks. Each encoding block consists of $3\times 3$ conv $\rightarrow$ LeakyReLU $\rightarrow 2\times 2$ max-pool, doubling channels; each decoding block consists of $2\times 2$ up-sample $I(R;Z)$ 0 conv $I(R;Z)$ 1 LeakyReLU. Skip-connections bridge corresponding encoder and decoder layers. A final $I(R;Z)$ 2 convolution reduces the output to one channel, yielding the predicted, multiple-suppressed gather

$I(R;Z)$ 3

The defining single-stage input construction is

$I(R;Z)$ 4

where $I(R;Z)$ 5 is no longer manually chosen but treated as a trainable scalar. Rather than the two-stage warm-up+IDR strategy of Cheng et al. (2025), each forward pass therefore uses a single “noisier” input whose multiple content is modulated by a parameter learned jointly with the network. In this formulation, the dynamic part of the self-supervised loss is inseparable from the data construction step itself, because the loss depends on a trainable perturbation of the raw gather (Song et al., 29 Apr 2026).

3. Learnable scaling and composite uncertainty-weighted loss

The learnable scaling factor is defined directly in the input,

$I(R;Z)$ 6

It is initialized at a small non-zero value, for example $I(R;Z)$ 7, and updated by back-propagation alongside the network parameters $I(R;Z)$ 8. The stated purpose is to make the amplifier on MDC multiples dynamic, allowing the network to “discover” the correct amplitude match to the true multiples in $I(R;Z)$ 9 (Song et al., 29 Apr 2026).

The loss has two explicit components. The reconstruction term is

$d_{\rm raw}$ 0

which encourages the network to map the noisier input back to the original raw data, that is, to suppress only the multiples. The scaling-constraint term is

$d_{\rm raw}$ 1

which prevents $d_{\rm raw}$ 2 from collapsing to zero, described as the trivial solution, by gently anchoring it toward a mid-level value of $d_{\rm raw}$ 3 (Song et al., 29 Apr 2026).

The composite objective then uses homoscedastic uncertainty weighting following Kendall et al. (2018). Each loss term is associated with a learnable uncertainty $d_{\rm raw}$ 4, with $d_{\rm raw}$ 5 and $d_{\rm raw}$ 6 initialized, for example, to $d_{\rm raw}$ 7 and updated by gradient descent alongside $d_{\rm raw}$ 8 and $d_{\rm raw}$ 9. The factor $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 0 down-weights a loss term if its associated $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 1 grows large, interpreted as high uncertainty, while the $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 2 terms prevent $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 3. The resulting construction automatically balances the contributions of the multiple loss terms without manual tuning (Song et al., 29 Apr 2026).

A common misconception is that a dynamic SSL loss is simply “less constrained” than a static one. In this case the opposite is true: the trainable component $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 4 is not left unconstrained, but is coupled to the reconstruction term and explicitly regularized by $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 5. The dynamic aspect lies in joint optimization, not in the removal of structure from the objective.

4. Dynamic behavior as implicit regularization

The adaptive seismic formulation attributes an implicit regularization effect to the evolving scaling factor. As $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 6 changes during training, the energy of the injected MDC multiples varies from batch to batch, forcing the network to learn morphological features rather than over-fit a single amplitude ratio. The paper describes this as acting much like a data augmentation in amplitude space and improving generalization across datasets with different multiple strengths (Song et al., 29 Apr 2026).

The uncertainty-weighted composite loss provides a second adaptive mechanism. Because $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 7 and $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 8 respond to the evolving scales of $d_{\rm m}\;=\;\mathrm{MDC}(d_{\rm raw},d_{\rm raw})\,.$ 9 and $d_{\rm m}$ 0, neither term is allowed to dominate or vanish. The stated consequence is balanced optimization: the network cannot cheat by driving $d_{\rm m}$ 1 without incurring a large $d_{\rm m}$ 2 penalty, yet it is also not forced to over-emphasize the constraint at the expense of reconstruction (Song et al., 29 Apr 2026).

A cross-paper comparison shows that dynamic SSL objectives vary in what exactly is adapted.

Method	Dynamic component	Stated role
Adaptive multiple suppression (Song et al., 29 Apr 2026)	Trainable $d_{\rm m}$ 3; learnable $d_{\rm m}$ 4	Amplitude matching; automatic loss balancing
GUESS (Mohamadi et al., 2024)	Batch-derived $d_{\rm m}$ 5 in pseudo-whitening target	Data-dependent uncertainty in invariance enforcement
AdaDim (Kokilepersaud et al., 18 May 2025)	$d_{\rm m}$ 6	Shift from $d_{\rm m}$ 7 to $d_{\rm m}$ 8
DSA (Song et al., 2024)	$d_{\rm m}$ 9 and $f_{\theta}$ 0	Dynamic aggregation and separation, robust to outliers
VICReg-Dual-Gating (Krishna et al., 2023)	Budget loss tied to $f_{\theta}$ 1	Joint SSL and FLOPs control

GUESS replaces strict whitening with a pseudo-whitening target

$f_{\theta}$ 2

where $f_{\theta}$ 3 is a batch-derived cross-correlation from autoencoder latents; the loss is therefore uncertain about forcing all off-diagonal terms to zero and instead penalizes deviations from data-derived off-diagonals (Mohamadi et al., 2024). AdaDim uses

$f_{\theta}$ 4

so that effective rank drives a gradual shift from feature decorrelation to sample uniformity (Kokilepersaud et al., 18 May 2025). DSA defines dynamic pairwise weights

$f_{\theta}$ 5

so that pairwise attraction or repulsion becomes instance-wise and learnable rather than fixed by a preset contrastive template (Song et al., 2024). This comparison suggests that dynamic SSL is best understood as an adaptive control layer over invariance, redundancy reduction, or pairwise geometry rather than as a single named algorithm.

5. Theoretical context in representation learning dynamics

Theoretical analysis of SSL dynamics provides a useful backdrop for understanding why adaptive objectives are introduced. In "Representation Learning Dynamics of Self-Supervised Models" (Esser et al., 2023), a naïve extension of multivariate regression dynamics to SSL leads to trivial scalar representations and therefore dimension collapse. For linear two-layer networks with orthogonality constraints,

$f_{\theta}$ 6

optimization is performed on the product of two Grassmannians, and the projected continuous-time dynamics are

$f_{\theta}$ 7

The resulting width-independent dynamics couple representation coordinates through a data-dependent matrix $f_{\theta}$ 8, rather than allowing the output coordinates to evolve independently (Esser et al., 2023).

The same work shows that, under the constrained trace-form objective, if all eigenvalues of $f_{\theta}$ 9 are positive then $3\times 3$ 0, while if $3\times 3$ 1 has negative eigenvalues then $3\times 3$ 2 converges to the eigenvectors associated with the most negative eigenvalues. In particular, when a $3\times 3$ 3-dimensional embedding is requested and $3\times 3$ 4 has at least $3\times 3$ 5 negative eigenvalues, the flow locks in $3\times 3$ 6 orthogonal directions and avoids collapse (Esser et al., 2023). The theoretical point is not that every dynamic SSL loss implements Grassmann-manifold optimization, but that fixed, decoupled objectives can have pathological dynamics, whereas adaptive or constraint-aware objectives alter those dynamics.

AdaDim supplies a complementary empirical account. It reports that early training is characterized by sharp increases in $3\times 3$ 7 caused by feature decorrelation, with $3\times 3$ 8 rising as $3\times 3$ 9 tracks $\rightarrow$ 0 closely, while later training is characterized by more uniform eigenvalues, modest continued growth in $\rightarrow$ 1, and a plateau or decline in $\rightarrow$ 2. Its adaptive coefficient therefore moves from near $\rightarrow$ 3 to near $\rightarrow$ 4, shifting emphasis from $\rightarrow$ 5 to $\rightarrow$ 6 as the representation geometry changes (Kokilepersaud et al., 18 May 2025). Taken together, these results suggest that dynamic SSL losses are often motivated by training dynamics that are structurally non-stationary.

6. Empirical validation and significance

In the adaptive seismic study, synthetic layered and Otway models show that training epochs are reduced from $\rightarrow$ 7 on Otway and $\rightarrow$ 8 on the layered model versus SSL-MDC, attributed to single-stage training (Song et al., 29 Apr 2026). Shot-by-shot residuals show that the proposed method suppresses multiples at identical spatial-temporal positions to MDC predictions, with residuals qualitatively matching the true primaries. In Otway migration, raw migration exhibits spurious reflectors, both methods remove the artifact, SSL-MDC over-suppresses a legitimate reflector, and the adaptive method preserves it. Yellow-boxed regions in migrated images show superior structural consistency for the adaptive method, described as no new artifacts and correct layer geometry. In CMP-stack analysis, the extracted multiples, defined as the difference between raw and suppressed stacks, waveform-by-waveform at CMP 180 align almost perfectly with the MDC reference, demonstrating accurate amplitude and phase matching (Song et al., 29 Apr 2026).

Field-data results on Viking Graben Line 12 show that single-stage training with $\rightarrow$ 9 epochs again sufficed. In shot gathers, deeper-zone and far-offset multiples are more completely removed by the adaptive method than by SSL-MDC. In migration, raw data contain densely packed high-frequency events at depth; both methods clean them, but the adaptive result exhibits fewer artifacts and sharper layering. CMP-stack trace comparisons at CMP 10 again show excellent waveform alignment between the suppressed multiples and MDC-predicted multiples (Song et al., 29 Apr 2026).

These results define the practical significance of dynamic self-supervised loss in this domain. The method promotes $\rightarrow 2\times 2$ 0 to a trainable parameter, enforces it with a light $\rightarrow 2\times 2$ 1 constraint, and uses homoscedastic uncertainties to self-tune the balance between reconstruction and scaling constraint, yielding a fully automated, single-stage SSL pipeline that obviates manual amplitude tuning, eliminates the costly IDR stage, and delivers equal or improved multiple-suppression performance on both synthetic and field data (Song et al., 29 Apr 2026).

More broadly, the cited SSL literature suggests a common pattern. GUESS dynamically adapts the whitening target to data-derived uncertainty, AdaDim dynamically blends dimension-contrastive and sample-contrastive objectives, DSA dynamically learns which sample pairs should be aggregated or separated, and VICReg-Dual-Gating couples SSL to a budget-aware regularizer so that dense and gated encoders co-evolve from scratch (Mohamadi et al., 2024, Kokilepersaud et al., 18 May 2025, Song et al., 2024, Krishna et al., 2023). This suggests that dynamic self-supervised loss has become a general strategy for reducing the mismatch between a fixed objective and the changing statistical structure encountered during training.