Gradient-Informed Temporal Sampling (GITS)

• GITS is a family of gradient-informed temporal sampling methods that leverage gradient-derived utility signals to allocate temporal resources efficiently.
• It optimizes sampling by selecting coarse start indices in PDE training, echo-specific masks in MRI, and asynchronous events in event cameras for improved accuracy.
• Empirical results demonstrate that integrating gradient cues with coverage constraints reduces errors by up to 38% compared to conventional uniform sampling.

Searching arXiv for the cited GITS-related papers and closely related terminology. Gradient-Informed Temporal Sampling (GITS) denotes temporal allocation schemes in which sample selection, triggering, or acquisition design is driven by gradient-derived informativeness signals rather than by uniform schedules alone. The term is introduced explicitly for budgeted temporal-window selection in autoregressive PDE surrogate training, where pilot-model gradient norms are combined with temporal coverage objectives to optimize rollout accuracy (Wang et al., 18 Mar 2026). Closely related mechanisms appear in multi-echo gradient-echo MRI, where reconstruction-loss gradients are backpropagated through stochastic echo-wise sampling masks, and in event cameras, where asynchronous events are emitted when local spatial gradient states change (Zhang et al., 2021, Lehtonen et al., 2024). This suggests a broader technical pattern: temporal resources are concentrated where gradients indicate high downstream utility, subject to explicit sampling, bandwidth, or stability constraints.

1. Core formulation across domains

A useful unifying view is that GITS couples a temporal decision variable to a gradient-sensitive utility signal under a budgeted selection rule. The temporal decision variable differs by domain: coarse temporal start indices in PDE surrogate training, echo-dependent k-space masks in multi-echo MRI, and asynchronous event times in event cameras. The gradient signal also differs: parameter-space rollout gradients, end-to-end reconstruction-loss gradients through a straight-through estimator, or thresholded spatial image gradients.

Domain	Temporal object	Gradient-informed signal
PDE surrogate training	Shared temporal start indices	Pilot short-rollout gradient norm $s_k=\|g_k\|_2$
Multi-echo GRE MRI	Echo-specific sampling masks $U_j$	Reconstruction-loss gradients through ST-estimated stochastic masks
Event cameras	Event emission times	Changes in ternary spatial gradients $T_X,T_Y$

Despite this common structure, the three formulations are not identical. In the PDE setting, GITS is an explicit combinatorial selector with a monotone submodular objective and a greedy approximation guarantee (Wang et al., 18 Mar 2026). In MRI, “gradient-informed” refers to differentiable acquisition design through a stochastic binary mask layer, optimized jointly with a deep ADMM reconstructor (Zhang et al., 2021). In event cameras, the connection is more architectural than terminological: temporal samples occur when local spatial gradients cross thresholds, so event density tracks spatiotemporal gradient energy rather than raw intensity change (Lehtonen et al., 2024).

A common misconception is that GITS always refers to image-space gradients. The published uses show three distinct meanings of “gradient”: rollout-loss parameter gradients in neural simulators, backpropagated acquisition gradients in MRI, and spatial finite differences in vision sensing. The shared principle is not the representation of the gradient, but its role in steering temporal allocation.

2. Explicit GITS in PDE surrogate training

In its named form, GITS is a budgeted temporal-window selection method for training autoregressive neural simulators on pre-generated PDE trajectories in the offline shared temporal start-index setting (Wang et al., 18 Mar 2026). Let the admissible start-index pool on the coarse time axis be

$$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$

Under budget $K$, a subset $S\subseteq\mathcal{C}$ with $|S|=K$ is selected, and the same subset of starts is reused across all trajectories:

$$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$

The paper’s central claim is that one-step training loss is not the correct target when long-horizon rollout accuracy is the downstream objective. To address this, GITS combines a local model-aware signal with set-level temporal coverage. A lightweight pilot model is first trained on the full candidate pool for $E_p$ epochs, yielding parameters $\theta_p$. For each candidate start $U_j$0, a short rollout of length

$U_j$1

is used to define a candidate-specific short-rollout loss $U_j$2, and the local informativeness score is the pilot gradient norm

$U_j$3

This score is then regularized by two facility-location coverage terms: a global temporal coverage term based on

$U_j$4

and a sliding-window coverage term based on

$U_j$5

The joint objective is

$U_j$6

Because the first term is modular and the coverage terms are monotone submodular facility-location functions, $U_j$7 is monotone submodular, and greedy maximization under $U_j$8 achieves the standard $U_j$9 approximation guarantee (Wang et al., 18 Mar 2026).

The reported experiments use three PDEBench forward tasks—diff-sorp, diff-react, and rdb—and four surrogate backbones—U-Net, FNO, ConvLSTM, and Transformer—with history length $T_X,T_Y$0, train/validation/test trajectory split $T_X,T_Y$1, Adam with $T_X,T_Y$2, AMP, batch size $T_X,T_Y$3, gradient clipping $T_X,T_Y$4, residual $T_X,T_Y$5-prediction, and output clamping to $T_X,T_Y$6 (Wang et al., 18 Mar 2026). At sampling ratio $T_X,T_Y$7, the mean rollout nRMSE over the 12 dataset–backbone configurations is 0.193 for GITS, compared with 0.334 for coverage-only, 0.400 for uniform, 0.400 for PRISM, 0.548 for GradMatch, 0.624 for loss-only, and 0.665 for GLISTER. Across 36 configurations at ratios $T_X,T_Y$8, GITS is best in 27/36 cases and has the lowest overall mean nRMSE 0.219; relative to uniform, it yields a mean reduction of 38.3\% and wins in 30/36 configurations (Wang et al., 18 Mar 2026).

The ablation study is especially important for interpreting the method. Mean nRMSE over 36 configurations is 0.916 for loss-only, 1.172 for grad-only, 0.692 for loss-div, and 0.219 for GITS, with GITS best in 36/36 (Wang et al., 18 Mar 2026). This directly supports the claim that neither local gradients nor coverage alone suffice. The boundary analyses further delimit the regime of validity: a success case such as rdb/FNO/0.10 shows Spearman correlation 0.774 between pilot gradient and empirical utility, whereas a failure case such as diff-sorp/ConvLSTM/0.10 shows negative gradient–utility and loss–utility correlations, and diff-react/Transformer/0.05 exhibits over-dispersion under a highly concentrated utility landscape (Wang et al., 18 Mar 2026).

3. Gradient-informed echo-wise sampling in multi-echo GRE MRI

In multi-echo gradient-echo MRI for quantitative susceptibility mapping (QSM), the signal evolution across echo times $T_X,T_Y$9 is modeled as

$$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$0

where magnitude decays exponentially with TE and phase evolves linearly with TE (Zhang et al., 2021). The multi-coil, multi-echo forward acquisition model is

$$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$1

with echo-specific binary sampling mask $$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$2, Fourier operator $$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$3, coil sensitivity map $$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$4, and i.i.d. Gaussian noise $$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$5. Joint reconstruction minimizes

$$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$6

where $$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$7 is a learned regularizer that exploits cross-echo correlations.

The acquisition-design component extends LOUPE-ST to multi-echo sampling pattern optimization (SPO). For each echo $$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$8, learnable weights $$\mathcal{C}=\{k\in\mathbb{Z}\mid L\le k\le T_c-2\}.$$9 parameterize a probabilistic mask $K$0 through a sigmoid transform and sampling-ratio renormalization. Binary masks are then instantiated stochastically as

$K$1

and a straight-through estimator allows gradients from the reconstruction loss to update the mask parameters. In this sense, the temporal sampling design is gradient-informed: gradients from the end-to-end SSIM objective directly affect the echo-wise sampling probabilities (Zhang et al., 2021). The fixed under-sampling budget is 23\% of k-space, corresponding to approximately 4.35× acceleration, and the study is restricted to 2D Cartesian variable-density sampling with no explicit non-Cartesian or hardware trajectory constraints.

Reconstruction is performed with an unrolled deep ADMM network with $K$2 iterations. The regularizer is implemented by a CNN denoiser, the data-consistency subproblem is solved by conjugate gradient, and both denoiser weights and the ADMM penalty $K$3 are learned. To encode inter-echo structure, a Temporal Feature Fusion (TFF) block is inserted into the denoiser pathway. The recurrent module is repeated over the $K$4 echoes with shared weights; at each step it takes the current single-echo image and a hidden state, and the concatenated hidden states are passed to the denoiser. The input consists of 20 channels corresponding to real and imaginary parts across 10 echoes (Zhang et al., 2021).

The training objective jointly optimizes reconstruction parameters $K$5 and sampling parameters $K$6 by minimizing a channel-wise SSIM loss across ADMM iterations and echoes:

$K$7

After joint training, binary masks $K$8 are sampled from the learned probabilities $K$9 and fixed; the deep ADMM network is then further trained alone with those fixed masks. The implementation uses PyTorch + Adam, batch size 1, 100 epochs, initial learning rate $S\subseteq\mathcal{C}$0, and RTX 2080Ti hardware. The dataset comprises 7 subjects, acquired with 3D MEGRE on a 3T GE scanner, 32-channel head coil, matrix size $S\subseteq\mathcal{C}$1, resolution $S\subseteq\mathcal{C}$2, 10 echoes, TE1 = 1.972 ms, and echo spacing = 3.384 ms. Coils are compressed to 8 virtual coils, and ESPIRiT sensitivities are estimated from a $S\subseteq\mathcal{C}$3 auto-calibration region on the first echo and reused for all echoes (Zhang et al., 2021).

The ablation results on echo-combined image metrics show consistent gains from both acquisition and reconstruction modules:

Method	PSNR	SSIM
Deep ADMM	40.95 ± 3.72	0.9820 ± 0.0257
+ single SPO	41.77 ± 3.24	0.9839 ± 0.0091
+ TFF	42.24 ± 3.28	0.9864 ± 0.0074
+ TFF + single SPO	42.77 ± 3.38	0.9867 ± 0.0076
+ TFF + multi SPO	43.75 ± 3.02	0.9894 ± 0.0058

All improvements with added blocks are statistically significant ($S\subseteq\mathcal{C}$4) except the final row serving as the best overall (Zhang et al., 2021). Against LLR and Multi-echo MoDL, the best Deep ADMM + TFF configuration also leads under manual variable density, single SPO, and multi SPO. The qualitative QSM analysis reports progressive improvements, including veil structures, from manual variable density to single SPO to multi SPO (Zhang et al., 2021). A plausible implication is that echo-specific temporal diversity in k-space, rather than a single shared pattern, is materially beneficial when the reconstruction network can exploit signal evolution across TE.

4. Gradient-triggered temporal sampling in event cameras

The event-camera paper introduces gradient events, a ternary asynchronous event type that encodes temporal changes in local spatial gradients rather than in brightness directly (Lehtonen et al., 2024). For a grayscale image $S\subseteq\mathcal{C}$5, the forward-difference gradients are

$S\subseteq\mathcal{C}$6

with zero padding at the last column or row. A spatial threshold matrix $S\subseteq\mathcal{C}$7 takes values in a threshold set $S\subseteq\mathcal{C}$8; in the reported experiments,

$S\subseteq\mathcal{C}$9

with periodic assignment $|S|=K$0 for $|S|=K$1 (Lehtonen et al., 2024).

These gradients are ternarily quantized:

$|S|=K$2

Quantized amplitudes are $|S|=K$3 and $|S|=K$4. Gradient events $|S|=K$5 then encode changes from a previous ternary state $|S|=K$6 to the current state $|S|=K$7 via a lossless $|S|=K$8 mapping. The receiver can decode the new ternary gradient from $|S|=K$9, so the event stream is a temporal encoder of quantized spatial gradients (Lehtonen et al., 2024).

This architecture yields what can reasonably be described as a GITS-like temporal allocation rule. Events are emitted only when spatial gradients cross thresholds and change ternary state, so temporal samples concentrate at edges and textured structures while global oscillatory illumination is largely cancelled in the logarithmic domain. The paper quantifies this concentration behavior: without resolution compression (RC), overall gradient-event probability is approximately $$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$0, versus brightness-event probability $$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$1 across ECD, MVSEC, and HQF; with RC, combined probability is $$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$2 (Lehtonen et al., 2024). This suggests that the method can deliver higher information content per event even when event bandwidth is comparable to, or lower than, conventional brightness-event streams.

Reconstruction proceeds through the discrete Laplacian derived from quantized gradients and a Poisson solve. Using

$$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$3

the paper reconstructs grayscale frames with successive over-relaxation (SOR) using $$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$4, $$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$5, and $$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$6 across all quantitative results. The reconstructed image is zero-centered and then mean-adjusted to the ground-truth mean, because absolute mean intensity is unobservable from gradients alone (Lehtonen et al., 2024).

Quantitatively, gradient-event reconstruction outperforms brightness-event methods on the event-to-video benchmarks. Without RC, the reported values are ECD: MSE 0.002, SSIM 0.850, LPIPS 0.068; MVSEC: MSE 0.017, SSIM 0.676, LPIPS 0.212; HQF: MSE 0.013, SSIM 0.815, LPIPS 0.125. With RC, they are ECD: 0.002 / 0.828 / 0.091; MVSEC: 0.016 / 0.606 / 0.255; HQF: 0.012 / 0.781 / 0.159 (Lehtonen et al., 2024). On ECD, the best representative brightness-event baseline has MSE $$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$7 and SSIM $$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$8, whereas gradient events achieve MSE 0.002 and SSIM 0.850. The method is also computationally lightweight: per-pixel compute is $$\mathcal{D}(S)=\left\{\bigl(x_{k-L+1:k}^{(n)},x_{k+1}^{(n)}\bigr)\mid n=1,\dots,N,\;k\in S\right\}.$$9, storage requires two ternary states per pixel plus a small threshold pattern, and the architecture is described as comparator- and LUT-friendly (Lehtonen et al., 2024).

5. Comparative interpretation, misconceptions, and boundary conditions

The three formulations clarify that GITS is not a single algorithm but a family of gradient-steered temporal allocation strategies. In PDE surrogate training, the selected objects are coarse start indices shared across trajectories, and the objective is explicit rollout nRMSE (Wang et al., 18 Mar 2026). In multi-echo MRI, the selected objects are binary k-space samples distributed across echo times under a fixed 23\% budget, and the objective is end-to-end SSIM-driven reconstruction and downstream QSM quality (Zhang et al., 2021). In event cameras, the selected objects are asynchronous events emitted when ternary gradient states change, and the objective is improved grayscale reconstruction under bandwidth pressure and oscillatory illumination (Lehtonen et al., 2024).

A second misconception is that local informativeness alone is sufficient. The PDE ablations directly refute this: grad-only has mean nRMSE 1.172, far worse than full GITS at 0.219, and even loss-div remains at 0.692 (Wang et al., 18 Mar 2026). The corresponding lesson in MRI is that a shared single sampling pattern across echoes is inferior to echo-specific multi SPO, and the corresponding lesson in event cameras is that raw brightness change is not the most informative trigger when global flicker can flood the stream (Zhang et al., 2021, Lehtonen et al., 2024). This suggests that temporal diversity or coverage regularization is a recurring requirement when gradient-derived utilities are spatially or temporally clustered.

The boundary conditions are domain-specific. In PDE training, GITS can fail when pilot gradients are misaligned with downstream utility or when coverage becomes over-dispersive under an extremely concentrated utility landscape (Wang et al., 18 Mar 2026). In MRI, the approach is limited to Cartesian 2D variable-density masks with a fixed under-sampling ratio, relies on straight-through discretization heuristics, reuses coil sensitivities estimated from the first echo for all echoes, and is validated on a relatively small dataset of 7 subjects (Zhang et al., 2021). In event cameras, pure gradients do not recover absolute intensity mean, textureless regions produce few or no events, near-threshold noise can induce spurious toggling, and RC can introduce aliasing in high-frequency regions (Lehtonen et al., 2024).

6. Implementation regimes and reproducibility

The PDE formulation of GITS is fully specified at the algorithmic level. The reported practical defaults are short rollout horizon $E_p$0, pilot epochs $E_p$1, and coverage weights $E_p$2. Kernel scales are derived from budget-implied target spacing:

$E_p$3

Mean selector time per configuration over 36 runs is reported as 13.3 s for GLISTER, 11.6 s for PRISM, and 10.9 s for GITS; at ratio 0.05, downstream training time is 272.5 s for GITS versus 233.0 s for uniform (Wang et al., 18 Mar 2026). The experiments use public PDEBench data, PyTorch 2.1, and seeds $E_p$4, but no anonymized code repository accompanied the submission.

The MRI implementation is likewise reproducible at the hyperparameter level. The unrolled reconstructor uses $E_p$5 ADMM iterations; when TFF is absent, the denoiser’s number of convolutional kernels is increased to match memory consumption. The QSM pipeline fits the field $E_p$6 across echoes by nonlinear Levenberg–Marquardt and then computes susceptibility by MEDI dipole inversion (Zhang et al., 2021). Code availability is not stated, although the appendix shows learned sampling patterns.

The event-camera implementation is minimal in parameter count. The reported configuration fixes the threshold mosaic to $E_p$7, the reconstruction parameters to $E_p$8, $E_p$9, $\theta_p$0, and evaluates through the updated evreal event-to-video pipeline on ECD, MVSEC, and HQF (Lehtonen et al., 2024). The resulting system has only local neighborhood dependencies and nearest-neighbor access, making it naturally compatible with GPU- and FPGA-oriented deployments.

Taken together, these works place GITS at the intersection of data valuation, acquisition design, and asynchronous sensing. In its explicit form, it is a monotone-submodular, coverage-regularized selector for rollout-oriented PDE surrogate training (Wang et al., 18 Mar 2026). In related embodiments, it appears as reconstruction-gradient-driven echo-wise mask learning in multi-echo MRI and as thresholded gradient-state triggering in event cameras (Zhang et al., 2021, Lehtonen et al., 2024). The common technical theme is consistent: temporal samples are most useful when they are not merely numerous, but selectively concentrated where gradient-derived signals indicate that they will most improve the downstream objective.