Glocal-IB for Time Series Imputation

• Glocal Information Bottleneck is a model-agnostic paradigm that combines local point-wise reconstruction with a mutual-information based global alignment.
• It employs an information bottleneck formulation by minimizing KL divergence and applying contrastive objectives to maintain coherent latent representations.
• Experimental results show significant improvements in imputation accuracy under high missingness, achieving up to 40% MSE reduction across multiple datasets.

Glocal Information Bottleneck (Glocal-IB) is a model-agnostic training paradigm for time series imputation (TSI) that augments local reconstruction objectives with a mutual-information-based global alignment to address an optimization dilemma that becomes acute under high-rate missingness. In the formulation introduced in "Glocal Information Bottleneck for Time Series Imputation" (Yang et al., 6 Oct 2025), conventional point-wise objectives such as MAE and MSE are treated as preserving local information, whereas the coherence of the latent representation distribution is treated as global information. The central claim is that, under severe masking, purely local reconstruction can yield low training loss while producing poor imputations and distorted latent representations at inference; Glocal-IB addresses this by combining regularization, local reconstruction, and explicit latent alignment between masked inputs and their originally observed counterparts (Yang et al., 6 Oct 2025).

1. Problem setting and the optimization dilemma

Time series imputation aims to recover missing values in partially observed temporal data. The paper formalizes the setting with an original multivariate time series $X \in \mathbb{R}^{N \times T}$, a binary mask $M \in \{0,1\}^{N \times T}$, and a masked input $X^{o} = X \odot M$; the model outputs an imputed sequence $\hat{X}$, with entries satisfying $M=0$ constituting the imputation targets (Yang et al., 6 Oct 2025). The work focuses on high missingness regimes, including missing rates of $10\%$, $30\%$, $50\%$, $70\%$, and $90\%$.

The paper distinguishes two kinds of signal. Local information is the point-wise reconstruction accuracy of masked entries, typically captured by MAE or MSE. Global information is the distributional structure and semantic regularities reflected in the latent representation $M \in \{0,1\}^{N \times T}$0, including properties such as coherent temporal dynamics or smooth manifold structure. The reported optimization dilemma is that existing encoder-decoder TSI models trained only with point-wise reconstruction can exhibit low training loss yet poor imputations at inference, together with distorted latent representation distributions when inputs are heavily masked (Yang et al., 6 Oct 2025).

This diagnosis is important because it reframes failure under high missingness as more than a numerical reconstruction problem. The paper argues that local objectives alone do not provide global guidance, so the model can overfit local noise induced by masking instead of learning the underlying data structure. In that sense, Glocal-IB is not merely a new loss term but a reformulation of TSI training around the tension between local recovery and global latent stability.

2. Information-bottleneck formulation

The method is grounded in the classical Information Bottleneck (IB) principle, which seeks latent representations that are compact yet informative. In its standard form, the IB objective is

$M \in \{0,1\}^{N \times T}$1

with the usual Markov assumption $M \in \{0,1\}^{N \times T}$2 (Yang et al., 6 Oct 2025). For TSI, the paper specializes the setup by treating $M \in \{0,1\}^{N \times T}$3 as input and $M \in \{0,1\}^{N \times T}$4 as target, yielding

$M \in \{0,1\}^{N \times T}$5

In this specialization, $M \in \{0,1\}^{N \times T}$6 is the encoder, $M \in \{0,1\}^{N \times T}$7 is the decoder, and $M \in \{0,1\}^{N \times T}$8 is the prior. The interpretation given in the paper is direct: minimizing $M \in \{0,1\}^{N \times T}$9 suppresses noise in the latent code tied to corrupted inputs, while maximizing $X^{o} = X \odot M$0 retains information required for recovery (Yang et al., 6 Oct 2025).

Glocal-IB implements this principle through the composite objective

$X^{o} = X \odot M$1

The three terms are defined as follows.

The regularization term is

$X^{o} = X \odot M$2

which upper-bounds $X^{o} = X \odot M$3. The local term is

$X^{o} = X \odot M$4

under a Gaussian decoder $X^{o} = X \odot M$5, and is typically evaluated over masked entries. The global term is the defining extension of Glocal-IB: a Global Alignment loss derived from a tractable mutual information approximation, used to align latent representations of masked inputs with those of their originally observed counterparts (Yang et al., 6 Oct 2025).

The encoder is parameterized as a diagonal Gaussian,

$X^{o} = X \odot M$6

and the latent is obtained by reparameterization,

$X^{o} = X \odot M$7

At inference, the paper states that one can use $X^{o} = X \odot M$8 deterministically or sample $X^{o} = X \odot M$9 (Yang et al., 6 Oct 2025).

3. Global alignment and latent-structure preservation

The distinctive component of Glocal-IB is the Global Alignment loss. Its purpose is to enforce that the latent representation derived from masked inputs, $\hat{X}$0, closely matches the embedding of the corresponding originally observed input, $\hat{X}$1, at the same timestamp. The paper states that this suppresses spurious variability introduced by masking and stabilizes the global latent distribution (Yang et al., 6 Oct 2025).

Two implementations are provided. The first is an InfoNCE-style mutual-information lower bound:

$\hat{X}$2

where

$\hat{X}$3

In this construction, positives are encodings of the original sequence at the same timestamp as $\hat{X}$4, and negatives are other timestamps within the mini-batch. The second is a simplified alignment objective:

$\hat{X}$5

described as a lightweight alternative inspired by recent contrastive learning simplifications (Yang et al., 6 Oct 2025).

The mutual-information perspective is explicit throughout the derivation. The KL term is presented as a variational upper bound on $\hat{X}$6, while both the local reconstruction term and the contrastive global term act as lower-bound surrogates for $\hat{X}$7. The paper’s theoretical position is that local reconstruction alone can bias learning toward memorizing numerical details, including noise induced by missingness, whereas the alignment term preserves global semantics by anchoring the masked latent to the original latent (Yang et al., 6 Oct 2025).

A recurring implication of this formulation is that Glocal-IB is intended to improve generalization under missingness, not merely to reduce training error. The latent space is treated as an object of optimization in its own right, and the reported visualizations support that emphasis: Glocal-IB maintains a coherent latent distribution from $\hat{X}$8 to $\hat{X}$9 missingness, whereas SAITS, TimesNet, and GPVAE exhibit distortion or collapse as missingness increases (Yang et al., 6 Oct 2025).

4. Training procedure, applicability, and implementation

The training pipeline begins by sampling a mini-batch of fully observed sequences $M=0$0, generating random masks $M=0$1 at the desired missing rate, and forming masked inputs $M=0$2 (Yang et al., 6 Oct 2025). The masked sequence is encoded to obtain $M=0$3 and $M=0$4, from which $M=0$5 is sampled or replaced by $M=0$6. The decoder then reconstructs $M=0$7, and the local loss is computed over missing entries:

$M=0$8

The regularization loss is computed as the closed-form KL divergence to the standard Gaussian prior. For global alignment, the same encoder backbone is applied to the original unmasked input to obtain $M=0$9, and positive pairs are formed between $10\%$0 from $10\%$1 and $10\%$2 from $10\%$3 at the same timestamp. Negatives, in the InfoNCE variant, are drawn from other timestamps in the batch (Yang et al., 6 Oct 2025).

The method is described as model-agnostic. It can be plugged into any encoder-decoder TSI backbone, requires only one additional MLP projector for alignment, and does not require an external foundation model because the encoder itself provides the original embeddings (Yang et al., 6 Oct 2025). This is a notable design choice relative to heavier alignment approaches.

The experimental implementation uses a vanilla 2-layer Transformer with hidden dimension $10\%$4 and sequence length $10\%$5, optimized with Adam at learning rate $10\%$6, batch size $10\%$7, for $10\%$8 epochs. Training uses random point-wise MCAR masking, while block-wise missingness is also evaluated. Inference feeds masked $10\%$9 through the encoder to get $30\%$0 via $30\%$1, then decodes $30\%$2 (Yang et al., 6 Oct 2025).

The overhead relative to standard reconstruction training consists of one additional encoder forward pass for $30\%$3, the alignment term whose InfoNCE form is linear in the number of negatives, and a lightweight MLP projector. The paper states that memory and runtime overheads are marginal compared to foundation-model alignment approaches (Yang et al., 6 Oct 2025). Hyperparameter guidance is also specific: small $30\%$4 values are recommended, with performance degrading if $30\%$5 due to over-regularization; $30\%$6 is typically in $30\%$7; and $30\%$8 should be tuned to stabilize latent alignment without overwhelming reconstruction.

5. Experimental setting and quantitative results

The empirical study evaluates Glocal-IB on nine datasets: ETTh1, ETTh2, ETTm1, ETTm2, Beijing Air, PEMS-Traffic, Electricity, Weather, and METR-LA, with standard splits from PyPOTS and input length $30\%$9 (Yang et al., 6 Oct 2025). The dataset descriptions given in the paper are: ETTh1, ETTh2, ETTm1, ETTm2 as electricity transformer datasets with $50\%$0 variables and hourly or minutely resolution; Beijing Air with $50\%$1 variables $50\%$2 $50\%$3 stations; PEMS-Traffic with $50\%$4 occupancy sensors; Electricity with $50\%$5 clients; Weather with $50\%$6 variables and $50\%$7-minute sampling; and METR-LA with $50\%$8 sensors and $50\%$9-minute speeds.

Baselines include SAITS, Transformer, PatchTST, iTransformer, DLinear, FreTS, TimeMixer, TimesNet, and GPVAE; USGAN and TCN are included in missing-pattern and efficiency analyses. Performance is measured with MAE and MSE, lower being better, across five missing rates $70\%$0 and averaged (Yang et al., 6 Oct 2025).

Dataset	Glocal-IB	Selected baselines
ETTh1	MAE 0.283, MSE 0.197	SAITS 0.402 / 0.376; Transformer 0.399 / 0.373; TimesNet 0.602 / 0.702
ETTh2	MAE 0.249, MSE 0.132	SAITS 0.340 / 0.256; Transformer 0.307 / 0.218
ETTm1	MAE 0.157, MSE 0.069	SAITS 0.206 / 0.099; Transformer 0.202 / 0.096
Weather	MAE 0.096, MSE 0.056	SAITS 0.136 / 0.093; Transformer 0.139 / 0.091
Beijing Air	MAE 0.223, MSE 0.320	SAITS 0.256 / 0.353; Transformer 0.268 / 0.375

The paper reports consistent improvements across all nine datasets, with up to about $70\%$1 MSE reduction on several ETT subsets relative to strong baselines (Yang et al., 6 Oct 2025). It also states that, under block-wise missingness, Glocal-IB achieves the lowest MAE by a substantial margin, indicating robustness to structured data loss. Efficiency analyses further report that Glocal-IB adds only marginal overhead in memory and runtime compared to “Ori” training and is far cheaper than foundation-model alignment such as Time-MoE-based alignment.

The ablation results are central to the paper’s argument. Removing either $70\%$2 or $70\%$3 significantly degrades performance at higher missing rates, which the authors interpret as evidence that both compactness regularization and global alignment are critical (Yang et al., 6 Oct 2025). The study also reports that the InfoNCE variant tends to be stronger, while the simplified alignment remains effective and cheaper.

6. Interpretation, relation to prior work, and limitations

The paper’s theoretical interpretation is that local-only objectives fail because maximizing $70\%$4 through point-wise reconstruction alone biases learning toward local numerical details, including structured noise introduced by missingness. Under high masking, the encoder observes corrupted inputs at both train and test time, and without global guidance the latent representation can overfit to noise patterns rather than preserve a coherent global distribution (Yang et al., 6 Oct 2025). This provides the conceptual basis for the “glocal” combination of local reconstruction and global alignment.

In relation to prior work, the paper situates Glocal-IB against two families of methods. First, IB-based TSI methods such as GPVAE, CIB, and MIWAE are described as relying on reconstruction to increase $70\%$5 while lacking explicit global alignment, leaving them vulnerable to the optimization dilemma (Yang et al., 6 Oct 2025). Second, foundation-model alignment approaches such as Time-MoE are described as able to inject external semantics but as being heavy, and in TSI appearing less beneficial, likely because forecasting pretraining objectives do not offer strong semantic constraints for imputation. Glocal-IB is positioned as lightweight, self-contained, and directly targeted at global mutual-information alignment between masked and original inputs.

Several practical conditions for benefit are identified. The masking mechanism used during training should be representative of test-time missingness; the encoder must be sufficiently expressive to model both masked and original inputs; $70\%$6 must be tuned so that the global term does not overwhelm local reconstruction; and $70\%$7 should be kept small to avoid over-regularization (Yang et al., 6 Oct 2025). These conditions are not presented as formal guarantees, but they delimit the regime in which the method is expected to work well.

The stated limitations are also specific. Gains under extreme missingness, such as $70\%$8, are smaller because when only about $70\%$9 is observed the alignment signal is inherently weak. The experiments focus on representative backbones—Transformer, SAITS, and TimesNet—and nine datasets; extending the analysis to broader architectures and larger-scale data is identified as a natural next step (Yang et al., 6 Oct 2025). A plausible implication is that Glocal-IB addresses a structural failure mode of TSI objectives, but its effectiveness still depends on the amount of observable signal available for alignment.