Snippet Similarity-Weighted Reconstruction

• The paper's main contribution is a self-supervised framework that fuses snippet-level contrastive learning with similarity-weighted reconstruction to capture robust data representations.
• The methodology leverages a bidirectional LSTM encoder, a projector for snippet aggregation, and a softmax-based similarity metric to mitigate noise and privacy constraints in fragmented time-series data.
• Empirical results demonstrate a 31.9% reduction in RMSE under distribution shifts, highlighting the framework's effectiveness on privacy-friendly and feature-sparse EV charging records.

Snippet similarity-weighted masked input reconstruction is an advanced self-supervised representation learning framework in which local, fragmented input segments (“snippets”) are masked and reconstructed using similarity information across snippets. This approach is particularly well-suited for privacy-preserving, feature-sparse, and noisy data regimes, such as electric vehicle (EV) charging records (Arunan et al., 5 Oct 2025). The technique leverages contrastive learning to establish high-level associative relationships among snippets, then utilizes a similarity-weighted decoding mechanism to enhance reconstruction, resulting in robust, generalizable representations even under severe distribution shift and privacy constraints.

1. Formulation and Algorithmic Structure

The framework processes a collection of unlabeled time-series snippets $X = \{x_1, x_2, ..., x_N\}$, where each $x_i \in \mathbb{R}^{T \times C}$ (with $T$ time points and $C$ sensor channels for battery monitoring). For each snippet $x_i$, subsequences are masked using a geometric strategy (e.g., contiguous segments set to zero), producing $x^*_i$. Both original and masked snippets pass through a bidirectional LSTM encoder $f(\cdot)$, yielding point-wise features $p_i, p^*_i \in \mathbb{R}^{T \times d_f}$.

A projector $h(\cdot)$ condenses point-wise representations into snippet-wise encodings $s_i, s^*_i \in \mathbb{R}^{1 \times d_h}$:

$h(\{p_i, p^*_i\}_{i=1}^N) = \{s_i, s^*_i\}_{i=1}^N$

Reconstruction is performed for masked snippet $i$ via similarity weighting over other snippets, using cosine similarity $D_{s_i, s'_j}$ with temperature $\tau$:

$$\hat{p}_i = \sum_{j \neq i} \frac{\exp(D_{s_i, s'_j} / \tau)}{\sum_{k \neq i} \exp(D_{s_i, s'_k} / \tau)} p'_j$$

where $p'_j$ are the point-wise features of snippet $j$, and $S$ is the set of all snippets.

A decoder $d(\cdot)$ reconstructs the masked input:

$\hat{x}_i = d(\hat{p}_i)$

The final reconstruction loss is

$$\mathcal{L}_r = \sum_{i=1}^N \|x_i - \hat{x}_i\|^2_2$$

This is jointly trained with contrastive loss $\mathcal{L}_c$ (see below), with overall pre-training objective:

$$\mathcal{L}_\text{pretrain} = \mathcal{L}_r + \beta \mathcal{L}_c$$

$\beta$ is an uncertainty-weighted coefficient automatically tuned per loss component.

2. Contrastive Learning for High-Level Snippet Similarity

A snippet-wise contrastive loss is applied to explicitly enforce associativity between original and masked versions of each snippet. Positive pairs $(s_i, s^*_i)$ are pulled close in representation space; negatives (other snippet pairs) are pushed apart:

$$\mathcal{L}_c = - \sum_{s \in S} \sum_{s' \in S^+} \log \left( \frac{\exp(D_{s, s'}/\tau)}{\sum_{s'' \neq s} \exp(D_{s, s''}/\tau)} \right)$$

where $S^+$ is the set of positives for $s$ (original and its own masked version). This contrastive pre-training ensures that high-level similarity relationships across fragmented snippets are captured, even with noisy EV data.

3. Representation Learning: Point-wise and Snippet-wise Fusion

The architecture learns representations at two levels:

Point-wise (granular temporal structure): The encoder $f(\cdot)$ models per-time-point battery charging behavior, extracting sequential dependencies and local patterns.

Snippet-wise (high-level associative): The projector $h(\cdot)$ aggregates across time, enabling the model to compare global charging behaviors between snippets. The similarity-weighted reconstruction fuses the masked snippet’s own context with related patterns from other similar snippets.

This dual structure enables the model to generalize across data from varying manufacturers and battery age regimes, yielding robust features even under severe distribution shifts.

4. Empirical Performance and Domain Robustness

Empirically, the snippet similarity-weighted masked input reconstruction method achieves strong performance on large-scale field EV data (Arunan et al., 5 Oct 2025). Under age-induced distribution shifts (“Distribution 3”), test error (measured in RMSE) is reduced by 31.9% relative to the best previous benchmark. The performance remains consistent across in-distribution and out-of-distribution settings, even when only 10% of labeled data is used for fine-tuning.

Key factors for this robustness include:

• Similarity-weighted fusion suppresses noisy, unrelated snippets during reconstruction.
• Contrastive pre-training captures invariant relationships across diverse operational regimes.
• The approach is optimized for privacy-friendly data, which is inherently fragmented and lacks dense contextual information.

5. Privacy-Preserving Learning and Label Efficiency

The method is particularly suited to privacy-preserving regimes:

• Training is performed solely on fragmented charging records, which omit sensitive details.
• Unlabeled data is effectively used for self-supervised pre-training, reducing reliance on labeled records that may contain privacy-invasive operational metadata.
• By leveraging cross-snippet similarity, the model improves reconstruction and feature learning without requiring the full sequence context typically unavailable in privacy-compliant datasets.

6. Technical and Methodological Context

This framework is the first self-supervised capacity estimation pre-training model specifically designed for privacy-friendly EV charging data (Arunan et al., 5 Oct 2025). It contrasts with prior self-supervised methods (such as masked language/image modeling and conventional contrastive learning), by integrating snippet-wise similarity into both the objective and the decoding process. This design supports generalization to other domains characterized by fragmented, noisy, and low-feature data—where preserving privacy is critical.

7. Related Research Directions and Potential Extensions

This suggests avenues for future work, including:

• Application of similarity-weighted masked reconstruction to other privacy-friendly time-series domains (e.g., medical sensor data).
• Investigation of more advanced masking strategies (e.g., adaptive or context-driven) to further optimize reconstruction quality.
• Extension of the snippet-wise contrastive fusion to multimodal data or graph representation domains, building on related ideas in self-supervised learning.

Table: Framework Components

Component	Role	Technical Details
Encoder $f(\cdot)$	Point-wise representation	BiLSTM, per-time-point modeling
Projector $h(\cdot)$	Aggregates to snippet-wise vector	MLP, reduces T × $d_f$ to 1 × $d_h$
Contrastive loss $\mathcal{L}_c$	Associates original/masked snippets	Cosine similarity, temperature $\tau$
Similarity-weighted fusion	Guides snippet reconstructions	Softmax-normalized cosine weighting
Decoder $d(\cdot)$	Reconstructs masked input	MLP, $d_f$ → $C$ output per timepoint

The above structure allows the framework to simultaneously model fine-grained temporal details and robust, transferable high-level relationships, leading to improved battery capacity estimation and generalization under strict privacy constraints (Arunan et al., 5 Oct 2025).