---
title: Simple MixUp for Time-Series
url: https://www.emergentmind.com/topics/embarrassingly-simple-mixup-for-time-series
type: topic
---

# Simple MixUp for Time-Series

Embarrassingly simple MixUp for time-series is a vicinal risk-based data augmentation paradigm that synthesizes virtual examples by convexly mixing pairs of time-series and their corresponding labels in either raw or latent space. By leveraging the element-wise, label-weighted interpolation, MixUp regularizes neural models for time-series classification, self-supervised representation learning, and transfer scenarios without requiring domain-specific transformations or hyperparameter tuning. This methodology has been further extended and systematically evaluated across numerous architectures, datasets, and training regimes, positioning MixUp and its latent variants as robust and scalable augmentation baselines in the time-series domain.

## 1. Formalization and Methodological Variants

The MixUp procedure operates on pairs of labeled time-series $(x_i, y_i)$ and $(x_j, y_j)$. A mixing coefficient $\lambda \sim \mathrm{Beta}(\alpha, \alpha)$ or $\lambda \sim \mathrm{Uniform}(0,1)$ is sampled, then the synthetic input and label are constructed as:
\[
\tilde{x} = \lambda x_i + (1-\lambda)x_j, \qquad \tilde{y} = \lambda y_i + (1-\lambda) y_j
\]
where $x \in \mathbb{R}^{T \times C}$ for multivariate sequences. Label mixing creates soft targets, encouraging smooth decision boundaries and mitigating overconfidence.

**MixUp++** introduces two modifications: (i) all minibatches include both real and mixed examples; (ii) $k$ distinct random MixUp pairs are generated per batch using independent permutations and mix ratios. The supervised loss is
\[
\mathcal{L}_\mathrm{sup} = 
\frac{1}{N} \sum_{i=1}^N \ell(f(x_i), y_i) + \frac{1}{kN} \sum_{m=1}^k \sum_{i=1}^N \ell(f(\tilde{x}_i^{(m)}), \tilde{y}_i^{(m)})
\]
[2304.04271].

**LatentMixUp++** replaces input mixing with hidden feature mixing. For encoder $h_\theta$, projection head $g_\phi$, and input pair $(x_i, y_i), (x_j, y_j)$,
\[
\tilde{z} = \lambda h_\theta(x_i) + (1-\lambda) h_\theta(x_j), \qquad \tilde{y} = \lambda y_i + (1-\lambda)y_j
\]
The prediction is via $g_\phi(\tilde{z})$. This variant leverages more linear latent manifolds, benefiting from model-internal representations [2304.04271].

## 2. Integration into Contrastive and Supervised Frameworks

In supervised classification, MixUp or its variants are applied as preprocessing per minibatch and require only modifications to the input pipeline and loss function (cross-entropy on soft labels). Integration is architecture-agnostic, applicable to CNNs (e.g., InceptionTime, FCN), RNNs, Transformers, and ResNet-style 1D CNNs [2304.04271, 2201.11739, 2309.09970].

In self-supervised or contrastive learning, MixUp is combined with triplet or contrastive losses by mixing views before encoder passage [2203.09270]. The MixUp-Normalized Temperature-Scaled Cross-Entropy (MNT-Xent) loss is used:
\[
\ell_i = -\lambda \log\!
        \frac{\exp(D_C(\tilde{z}_i, z_i^{(1)})/\tau)}
        {\sum_{k=1}^N [\exp(D_C(\tilde{z}_i, z_k^{(1)})/\tau) + \exp(D_C(\tilde{z}_i, z_k^{(2)})/\tau)]}
      -(1-\lambda) \log\!
        \frac{\exp(D_C(\tilde{z}_i, z_i^{(2)})/\tau)}
        {\sum_{k=1}^N [\exp(D_C(\tilde{z}_i, z_k^{(1)})/\tau) + \exp(D_C(\tilde{z}_i, z_k^{(2)})/\tau)]}
\]
where $D_C$ denotes cosine similarity and $\tau$ the temperature parameter. The positive weights are soft, directly encoding the mix ratio [2203.09270].

## 3. Implementation Details and Hyperparameters

MixUp for time-series is "embarrassingly simple" in that no domain-specific expertise or sensitive tuning is required:
- Mixing coefficient: $\lambda \sim \mathrm{Beta}(\alpha, \alpha)$, commonly $\alpha=0.2$ or $\alpha=0.4$.
- Number of mixes per batch: $k=1$ or $k=2$; $k > 2$ can degrade performance.
- Networks: Standard architectures (FCN, InceptionTime, ResNet-18, Transformers) without architectural changes.
- Training: Adam optimizer ($\text{lr}=10^{-3}$), batch size 128–256, up to 1000 epochs.
- Minimal additional computational overhead, with $<$1 ms per batch in practice [2309.09970, 2203.09270, 2304.04271].
- Requires time-series to be resampled or padded to common length $L$ for element-wise mixing [2309.09970].

Semi-supervised extensions include pseudo-labeling: confident model predictions above threshold $\tau$ (e.g., 0.99) on unlabeled data are included, and MixUp is applied jointly over labeled and pseudo-labeled data [2304.04271].

## 4. Empirical Results

The effectiveness of MixUp and its variants has been extensively validated across multivariate, univariate, and physiological time-series datasets. The following summarizes key results from representative studies:

| Dataset Family     | Baseline Acc (%) | MixUp Acc (%) | LatentMixUp++ (best) | Reference      |
|--------------------|------------------|---------------|----------------------|---------------|
| UCI-HAR           | 92.95 ± 0.83     | 92.63 ± 0.56  | **94.44 ± 0.72**     | [2304.04271]  |
| Sleep-EDF         | 80.57 ± 0.34     | 79.14 ± 0.96  | **81.12 ± 0.47**     | [2304.04271]  |
| PTB-XL            | 77.94            | 78.91         | —                    | [2309.09970]  |
| PAMAP2            | 93.62            | **95.31**     | —                    | [2309.09970]  |

Additional findings:
- On the 128 UCR (univariate) and 30 UEA (multivariate) datasets, MixUp contrastive pretraining achieved $0.759$ (UCR) and $0.627$ (UEA) kNN accuracy, exceeding all baselines [2203.09270].
- Significant improvements (up to +10.5% points) observed where interpolated series remain plausible (continuous signals) [2201.11739].
- LatentMixUp++ yields the largest benefits in low-label regimes: up to +15% relative improvement with only 1% labeled data; ablation shows diminishing gains as $k$ increases beyond 2 due to overwhelming synthetic samples [2304.04271].
- Mix-based methods outperformed single-sample augmentations (jitter, scale, warping), and CutMix sometimes provided further gains, especially in class-imbalanced settings [2309.09970].

## 5. Analysis of Efficacy and Domain Adaptation

Several properties underlie the robustness and transferability of MixUp-based approaches:
- **No domain-specific assumptions or transformations** are required; MixUp performs consistently across diverse sensor modalities and application domains by interpolating in raw or latent spaces [2309.09970].
- **Vicinal Risk Minimization**: Mixing expands the local neighborhood manifold, filling gaps between real samples and regularizing the learned function.
- **Soft-label smoothing**: Mixed labels prevent overfitting to hard class boundaries, improve calibration, and act similarly to distillation [2203.09270].
- **Latent space interpolation**: LatentMixUp++ leverages more linear, task-aligned representations, mitigating issues with destructive raw-space interpolation (e.g., cancellation under phase shifts).
- **Minimal sensitivity to hyperparameters**: A single $\alpha$ value is effective across datasets; no extensive parameter search is required [2309.09970].

MixUp's effectiveness is greatest when interpolated samples remain on-manifold and the datasets benefit from regularized decision boundaries. Detrimental effects can arise if interpolation yields unrealistic signals, such as when discriminative features are highly localized or sharp spiking patterns are averaged out [2201.11739].

## 6. Extensions, Variants, and Best Practices

Beyond raw-space MixUp, related mix-based augmentations have been formulated:
- **CutMix**: Random segments ("patches") are swapped between series, preserving uninterpolated context and further diversifying signal morphologies. In time-series, contiguous intervals are exchanged across all channels [2309.09970].
- **Manifold MixUp**: Mixing occurs in hidden layers deeper in the network, introducing additional regularization by perturbing high-level feature representations [2309.09970].
- **Semi-supervised MixUp**: Combining labeled data and pseudo-labeled unlabeled samples with mix-based augmentation yields further gains, especially in data-scarce regimes [2304.04271].
- **Multiple MixUp per batch**: Empirical analysis recommends $k=2$ for jointly optimizing over real and synthetic pairs; higher $k$ can induce over-regularization [2304.04271].

Best practices:
- Standardize sequence lengths across minibatch.
- Apply MixUp per minibatch after shuffling; use a fixed $\alpha$.
- Include both real and mixed samples in each training batch.
- For unbalanced classes, combine MixUp with class-balanced sampling.
- Integrate CutMix or Manifold MixUp as needed for increased augmentation diversity.

## 7. Limitations and Ongoing Challenges

While "embarrassingly simple" MixUp strategies are robust and adaptable, several open issues remain:
- **Interpolation realism**: For tasks with highly non-overlapping temporal events, MixUp can degrade discriminative features.
- **Performance with many synthetic samples**: Excessive augmentation relative to real data can harm performance; careful selection of $k$ is advised [2304.04271].
- **Extensibility**: Application to other time-series tasks (forecasting, anomaly detection), non-convolutional architectures, or integration with consistency-based semi-supervised objectives is an open direction [2304.04271].
- **Efficiency**: Computational cost increases linearly with $k$; sampling strategies or dynamic mix ratios may offer improvements.

In summary, MixUp and its latent and manifold extensions provide universal, high-performing augmentation strategies for time-series learning with minimal domain expertise or tuning required, consistently improving classification and representation learning outcomes across varied benchmark and clinical datasets [2304.04271, 2309.09970, 2203.09270, 2201.11739].

Source: https://www.emergentmind.com/topics/embarrassingly-simple-mixup-for-time-series