CPiRi: Channel Permutation-Invariant Forecasting

• CPiRi is a multivariate time series forecasting framework that incorporates channel permutation invariance for robust predictions in dynamic channel settings.
• It employs a frozen univariate temporal encoder paired with a lightweight permutation-equivariant spatial module that captures cross-channel relations.
• Channel shuffling during training enforces content-driven interactions, balancing channel independence with explicit relational modeling.

Searching arXiv for CPiRi and closely related set/permutation-invariant references. CPiRi, short for Channel Permutation-Invariant Relational Interaction, is a multivariate time series forecasting framework that combines a frozen pre-trained univariate temporal encoder, a lightweight spatial interaction module, and a channel shuffling training strategy to achieve channel permutation invariance (CPI) while retaining explicit cross-channel modeling (Xu et al., 28 Jan 2026). It is formulated for deployment regimes with structural co-drift and distributional co-drift, where channels may be reordered, added, or removed and correlation structure may shift over time, yet forecasting performance is expected to remain stable without retraining (Xu et al., 28 Jan 2026).

1. Problem formulation and motivation

CPiRi is defined in the standard multivariate time series forecasting setting. Let $C$ denote the number of channels, $L$ the look-back window length, and $T$ the forecast horizon. At time $t$, the observation is $X_t \in \mathbb{R}^C$; the model input is $X=\{X_1,\dots,X_L\}\in\mathbb{R}^{L\times C}$, and the target is $$Y=\{X_{L+1},\dots,X_{L+T}\}\in\mathbb{R}^{T\times C}$$ (Xu et al., 28 Jan 2026). The objective is to learn a forecaster $F$ that maps $X\mapsto \hat Y$ with low prediction error.

The framework is motivated by a limitation in the dominant partition of prior methods into channel-dependent and channel-independent models. Channel-dependent models, including GNN-based methods such as MTGNN and spatial-temporal Transformers such as Informer, Crossformer, STID, Timer-XL, and iTransformer, jointly process the full multivariate sequence and explicitly model cross-channel structure (Xu et al., 28 Jan 2026). Their strength is relational expressivity; their limitation is that they often assume a fixed channel index ordering or topology and can therefore overfit to training-time channel arrangements. The paper reports that Informer’s error increases by more than 400% when channels are shuffled at test time, illustrating this brittleness under channel changes (Xu et al., 28 Jan 2026).

Channel-independent models such as DLinear, PatchTST, Chronos-Bolt, and Sundial apply the same temporal model to each channel separately (Xu et al., 28 Jan 2026). These models are naturally permutation-invariant and robust to heterogeneous channels, but they ignore explicit inter-channel dependencies. CPiRi is designed to resolve this trade-off: it keeps the robustness and flexibility associated with channel-independent processing while recovering the multivariate advantage of cross-channel reasoning (Xu et al., 28 Jan 2026).

In practical terms, the target setting includes traffic systems, sensor networks, healthcare monitoring, finance, and retail demand forecasting, where sensors fail, devices are added, market instruments enter and exit, or data pipelines simply reindex channels (Xu et al., 28 Jan 2026). Within that setting, CPI is elevated from a convenience to a primary modeling requirement.

2. Channel permutation invariance as a forecasting property

The distinctive property of CPiRi is channel permutation invariance. In the formulation used by the paper, prediction quality should remain almost unchanged when channels are permuted, and the system should generalize to new channels without retraining (Xu et al., 28 Jan 2026). This is operationalized through a diagnostic: train with a fixed order and evaluate after shuffling channels at test time. Traditional channel-dependent models show catastrophic degradation under this diagnostic, indicating memorization of positional patterns rather than semantic relations (Xu et al., 28 Jan 2026).

The paper distinguishes permutation-invariance from permutation-equivariance. If $H=\{h_1,\dots,h_C\}$ is the set of per-channel features, a function $L$0 is permutation-equivariant if

$L$1

for any permutation $L$2 of the channel indices. A function $L$3 is permutation-invariant if

$L$4

Within CPiRi, the spatial module is permutation-equivariant, while the encoder and decoder operate independently on each channel; their composition yields a forecasting pipeline that is equivariant at the channel level, so shuffling the input channels induces the same shuffle in the output predictions (Xu et al., 28 Jan 2026).

A central implication is that cross-channel structure must be inferred from content rather than order. Because channel order is randomized during training, the spatial module cannot rely on shortcuts such as a fixed association between channel index and semantic identity. The only stable solution is content-driven relational reasoning over channel embeddings (Xu et al., 28 Jan 2026). This suggests that CPI in CPiRi is not merely an invariance constraint but also a regularizer against spurious topology memorization.

3. Architectural decomposition: frozen temporal backbone plus spatial relation module

CPiRi uses a three-stage spatio-temporal decoupling architecture consisting of a frozen temporal encoder, a permutation-equivariant spatial module, and a frozen decoder (Xu et al., 28 Jan 2026). Temporal modeling is offloaded to a pre-trained univariate foundation model, while cross-channel interaction is handled by a small trainable component.

For each channel $L$5, the input series

$L$6

is processed independently by the frozen Sundial encoder, and CPiRi uses the final patch representation as a temporal summary,

$L$7

Collecting all channels yields $L$8 (Xu et al., 28 Jan 2026). Because this stage is channel-independent and frozen, it is trivially permutation-invariant and preserves temporal priors learned during pretraining.

The second stage treats $L$9 as a set of channel embeddings and applies a standard Transformer encoder block with multi-head self-attention and feed-forward layers: $T$0

$T$1

$T$2

The key structural property is that multi-head self-attention over channels is permutation-equivariant, so

$T$3

for any channel permutation $T$4 (Xu et al., 28 Jan 2026). Each enriched representation $T$5 therefore depends on $T$6 and on the set of all channel features, but not on an absolute index identity.

In the third stage, each $T$7 is decoded independently by the frozen Sundial decoder: $T$8 Stacking channelwise outputs yields $T$9 (Xu et al., 28 Jan 2026). By default, the spatial Transformer is the only trainable part of the framework.

This architecture is described in the paper as radically decoupled. The practical consequence is that a strong temporal backbone can be reused across datasets, while only a lightweight spatial block is adapted for a particular multivariate environment (Xu et al., 28 Jan 2026).

4. Training objective and theoretical grounding

CPiRi enforces CPI through a channel shuffling strategy during training (Xu et al., 28 Jan 2026). For each mini-batch $t$0, a permutation $t$1 is sampled uniformly from the symmetric group over the channel dimension, both input and target are permuted, the CPiRi pipeline is applied to the permuted input, and loss is computed against the correspondingly permuted target. The training objective is

$t$2

The paper’s theoretical argument is that equivariant architecture and stochastic channel shuffling act jointly. Since the encoder and decoder are channelwise independent and the spatial module is permutation-equivariant by construction, any deviation from equivariant behavior leads to systematically higher loss over the distribution of random permutations (Xu et al., 28 Jan 2026). In that sense, the shuffling procedure functions as permutation-invariant regularization via data augmentation.

The theory section explicitly connects the framework to DeepSets as introduced by Zaheer et al. (Zaheer et al., 2017). The paper states that any permutation-equivariant function over a set can be expressed in the form

$t$3

where $t$4 is a symmetric aggregation (Xu et al., 28 Jan 2026). Self-attention is presented as consistent with this structural form because it computes content-based interactions across the set and aggregates transformed values through weighted sums. The architectural choice and the training objective are therefore aligned: the former supplies the correct symmetry class, and the latter encourages the learned function to realize that symmetry in practice (Xu et al., 28 Jan 2026).

An important ablation concerns unfreezing the temporal encoder. The paper reports that joint fine-tuning can produce slight gains on some datasets, but training becomes 5× more memory-consuming, representation quality shows signs of overfitting and reduced separability, and the decoupling that supports CPI is weakened (Xu et al., 28 Jan 2026). This is one of the clearest indications that CPiRi’s design is not simply a modular implementation detail but a structural prior.

5. Empirical behavior: accuracy, robustness, and inductive generalization

The experimental study uses nine datasets: METR-LA, PEMS-BAY, PEMS-04, PEMS-08, SD, Electricity, and three large-scale subsets from LargeST—GBA with 2,352 channels, GLA with 3,834 channels, and CA with 8,600 channels (Xu et al., 28 Jan 2026). Evaluation follows BasicTS+ protocols, and the primary metric is WAPE

$t$5

with MAE also reported (Xu et al., 28 Jan 2026).

The headline empirical result is that CPiRi is state-of-the-art on PEMS-BAY, PEMS-04, PEMS-08, SD, and Electricity, while remaining competitive on METR-LA, where STID and Crossformer benefit from extra exogenous features such as holidays that CPiRi does not use (Xu et al., 28 Jan 2026). On SD, CPiRi improves WAPE by more than 12% over Sundial and more than 30% over Timer-XL (Xu et al., 28 Jan 2026).

The most distinctive evidence concerns robustness under channel shuffling.

Setting	Comparator	Reported outcome
Test-time shuffle on PEMS-08	Informer	WAPE 13.02% $t$6 118.19%
Test-time shuffle on PEMS-08	STID	WAPE 10.90% $t$7 65.18%
Test-time shuffle on PEMS-08	Crossformer	WAPE 11.43% $t$8 39.85%
Test-time shuffle across datasets	CPiRi	WAPE changes by less than 0.25%
Partial shuffle on PEMS-08	CPiRi	WAPE remains exactly 9.43% at 0%, 25%, 50%, 75%, 100% permutation

These results support the paper’s claim of essentially perfect CPI (Xu et al., 28 Jan 2026). They also show that simple shuffling during training is insufficient if the architecture itself still encodes positional dependence through channel-specific positional encodings or fixed graph structure (Xu et al., 28 Jan 2026).

The paper also evaluates inductive generalization to unseen channels by training on only a subset of channels and testing on the full set without retraining. On PEMS-08, when training on 25% of channels, CPiRi with shuffling obtains WAPE $t$9, compared with WAPE $X_t \in \mathbb{R}^C$0 without shuffling (Xu et al., 28 Jan 2026). The paper further reports only 2% absolute accuracy loss when training on 25% rather than 100% of channels, alongside a 70% reduction in training time (Xu et al., 28 Jan 2026). This suggests that the model learns dataset-transferable relational structure rather than a fixed channel inventory.

Ablations reinforce the dependence on all three design elements. Without pretraining, the same architecture collapses on SD from 12.25% WAPE to 64.84% (Xu et al., 28 Jan 2026). Without the spatial module, CPiRi degenerates to the channel-independent Sundial baseline, and on PEMS-08 WAPE degrades from 9.43% to 22.69% (Xu et al., 28 Jan 2026). Replacing attention with mean pooling also degrades performance, indicating that content-aware interaction rather than global averaging is required (Xu et al., 28 Jan 2026).

6. Scalability, implementation, and deployment profile

CPiRi is explicitly positioned as a practically efficient alternative to fully coupled spatio-temporal attention models (Xu et al., 28 Jan 2026). Its complexity is

$X_t \in \mathbb{R}^C$1

where the $X_t \in \mathbb{R}^C$2 term comes from the temporal Sundial backbone and the $X_t \in \mathbb{R}^C$3 term from a single channelwise attention layer (Xu et al., 28 Jan 2026). This is contrasted with models such as iTransformer and Timer-XL, which use $X_t \in \mathbb{R}^C$4 spatio-temporal attention and therefore scale poorly on large channel sets (Xu et al., 28 Jan 2026).

On the CA subset with 8,600 channels and $X_t \in \mathbb{R}^C$5, the reported compiled inference cost is approximately 0.41 s and about 8 GB of GPU memory for CPiRi, compared with approximately 0.40 s and 5.17 GB for Sundial, approximately 17.7 GB for iTransformer, and approximately 75 GB for Timer-XL, which is often out-of-memory (Xu et al., 28 Jan 2026). On the same dataset, Timer-XL records 42.75% WAPE, Sundial 23.60%, and CPiRi 12.68% (Xu et al., 28 Jan 2026). The practical interpretation is that CPiRi is only slightly heavier than a purely channel-independent backbone while remaining substantially more scalable than large channel-dependent baselines.

Implementation details are fully specified. Experiments use BasicTS+, NVIDIA A800 80GB, PyTorch 2.3.1, and CUDA 12.1; all runs use $X_t \in \mathbb{R}^C$6, learning rate $X_t \in \mathbb{R}^C$7, weight decay $X_t \in \mathbb{R}^C$8, gradient clipping 3.0, and dropout 0.3 in the spatial block (Xu et al., 28 Jan 2026). The spatial module is one Transformer encoder block with LayerNorm and GELU, and the Sundial encoder and decoder are loaded from an official pretrained checkpoint and kept frozen by default (Xu et al., 28 Jan 2026). The released codebase is JasonStraka/CPiRi (Xu et al., 28 Jan 2026).

The deployment pattern described by the paper is straightforward: encode each channel independently with the foundation model, apply the learned spatial module to the resulting set of embeddings, and decode each channel independently (Xu et al., 28 Jan 2026). Because the system is CPI, inference does not require aligning channels to a canonical order, and training on partial channel sets can support later deployment on larger or changed channel inventories (Xu et al., 28 Jan 2026).

7. Relation to prior work, scope, and terminological ambiguity

Relative to channel-dependent models, CPiRi differs by not binding spatial reasoning to a fixed graph, topology, or index ordering (Xu et al., 28 Jan 2026). Relative to channel-independent models, it differs by reintroducing explicit cross-channel interaction through a small relational head rather than treating each channel as a fully isolated task (Xu et al., 28 Jan 2026). Relative to set-based methods, its spatial component can be viewed as a set Transformer over channels, and the paper explicitly situates its theory alongside DeepSets (Zaheer et al., 2017).

A recurrent misconception addressed by the experiments is that architectural CPI alone or train-time shuffling alone is sufficient. The reported results suggest that both are necessary: models with position-sensitive architectures still fail badly under shuffled evaluation even when augmented with shuffled training data, whereas CPiRi pairs a permutation-equivariant spatial module with a shuffling objective (Xu et al., 28 Jan 2026). Another misconception is that channel-independent robustness is adequate in strongly coupled multivariate domains; the large gap between CPiRi and the Sundial baseline on PEMS-08 shows that explicit inter-channel modeling remains essential (Xu et al., 28 Jan 2026).

The paper also states several limitations. CPiRi depends on a good pre-trained temporal encoder; without it, performance collapses (Xu et al., 28 Jan 2026). The spatial modeling is currently static, since the framework uses a fixed Transformer block rather than a dynamic fusion mechanism, and the authors note that this may be suboptimal for abrupt trend shifts (Xu et al., 28 Jan 2026). Although $X_t \in \mathbb{R}^C$9 is manageable for thousands of channels, extremely large channel sets may require sparsification through sparse attention or clustering (Xu et al., 28 Jan 2026).

Finally, the acronym is potentially ambiguous. In this context, CPiRi denotes Channel Permutation-Invariant Relational Interaction for multivariate time series forecasting (Xu et al., 28 Jan 2026). It should not be conflated with codeword position index based SCMA (CPI-SCMA) in wireless multiple access (Lai et al., 2018), nor with communication price of privacy (cPoP) in PIR over distributed storage codes (Kumar et al., 2016). Here, the term refers specifically to a forecasting architecture whose central contribution is order-agnostic, interaction-aware prediction under changing channel sets (Xu et al., 28 Jan 2026).