SICL: Subject-Invariant Contrastive Learning

• SICL is a family of methods that suppresses subject-specific variation to enhance cross-subject generalization in applications like human activity recognition and EEG emotion analysis.
• It refines contrastive learning by modifying pair construction, re-weighting negatives, or adding adversarial objectives to emphasize task-relevant features over identity cues.
• Empirical studies show improved accuracy and reduced subject bias, demonstrating SICL’s effectiveness across biosignals, HAR, and other cross-subject tasks.

Searching arXiv for papers on subject-invariant contrastive learning and closely related cross-subject contrastive methods. arXiv query: "Subject Invariant Contrastive Learning Human Activity Recognition (Yarici et al., 4 Jul 2025)" Subject-Invariant Contrastive Learning (SICL) denotes a family of contrastive representation-learning methods that aim to suppress subject-specific variation while preserving task-relevant structure. Its motivating problem is cross-subject generalization: in human activity recognition, biosignals, and related domains, models trained with standard contrastive or supervised objectives can encode who produced a signal rather than what physiological state, affective state, or activity it represents. In that setting, SICL modifies pair construction, negative handling, latent-space structure, or auxiliary objectives so that the learned representation becomes more stable across subjects and more discriminative for downstream semantics (Yarici et al., 4 Jul 2025, Cheng et al., 2020, Shen et al., 2021).

1. Problem formulation and historical emergence

An early biosignal-specific formulation appears in "Subject-Aware Contrastive Learning for Biosignals" (Cheng et al., 2020), which distinguishes subject-specific self-supervised learning from subject-invariant self-supervised learning. The paper starts from the observation that EEG and ECG datasets often have noisy or scarce labels, small numbers of subjects, and large signal variability caused by physiology, sensor placement, recording conditions, and subject-specific dynamics. Under those conditions, contrastive learning can exploit subject identity as an easy discriminative cue rather than the intended task structure (Cheng et al., 2020).

In cross-subject EEG emotion recognition, the same issue is framed as inter-subject variability in emotion-related EEG responses. "Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition" (Shen et al., 2021) studies the realistic setting in which a model is trained on some subjects and must generalize to different, unseen subjects at test time. Its central motivation comes from inter-subject correlation findings in neuroscience: when subjects receive the same naturalistic or emotional stimulus, their neural activity can exhibit synchronized structure, suggesting that subject invariance can be grounded in stimulus-locked correspondence rather than generic domain confusion (Shen et al., 2021).

The term Subject-Invariant Contrastive Learning is made explicit in "Subject Invariant Contrastive Learning for Human Activity Recognition" (Yarici et al., 4 Jul 2025). There the problem is cross-subject distribution shift in sensor-based HAR: different people perform the same activity differently because of age, physical condition, gait style, limb length and body dynamics, execution speed and amplitude, and natural behavioral habits. The paper argues that standard contrastive learning often embeds subject-specific variations rather than activity-specific features, which harms transfer to unseen subjects (Yarici et al., 4 Jul 2025).

Taken together, these works establish SICL as a response to a recurring failure mode of contrastive learning under subject variability. The common goal is not personalization, but the opposite: representations that retain semantics while reducing subject dependence. This suggests that SICL is best understood as a cross-subject domain-generalization strategy implemented through contrastive objectives rather than as a single canonical loss.

2. Core formulations and pair semantics

The main SICL formulations in the literature differ in how they define positives, negatives, and the mechanism by which subject information is suppressed.

Formulation	Pair semantics	Invariance mechanism
CLISA (Shen et al., 2021)	Positive pair: same time segment of the same trial across two different subjects; negatives: all other samples in the minibatch	Cross-subject alignment grounded in shared stimulus segment identity
SICL for HAR (Yarici et al., 4 Jul 2025)	Standard positive pair from two augmentations of the same sample; negatives split into same-subject and different-subject subsets	Re-weight same-subject negatives with $Q_{Si}$
Subject-invariant SSL / hybrid adversarial methods (Cheng et al., 2020, Lin et al., 3 Apr 2026)	Standard augmentative or class-based positives; subject labels used by an auxiliary discriminator	Adversarial removal of subject identity from encoder features

In CLISA, positive pairs are not created by data augmentation and are not matched by emotion label. A positive pair consists of EEG segments from two different subjects corresponding to the same time segment of the same trial, so the inter-subject alignment is anchored to shared stimulus timing. With subjects $A$ and $B$, the minibatch is

$D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$

and for anchor $X_i^A$ the sole positive is $X_i^B$, while the negatives are

$\{X_j^s \mid j\neq i,\ s\in\{A,B\}\},$

giving $2(N-1)$ negatives per anchor. The contrastive loss is a temperature-scaled normalized cross-entropy over cosine similarity, with the total minibatch loss

$L=\sum_{i=1}^{N}(l_i^A+l_i^B).$

This formulation defines subject invariance through same-stimulus inter-subject alignment rather than adversarial domain confusion (Shen et al., 2021).

In HAR SICL, the baseline remains an InfoNCE-style contrastive objective, but negatives are decomposed according to subject identity. The denominator becomes

$$D_i = Q_{Si} \sum_{s \in S(i)} \exp\left( \frac{z_i \cdot z_s}{\tau} \right) + \sum_{k \notin S(i)} \exp\left( \frac{z_i \cdot z_k}{\tau} \right),$$

where $A$0 denotes negatives from the same subject as anchor $A$1. The stated purpose is to re-weight negative pairs drawn from the same subject to suppress subject-specific cues and emphasize activity-specific information (Yarici et al., 4 Jul 2025).

A third formulation uses adversarial subject removal. In the 2020 biosignal paper, a subject classifier $A$2 is attached to the latent representation $A$3, and the encoder is trained jointly with the contrastive loss and a subject-confusion regularizer: $A$4 In the 2026 sEMG fatigue model, subject is explicitly treated as a domain, and a GRL-equipped domain classifier is combined with supervised contrastive learning: $A$5 Here the subject invariance mechanism is the domain-adversarial branch, while the contrastive term sharpens class structure (Cheng et al., 2020, Lin et al., 3 Apr 2026).

These formulations are not equivalent. CLISA aligns across subjects directly; HAR SICL modifies the treatment of same-subject negatives; adversarial hybrids remove subject information with an auxiliary discriminator. A plausible synthesis is that SICL is defined less by one specific loss than by the explicit use of subject metadata or cross-subject correspondence to prevent the embedding from organizing around person identity.

3. Learning pipelines, architectures, and representation design

SICL systems are typically staged rather than monolithic. CLISA uses two phases: a contrastive pretraining phase that learns aligned spatiotemporal EEG representations, followed by a prediction phase that uses only the trained base encoder to extract latent signals and convert them into differential entropy features,

$A$6

which are then smoothed with a linear dynamical system and classified by a three-layer multilayer perceptron. The encoder itself uses a spatial convolution

$A$7

followed by a temporal convolution

$A$8

so that subject alignment is learned directly on EEG time series before classification (Shen et al., 2021).

In biosignals, augmentations remain central when cross-subject correspondence is unavailable. The 2020 subject-aware SSL framework is MoCo-like: two augmented views $A$9 and $B$0 are encoded into query and key embeddings, and InfoNCE is combined with subject-aware negative sampling or adversarial subject confusion. The reported time-series augmentations include temporal cutout, temporal delay, Gaussian noise, bandstop filtering, signal mixing, and, for multichannel EEG, spatial rotation, spatial shift, sensor dropout, and sensor cutout. The paper reports that for EEG the most effective were temporal cutout, temporal delay, and signal mixing (Cheng et al., 2020).

Sleep staging provides a more ambiguous case. The MViTime model uses a 1D adaptation of MobileViT for 30-second single-channel Fpz-Cz epochs from Sleep-EDF, with cropping and permutation augmentations applied in a two-stage procedure of contrastive pretraining followed by supervised fine-tuning. However, the paper’s cross-subject branch remains only partially SICL-like: positives are still defined as different transformations of the same subject’s selected stage-specific signal, while transformed signals from different subjects are treated as negatives. The work is therefore better characterized as ISC-motivated cross-subject contrastive organization than as explicit same-stage cross-subject positive alignment (Zhang et al., 2023).

In HAR, SICL is intentionally lightweight. The loss can be inserted into existing pipelines using a three-layer 1D CNN plus transformer self-attention for inertial or foot-pressure data, or a lightweight convolutional co-occurrence network for skeletons. The paper emphasizes that the method requires no additional encoder, no adversarial branch, and no new modality-specific architecture; the extra cost is batch-wise bookkeeping for same-subject negatives and the computation of $B$1 or $B$2 (Yarici et al., 4 Jul 2025).

The sEMG fatigue model represents a different design point: it uses an Inception-attention feature extractor over six-channel $B$3 time-frequency images, a fatigue classifier, and a GRL-based domain classifier. The embedding is a 128-dimensional feature vector, and the whole system is trained end-to-end with fatigue cross-entropy, supervised contrastive loss, and domain cross-entropy (Lin et al., 3 Apr 2026).

4. Empirical evidence and evaluation practice

The strongest direct evidence for SICL comes from cross-subject evaluation in which test subjects are unseen during both pretraining and classifier training. In CLISA, THU-EP uses 10-fold cross-subject cross-validation and SEED uses leave-one-subject-out evaluation. On THU-EP binary positive-vs-negative classification, CLISA achieves $B$4, compared with $B$5 for raw DE+MLP, $B$6 for subspace alignment, $B$7 for CorrCA, and $B$8 for SeqCLR; on the harder nine-class THU-EP task, CLISA reaches $B$9, and on SEED it obtains $D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$0. The paper also reports the best ROC/AUC on THU-EP binary classification, with CLISA at 0.758 versus 0.667–0.682 for baselines (Shen et al., 2021).

The HAR SICL paper reports consistent gains across datasets, modalities, and contrastive objectives under cross-subject evaluation. The headline claim is improvement by up to 11% over traditional contrastive learning methods. In unimodal linear evaluation, for example, on UTD-MHAD inertial data SimCLR gives 0.7173 and SICL gives 0.7359; on MMAct skeleton data SimCLR gives 0.7257 and SICL gives 0.7413; on DARai foot-pressure data SimCLR gives 0.1172 and SICL gives 0.1316. The same principle also improves multimodal CMC and supervised contrastive learning, with SI-SupCon reaching 0.2031 on DARai inertial data versus 0.1715 for SupCon and 0.1683 for cross-entropy (Yarici et al., 4 Jul 2025).

The biosignal SSL paper shows a more nuanced pattern. For EEG linear evaluation, subject-invariant SSL improves frozen representation quality, and in intrasubject testing reaches 79.4 for 2-class and 50.3 for 4-class, compared to 77.6 and 46.6 for base SSL. The paper also states that using labels from 4 subjects for subject-invariant SSL gave performance similar to supervised end-to-end learning with 16 subjects. At the same time, subject-specific SSL rather than subject-invariant SSL gives the best fine-tuning result for EEG, and on ECG only mild adversarial regularization helps because label distribution is uneven across subjects (Cheng et al., 2020).

The sEMG fatigue study provides a hybrid SICL-like result under subject-wise 4-fold cross-validation. The full IADAN model achieves 93.54% accuracy, 92.69% recall, and 92.69% F1-score. On Fold 1, the loss ablation is especially revealing: fatigue cross-entropy alone gives 62.44 accuracy, $D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$1 domain cross-entropy gives 81.87, $D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$2 supervised contrastive gives 87.47, and the full $D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$3 objective gives 95.28. The corresponding t-SNE visualization is described as showing three distinct fatigue clusters with subject IDs mixed within clusters, which is the qualitative pattern expected of subject-invariant features (Lin et al., 3 Apr 2026).

Sleep staging offers weaker evidence. MViTime reports 87.8 accuracy and 82.7 F1 on EDF-20, and 84.9 accuracy and 79.6 F1 on EDF-78, while held-out subject examples show improvements in difficult S1 cases and larger gains when 58 additional subjects are used for contrastive pretraining. However, the paper does not provide a full LOSO evaluation over the entire cohort, does not fully specify the main benchmark protocol, and does not construct same-stage cross-subject positives. Its evidence for subject invariance is therefore suggestive rather than definitive (Zhang et al., 2023).

5. Theoretical extensions and domain-adjacent uses

Several papers broaden SICL beyond the specific losses used in biosignals and HAR. "Invariance-adapted decomposition and Lasso-type contrastive learning" (Koyama et al., 2022) proposes an invariance-adapted latent space in which each transformation $D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$4 fixes a subset of coordinates, and the latent space is decomposed into minimal intersections of invariant spaces and their complements. This yields a family of components $D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$5 analogous to frequencies in harmonic analysis. The paper is not about subject identity directly, but it suggests that subject invariance need not mean collapsing the entire embedding into one uniformly invariant space. A plausible implication is that subject-specific, session-specific, and task-invariant coordinates can be localized to different blocks rather than globally entangled (Koyama et al., 2022).

"Structured Contrastive Learning for Interpretable Latent Representations" (Shen et al., 18 Nov 2025) makes that partition explicit by dividing $D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$6 into invariant, variant, and free subspaces,

$D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$7

and applying a push-pull objective in which positive pairs are pulled together in $D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$8 and pushed apart in $D=\{X_i^s \mid i=1,2,\dots,N;\ s\in\{A,B\}\},$9. The paper studies ECG phase invariance and IMU rotation robustness rather than subject identity, but it is directly relevant to SICL because subject identity can be interpreted as the variant nuisance factor while task-relevant content occupies the invariant subspace (Shen et al., 18 Nov 2025).

SepCLR separates common from salient patterns through two encoders and a k-JEM joint-entropy term. Its constrained InfoMax objective,

$X_i^A$0

is framed for contrastive analysis rather than subject invariance, but it provides a principled template for separating shared structure from target-specific variation and for preventing leakage between branches. This suggests one route to SICL in weakly supervised settings: an invariant branch for cross-subject common structure and a separate nuisance branch for subject- or domain-specific residuals (Louiset et al., 2024).

Application domains beyond biosignals make the boundaries of the term clearer. "CustomContrast" explicitly argues that an ideal subject representation in subject-driven text-to-image customization should decouple subject intrinsic attributes from irrelevant attributes such as pose, view, lighting, style, size, position, and background, using crossmodal semantic contrastive learning and multiscale appearance contrastive learning (Chen et al., 2024). By contrast, "Consistent Subject Generation via Contrastive Instantiated Concepts" is not subject-invariant in the usual sense: it learns a mechanism that keeps identity consistent under prompt and noise changes, so its representation is subject-consistent and subject-indexed rather than subject-invariant (Hsin-Ying et al., 31 Mar 2025). "SiCL: Silhouette-Driven Contrastive Learning" for person re-identification under clothes change is similarly narrower: it learns cross-clothes invariance by aligning RGB and silhouette information, which is closely related to subject-invariant learning in spirit but more precisely targets clothes invariance rather than generic subject invariance (Li et al., 2023).

6. Assumptions, limitations, and recurrent misconceptions

A central limitation of SICL is that subject invariance is rarely free. CLISA depends critically on common stimuli across subjects and on matched segment timing within those stimuli. Its positive-pair definition is therefore especially natural for paradigms in which all participants watch the same videos, but it is not a generic method for settings where subjects perform different tasks or view different items (Shen et al., 2021).

Other SICL variants require subject metadata. HAR SICL needs subject IDs during training to determine whether a negative belongs to the same subject as the anchor. In the self-supervised version, activity labels are not needed, but subject IDs are; in the supervised contrastive version, both are used. This means SICL is often self-supervised only with respect to task labels, not label-free in the strict sense (Yarici et al., 4 Jul 2025).

A second limitation is that removing subject information can remove useful signal. The 2020 biosignal paper states this most clearly on ECG: only lower adversarial weights worked well because beat and rhythm labels were unevenly distributed across subjects, so stronger invariance removed partly predictive subject-related information. The paper’s broader conclusion is that the desired degree of invariance depends on whether subject identity is nuisance variation or partly task-relevant signal (Cheng et al., 2020).

Protocol quality also varies widely. The sleep staging paper is directly motivated by inter-subject correlation and cross-subject robustness, yet it does not clearly formalize the main benchmark split, demonstrates held-out-subject evaluation only on three example subjects, and defines positives as within-subject augmentation pairs while treating different subjects as negatives. In strict SICL terms, it is therefore a partial match rather than a fully realized same-stage cross-subject alignment method (Zhang et al., 2023).

A recurring misconception is that any subject-centered contrastive method is subject-invariant. The literature itself distinguishes several cases. CoCoIns preserves identity across nuisance changes rather than removing it; SiCL for re-identification seeks clothes invariance; CustomContrast decouples intrinsic subject attributes from irrelevant attributes inside a personalization pipeline; and Structured Contrastive Learning studies nuisance partitions without subject labels. These are adjacent and often informative, but they are not identical to the biosignal and HAR sense of SICL in which the goal is zero-shot transfer to unseen subjects (Hsin-Ying et al., 31 Mar 2025, Li et al., 2023, Chen et al., 2024).

Another recurrent issue is evidence for invariance itself. Some papers provide t-SNE plots, subject-ID accuracy reductions, or held-out-subject results; others do not provide explicit subject-classification leakage tests, domain discrepancy measures, or exhaustive subject-wise evaluation. The strongest empirical SICL papers therefore combine cross-subject protocols with analyses showing reduced subject clustering and preserved task separability. Across the literature, this suggests that SICL is best evaluated not only by downstream accuracy but also by whether subject identity becomes less recoverable from the learned representation while task semantics remain discriminative.

In aggregate, Subject-Invariant Contrastive Learning is not one algorithm but a design principle for cross-subject representation learning. Its strictest forms define positives through cross-subject semantic correspondence, its lightweight forms re-weight same-subject negatives, its hybrid forms combine contrastive losses with adversarial subject confusion, and its more theoretical extensions argue for structured latent spaces that explicitly separate invariant and variant factors. The unifying criterion is constant across these variants: the representation should generalize across people by reducing the extent to which subject identity serves as the organizing axis of the embedding.