Privileged Contrastive Pretraining (PriCon)

• PriCon is a learning paradigm that exploits richer, training-only information to craft robust, deployable representations under partial observability.
• It integrates varied contrastive methods—including SimSiam-style alignment, teacher–student KL distillation, and adaptive InfoNCE—to suit domain-specific tasks.
• Applications span humanoid control, multimodal affect modelling, and PointGoal navigation, achieving faster convergence and improved transfer performance.

Searching arXiv for the cited PriCon papers to ground the article in current preprints. I’ll look up the specific arXiv entries for PriCon and closely related formulations. [Tool usage placeholder: arXiv search executed for (Yuan et al., 15 Dec 2025, Pinitas et al., 30 Jul 2025), and (Zhalehmehrabi et al., 3 Jun 2026) to verify titles and topical correspondence.] Privileged Contrastive Pretraining (PriCon) is a general paradigm for learning deployable representations by aligning a view that is available at inference with a richer privileged view that is available only during training. Across current formulations, the common objective is to use privileged information to shape a latent space that remains computable from the deployable modality at test time. In humanoid whole-body control, PriCon aligns proprioceptive observations with privileged simulator state via a SimSiam-style objective (Yuan et al., 15 Dec 2025). In multimodal affect modelling, PriCon denotes a two-stage teacher–student framework that combines supervised contrastive learning (SCL) with Learning Using Privileged Information (LUPI) (Pinitas et al., 30 Jul 2025). In PointGoal navigation, privileged LiDAR is used during representation pretraining to modulate an InfoNCE objective for monocular RGB, after which the encoder is frozen for reinforcement learning (Zhalehmehrabi et al., 3 Jun 2026). Taken together, these works suggest that PriCon is best understood as a representational strategy for partial observability, modality asymmetry, and deployment-time sensor constraints rather than as a single fixed loss.

1. Definition and conceptual scope

PriCon addresses settings in which training-time information is richer than deployment-time information. The privileged view may be a full simulator state, accurate contact and kinematic quantities, body mass and base orientation for humanoid control, fine-grained audiovisual or physiological features for affect modelling, or LiDAR geometry for navigation (Yuan et al., 15 Dec 2025). The deployable or prevalent view is the one available at test time: proprioception on the robot, raw frames in affective computing, or monocular RGB together with standard task-related inputs in navigation (Yuan et al., 15 Dec 2025, Pinitas et al., 30 Jul 2025, Zhalehmehrabi et al., 3 Jun 2026).

The central asymmetry is explicit in all three instantiations. In humanoid WBC, the problem is framed as a POMDP $M=(S,O,A,P,\Omega,R,\gamma)$ with partial observability $o_t \subset s_t$, where privileged variables such as accurate root linear velocity, body mass distribution, base quaternion, and terrain/contact indicators are accessible in simulation or to the critic but cannot be robustly sensed on-board at runtime (Yuan et al., 15 Dec 2025). In affective computing, the distinction is between in-vitro multimodal acquisition and in-vivo unimodal deployment, where privacy constraints, sensor availability, and noise often restrict inference to a single less informative modality such as raw frames (Pinitas et al., 30 Jul 2025). In navigation, geometry is treated as more stable across scenes than appearance, and privileged LiDAR is used only during pretraining so that the deployed policy can rely on RGB, goal position, and proprioception without LiDAR (Zhalehmehrabi et al., 3 Jun 2026).

A plausible implication is that PriCon formalizes a recurring design pattern: exploit training-only channels to induce a control- or task-relevant latent, then discard those channels at inference. The details of how that latent is induced, however, differ substantially across domains.

2. Core training formulations

PriCon does not correspond to one canonical objective. The published formulations span supervised contrastive pretraining, Siamese consistency with stop-gradient, and adaptive-temperature InfoNCE.

In multimodal affect modelling, PriCon first pretrains teacher representations with supervised contrastive learning. For a batch index set $I$, positives $P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$, comparands $A_i=I\setminus\{i\}$, and teacher representation $r_i\in \mathbb{R}^d$, the loss is

$$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$

The pretrained teacher is then used in a LUPI stage, where the student sees only prevalent inputs $x$, the teacher sees privileged or fused inputs $\tilde{x}$, and the student minimizes

$$L_P = (1-\alpha)\, L_{CE}(f(x), y) \;+\; \alpha\, L_{KL}\big(\operatorname{softmax}(g(\tilde{x})),\operatorname{softmax}(f(x))\big).$$

The teacher’s parameters are fixed during student training, and the distillation occurs at the probability level via KL divergence rather than embedding-level MSE or margin regularizers (Pinitas et al., 30 Jul 2025).

In humanoid WBC, PvP realizes PriCon through SimSiam-style privileged–proprioceptive alignment. Let $o_t \subset s_t$0 denote the privileged state with privileged-only fields zeroed, leaving only proprioceptive components. With shared encoder $o_t \subset s_t$1 and predictor $o_t \subset s_t$2,

$o_t \subset s_t$3

The PvP loss is the symmetric negative cosine similarity with stop-gradient:

$o_t \subset s_t$4

$o_t \subset s_t$5

This formulation uses no negatives, no temperature $o_t \subset s_t$6, no large batches, and no momentum encoder; the positive pair is generated by zero-masking privileged channels rather than by hand-crafted augmentation (Yuan et al., 15 Dec 2025).

In PointGoal navigation, PriCon uses an InfoNCE objective with pair-specific temperatures derived from privileged LiDAR geometry. For encoder $o_t \subset s_t$7, projector $o_t \subset s_t$8, normalized embedding $o_t \subset s_t$9, and positive index $I$0 for anchor $I$1,

$I$2

LiDAR vector $I$3 induces a weighted distance and similarity

$I$4

$I$5

For negatives, the temperature is an adaptive value derived from this geometric similarity and then mapped to $I$6; positives use a small constant $I$7 (Zhalehmehrabi et al., 3 Jun 2026).

These formulations share a common role for privileged information but differ in what privileged information does. In affect modelling it strengthens a teacher whose logits supervise a student; in humanoid control it defines modality-complementary paired views for direct latent alignment; in navigation it modulates pairwise repulsion and attraction in contrastive pretraining.

3. PriCon in humanoid whole-body control

The paper “PvP: Data-Efficient Humanoid Robot Learning with Proprioceptive-Privileged Contrastive Representations” defines PvP as a concrete instantiation of PriCon tailored to humanoid whole-body control (Yuan et al., 15 Dec 2025). The policy consumes proprioceptive observations $I$8 available at train and test time, while the value network consumes privileged states $I$9 available only during training. For velocity tracking on LimX-Oli-31dof-Velocity, the proprioceptive input is a stack of 5 consecutive frames containing base angular velocity, projected gravity, velocity commands, joint positions, joint velocities, actions, and gait; for motion imitation on LimX-Oli-31dof-Mimic, the input is a single frame containing base angular velocity, projected gravity, joint positions, joint velocities, actions, and mimic reference (Yuan et al., 15 Dec 2025). Training-only privileged state additionally includes quantities such as base linear velocity, base position along $P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$0, body mass, base quaternion, previous actions, and other privileged fields depending on the task (Yuan et al., 15 Dec 2025).

PvP’s distinctive design within PriCon is the zero-masked privileged pairing. The positive pair is $P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$1, where $P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$2 is the same vector with privileged-only fields zeroed, leaving only deployable proprioceptive components. This generates modality-complementary views without domain-specific augmentation (Yuan et al., 15 Dec 2025). The architecture consists of MLP policy and value encoders with the same architectural class but different inputs. The policy network uses Linear(on-demand, 512) → [ELU](https://www.emergentmind.com/topics/exponential-linear-unit-elu) → Linear(512,256) → ELU → Linear(256,128) → ELU → Linear(128,31), and the value network uses Linear(on-demand, 512) → ELU → Linear(512,256) → ELU → Linear(256,128) → ELU → Linear(128,1). PvP attaches to the policy encoder $P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$3, and the predictor $P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$4 is an MLP head not shared with the policy head. No LSTM or RNN is used; velocity tracking instead relies on short temporal stacking of proprioceptive frames (Yuan et al., 15 Dec 2025).

The reinforcement-learning substrate is PPO with GAE. The policy acts from $P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$5, while the value network receives privileged $P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$6 for advantage estimation. Joint optimization uses

$P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$7

or, with interval updates,

$P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$8

where $P_i=\{p\in I\setminus\{i\}: y_p=y_i\}$9 every $A_i=I\setminus\{i\}$0 steps and $A_i=I\setminus\{i\}$1 otherwise (Yuan et al., 15 Dec 2025). Training is online and end-to-end with PPO on single-GPU IsaacLab, with Adam, $A_i=I\setminus\{i\}$2, $A_i=I\setminus\{i\}$3, GAE $A_i=I\setminus\{i\}$4, entropy coefficient $A_i=I\setminus\{i\}$5, value coefficient $A_i=I\setminus\{i\}$6, clip range $A_i=I\setminus\{i\}$7, max grad norm $A_i=I\setminus\{i\}$8, 4 mini-batches, 5 learning epochs, desired KL $A_i=I\setminus\{i\}$9, 4096 environments per worker, 32 steps per episode, and up to 30000 episodes (Yuan et al., 15 Dec 2025).

The empirical claims are task-specific. On the LimX Oli robot, PvP markedly accelerates convergence in velocity tracking relative to PPO+SimSiam, PPO+SPR, and PPO+VAE, and reaches higher final returns in motion imitation. It converges faster on the action smoothness penalty, achieves the best tracking KPIs across linear velocity, angular velocity, and imitation metrics, and transfers from IsaacLab to MuJoCo and to real-robot trials on LimX Oli (Yuan et al., 15 Dec 2025). The ablations further report that policy-encoder SRL is more stable than value-encoder SRL, interval $r_i\in \mathbb{R}^d$0 yields the overall best performance in imitation, and a moderate $r_i\in \mathbb{R}^d$1 worked well, with $r_i\in \mathbb{R}^d$2 and $r_i\in \mathbb{R}^d$3 chosen (Yuan et al., 15 Dec 2025).

4. PriCon in multimodal affect modelling

The paper “Privileged Contrastive Pretraining for Multimodal Affect Modelling” defines PriCon explicitly as a two-stage teacher–student framework designed to bridge the in-vitro to in-vivo generalization gap in affective computing (Pinitas et al., 30 Jul 2025). The framework integrates SCL for teacher pretraining with LUPI for transfer to a student that sees only the prevalent modality at test time. In the reported setup, raw frames are prevalent, while privileged information includes handcrafted audiovisual and physiological features in RECOLA, and high-level gameplay features in AGAIN (Pinitas et al., 30 Jul 2025).

The student $r_i\in \mathbb{R}^d$4 is a frames-only CNN over grayscale frames with five convolutional layers having filter sizes $r_i\in \mathbb{R}^d$5, strides 2 for the first four layers and 1 for the fifth, $r_i\in \mathbb{R}^d$6 kernels, ReLU activations, a dense layer of 768 ReLU units, and a 2-unit SoftMax decision layer, with dropout $r_i\in \mathbb{R}^d$7 before the output. The privileged teacher $r_i\in \mathbb{R}^d$8 is an ANN with a 30-unit hidden layer using logistic activation and a 2-unit SoftMax output, again with dropout $r_i\in \mathbb{R}^d$9. The fusion teacher $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$0 concatenates penultimate frame and feature representations, adds a 30-unit ReLU alignment layer on the frame side, fuses via a linear 60-unit ReLU layer, and outputs a 2-unit SoftMax (Pinitas et al., 30 Jul 2025).

The data regime is also explicit. RECOLA contains audio, video, and physiology from dyadic interactions, totaling 9.5 hours and 46 participants, of whom 23 are publicly released. Features comprise 40 visual, 130 audio, and 116 physiological handcrafted descriptors. Sessions are segmented into overlapping windows of length $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$1 seconds with step 400 ms, using 5 grayscale frames of size $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$2 per second. Binary high/low arousal and valence labels are constructed by median and threshold procedures with $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$3 (Pinitas et al., 30 Jul 2025). AGAIN uses raw gameplay frames and privileged feature sets of 14 general plus game-specific features, with 33 for Endless, 39 for Pirates!, and 47 for Run’N’Gun!, over 120 participants. Sessions are segmented with window length $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$4 seconds and step 500 ms; binarization uses participant-relative anchoring with $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$5 (Pinitas et al., 30 Jul 2025).

The results are reported as average 5-fold accuracies with participant-wise splits and significance via paired $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$6-tests at $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$7. PriCon students consistently outperform both standard LUPI and end-to-end frames-only baselines in many settings. Examples given in the paper include RECOLA arousal at 1 s, where $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$8 improves from 59.41 to 63.61 under LUPI, and PriCon further improves $$L_{SC} = \sum_{i\in I} \frac{-1}{|P_i|}\sum_{p \in P_i} \log \frac{\exp(r_i\cdot r_p/\tau)} {\sum_{a\in A_i}\exp(r_i\cdot r_a/\tau)}.$$9 from 60.75 to 63.74; and AGAIN Run’N’Gun! at 2 s, where $x$0 improves from 69.80 to 72.05 under PriCon (Pinitas et al., 30 Jul 2025). The paper further reports that students often approach all-modality teacher performance despite using only frames at test time, whereas gains for valence are more modest and variable (Pinitas et al., 30 Jul 2025).

This variant makes clear that PriCon need not align teacher and student at the representation level. Here, privileged contrastive pretraining is realized by shaping the teacher’s latent geometry through label-aligned supervised contrastive learning and then transferring that structure via KL-based distillation.

5. PriCon in PointGoal navigation

The paper “Robust Scene Transfer for PointGoal Navigation via Privileged Sensor Guided Contrastive Learning” states that its official name is sensor-guided adaptive contrastive learning for visual representation pretraining and that this is exactly privileged contrastive pretraining, with privileged LiDAR guiding pretraining for monocular RGB (Zhalehmehrabi et al., 3 Jun 2026). The task is PointGoal navigation: the agent receives an RGB image, goal position in the robot’s local frame in polar coordinates, and proprioception consisting of linear and angular velocities, and outputs continuous linear and angular velocities (Zhalehmehrabi et al., 3 Jun 2026).

The representation pipeline is separated from policy optimization. A ResNet-50 backbone initialized with ImageNet weights maps RGB images to features, and a two-layer MLP projection head outputs a $x$1 embedding with ReLU and BatchNorm; embeddings are $x$2-normalized before cosine similarity. Input resolution is $x$3 RGB. The final model uses no image augmentations. Pretraining uses batch size 8192, Adam with learning rate $x$4 and linear decay, $x$5, $x$6, and positive temperature $x$7 (Zhalehmehrabi et al., 3 Jun 2026).

Privileged LiDAR does not determine positive pairs. Positive pairs are assigned through a scene-invariant pairing strategy in which trajectories collected in a reference environment are replayed in visually distinct environments with identical geometry, so frames at the same trajectory step across environments provide navigation-relevant counterparts. LiDAR is used only to compute the geometry-aware similarity that sets negative-pair temperatures (Zhalehmehrabi et al., 3 Jun 2026). The LiDAR weighting scheme assigns $x$8 to beams within the camera field of view and smoothly downweights beams outside the field of view toward $x$9 via a sigmoid schedule over azimuth (Zhalehmehrabi et al., 3 Jun 2026).

After pretraining, the encoder is frozen and used as the perceptual backbone for SAC. The policy receives

$\tilde{x}$0

where $\tilde{x}$1 is the frozen normalized embedding. The policy itself is a two-layer MLP with hidden dimension 256. SAC uses actor and critic learning rates $\tilde{x}$2, batch size 1024, replay buffer $\tilde{x}$3, discount $\tilde{x}$4, target update $\tilde{x}$5, and automatic entropy tuning. Reward shaping uses success and collision rewards of $\tilde{x}$6 and $\tilde{x}$7 respectively, together with penalties on change in geodesic distance, reverse velocity, and angular velocity (Zhalehmehrabi et al., 3 Jun 2026).

The evaluation emphasizes scene transfer. Policies are trained in Photo-studio and evaluated on seven unseen indoor and outdoor environments under appearance and semantic shifts. Reported metrics are Success Rate (SR) and Success weighted by Path Length (SPL), averaged over 5 seeds and 100 episodes per test environment (Zhalehmehrabi et al., 3 Jun 2026). Key reported numbers include: under appearance shift with uniform floors and backgrounds, PriCon achieves SR 82.6% and SPL 78.8%, versus CLIP-ViT at SR 52.0% and SPL 44.7%, and SimCLR at SR 39.4% and SPL 33.4%; under hard scene shift with simple floors, PriCon achieves SR 77.7% versus SimCLR 51.1%; and under hard scene shift with textured floors, PriCon achieves SR 49.4% versus SimCLR 45.7%, DINOv2 15.2%, and CLIP-ResNet 10.1% (Zhalehmehrabi et al., 3 Jun 2026). Ablations attribute gains to scene-invariant pairing without augmentations, cross-stage domain mismatch, and adaptive temperatures, with SR rising from 48.38% for SimCLR to 63.56% for full PriCon (Zhalehmehrabi et al., 3 Jun 2026).

6. Comparative structure, practical implications, and limitations

Across these papers, PriCon is unified by three recurrent elements: a privileged view available only during training, a deployable view available at inference, and a contrastive or contrastive-derived mechanism that makes the deployable representation reflect privileged structure. PvP explicitly summarizes this as PriCon’s three pillars: privileged view, deployable view, and contrastive alignment (Yuan et al., 15 Dec 2025). The affective-computing formulation re-expresses the same idea in teacher–student terms: privileged modalities strengthen a teacher, and the student ultimately operates with frames only (Pinitas et al., 30 Jul 2025). The navigation formulation again preserves the asymmetry: LiDAR shapes the RGB encoder only during pretraining, and deployment uses RGB alone together with goal and proprioceptive inputs (Zhalehmehrabi et al., 3 Jun 2026).

The differences are equally important. PvP performs joint online learning with PPO and shared minibatches; there is no offline pretraining or encoder freezing (Yuan et al., 15 Dec 2025). The affect model uses a two-stage pipeline in which teachers are pretrained by SCL, then fixed or probed during student training (Pinitas et al., 30 Jul 2025). The navigation model separates stages even more sharply: the encoder is pretrained independently, frozen, and then used as a backbone for SAC under a cross-stage domain mismatch that intentionally decouples pretraining and policy-learning environments (Zhalehmehrabi et al., 3 Jun 2026). This suggests that PriCon is compatible with online RL, offline pretraining, and classical teacher–student distillation, provided the privileged signal is confined to training.

Several practical conclusions recur in the source materials. In humanoid WBC, privileged pairing is described as powerful, policy-side SRL as more stable than value-side SRL, and interval scheduling such as $\tilde{x}$8 as useful for avoiding early SRL overfitting on low-quality rollouts (Yuan et al., 15 Dec 2025). In affect modelling, the mixing coefficient $\tilde{x}$9 balances task supervision and teacher alignment and is selected from $$L_P = (1-\alpha)\, L_{CE}(f(x), y) \;+\; \alpha\, L_{KL}\big(\operatorname{softmax}(g(\tilde{x})),\operatorname{softmax}(f(x))\big).$$0 on the validation set (Pinitas et al., 30 Jul 2025). In navigation, heavy augmentations are explicitly avoided because they can conflict with geometry consistency, and the large batch size and pairwise temperature computations are noted as memory-intensive (Zhalehmehrabi et al., 3 Jun 2026).

The limitations are domain-specific but structurally related. PvP warns that noisy or biased privileged estimates may mislead the encoder, that distribution mismatch between simulation privileged states and real-world dynamics may harm generalization, and that conclusions are currently limited to PPO rather than off-policy or model-based RL (Yuan et al., 15 Dec 2025). The affective-computing PriCon notes dependence on reliable labels for SCL, sensitivity of performance gains for valence, and the need for privileged modalities during teacher training (Pinitas et al., 30 Jul 2025). The navigation formulation assumes time-synchronized and extrinsically calibrated LiDAR–camera rigs, identical underlying obstacle layouts for scene-invariant pairing, and substantial compute due to batch size 8192 and per-pair temperature calculations (Zhalehmehrabi et al., 3 Jun 2026).

A common misconception would be to treat PriCon as synonymous with contrastive pretraining in the narrow SimCLR sense. The published record does not support that simplification. One PriCon variant uses supervised contrastive teacher pretraining plus KL distillation, another uses SimSiam-style zero-masked privileged pairing with no negatives and no temperature, and a third uses LiDAR-guided adaptive temperatures within InfoNCE (Pinitas et al., 30 Jul 2025, Yuan et al., 15 Dec 2025, Zhalehmehrabi et al., 3 Jun 2026). What remains stable across these variants is not the exact loss but the training-time use of privileged information to make an inference-time representation more robust, more task-relevant, or more transferable.