Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policy (2511.09737v1)

Abstract: Generalization to unseen environments is a significant challenge in the field of robotics and control. In this work, we focus on contextual reinforcement learning, where agents act within environments with varying contexts, such as self-driving cars or quadrupedal robots that need to operate in different terrains or weather conditions than they were trained for. We tackle the critical task of generalizing to out-of-distribution (OOD) settings, without access to explicit context information at test time. Recent work has addressed this problem by training a context encoder and a history adaptation module in separate stages. While promising, this two-phase approach is cumbersome to implement and train. We simplify the methodology and introduce SPARC: single-phase adaptation for robust control. We test SPARC on varying contexts within the high-fidelity racing simulator Gran Turismo 7 and wind-perturbed MuJoCo environments, and find that it achieves reliable and robust OOD generalization.

• The paper presents SPARC, a unified single-phase adaptation approach that combines context encoding and history-based adaptation to generalize to unseen environments.
• Experimental results in Gran Turismo 7 and MuJoCo show SPARC’s superior performance with high lap completion rates and near-oracle reward returns under OOD conditions.
• The study highlights practical benefits such as on-device continual learning and distributed training, paving the way for real-world robotics and autonomous vehicle control.

Out-of-Distribution Generalization with SPARC: Robust Racing with a Unified Adaptation Paradigm

Introduction

This paper presents SPARC (Single-Phase Adaptation for Robust Control), a context-adaptive RL framework that targets robust generalization to out-of-distribution (OOD) environments without privileged context information during deployment. SPARC directly addresses the limitations of prior two-phase adaptation algorithms, such as Rapid Motor Adaptation (RMA), by unifying context encoding and history-based adaptation into a single, streamlined training curriculum. The approach is empirically validated on high-fidelity Gran Turismo 7 racing simulations, encompassing 100 unseen vehicle models, and on MuJoCo tasks subjected to OOD wind perturbations. The methodology enables agents to adapt policies to OOD contexts using only histories of interactions, bypassing the need for explicit context input at test time.

SPARC Algorithmic Design

SPARC implements a joint training procedure, simultaneously updating an expert policy $\pi^{ex}$ (with access to privileged context during training) and an adapter policy $\pi^{ad}$ (relying exclusively on observation-action history). The adapter employs a history encoder $\phi$ trained to approximate the expert's context encoding $\psi(c)$ via supervised regression, where $c$ is the contextual variable accessible only during training. Critically, the context encoding target is non-stationary, as both policies co-adapt, in contrast to the static per-checkpoint target in RMA.

Figure 1: SPARC’s architecture: expert and adapter policies are trained in a single phase using context and history inputs. The adapter, trained to regress the expert’s contextual representation, enables robust adaptation to OOD environments.

The modular structure supports off-policy learning, asynchronous distributed rollout collection, and continual learning on-device, supporting practical deployment in resource-constrained or privacy-sensitive systems. Experience is collected with the adapter policy to ensure policy-correct learning dynamics for the final deployed model.

Experimental Protocol

SPARC is evaluated both in MuJoCo continuous control environments (HalfCheetah, Hopper, Walker2d) with systematic wind perturbations, and in Gran Turismo 7, where the training distribution covers 400 car models and OOD generalization is assessed on 100 held-out vehicles. Track diversity is high, spanning tarmac, concrete, and mixed dirt surfaces. Lap times are normalized against the built-in AI (BIAI) to account for inherent vehicle speed variability. The adapter policy is deployed for inference on OOD settings.

Performance Results

SPARC consistently outperforms strong baselines—including Only Observation, History Input (pure TCN history), and RMA—on OOD metrics such as lap completion rate and normalized lap times. On Grand Valley, SPARC completes laps for 98.06% of OOD vehicles, outperforming RMA and history-based baselines.

Figure 2: SPARC attains the highest OOD lap completion rate and fastest normalized lap times on Gran Turismo 7’s Grand Valley track.

A detailed analysis under engine power and mass perturbations confirms SPARC’s dominance: 99.9% lap completion over the evaluated OOD grid, surpassing both RMA and Oracle baselines (Oracle given only car context, no history). SPARC demonstrates improved performance over Oracle due to superior exploitation of history in partial observability.

Figure 3: SPARC yields higher average returns across extensive OOD wind perturbations in MuJoCo HalfCheetah compared to RMA.

In MuJoCo wind experiments, SPARC exhibits superior reward returns in 2 out of 3 environments. The method robustly manages continuous context drift, as visualized by delta reward maps, and produces near-Oracle performance.

Ablation and Sensitivity Analysis

SPARC’s training design allows direct investigation of architectural choices:

• Rollout Policy Selection: Experience collection with the adapter policy yields improved OOD generalization vis-à-vis naively using the expert policy.
• History Length Sensitivity: A history buffer size of $H=50$ maximizes OOD reward, balancing information sufficiency with computational tractability.

  Figure 4: Return in MuJoCo wind perturbations peaks at history length $H=50$ for SPARC, indicating the optimal context resolution.

SPARC supports on-device indefinite continual learning, eliminating brittle model checkpoint selection needed by two-phase methods.

Transfer to Novel Dynamics

SPARC demonstrates superior generalization to previously unseen simulator physics. After a physics update in Gran Turismo, SPARC’s agent shows minimal degradation in lap completeness and times, outperforming all baselines—highlighting robust transfer when contextual priors are partially informative or missing.

Figure 5: SPARC and Oracle achieve the lowest performance drops after simulator physics change; SPARC is more reliable when true context is incomplete.

Implementation Considerations

The full SPARC pipeline is trained with QR-SAC as the base RL algorithm. Training is distributed and asynchronous, scalable to multi-GPU/cloud clusters or local on-device execution. Architectures are wide FC + TCN stacks (for observation and history encoders). Wall-clock times: 6 days for Gran Turismo runs (on 20 PS4/PS5 units), and 14 hours for MuJoCo on NVIDIA A100. Batch sizes, encoder widths, and learning rates are tuned per context/task. SPARC’s modular interfaces support integration with simulation and physical robotics systems alike.

Implications and Future Directions

SPARC delivers strong OOD generalization without requiring test-time privileged context, unlocking practical deployment for real-world robotics, vehicle control, and continuous adaptation settings. The work suggests single-phase adaptation can replace two-phase pipelines where context is unavailable, and demonstrates that historical state-action trajectories can effectively encode latent context for robust policy adaptation.

Future lines of inquiry include:

• Physical real-world robot deployment and learning with ongoing context drift.
• Exploration of stronger or higher-dimensional context spaces.
• Integration with pixel-based and visual context encoding (using masking, augmentation, or attention frameworks).
• Scaling to decentralized, edge devices in automotive or industrial systems.
• Formal theoretical analysis of convergence properties and guarantees under non-stationary context encoding.

Conclusion

SPARC offers a principled, efficient method for robust adaptation in contextual RL tasks, demonstrating consistent superior performance in OOD generalization for both simulated vehicles and continuous control agents. Its unified architecture simplifies implementation and maintenance, enables continual learning, and enhances transfer to unseen contexts and dynamics. SPARC establishes a new baseline for real-world adaptable RL agents in the absence of explicit context—facilitating reliable autonomy in uncertain environments.