- The paper introduces Intern-S2-Preview-397B, a scientific agentic foundation model trained through multimodal pre-training, multi-task and agentic reinforcement learning, and on-policy distillation to support verifiable long-horizon workflows.
- Intern-S2-Preview-397B delivers strong scientific results, including 56.92 on Biology-Instructions, 67.88 on MP20, 4.36% on ProteinBinder-9, and improved time-series understanding and forecasting across all seven evaluated forecasting tasks.
- The system adds practical scaling and specialization methods—including compressed time-series encoding, continuous forecasting, Memory Decoder adapters, speculative decoding, GEPO, and harness-agnostic agent training—while remaining weaker than leading closed models on difficult long-horizon terminal tasks.
Intern-S2-Preview is a series of scientific agentic foundation models from Shanghai AI Laboratory, with Intern-S2-Preview-397B as the primary evaluated model (2608.13505). The system targets a gap the authors identify in existing model families: general-purpose LLMs lack specialization for heterogeneous scientific modalities and verifiable tool interaction, while scientific multimodal models are typically evaluated as static question-answering systems rather than long-horizon agents. The paper's contribution is threefold: a training pipeline spanning scientific multimodal pre-training through multi-task and agentic reinforcement learning to on-policy distillation; architectural extensions for time-series understanding and forecasting plus a modular Memory Decoder for domain specialization; and evaluations across scientific, multimodal, agentic, and general-purpose benchmarks showing competitive or leading results.
Architecture: time-series modules and Memory Decoder
The time-series encoder upgrades Intern-S1-Pro's design with compressive patching (normalization, CNN local features, Q-Former temporal compression) and a channel-wise Transformer encoder that models inter-channel dependencies before the global temporal encoder. Maximum supported input length increases from roughly 240,000 to 300,000 time steps, with approximately 5–6× faster inference at maximum length and GPU memory reduced to around 20% of the previous version. The upgrade also enables modelling of high-frequency short signals, extending disciplinary coverage to radar signal analysis (~MHz).
A new forecasting module adds a dedicated numerical generation branch conditioned on LLM semantic context and time-series encoder representations via cross-attention, with a horizon predictor that reaches 99% accuracy in inferring prediction length from instructions. Generating forecasts through a continuous-valued branch rather than discrete text tokens preserves numerical fidelity — a design choice validated by the forecasting results discussed below.
Memory Decoder is presented as a separate extension rather than a component of the base model. It attaches a domain-specific parametric memory to the frozen 397B backbone, with dynamic next-token distribution fusion controlled by a lightweight token-level router. The memory is trained by distilling retrieval-based evidence from a token-level datastore (in the spirit of kNN-LM) combined with gold-answer supervision; the router is trained with a signed linear regularizer that pushes fusion weight up on domain data and down on general data. The motivation is that backbone fine-tuning for each new domain risks perturbing general reasoning and agentic capabilities, whereas memory attachment avoids this trade-off. The empirical support for this claim is a single-domain study (biology), which is a notable scope limitation.
Pre-training
Three data pipelines strengthen the scientific foundation. Visual Pre-training (VP) learns from rendered scientific page images via contrastive next-latent prediction over raster-scanned foreground visual features, interleaved with text under a joint objective. Because supervision derives from visual features directly, VP requires neither OCR, layout parsing, nor paired annotations, and preserves figures, tables, equations, and layout that text extraction loses.
Interleaved image-text data is constructed from PDFs via MinerU2.5-Pro parsing, cropping of images, equations, and tables as visual units, and layout-aware sequence assembly. A Toolformer-inspired visual-gain filter computes the perplexity difference between text-only and interleaved conditions, retaining only pages where visual content meaningfully reduces perplexity; chunks are capped at 256k tokens with 512-token overlap, focused on life sciences, chemistry, and materials science.
Finally, a large-scale image retrieval pipeline — an 8B embedding model producing 1024-dimensional vectors stored in sharded Milvus collections at hundreds-of-millions scale — supports text-to-image and image-to-image recall with reranking, raising the ratio of high-quality multimodal samples.
Post-training: scalable and stable RL
The post-training pipeline proceeds from SFT through scalable multi-task RLVR, parallel black- and white-box agentic RL, and on-policy distillation. Several systems contributions address RL scaling:
Partial rollout with off-policy correction. A co-located system built on XTuner and LMDeploy pauses in-flight rollouts when a training batch is complete, retains prefixes, trains on the same GPUs, then resumes generation after weight synchronization. Tokens generated across policy updates are corrected with clipped importance weights used as detached multipliers in a REINFORCE objective — a formulation the authors argue retains nonzero gradients from all tokens, unlike PPO-style surrogate clipping. Trajectory staleness is bounded at three policy updates. For MoE policies, Rollout Routing Replay (R3) aligns expert paths, and a bidirectional binary-KL mask filters residual numerical outliers after FP8/BF16/FP32 operator-level precision alignment.
Adaptive length regularization. Advantages of positive responses are reweighted by a length-dependent decay, activated only when a query's positive-response ratio exceeds a threshold τ, and never applied to negative responses. The design explicitly avoids penalizing failed trajectories' lengths to preserve exploration. Reported results on Intern-S2-Preview-35B show comparable reward curves with substantially shorter outputs.
Online speculative decoding. The draft model is retrained each RL iteration on the current policy's token distributions using a hybrid LK loss combining forward KL and total-variation distance, with an acceptance-rate-adaptive mixing coefficient. This yields approximately 2× rollout speedup and 1.7× end-to-end RL training speedup, losslessly, since verification preserves the policy's sampling distribution.
GEPO for multi-task stability. Group-level Entropy-Controlled Policy Optimization uses group-level entropy as a diagnostic to attenuate positive advantages in low-entropy groups and negative advantages in high-entropy groups, with asymmetric coefficients because low-entropy groups are more susceptible to length collapse. This preserves task-dependent exploration regimes rather than forcing a shared entropy target.
The unified objective combines leave-one-out REINFORCE advantages, GEPO shaping, adaptive length regularization, clipped importance weights, R3 routing replay, and BKL masking, optimized with Muon at learning rate 10−6, rollout batches of 8,192 responses, and a 65,536-token maximum generation length.
Agentic RL: harness × task abstraction
The agentic framework decouples agent runtimes (harnesses) from executable task distributions. A unified rollout runtime composes harness–task pairs under a shared session contract with a shared sandbox provider; Agent Gateway adapters integrate white-box loops and black-box harnesses including OpenClaw, Claude Code, OpenCode, OpenHands, and Mini-SWE. A client-transparent serving layer accepts OpenAI Chat Completions, Responses, and Anthropic Messages protocols while capturing token IDs, log probabilities, and MoE router experts — a token-in-token-out (TITO) interface. Trace-aware experience assembly organizes sessions as incremental PrefixTrees that preserve exact token-level lineage of model calls, joining semantic trajectories in the Replay Buffer with model-execution evidence.
Task construction draws on curated coding and terminal collections (SWE-smith, SWE-Gym, R2E-Gym, SWE-rebench-V2, Scale-SWE, Nemotron terminal tasks, ClawGym — over 215,000 tasks in total) and a self-evolving task-synthesis system that composes community skills into a skill-state graph, generates task bundles through stage-wise validated synthesis, and updates sampling and synthesis from observed agent failures. Training applies session-aware outcome credit (one group-relative advantage per session across all policy-generated segments), process-aware advantage control (adv penalties applied only to positive advantages for malformed tool calls, protocol violations, and abnormal terminations), and verifier hardening: gold patches and held-out tests are isolated from agent workspaces, git metadata is sanitized, and canonical tests are restored after agent execution, with conservative all-correct grading semantics.
The paper reports reward improvements across SWE, general-purpose, and terminal tasks under multiple harnesses, but is careful to note that shared upward trends indicate the common training stack works across harness–task compositions, not that absolute reward scales are comparable.
On-policy distillation
Rather than many fine-grained domain experts, the paper trains two broad experts — a reasoning expert via mixed reasoning RL and an agentic expert via agentic RL — and consolidates them into one student. The authors state their preliminary evaluation found fine-grained multi-teacher specialization costly with limited additional benefit, though this claim is supported only by preliminary results. A warmup stage (teacher-generated trajectory SFT, following Nemotron 3 Ultra) reduces student–teacher distribution mismatch before OPD begins. The OPD objective minimizes reverse KL on student-induced states; because teachers share the SFT origin with the student, transmitting only the sampled-token teacher log-probability suffices, reducing the teacher payload from O(HV) or O(Hk) to O(H) — a substantial systems saving at 256k-token context lengths. The OPD loss reuses the clipped importance-weighted REINFORCE form of reasoning RL, differing only in the advantage construction (token-level teacher–student log-probability difference against a frozen proximal student).
Evaluation results
Intern-S2-Preview-397B achieves leading results on several scientific benchmarks, with the largest margins on sequence- and structure-oriented tasks:
| Benchmark |
Intern-S2-Preview-397B |
Best competing model |
| Biology-Instructions |
56.92 |
13.87 (GPT-5.5) |
| Mol-Instructions |
52.37 |
40.49 (GPT-5.5) |
| MP20 |
67.88 |
16.75 (Gemini-3.1-Pro) |
| ProteinBinder-9 |
4.36% |
2.40% (Claude-Opus-4.8) |
| SciReasoner |
63.97 |
61.15 (GPT-5.5) |
| MolecularIQ |
61.49 |
76.41 (GPT-5.5) |
| TOMG-Bench |
65.66 |
69.89 (GPT-5.5) |
The margins on MP20 and Biology-Instructions are striking — roughly 4× the best general-purpose competitor — and indicate that specialized pre-training and time-series/multimodal modules, rather than scale alone, drive these gaps. Conversely, GPT-5.5 retains the lead on MolecularIQ and TOMG-Bench, showing the advantage is not uniform across chemistry.
On general benchmarks, the model is best among open models on MMLU-Pro (89.75), SimpleQA-Verified (69.90), MMMU-Pro (80.46), and ChartQAPro (69.65), but trails closed models on HMMT-2026, Terminal-Bench 2.1 (67.42 vs. 84.60 for Claude-Opus-4.8), and WildClawBench (44.68 vs. 64.72). On science-oriented agentic tasks (SciCode, SGI-Bench, ResearchClawBench) it generally surpasses DeepSeek-V4-Pro and Qwen3.5-397B but ranks behind GLM-5.2.
The time-series results are among the strongest claims in the paper. On SciTS understanding, Intern-S2-Preview-397B surpasses the trillion-parameter Intern-S1-Pro on seven of nine shared tasks despite less than half the parameters, with the PHU01 F1 rising from 36.8 to 66.9. On SciTS forecasting, it outperforms specialized time-series foundation models (Moirai, TimeMoE, Chronos-bolt, UniTS, TimeOmni) across all seven tasks — e.g., MAPE of 32.8 on MEG03 versus 37.5 for the best baseline — while text and VL LLMs frequently fail on long horizons due to output-capacity and instruction-following limits. A competitive zero-shot MASE of 0.785 on GIFT-Eval supports generality beyond scientific signals.
The Memory Decoder evaluation instantiates a 4B biology memory (Intern-MemDec-4B), improving Biology-Instructions average from 56.92 to 60.32 while remaining close to the frozen backbone on MMLU-Pro, Mol-Instructions, MMMU-Pro, MicroVQA, IMO-Answer-Bench, and SFE. This supports the modular-specialization claim, though the evidence covers one domain and one memory size.
Limitations and open questions
The paper is explicit that Intern-S2-Preview remains a preview system. Several limitations bear directly on the results above. The Memory Decoder study covers only biology; whether the approach generalizes across scientific subfields, and how multiple attached memories interact, is unresolved. The claim that two broad expert teachers suffice for distillation rests on preliminary evaluation rather than systematic ablation. Agentic reward trajectories show harness-dependent transients, and the authors note absolute reward scales are not comparable across harnesses, so cross-harness capability claims rest on qualitative trends. On the hardest general agentic evaluations (Terminal-Bench 2.1, WildClawBench), a gap of 15–20 points to the strongest closed model persists, indicating long-horizon reliability remains the principal weakness. The conclusion identifies reliability over longer scientific workflows, expansion of domain memories and task environments, verifier strengthening, and deeper integration with specialized scientific tools as the specific open directions.
Conclusion
Intern-S2-Preview-397B demonstrates that a single foundation model can combine scientific multimodal understanding, domain-specific reasoning, scientific generation, and tool-grounded agentic interaction, with the strongest results in sequence-level biology, materials structure generation, protein binder design, and time-series modelling and forecasting. The training infrastructure — partial rollout with off-policy correction, online speculative decoding, GEPO, and the harness-agnostic agentic RL stack with trace-aware experience assembly — is itself a substantive contribution for scaling RL over heterogeneous long-horizon tasks. The evaluation supports the paper's central position that scientific capability should be measured by executable, verifiable, iterative workflows rather than isolated benchmark answers, while the remaining gaps on long-horizon terminal and research-agent benchmarks delineate precisely where current methods fall short.