---
title: 'LoongBench: Scalable Synthetic Data for RL'
url: https://www.emergentmind.com/topics/loongbench
type: topic
---

# LoongBench: Scalable Synthetic Data for RL

LoongEnv is a modular, domain-agnostic environment for scalable synthetic data generation and automatic verification across diverse, reasoning-intensive domains. It forms a core component of the Loong Project’s open-source framework for reinforcement learning (RL) with verifiable reward, aimed at advancing long-chain-of-thought (CoT) capabilities in large language models (LLMs) [2509.03059]. LoongEnv supports the autonomous creation of question–answer–code triples, enabling extensive RL alignment for mathematical, scientific, and logical reasoning benchmarks.

## 1. System Architecture and Data Flow

LoongEnv is architected as a multi-agent, modular pipeline, structurally decoupled from the downstream LLM agent and designed to plug into RL loops as an “environment.” The overall agent–environment cycle is structured as follows:

- **Seed Corpus**: A curated dataset (LoongBench; 8,729 human-vetted QA–code triples spanning 12 domains) is ingested as the initial substrate.
- **Synthetic Generation**: Based on seed samples, LoongEnv generates new questions using various prompting strategies (Few-Shot, Self-Instruct, Evol-Instruct).
- **Coder Agent**: Constructs executable code to answer the new question; code is run to yield a synthetic answer.
- **Verifiers**: Judge module ensures code executability and semantic equivalence between the code-produced and LLM-generated answers.
- **Reward Signal**: RL loop rewards the LLM agent for generating a valid CoT answer matching the verifiable answer from the code execution.

The pipeline supports dynamic scaling, pluggable question synthesis techniques, domain-specific verification, and interaction with the RL agent for feedback-based sample selection.

## 2. Prompting and Generation Strategies

LoongEnv implements three primary automated data-generation paradigms:

1. **Few-Shot Prompting**: Human seed examples are used as an in-context demonstration for synthesizing further problem instances with similar structure. This approach yields high correctness and executability (e.g., Physics: 93.9% pass rate, Logic: 92.6% pass rate).

2. **Self-Instruct**: Recursively prompts instruction-tuned models to enhance novelty and diversity beyond surface-level copying. While this increases the semantic range (e.g., lower average embedding similarity to seeds), it sometimes decreases executability (higher judge rejection rates).

3. **Evol-Instruct**: Iteratively mutates existing questions via targeted generalization, specification, or complexity scaling. This approach is optimized for generating high-difficulty, structurally complex problems, commonly resulting in richer reasoning chains but increased execution failure rates (e.g., Logic: 55% not executable, Physics: 14% not executable).

This collective strategy set facilitates both wide semantic coverage and control over difficulty and forms the basis for large-scale synthetic reasoning data creation.

## 3. Automatic Verification and Quality Control

LoongEnv rigorously filters synthetic outputs through multi-stage, automated judging:

- **Executability**: All generated code is run in a sandbox; only samples that execute without error are retained.
- **Semantic Verification**: A judge agent (possibly domain-specialized) assesses if the code and generated answer semantically solve the original question; malformed or unverifiable problems are rejected.
- **Empirical Outcomes**: Few-shot samples exhibit the highest execution and verification rates; self-instruct and evol-instruct methods yield a tradeoff, increasing diversity and difficulty at the expense of reliability.

This process results in a dataset enriched in high-quality, verifiable question–answer–code triples that are suitable for RL.

## 4. Agent–Environment Loop for Reinforcement Learning

LoongEnv is designed as an RL-ready environment, supporting alignment and skill acquisition for reasoning LLMs via the RLVR (RL with verifiable reward) paradigm:

- **Agent Action**: The LLM agent receives the synthetic question and produces a chain-of-thought and final answer.
- **Environment Response**: The generated answer is semantically compared to the code-executed answer from LoongEnv.
- **Reward Assignment**: Only answers verified as correct (by the judge) receive reward signal. Formally,
  $$
  \text{Reward}(q, c, a_{\text{agent}}) =
  \begin{cases}
  1 & \text{if } V(a_{\text{code}}, a_{\text{agent}}) = 1 \\
  0 & \text{otherwise}
  \end{cases}
  $$
  where $V$ denotes the judgment of equivalence.

This architecture allows scalable RL alignment in previously under-resourced or high-supervision-requirement domains.

## 5. Domain Coverage, Diversity, and Difficulty

LoongEnv leverages the LoongBench seed set and supports extension across 12 diverse domains: advanced mathematics, logic, chemistry, programming, physics, finance, medicine, security, board games, and others.

### Empirical Findings

- **Correctness**: Few-shot generation has the highest pass rate; evol-instruct leads to greater failure but enhances coverage of “hard” and edge-case reasoning.
- **Diversity**: Measured by pairwise embedding similarity and cluster analysis (e.g., t-SNE projection), Self-Instruct and Evol-Instruct variants yield greater semantic drift from seeds; Evol-Instruct maintains high structural similarity but increases solution complexity.
- **Difficulty**: Expectation accuracy of SOTA models (e.g., GPT-4.1-mini, DeepSeek-r1) drops substantially on Evol-Instruct data, quantifying its added challenge (e.g., GPT4.1-mini: 62.0% vs. 92.0% accuracy on few-shot logic).

| Model         | Few-shot | Self-Instruct | Evol-Instruct | Seed Dataset |
|---------------|----------|---------------|---------------|--------------|
| GPT4.1-mini   | 92.0%    | 83.0%         | 62.0%         |   71.8%      |
| DeepSeek-r1   | 93.2%    | 87.4%         | 70.3%         |   77.4%      |

This gradient in difficulty and diversity is systematically controlled through LoongEnv’s design, supporting targeted curriculum construction for RL.

## 6. Innovations and Distinguishing Characteristics

Key innovations of LoongEnv, relative to prior synthetic data generators (especially those limited to mathematics or programming), include:

- **Executable Reasoning**: Always produces executable code along with text answers, enabling rigorous, automated verification.
- **Multi-domain and Multi-agent Support**: Expands scalable synthetic data creation beyond traditional domains, with plug-in generation and verification modules.
- **Evolutionary and Recursive Prompting**: Introduces structures for controlled, progressive escalation in question difficulty and novelty.
- **RLVR Integration**: Native support for reinforcement learning via verifiable rewards, reducing annotation bottleneck and improving reasoning alignment in complex problem spaces.

## 7. Implications and Future Directions

LoongEnv establishes a systematic, automated workflow for high-fidelity reasoning data generation across scientific and logical domains. Its design permits:

- Automated scaling of RL alignment for LLMs in fields where verifiable data are scarce and manual annotation costs are prohibitive.
- Fine-grained curricular control over training samples, including reliability–difficulty tradeoffs.
- Expansion to novel, under-explored domains via new seed corpora and verifier plugins.

A plausible implication is that LoongEnv workflows, when embedded in RL pipelines, may shift the development paradigm for LLM reasoning—moving from reliance on narrow human-labeled datasets towards scalable, domain-extensible, and verification-driven alignment approaches. However, challenges persist regarding the reliability of advanced synthesis strategies (notably Evol-Instruct) and the need for further improvement in judge and code execution robustness.

---

LoongEnv’s implementation, seed corpora, benchmarking scripts, and documentation are provided at [https://github.com/camel-ai/loong](https://github.com/camel-ai/loong) [2509.03059], enabling empirical validation and extension by the research community.

Source: https://www.emergentmind.com/topics/loongbench