LoongEnv: Automated Synthetic Data System

• LoongEnv is a modular synthetic data generation environment that automates creating and verifying question-answer-code triples for formal reasoning.
• It employs few-shot, self-instruct, and evol-instruct strategies to generate diverse, high-quality, and executable datasets.
• LoongEnv integrates reinforcement learning with code execution and answer verification to bolster LLM alignment and systematic benchmarking.

LoongEnv is a modular synthetic data generation environment engineered for scalable, automated creation and verification of question-answer-code triples in reasoning-intensive domains. Developed as a central component of the Loong Project (Huang et al., 3 Sep 2025), LoongEnv abstracts and automates dataset expansion beyond human-crowdsourced seeds (LoongBench), enabling reinforcement learning (RL) alignment for LLMs in mathematics, science, logic, programming, and other formal reasoning domains. LoongEnv supports multi-agent prompting, programmatic code generation, automatic execution-based answer verification, and rigorous filtering for data fidelity.

1. Modular Architecture and Agent-Environment Loop

LoongEnv operates as an environment in a formal agent-environment RL framework. Its architecture comprises:

• Seed Dataset Source: Human-vetted question-answer-code triples, e.g., LoongBench, across domains such as mathematics, chemistry, logic, and physics.
• Synthetic Generation Modules: Automated agents synthesize new questions by applying prompt engineering or recursive evolution strategies to seed examples.
• Code Generation and Execution: Each synthetic question is paired with generated code, which is executed in a sandbox for answer extraction.
• Verification Subsystem: Dedicated judge agents evaluate question meaningfulness, code correctness, and semantic alignment between code-executed answers and agent inference outputs.
• RL Integration: The synthetic pipeline creates tasks where LLM agents generate chain-of-thought (CoT) solutions and are rewarded only if their natural language answers align with execution-based ground truth.

This modular structure decouples generation, verification, and judging, facilitating domain transfer, reproducibility, and scalability.

2. Prompting Strategies for Synthetic Data Expansion

LoongEnv supports three principal strategies for question synthesis:

1. Few-Shot Prompting: Adapts Brown et al. (2020), using in-context human-generated QA pairs as templates to synthesize new questions closely resembling original styles. This approach yields high executability and correctness rates.
2. Self-Instruct: Implements recursive chain-of-thought instruction prompting per Wang et al. (2023), fostering diversity and novelty in generated questions, albeit with increased risk of malformed queries or answers that are harder to verify.
3. Evol-Instruct: Applies evolutionary operators (generalization, specification, complexity scaling) to seed examples, producing semantically similar but structurally more challenging problem instances [Xu et al., 2023]. Evol-Instruct increases reasoning difficulty and exposes LLM boundaries.

The generation workflow applies quality control filters: code execution success, judge agent validation, and semantic correctness, retaining only verified outputs.

3. Automated Question-Answer-Code Triple Generation

LoongEnv's pipeline systematically expands data from seed examples:

• Question Synthesis Agent produces domain-meaningful natural language problems according to selected prompting strategy.
• Code Generation Agent constructs explicit, often Python-based, code routines to solve the new question.
• Code Execution yields numerical or symbolic ground-truth answers via deterministic computation.
• Verifier Module checks both question legibility and the agreement of CoT answer with execution outcome, via plugin mechanisms or semantic comparators (e.g., MathVerifier, Llama-as-Judge).

The resulting synthetic data is rigorous, diversified, and readily extensible to new domains, supporting RL and supervised learning applications.

4. RL Integration and Verifiable Reward Mechanism

LoongEnv is explicitly designed to support RLVR (Reinforcement Learning with Verifiable Reward) alignment protocols for LLMs:

• LLM agent outputs a chain-of-thought solution and final answer.
• Environment executes the code and supplies an answer.
• Judge agent assesses semantic equivalence between the agent's answer and execution-generated truth.
• Reward is assigned using:

$$\text{Reward}(q, c, a_{\text{agent}}) = \begin{cases} 1 & \text{if } V(a_{\text{code}}, a_{\text{agent}}) = 1 \ 0 & \text{otherwise} \end{cases}$$

where $V$ is the verification function.

This closed-loop enables automated RL training over unbounded question domains, transcending the limits of human annotation.

5. Analysis of Data Diversity, Correctness, and Difficulty

Empirical benchmarks in the foundational paper report:

• Correctness: Few-shot generation achieves highest executability and pass rates, while Self-Instruct and Evol-Instruct trade increased diversity for higher judge rejection and execution failure rates (e.g., Logic domain: 92.6% pass for few-shot vs. 55% not executable for Evol-Instruct).
• Diversity: Semantic embedding analyses (pairwise cosine similarity) and t-SNE clustering verify that Self-Instruct and Evol-Instruct strategies yield greater sample dispersion, complexity, and semantic coverage than direct few-shot expansion.
• Difficulty: Model accuracy decreases with increased problem complexity (e.g., GPT-4.1-mini: 92.0% on few-shot, 62.0% on Evol-Instruct), indicating successful generation of higher-difficulty examples.

Model	Few-shot	Self-Instruct	Evol-Instruct	Seed Dataset
GPT4.1-mini	92.0%	83.0%	62.0%	71.8%
DeepSeek-r1	93.2%	87.4%	70.3%	77.4%

This stratification enables targeted curriculum design for RL training.

6. Domain Coverage, Innovations, and Research Implications

LoongEnv advances synthetic data environments via:

• Coverage of 12 formal reasoning domains with extensible plug-and-play modules.
• Automated code-based executable reasoning, not restricted to direct answer generation.
• Multiple integrated prompting paradigms, facilitating both curriculum and robustness.
• RLVR-ready agent-environment architecture, suitable for annotation-free, scalable LLM alignment.
• Empirical demonstration of increased problem diversity, complexity, and diagnostic power for LLM benchmarking.
• Enables systematic investigation of LLM failure modes, generalization, and progression under reward-driven learning.

A plausible implication is that LoongEnv's evolutionary strategies are particularly valuable for probing the outer bounds of model reasoning, while strict verification maintains data integrity for alignment objectives.

7. References and Availability

LoongEnv is documented in the Loong Project (Huang et al., 3 Sep 2025), with code and resources available at https://github.com/camel-ai/loong. For full empirical analyses, strategies, and protocol designs, see the cited arXiv article and associated supplementary material.