LogicEnvGen: Logic-Driven Env Generation

• LogicEnvGen is a logic-driven environment generation framework that creates diverse test cases by synthesizing environments from decision-tree-based agent tasks.
• It leverages LLM-guided analysis and constraint solving to instantiate environments that meet both physical and logical criteria, outperforming traditional perceptual generators.
• Empirical results demonstrate that LogicEnvGen significantly increases logical coverage and fault detection, establishing it as a robust methodological benchmark in embodied AI.

LogicEnvGen is a logic-driven environment generation framework for embodied AI, designed to maximize logical diversity in generated test cases by synthesizing environment instances according to the logical structure of agent tasks. Engineered around decision-tree-derived behavioral trajectories, its architecture and evaluation methodology address the shortfalls of perceptually-focused generators by introducing a rigorous, quantitative framework for environment diversity, usability, and fault detection. Empirical validation demonstrates its superiority across all key metrics relative to established baselines, establishing LogicEnvGen as a cornerstone methodology for environment-centric agent evaluation in embodied and interactive AI research (Wang et al., 20 Jan 2026).

1. Motivation and Paradigm Shift in Environment Generation

Traditional simulated environment generators for embodied agent evaluation have prioritized visual realism and surface-level object diversity, often disregarding the need for logical diversity—distinct decision situations that probe the conditional structure of agent policies. This limits the comprehensiveness of test coverage, resulting in undetected agent failure modes lurking in uninstantiated logical branches. LogicEnvGen addresses this gap by adopting a top-down, task-logic-driven paradigm: it explicitly computes the behavioral decision tree associated with a given task, synthesizes branch-complete logical trajectories, and instantiates corresponding environment situations, thereby ensuring broad logical coverage and maximized fault exposure potential (Wang et al., 20 Jan 2026).

2. Architecture and Logical Trajectory Synthesis

The LogicEnvGen workflow commences with LLM-guided analysis of the agent-executed task to yield a complete decision-tree-structured behavior plan, encoding the sequence of branching conditions and possible actions. From this plan:

• All root-to-leaf paths (γ₁,…,γ_M) are enumerated, each representing a semantically distinct logical trajectory—a specific succession of task-relevant preconditions, external states, and agent decisions.
• A heuristic selection algorithm is applied to prune redundant or equivalent trajectories, minimizing simulation redundancy while preserving logical diversity.
• For each retained logical trajectory, a concrete environment instance is synthesized, instantiating spatial, object, and state variables to realize the trajectory's branch-conditions.
• Constraint solvers ensure these instances satisfy all necessary physical constraints (e.g., room layouts, object support, spatial relations).

This approach guarantees the environment set exercises nearly all logical distinctions encoded in the agent's behavioral specification, subject to feasibility under physical and domain constraints.

3. LogicEnvEval: Quantitative Environment Evaluation Benchmark

To systematically evaluate the logical integrity and efficacy of environment generators, LogicEnvGen introduces LogicEnvEval, a four-metric benchmark:

Metric	Measures	Formal Definition
PhyPR (Physics Pass Rate)	Physical plausibility (per floor-plan, entity, relation)	$\text{PhyPR}_* = \frac{|E_*|}{|E|}$ for $* \in \{\text{FP, EN, RE}\}$
LogCov (Logic Coverage)	Fraction of unique decision-tree branches instantiated	$$\text{LogCov}(E) = \frac{|\mathcal{D}_E|}{|\mathcal{D}_\mathrm{total}|}$$
SceVR (Scenario Validity Rate)	Executability of correct agent policy	$\text{SceVR}(E) = \frac{|E_\mathrm{valid}|}{|E|}$
FauDR (Fault Detection Rate)	Fraction of faulty agent policies exposed	$\text{FauDR}(E) = \frac{\sum_{i=1}^N \delta_i}{N}$

Each metric operationalizes a distinct requirement:

• PhyPR enforces baseline testbed sanity by screening for environments that violate physical or spatial constraints.
• LogCov quantifies logical coverage, penalizing generators that fail to exercise rare or edge-case behavioral branches.
• SceVR eliminates invalid scenarios where the reference policy cannot complete the task due to missing preconditions or misconfigurations.
• FauDR directly measures the generator's power to expose faults in a benchmark suite of known-bad agent implementations.

4. Computation and Empirical Validation of Environment Metrics

Each LogicEnvEval metric is associated with a concrete computational protocol:

• PhyPR: Automated rule-based checkers process floor plans and 3D layouts to determine compliance in the floor-plan, entity, and spatial relation categories.
• LogCov: Root-to-leaf traversal of the decision tree matched with environment metadata identifies covered branches, tallying their union over the environment set.
• SceVR: Simulation of the (gold-standard) correct agent over each environment flags those where execution succeeds without logical or configuration failures.
• FauDR: Faulty agent policies are batch-simulated in all valid environments; whenever at least one environment triggers a failure per policy, this is recorded.

Experimental benchmarking across multiple LLM backbones (DeepSeek-v3, Gemini-2.5 Flash, Qwen2.5-72B) and comparative baselines (CoT-prompting, IFG, Holodeck) yields:

• PhyPR: LogicEnvGen and Holodeck both at 100%; CoT and IFG as low as 30–52%.
• LogCov: LogicEnvGen achieves 94.79–99.06%, surpassing all baselines (CoT: 63–86%, IFG: 91–96%, Holodeck: 37%).
• SceVR: LogicEnvGen at 92.78–99.06%; best baseline at 90.72%, weakest at 64.00%.
• FauDR: LogicEnvGen exposes ≈94.7% of faulty behavior trees; best baseline at ≈90.7%, worst at ≈26.7%.

A strong positive correlation is observed between LogCov and FauDR, validating the hypothesis that logical test coverage is predictive of fault-finding effectiveness (Wang et al., 20 Jan 2026).

5. Collective Role and Significance of LogicEnvEval Metrics

The composite use of PhyPR, LogCov, SceVR, and FauDR constitutes a multidimensional evaluation scaffold for simulated environment generators:

• High PhyPR ensures physical plausibility, filtering out non-credible testbeds.
• Elevated LogCov guarantees that the full logical complexity of agent policies is stress-tested.
• High SceVR assures that valid environment-task pairs are non-trivial and executable by correct agents.
• Major FauDR improvements directly reflect increased likelihood of detecting agent failure modes otherwise hidden in unexercised logical branches.

Optimal environment generators must simultaneously excel across all four axes; lack of logical diversity or physical validity renders the test suite incomplete or misleading for robust embodied agent assessment.

6. Practical Implications and Efficacy

LogicEnvGen's systematic logical trajectory synthesis and constraint-based environment instantiation provide a significant methodological leap over ad hoc or visually-biased environment sampling. Its effectiveness is confirmed through:

• Measured diversity improvements (1.04–2.61× higher LogCov than baselines),
• Fault detection improvements (+4.0% to +68.0% FauDR over baselines),
• Efficient operation (~50 min per full benchmark run).

The LogicEnvEval framework, by quantifying environment generation not only in terms of physical realism but also logical task structure and fault exposure capability, serves as a robust foundational standard for environment generator comparison and embodied AI agent evaluation (Wang et al., 20 Jan 2026).

7. Conclusion and Future Directions

LogicEnvGen establishes a formal, logic-complete and empirically validated methodology for scenario generation in embodied AI. By operationalizing logical diversity through decision-tree-derived test suites and rigorously quantifying generator quality across environment validity, coverage, and fault-detection, it addresses critical gaps in agent testbench construction. Future research may extend logic-driven methods to dynamic or multi-agent tasks, synthesize richer hierarchical environment/task/agent interactions, and further automate trajectory selection for scalable evaluation frameworks. The LogicEnvEval metrics framework is poised to become a field-standard for systematic, comprehensive evaluation of simulated environment generators.