LLMs for Engineering: Teaching Models to Design High Powered Rockets (2504.19394v2)

Abstract: LLMs have transformed software engineering, but their application to physical engineering domains remains underexplored. This paper evaluates LLMs' capabilities in high-powered rocketry design through RocketBench, a benchmark connecting LLMs to high-fidelity rocket simulations. We test models on two increasingly complex design tasks: target altitude optimization and precision landing challenges. Our findings reveal that while state-of-the-art LLMs demonstrate strong baseline engineering knowledge, they struggle to iterate on their designs when given simulation results and ultimately plateau below human performance levels. However, when enhanced with reinforcement learning (RL), we show that a 7B parameter model outperforms both SoTA foundation models and human experts. This research demonstrates that RL-trained LLMs can serve as effective tools for complex engineering optimization, potentially transforming engineering domains beyond software development.

• The paper introduces RocketBench, a simulation benchmark utilizing RocketPy and JSON interfaces to enable LLMs to configure complex rocket designs.
• The evaluation shows that standard LLMs excel in baseline design knowledge but falter in iterative optimization through prompting.
• Reinforcement learning with GRPO on a 7B model (Qwen 2.5) significantly outperformed human experts and larger models in both target altitude and precision landing challenges.

This paper, "LLMs for Engineering: Teaching Models to Design High Powered Rockets" (2504.19394), explores the application of LLMs to physical engineering design tasks, a domain less explored compared to software engineering. The research introduces RocketBench, a benchmark designed to evaluate LLMs' capabilities in high-powered rocketry design by connecting them to RocketPy, a high-fidelity trajectory simulation tool.

Key Contributions:

1. RocketBench: A practical benchmark environment built on RocketPy, enhanced with design rule checks (DRCs) and timeout mechanisms. It provides a structured JSON interface for LLMs to configure rocket parameters (motor, body, nose cone, fins, tail, parachutes, launch, payload).
2. Analysis of Current LLMs: Evaluation of state-of-the-art foundation models (GPT-4o, Claude 3.7, Deepseek v3, o1) on rocket design tasks using an iterative prompting protocol, revealing their strengths in baseline knowledge but limitations in iterative design optimization based on simulation feedback.
3. Reinforcement Learning (RL) Enhancement: Demonstration that applying RL (specifically Group Relative Policy Optimization - GRPO) to a 7B parameter LLM (Qwen 2.5) enables it to significantly outperform both larger foundation models and a human expert in complex rocket design challenges.

Methodology and Implementation:

The core of the research is the RocketBench environment, which simulates rocket flights using a 6-degrees-of-freedom model accounting for various physics (variable mass, aerodynamics, parachutes). Key practical additions to RocketPy for this work include:

• Design Rule Checks (DRCs): Implemented to prevent physically impossible designs (e.g., body diameter less than motor diameter) that would cause simulation failures.
• Timeout Mechanisms: Added to handle excessively long computations from unrealistic parameters.
• JSON Interface: A standardized format allowing LLMs to easily specify all rocket parameters, abstracting the complexity of the RocketPy library. The configurable parameters cover a wide range of design choices for different components, as detailed in Table 1 of the paper.
• Pre-curated Components: Motors and materials are limited to realistic, commercially available options to ensure manufacturability and physical realism. Detailed specifications (dimensions, mass, thrust profiles, cost) are provided to the models.
• Material Stress Simulation: Evaluates structural integrity throughout the flight profile, adding a critical engineering constraint.
• Economic Model: Calculates total cost based on motor selection and material volume, forcing models to consider performance-cost trade-offs, similar to real-world engineering projects.

Competition Tasks:

Two tasks, inspired by real rocket competitions, were used:

1. Target Altitude Challenge: Design a rocket to reach a target altitude (3048m) while ensuring structural integrity, minimizing horizontal drift, controlling cost, and ensuring safe landing velocity. The reward function is a weighted sum of metrics including altitude accuracy (50%), structural integrity (10%), horizontal drift (10%), cost efficiency (15%), and landing safety (15%).

   $$R_{\text{total}} = 0.5 \cdot R_{\text{altitude}} + 0.1 \cdot R_{\text{structural}} + 0.1 \cdot R_{\text{drift}} + 0.15 \cdot R_{\text{cost}} + 0.15 \cdot R_{\text{landing}}$$

2. Precision Landing Challenge: Design a rocket to land as close as possible to a target location (5.65km from launch, 4000m horizontal, 4000m vertical) while maintaining structural integrity, cost efficiency, and safety. This task is more complex as it requires reasoning about the entire trajectory, including parachute deployment timing and wind drift. The reward function heavily weights landing accuracy (75%), with structural integrity (15%), cost (5%), and safety (5%) as secondary objectives.

   $$R_{\text{total}} = 0.75 \cdot R_{\text{landing}} + 0.05 \cdot R_{\text{structural}} + 0.05 \cdot R_{\text{cost}} + 0.05 \cdot R_{\text{safety}}$$

LLM Evaluation Approaches:

• Iterative Prompting Protocol: Foundation models were given an initial prompt describing the task, environment, components, and reward function code. Subsequent prompts included the model's previous design and detailed simulation results (performance metrics, structural status, cost, etc.) to allow for iterative refinement.
• Reinforcement Learning (RL): A Qwen 2.5 7B model was trained using GRPO. The model received the environment specifications and reward function in the prompt and generated design parameters as its action. The simulation outcome provided the reward signal used for training.

Results and Practical Implications:

• Foundation Models (Iterative Prompting): While models like Claude 3.7 and o1 showed strong baseline engineering knowledge, often outperforming a human expert's initial designs, they struggled to iteratively improve and ultimately plateaued below the human expert's peak performance on both tasks, especially in the more complex Precision Landing Challenge. This suggests standard LLMs have good declarative engineering knowledge but lack effective iterative optimization strategies when faced with complex feedback.
• RL-trained LLM: The 7B Qwen 2.5 model trained with RL showed dramatic and consistent improvement over training steps.
  • On the Target Altitude Challenge, it reached a peak score of 79.98, surpassing the best human score (76.57).
  • On the Precision Landing Challenge, it achieved a peak score of 95.6, significantly exceeding the human expert (91.6) and the best foundation model (71.78). The RL model achieved landings within 12 meters of the target.
• Sample Efficiency: The RL training was highly sample-efficient, requiring only approximately 3,000 simulation samples to reach and surpass human-level performance. This is a crucial practical advantage for engineering tasks where simulations can be computationally expensive. LLMs' pre-trained engineering knowledge helps avoid the extensive random exploration typically needed in traditional RL.

Limitations and Future Work:

The authors acknowledge limitations: the simulation is a proxy and doesn't capture all real-world factors; model performance shows high variance; and the human baseline is based on a single expert with limited attempts.

Despite limitations, the research demonstrates that RL-enhanced LLMs can serve as powerful engineering tools. Current bottlenecks for wider adoption are creating robust simulation environments that interface well with LLMs and developing appropriate, potentially abstract, reward models.

Engineering Impact and Safety Concerns:

The paper suggests a future where RL-trained LLMs act as "next-generation CAD tools," automating design space exploration and optimization, freeing human engineers for higher-level innovation. This could accelerate progress across various engineering fields.

However, the research also raises safety concerns, as the ability to rapidly optimize designs using readily available LLMs and simulation tools could lower the barrier to developing potentially dangerous technologies. This highlights the need for new governance and regulatory frameworks beyond just limiting training compute.

In summary, this research presents a practical framework (RocketBench) and methodology (RL on LLMs) for applying AI to complex physical engineering design. It successfully demonstrates that while current foundation models struggle with iterative optimization, RL can leverage LLMs' baseline knowledge to achieve and surpass human expert performance with remarkable sample efficiency, opening up significant possibilities for transforming engineering practice.