Optimal exploration–exploitation trade-off in the meta-policy for LLM-generated control code

Determine the optimal balance between exploration of new code structures and exploitation of promising solutions in the meta-policy that guides large language model-based generation and iterative refinement of base-policy code within the proposed hierarchical decision-making framework for smart energy systems.

Background

The paper proposes a hierarchical decision-making framework for smart energy systems in which a meta-policy orchestrates the generation and refinement of executable base-policy code by LLMs. The approach is demonstrated on a simplified microgrid with battery storage control, yielding up to 15% cost savings relative to a baseline without battery operation.

Despite encouraging results, the authors observe substantial variability across runs due to LLM stochasticity and meta-policy choices for task generation. This motivates an explicit open research question on how to tune the exploration of novel code structures versus the exploitation of promising existing solutions to improve performance consistency and robustness.