- The paper shows that native function calling enables GPT-4o-mini and Gemini 2.5 Flash to achieve 100% success and near-MILP-optimal scheduling across 12 tariff days, while text-only parsing causes severe reliability losses.
- The paper finds that model performance diverges under conflicts: Claude Sonnet 4.6 consistently respects power caps and infeasibility, whereas GPT-4o-mini violates critical constraints, demonstrating the need for deterministic feasibility validation.
- The paper reports that weather-aware scheduling saves ยฃ0.14 per overcast day but can cost ยฃ0.15 more on sunny days with negative prices, while seven-day deployments capture 96.7โ98.0% of oracle savings.โโโโ
Motivation and positioning
Home Energy Management Systems (HEMS) can shift flexible residential loadsโEV charging, wet appliances, heatingโagainst dynamic price and generation signals, but adoption is limited by the burden of translating household preferences into technical scheduling parameters. This paper evaluates whether LLM agents can serve as a practical natural-language interface for multi-appliance scheduling, benchmarked against a mixed-integer linear programming (MILP) ground truth. The work positions itself against the agentic HEMS of El Makroum et al., which demonstrated feasible LLM orchestration with open-source models on Austrian day-ahead prices, and identifies four gaps: model coverage was restricted to open-source checkpoints (two of which failed multi-appliance coordination), optimization used price-only signals without weather or photovoltaic (PV) awareness, evaluation covered a single tariff day, and constraint-conflict behavior was never observed.
The paper addresses all four gaps with a single tool-calling ReAct agent grounded in live half-hourly Octopus Agile retail prices from Great Britainโincluding negative-price eventsโOpen-Meteo weather forecasts, an irradiance-to-generation PV model, household usage data, and retrieval-augmented generation (RAG). Three commercial models at comparable price/latency tiers are evaluated: GPT-4o-mini, Gemini 2.5 Flash, and Claude Sonnet 4.6. Notably, the architecture is deliberately flat rather than hierarchical: a single agent handles all coordination, testing whether native function calling removes the need for manual task decomposition used in earlier deployments.
System design
The agent operates over a six-tool action space: Agile price retrieval (48 half-hourly slots), weather forecast retrieval, PV generation prediction via a standard temperature-derated irradiance model (4.0 kWp nameplate, performance ratio 0.80), cheapest-window computation, an appliance-scheduling commit tool (the only invocation treated as a scheduling decision by the scorer), and an infeasibility-reporting tool. Two methodological details deserve emphasis. First, temporal grounding is pinned explicitly per run because models resolve relative dates like "tomorrow" inconsistentlyโa failure the authors observed directly during development. Second, inputs pass pre-LLM validation including prompt-injection screening, though adversarial red-teaming is explicitly out of scope.
The ground-truth formulation minimizes net cost with import at price Ctโ and export at a flat Smart Export Guarantee rate F=5 p/kWh, subject to single-start, horizon, calendar-deadline, and household power-cap constraints. Two features make this harder than prior price-only formulations: the shared PV resource couples appliance decisions, and the power cap prevents independent per-appliance optimization. Under negative Agile prices (Ctโ<F), simultaneous import/export becomes fictitiously profitable, so complementarity is enforced with tightly bounded big-M direction binaries only in those slots. All instances solve in under one second with PuLP/CBC.
Experiment 1: function calling achieves MILP parity; text parsing does not
Across 12 stratified Agile tariff days spanning low-, medium-, and high-volatility regimes (36 runs per cell, temperature 0), all three models achieve 100% success and near-MILP optimality with native function calling. GPT-4o-mini and Gemini 2.5 Flash match the optimum in every run (ฮณ=0.00%); Claude Sonnet 4.6 reaches 86% optimality with a mean gap of 0.12%. GPT-4o-mini dominates on efficiency at $0.0013 per successful schedule and 7.5 s latency.
Removing native function calling degrades every model sharply: GPT-4o-mini's optimality falls to 31% (ฮณ=2.17%), and Claude's success collapses to 31%. The most consequential finding here is a failure mode the authors term conversational short-circuiting: in 89% of Claude's failed single-appliance runs, the model issued zero tool calls and returned fluent proseโincluding formatted tables stating start times and costsโfor schedules it never committed. Because such output appears successful to a user, this constitutes a deployment-safety failure distinct from ordinary sub-optimality. The degradation is uniform across volatility terciles, indicating a structural interface effect rather than a data artifact.
Experiment 2: constraint conflicts separate the models
This adversarial suiteโsix scenario families covering deadline conflicts with compliance premiums of 5โ50%, power-cap overlap under a 9 kW cap, irregular calendars, provable infeasibility, instruction-versus-calendar conflict, and injected tool failuresโis the paper's most discriminating evaluation, and it reverses the benign-case ranking:
| Model |
Deadline |
Power cap |
Infeasible |
Instr. conflict |
| GPT-4o-mini |
1.00 |
0.00 |
0.50 / 1.00* |
0.00 |
| Gemini 2.5 Flash |
0.92 / 1.00 |
1.00 |
1.00 / 0.83* |
0.33 / 1.00* |
| Claude Sonnet 4.6 |
1.00 |
1.00 |
1.00 |
1.00 |
*Baseline / guided prompt.
GPT-4o-mini violates the 9 kW power cap in every run under both prompts, committing schedules that would exceed household supply, and follows casual overnight-charging instructions over binding departure deadlines in all runsโproducing schedules cheaper than feasible ones (mean gap โ152.9%). Prompt guidance repairs its infeasibility fabrication (S4) but not these safety-critical failures. Claude Sonnet 4.6 is the only model that serializes correctly under the cap, reports infeasibility universally, and prioritizes hard deadlines without any guidance. A shared failure persists across all three models: given a flight and a dentist appointment, no model infers that the flight defines the EV readiness deadline, suggesting free-text calendar interpretation remains unreliable for safety-critical inference. The authors draw the appropriate conclusion: prompt engineering is not a substitute for a deterministic post-hoc feasibility validator before actuation.
Experiment 3: regime-dependent value of weather awareness
Co-optimizing against forecast PV yields a nuanced result that contradicts naive self-consumption maximization. On overcast days, weather-aware scheduling saves ยฃ0.14/day and raises self-consumption by 3 percentage points. On mixed days it is equivalent to price-only (ยฃ0.003 difference). On sunny days it is ยฃ0.15/day more expensive despite higher self-consumption, because high-generation days coincide with negative midday Agile prices that make grid import preferable to self-consumption. Forecast-noise sweeps show graceful degradationโrealized cost spans ยฃ4.52โยฃ4.73/day across ยฑ10% to ยฑ50% perturbations, with no cliffโbut asymmetrically: over-forecasting solar costs more than under-forecasting at small error magnitudes, since optimistic forecasts commit load to generation that never materializes while pessimistic forecasts fall back toward already-near-optimal price-only behavior. Deployments with uncertain forecast quality should therefore prefer conservative solar estimates.
Experiment 4: sustained deployment captures 96.7โ98.0% of oracle savings
Over a seven-day rolling deployment simulation (three passes per model, 63 model-day runs), all agents capture 96.7โ98.0% of the extended-MILP oracle's savings relative to rule-based baselines. GPT-4o-mini's weekly cost of ยฃ24.63 sits within ยฃ0.01 of the price-only MILP. Projected to four weeks via bootstrap over day-level observations, agents cost approximately ยฃ98โยฃ100 versus ยฃ205 for an off-peak timerโroughly ยฃ1,270/year in projected savings for the representative household. The projection assumes week representativeness; seasonal effects are acknowledged as untested, and the confidence intervals appropriately reflect this. In the harder weekly joint-planning task (336 slots, ten cycles allocated across days and slots), the champion model produces feasible near-optimal plans, with residual losses concentrated in cycle-to-day allocation rather than within-day timing.
Limitations
The paper is candid about scope. Results rest on one household configuration, one market setting, simulated users, and fixed base-load and PV parameters; four-week costs are projections from a single archived week. Commercial model identifiers are pinned, but provider-side drift limits exact reproduction. Construct validity is the deepest concern: the MILP objective equates user value with net electricity cost, omitting comfort, habit, noise, and trust. Open-source backends were not re-run locally, so the privacy-relevant question of local-inference reliability remains quantified only through literature values obtained under a different market and architecture. Appliance-count scaling of flat versus hierarchical designs is left untested.
Conclusion
This work establishes that commercial function-calling LLMs close the multi-appliance coordination gap reported for open-source models, achieving MILP-optimal or near-optimal scheduling directly from natural language at negligible per-event cost. Its three substantive contributions beyond that confirmation are the demonstration that action-interface designโnot model scale aloneโgoverns orchestration reliability; the finding that efficiency-optimal and safety-optimal models differ, motivating deterministic feasibility validation as a deployment guardrail; and the regime-dependent characterization of weather-aware co-optimization under volatile retail prices with negative-price events. The conversational short-circuiting phenomenon and the shared calendar-inference failure are concrete, reusable findings for the broader agentic-systems evaluation literature.