- The paper demonstrates that retrieval augmentation is the key driver in enhancing ECM recommendation accuracy, with performance gains of up to 0.20 micro-F1.
- It systematically compares multiple LLMs and supervised tree methods, revealing that explicit reasoning adds cost without significant accuracy benefits.
- The study highlights that open-weight models with retrieval offer cost-effective, scalable solutions for real-world energy audit applications.
Retrieval over Reasoning for Energy-Retrofit Recommendation in LLM Systems
Problem Context and Motivation
The paper confronts the longstanding issue of automating building energy conservation measure (ECM) recommendation, a high-impact, structured prediction task typically requiring expert domain judgement and multi-label outputs. Using 10,422 New York City Local Law 87 (LL87) energy audit records as the testbed, the authors systematically dissect model, formulation, and data contributions to the accuracy, cost, and latency of ECM recommendation systems.
The motivating challenge lies in the lack of a strong supervised signal for ECM selection—auditor decisions are highly context-dependent, and general-purpose LLMs are not natively grounded in local building stock or historical auditor behavior. The work innovates by benchmarking eight leading LLMs across a retrieval × reasoning factorial, alongside classic supervised learners. The implications are broad for structured decision support in regulatory domains and for the cost-effective deployment of LLMs in grounded, enterprise workflows.
Upstream Task: Energy-Use-Intensity Regression
The precursor to ECM recommendation, building energy-use-intensity (EUI) regression, provides a resolved supervised problem. Across 14,332 building-years, tree ensemble methods (stacking ensemble, XGBoost, LightGBM, CatBoost) far outperform neural baselines (MLP, ResNet, 1D-CNN, LSTM, Transformer, TabNet) and probabilistic Gaussian processes, with the top ensemble attaining R2=0.757. Notably, even the most expressive deep models plateau well below tree ensemble performance, reaffirming literature on the dominance of tree-based methods on tabular building datasets.
This finding crucially reframes subsequent methodological choices: the limiting factor in EUI regression is the information content of the input (notably, prior-year EUI), not model expressivity. The same reasoning is brought to bear downstream, implying that for ECM recommendation, the solution is not merely larger models or deeper networks, but rather making available the right contextual knowledge.
Data and Benchmark Design
The core ECM recommendation task is cast as multi-label binary relevance over 19 validated ECM categories, each building associated with on average 2.8 recommended measures. This reframing—from single-label to multi-label—constitutes a pivotal advance, as the prior single-label approach artifactually suppressed accuracy to 25.9%; the multi-label tree baseline reaches micro-F1 of 0.571.
The experimental harness strictly controls for differences other than the prediction method. All arms (models × approaches) receive identical building features and must output a subset of the same ECM label set. Leakage is meticulously controlled: retrieval of neighbors occurs only from the training set, and a building's actual measures are never disclosed in its prompt, preventing local overfitting or information bleed.
Methods: Dissecting Retrieval and Reasoning
The paper's principal experimental design is a five-arm benchmark:
- Arm 1: Supervised gradient-boosted tree, one binary classifier per ECM, serving as gold standard
- Arms 2–5: Four LLM arms (plain, reasoning, RAG, RAG+reasoning) spanning the 2×2 grid of retrieval (kNN-based case insertions) and explicit reasoning (chain-of-thought or "thinking mode")
In retrieval arms, the addition is a simple k=5 nearest neighbor insertion (using standardized Euclidean distance) of audited buildings' ECM recommendations; reasoning arms request or natively enable step-by-step deliberation.
Eight LLMs were evaluated: Anthropic (Claude Opus 4.8, Sonnet 4.6, Haiku 4.5), OpenAI (GPT-5.5, GPT-5-mini), Google (Gemini 3.5 Flash), and two open-weight models (Llama 3.3 70B Instruct, Qwen3 235B) via hosted APIs.
Results: Retrieval is Decisive, Reasoning is Not
Main Findings
- Retrieval (RAG) is the dominant lever: Across all models, adding retrieval lifted micro-F1 by +0.11 to +0.20 (e.g., for Llama 3.3 70B, plain F1 = 0.314, RAG F1 = 0.511), with a statistically significant effect.
- Explicit reasoning yields no accuracy benefit: The addition of step-by-step reasoning or enabling "thinking modes" changed micro-F1 by at most ±0.018, while inflating cost up to 8.4× and latency accordingly.
- LLM cost-accuracy tradeoffs: A 70B open-weight Llama model with RAG attained 0.511 F1 at $0.00032/building—comparable to frontier GPT-5.5 with retrieval (0.526 F1 at$0.00318), but over 10× cheaper.
- Supervised tree baseline remains state-of-the-art in accuracy (F1 = 0.571) and computational cost (effectively zero) where training data exists.
Precision-Recall Decomposition
Plain and reasoning LLMs systematically over-recommend, with recall ≈ 0.82 but low precision (≈ 0.21), suggesting a severe false positive burden. Retrieval effectively suppresses ill-fitting ECMs, raising precision to ≈ 0.37 while maintaining high recall, directly addressing tendency to over-generation.
Supervised trees, conversely, are highly precise (precision 0.755) but conservative (recall 0.459), missing many true positives—highlighting complementary error profiles that could motivate hybrid strategies.
Ablation by Category Frequency
The effect of retrieval is heterogeneous across ECM categories:
- Common categories (>10% prevalence): All methods do reasonably well; retrieval provides minimal additional benefit.
- Mid-frequency categories: Retrieval is vital (e.g. Renewable Energy Systems: plain LLM 0.12 → RAG 0.47).
- Rare categories (<2% prevalence): LLMs (with or without retrieval) collapse to near-zero F1, while supervised trees retain useful performance due to direct estimation of category base rates. This exposes the vulnerability of RAG-based approaches to sample sparsity in structured recommendation tasks.
Cost and Deployment Implications
In cost-controlled deployments, model identity is diminished in relevance once retrieval is present. Open-weight RAG configurations (e.g., Llama 3.3 70B) provide near-frontier accuracy at an order of magnitude lower cost than API-gated, closed LLMs—strongly recommending open models for scaling over large building stocks when privacy regulations and data privacy concerns allow.
If labeled data and per-category base rate estimation are available, a supervised gradient-boosted tree remains both the highest-performing and most computationally efficient solution.
Negative Result: Preference Optimization
An additional experiment tested whether preference-optimized finetuning (using DPO, GRPO, and rejection sampling) on up to 21,703 auditor-derived preference pairs with a 7B Qwen policy could replace retrieval at inference. Despite robust optimization signals and various data balancing schemes, the fine-tuned model plateaued at 22–35% category match (vs. ≈50% for the retrieval-augmented LLM), indicating mode collapse toward majority categories and failure to internalize the rare-category vocabulary that auditor competence requires.
This clean negative result suggests that at current data and model scale, preference optimization does not substitute for retrieval grounding in knowledge-intensive structured prediction tasks.
Theoretical and Practical Implications
The evidence provided reframes deployment and research priorities for structured LLM classifiers in tabular or limited-feature environments:
- Information, not capacity, is the limiting constraint: Model size, depth, and inclusion of explicit reasoning did not drive improvements where relevant contextual information was absent from the prompt.
- Grounding with retrieval is critical for associative multi-label tasks in domains with sparse label supervision or where historical context is unavailable to the model.
- Task formulation (multi-label classification) drives more performance gain than model innovation: Recasting to reflect the true nature of auditor decisions (multi-label) outperformed all architectural manipulations.
- Hybrid strategies should be considered: LLM retrieval approaches perform well on common and mid-frequency categories, but rare categories necessitate fallback to supervised models or expert review. Routing based on category prior should become standard practice.
Future Directions
Improved rare-category recovery may be obtainable by designing retrieval indices with rare-category oversampling, feature-weighted kNN, or more advanced unsupervised neighbor selection. Exploring larger-scale preference optimization or reward functions enforcing diversity and rare-category emission is warranted. Extending the benchmark to tasks that require measure-level cost/benefit estimation or transfer to other regulatory datasets will additionally validate generality.
Conclusion
This paper provides conclusive evidence that, for ECM recommendation on NYC energy audits, retrieval augmentation outperforms both ungrounded LLM prompting and explicit reasoning, with nearly all improvement attributable to the retrieval of comparable, labeled neighbor records. Fine-tuned, open LLMs with retrieval match the cost-effectiveness of API-locked frontier models and approach supervised tree ensemble accuracy, at much lower inference costs. The findings have direct implications for large-scale deployment in energy efficiency programs and highlight retrieval as the necessary mechanism for grounding LLMs in structured, associative decision support tasks (2607.05440).