Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Low Emission Building Control with Zero-Shot Reinforcement Learning (2206.14191v3)

Published 28 Jun 2022 in cs.LG, cs.SY, and eess.SY

Abstract: Heating and cooling systems in buildings account for 31% of global energy use, much of which are regulated by Rule Based Controllers (RBCs) that neither maximise energy efficiency nor minimise emissions by interacting optimally with the grid. Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency, but existing solutions require access to building-specific simulators or data that cannot be expected for every building in the world. In response, we show it is possible to obtain emission-reducing policies without such knowledge a priori--a paradigm we call zero-shot building control. We combine ideas from system identification and model-based RL to create PEARL (Probabilistic Emission-Abating Reinforcement Learning) and show that a short period of active exploration is all that is required to build a performant model. In experiments across three varied building energy simulations, we show PEARL outperforms an existing RBC once, and popular RL baselines in all cases, reducing building emissions by as much as 31% whilst maintaining thermal comfort. Our source code is available online via https://enjeeneer.io/projects/pearl/

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Biewald, L. 2020. Experiment Tracking with Weights and Biases. Software available from wandb.com.
  2. Predictive Control for Linear and Hybrid Systems. Cambridge University Press.
  3. Optimal control of HVAC and window systems for natural ventilation through reinforcement learning. Energy and Buildings, 169: 195 – 205.
  4. Deep Reinforcement Learning in a Handful of Trials Using Probabilistic Dynamics Models. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, 4759–4770.
  5. Corradi, O. 2022. Electricity Maps: A real-time visualisation of the CO2 emissions of Electricity Consumption.
  6. EnergyPlus: creating a new-generation building energy simulation program. Energy and buildings, 33(4): 319–331.
  7. The efficient use of energy: Tracing the global flow of energy from fuel to service. Energy Policy, 38(1): 75–81.
  8. PILCO: A model-based and data-efficient approach to policy search. In Proceedings of the 28th International Conference on machine learning (ICML-11), 465–472.
  9. MB2C: Model-Based Deep Reinforcement Learning for Multi-zone Building Control. In Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, 50–59.
  10. Improving PILCO with Bayesian neural network dynamics models. In Data-Efficient Machine Learning workshop, ICML, volume 4, 25.
  11. Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In 2017 IEEE international conference on robotics and automation (ICRA), 3389–3396. IEEE.
  12. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, 1861–1870.
  13. Temporal Difference Learning for Model Predictive Control. arXiv 2203.04955.
  14. Gaussian Processes for Big Data. In Uncertainty in Artificial Intelligence, 282. Citeseer.
  15. Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning. CoRR, abs/1803.02291.
  16. Learning and Control Using Gaussian Processes. In 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS), 140–149.
  17. NeurOpt: Neural network based optimization for building energy management and climate control. In Proceedings of the 2nd Conference on Learning for Dynamics and Control, volume 120 of Proceedings of Machine Learning Research, 445–454. The Cloud: PMLR.
  18. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2): 99–134.
  19. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30.
  20. Data Center Cooling using Model-predictive Control. In Proceedings of the Thirty-second Conference on Neural Information Processing Systems (NeurIPS-18), 3818–3827. Montreal, QC.
  21. Levine, S. 2018. Reinforcement learning and control as probabilistic inference: Tutorial and review. arXiv preprint arXiv:1805.00909.
  22. Continuous control with deep reinforcement learning. In ICLR (Poster).
  23. Playing Atari with Deep Reinforcement Learning. CoRR, abs/1312.5602.
  24. Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), 7559–7566. IEEE.
  25. Energym: A Building Model Library for Controller Benchmarking. Applied Sciences, 11(8): 3518.
  26. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
  27. Mastering the game of go without human knowledge. nature, 550(7676): 354–359.
  28. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25.
  29. Zero-shot learning through cross-modal transfer. Advances in neural information processing systems, 26.
  30. Reinforcement Learning: An Introduction. The MIT Press, second edition.
  31. Energy optimization associated with thermal comfort and indoor air control via a deep reinforcement learning algorithm. Building and Environment, 155: 105 – 117.
  32. Deep Reinforcement Learning for Building HVAC Control. In Proceedings of the 54th Annual Design Automation Conference 2017, DAC ’17. New York, NY, USA: Association for Computing Machinery. ISBN 9781450349277.
  33. Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA.
  34. Model predictive path integral control using covariance variable importance sampling. arXiv preprint arXiv:1509.01149.
  35. A Review of Deep Reinforcement Learning for Smart Building Energy Management. IEEE Internet of Things Journal, 8(15): 12046–12063.
  36. Building HVAC scheduling using reinforcement learning via neural network based model approximation. In Proceedings of the 6th ACM international conference on systems for energy-efficient buildings, cities, and transportation, 287–296.
  37. Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning. Energy and Buildings, 199: 472–490.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube