Buildings account for approximately 40% of global energy consumption about half of which is used by heating, ventilation, and air-conditioning (HVAC) systems. Buildings are also responsible for one-third of global carbon-dioxide emissions. Furthermore, buildings’ micro-climate and air quality can directly affect productivity and decision-making performance of occupants in buildings. For these large-scale economic, environmental, and societal impacts, micro-climate control in buildings has become an important issue for governments, building managers, and home owners.
As a solution, smart buildings are fast becoming the norm across the globe for their energy efficiency and reduced operating costs. A smart building leverages sensory data, e.g., indoor and outdoor temperature and humidity, CO2 concentration, and occupancy status, and uses a combination of technologies to automate building energy management. The new generation of smart buildings aims to learn from data how to operate autonomously and with minimum user interventions. For instance, a learning thermostat can potentially learn how to adjust its set-point temperatures in coordination with other HVAC devices or based on its prediction of electricity tariffs in order to save energy and cost.
However, despite recent advances in Internet of Things (IoT) technology and data analytics, implementation of such smart buildings is impeded by the time-consuming process of data acquisition in buildings. This calls for algorithms that are very data-efficient. Furthermore, computational resources could be limited, whether the learning takes place in the cloud or locally on the HVAC device itself. This necessiates computationally efficient algorithms.
In this project, I developed a framework for designing learning-based controllers for HVAC systems in buildings to simultaneously reduce energy usage and improve occupants' comfort. By leveraging our domain knowledge about building thermodynamics, we constrained the space of admissible control solutions to a manageable size by parameterizing a high-dimensional control space into lower dimensional manifolds. Learning lower-dimensional control manifolds will be much more data-efficient, and hence, suitable for real-world applications. In addition, our control problem is formulated based on semi-Markov decision processes (SMDPs) rather than MDPs. This allows for continuous-time control as opposed to discrete-time control as in classic reinforcement learning. This consequently enables less-frequent learning updates, and hence, makes our algorithms computationally less expensive. Using our methodolgy, we designed a smart thermostat that can learn how to operate optimally in just about a week.
- Hosseinloo, A. H., Ryzhov, A., Bischi, A., Ouerdane, H., Turitsyn, K., & Dahleh, M. A., 2020. D ata-driven control of micro-climate in buildings: an event-triggered reinforcement learning approach. Applied Energy, 277, p.115451. [PDF]
- Hosseinloo, A.H. and Dahleh, M.A., 2020. Event-Triggered Reinforcement Learning; An Application to Buildings' Micro-Climate Control. In AAAI Spring Symposium: MLPS. [PDF]