du.sePublications
Change search
Link to record
Permanent link

Direct link
BETA
Publications (5 of 5) Show all publications
Han, M., May, R., Zhang, X., Wang, X., Pan, S., Yan, D., . . . Xu, L. (2019). A review of reinforcement learning methodologies for controlling occupant comfort in buildings. Sustainable cities and society, 51, Article ID 101748.
Open this publication in new window or tab >>A review of reinforcement learning methodologies for controlling occupant comfort in buildings
Show others...
2019 (English)In: Sustainable cities and society, ISSN 2210-6707, Vol. 51, article id 101748Article in journal (Refereed) Published
National Category
Building Technologies
Research subject
Complex Systems – Microdata Analysis
Identifiers
urn:nbn:se:du-30601 (URN)10.1016/j.scs.2019.101748 (DOI)000493744700053 ()2-s2.0-85070980900 (Scopus ID)
Available from: 2019-08-08 Created: 2019-08-08 Last updated: 2020-01-07Bibliographically approved
May, R., Zhang, X., Wu, J. & Han, M. (2019). Reinforcement learning control for indoor comfort: A survey. In: IOP Conference Series: Materials Science and Engineering: . Paper presented at 10th International Conference on Indoor Air Quality, Ventilation and Energy Conservation in Buildings, IAQVEC 2019; Bari; Italy; 5 September 2019 through 7 September 2019; Code 153083. , 609(6), Article ID 062011.
Open this publication in new window or tab >>Reinforcement learning control for indoor comfort: A survey
2019 (English)In: IOP Conference Series: Materials Science and Engineering, 2019, Vol. 609, no 6, article id 062011Conference paper, Published paper (Refereed)
Abstract [en]

Building control systems are prone to fail in complex and dynamic environments. The reinforcement learning (RL) method is becoming more and more attractive in automatic control. The success of the reinforcement learning method in many artificial intelligence applications has resulted in an open question on how to implement the method in building control systems. This paper therefore conducts a comprehensive review of the RL methods applied in control systems for indoor comfort and environment. The empirical applications of RL-based control systems are then presented, depending on optimisation objectives and the measurement of energy use. This paper illustrates the class of algorithms and implementation details regarding how the value functions have been represented and how the policies are improved. This paper is expected to clarify the feasible theory and functions of RL for building control systems, which would promote their wider-spread application and thus contribute to the social economic benefits in the energy and built environments.

National Category
Civil Engineering Computer and Information Sciences
Research subject
Complex Systems – Microdata Analysis
Identifiers
urn:nbn:se:du-31167 (URN)10.1088/1757-899X/609/6/062011 (DOI)2-s2.0-85074523665 (Scopus ID)
Conference
10th International Conference on Indoor Air Quality, Ventilation and Energy Conservation in Buildings, IAQVEC 2019; Bari; Italy; 5 September 2019 through 7 September 2019; Code 153083
Available from: 2019-12-06 Created: 2019-12-06 Last updated: 2019-12-06
May, R. (2019). The reinforcement learning method: A feasible and sustainable control strategy for efficient occupant-centred building operation in smart cities. (Licentiate dissertation). Borlänge: Dalarna University
Open this publication in new window or tab >>The reinforcement learning method: A feasible and sustainable control strategy for efficient occupant-centred building operation in smart cities
2019 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Over half of the world’s population lives in urban areas, a trend which is expected to only grow as we move further into the future. With this increasing trend in urbanisation, challenges are presented in the form of the management of urban infrastructure systems. As an essential infrastructure of any city, the energy system presents itself as one of the biggest challenges. As cities expand in population and economically, global energy consumption increases and as a result so do greenhouse gas (GHG) emissions. To achieve the 2030 Agenda’s sustainable development goal on energy (SDG 7), renewable energy and energy efficiency have been shown as key strategies for attaining SDG 7. As the largest contributor to climate change, the building sector is responsible for more than half of the global final energy consumption and GHG emissions. As people spend most of their time indoors, the demand for energy is made worse as a result of maintaining the comfort level of the indoor environment. However, the emergence of the smart city and the internet of things (IoT) offers the opportunity for the smart management of buildings. Focusing on the latter strategy towards attaining SDG 7, intelligent building control offers significant potential for saving energy while respecting occupant comfort (OC). Most intelligent control strategies, however, rely on complex mathematical models which require a great deal of expertise to construct thereby costing in time and money. Furthermore, if these are inaccurate then energy is wasted and the comfort of occupants is decreased. Moreover, any change in the physical environment such as retrofits result in obsolete models which must be re-identified to match the new state of the environment. This model-based approach seems unsustainable and so a new model-free alternative is proposed. One such alternative is the reinforcement learning (RL) method. This method provides a beautiful solution to accomplishing the tradeoff between energy efficiency and OC within the smart city and more importantly to achieving SDG 7. To address the feasibility of RL as a sustainable control strategy for efficient occupant-centred building operation, a comprehensive review of RL for controlling OC in buildings as well as a case study implementing RL for improving OC via a window system are presented. The outcomes of each seem to suggest RL as a feasible solution, however, more work is required in the form of addressing current open issues such as cooperative multi-agent RL (MARL) needed for multi-occupant/multi-zonal buildings.

Place, publisher, year, edition, pages
Borlänge: Dalarna University, 2019
Series
Dalarna Licentiate Theses ; 12
Keywords
Markov decision processes, Reinforcement learning, Control, Building, Indoor comfort, Occupant
National Category
Building Technologies
Research subject
Complex Systems – Microdata Analysis
Identifiers
urn:nbn:se:du-30613 (URN)978-91-88679-03-1 (ISBN)
Presentation
2019-11-01, B310, Borlänge, 10:00 (English)
Opponent
Supervisors
Available from: 2019-10-11 Created: 2019-10-07 Last updated: 2020-01-10Bibliographically approved
Han, M., Zhang, X., Xu, L., May, R., Pan, S. & Wu, J. (2018). A review of reinforcement learning methodologies on control systems for building energy. Borlänge: Högskolan Dalarna
Open this publication in new window or tab >>A review of reinforcement learning methodologies on control systems for building energy
Show others...
2018 (English)Report (Other academic)
Abstract [en]

The usage of energy directly leads to a great amount of consumption of the non-renewable fossil resources. Exploiting fossil resources energy can influence both climate and health via ineluctable emissions. Raising awareness, choosing alternative energy and developing energy efficient equipment contributes to reducing the demand for fossil resources energy, but the implementation of them usually takes a long time. Since building energy amounts to around one-third of global energy consumption, and systems in buildings, e.g. HVAC, can be intervened by individual building management, advanced and reliable control techniques for buildings are expected to have a substantial contribution to reducing global energy consumptions. Among those control techniques, the model-free, data-driven reinforcement learning method seems distinctive and applicable. The success of the reinforcement learning method in many artificial intelligence applications has brought us an explicit indication of implementing the method on building energy control. Fruitful algorithms complement each other and guarantee the quality of the optimisation. As a central brain of smart building automation systems, the control technique directly affects the performance of buildings. However, the examination of previous works based on reinforcement learning methodologies are not available and, moreover, how the algorithms can be developed is still vague. Therefore, this paper briefly analyses the empirical applications from the methodology point of view and proposes the future research direction.

Place, publisher, year, edition, pages
Borlänge: Högskolan Dalarna, 2018. p. 26
Series
Working papers in transport, tourism, information technology and microdata analysis, ISSN 1650-5581 ; 2018:02
Keywords
Reinforcement learning; Markov decision processes; building energy; control; multi-agent system
National Category
Control Engineering
Research subject
Complex Systems – Microdata Analysis, General Microdata Analysis - methods
Identifiers
urn:nbn:se:du-27956 (URN)
Available from: 2018-06-19 Created: 2018-06-19 Last updated: 2020-02-03Bibliographically approved
Han, M., May, R., Zhang, X., Wang, X., Pan, S., Yan, D. & Jin, Y.A novel reinforcement learning method for improving occupant comfort via window opening and closing.
Open this publication in new window or tab >>A novel reinforcement learning method for improving occupant comfort via window opening and closing
Show others...
(English)Manuscript (preprint) (Other academic)
National Category
Building Technologies
Research subject
Complex Systems – Microdata Analysis
Identifiers
urn:nbn:se:du-30942 (URN)
Available from: 2019-10-11 Created: 2019-10-11 Last updated: 2019-10-11Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-0551-9341

Search in DiVA

Show all publications