Appliance Usage Prediction for the Smart Home with an Application to Energy Demand Side Management - And Why Accuracy is not a Good Performance Metric for this Problem

Marc Wenninger, Jochen Schmidt, Toni Goeller

Abstract

Shifting energy peak load is a subject that plays a huge role in the currently changing energy market, where renewable energy sources no longer produce the exact amount of energy demanded. Matching demand to supply requires behavior changes on the customer side, which can be achieved by incentives such as Real-Time-Pricing (RTP). Various studies show that such incentives cannot be utilized without a complexity reduction, e. g., by smart home automation systems that inform the customer about possible savings or automatically schedule appliances to off-peak load phases. We propose a probabilistic appliance usage prediction based on historical energy data that can be used to identify the times of day where an appliance will be used and therefore make load shift recommendations that suite the customer’s usage profile. A huge issue is how to provide a valid performance evaluation for this particular problem. We will argue why the commonly used accuracy metric is not suitable, and suggest to use other metrics like the area under the Receiver Operating Characteristic (ROC) curve, Matthews Correlation Coefficient (MCC) or F 1 -Score instead.

References

  1. Arghira, N., Hawarah, L., Ploix, S., and Jacomino, M. (2012). Prediction of appliances energy use in smart homes. Energy, 48(1):128-134. 6th SDEWES Dubrovnik Conference SDEWES 2011.
  2. Barbato, A., Capone, A., Rodolfi, M., and Tagliaferri, D. (2011). Forecasting the usage of household appliances through power meter sensors for demand management in the smart grid. In IEEE Intern. Conf. SmartGridComm, 2011, pages 404-409.
  3. Barker, S., Mishra, A., Irwin, D., Cecchet, E., Shenoy, P., and Albrecht, J. (2012). Smart*: An Open Data Set and Tools for Enabling Research in Sustainable Homes. In Proceedings of SustKDD, Beijing, China.
  4. Basu, K., Hawarah, L., Arghira, N., Joumaa, H., and Ploix, S. (2013). A prediction system for home appliance usage. Energy and Buildings, 67:668 - 679.
  5. Beckel, C., Kleiminger, W., and Cicchetti, R. (2014). The eco data set and the performance of non-intrusive load monitoring algorithms. In Proceedings of the 1st ACM Intern. Conf. BuildSys 2014, pages 80-89.
  6. Chang, C., Verhaegen, P.-A., Duflou, J. R., Drugan, M. M., and Nowe, A. (2013). Finding days-of-week representation for intelligent machine usage profiling. Journ. of Industrial and Intelligent Information, 1(3):148-154.
  7. Cook, N. R. (2007). Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction. Circulation, 115(7):928-935.
  8. Duda, R. O., Hart, P. E., and Stork, D. G. (2000). Pattern Classification (2Nd Edition) . Wiley-Interscience.
  9. Hand, D. J. (2009). Measuring classifier performance: a coherent alternative to the area under the ROC curve. Machine Learning, 77(1):103-123.
  10. Hassan, N. U., Khalid, Y. I., Yuen, C., Huang, S., Pasha, M. A., Wood, K. L., and Kerk, S. G. (2016). Framework for minimum user participation rate determination to achieve specific demand response management objectives in residential smart grids. International Journal of Electrical Power & Energy Systems, 74:91-103.
  11. Hawarah, L., Ploix, S., and Jacomino, M. (2010). User behavior prediction in energy consumption in housing using bayesian networks. In Proc. of the 10th ICAISC, pages 372-379, Berlin, Heidelberg. Springer-Verlag.
  12. Heierman, III, E. O. and Cook, D. J. (2003). Improving home automation by discovering regularly occurring device usage patterns. In Proc. of the Third IEEE Intern. Conf. on Data Mining, ICDM 7803, pages 537- 540, Washington, DC, USA. IEEE Computer Society.
  13. Kang, Z., Jin, M., and Spanos, C. J. (2014). Modeling of end-use energy profile: An appliance-data-driven stochastic approach. In The 40th Annual Conf. of the IEEE Industrial Electronics Society, Dallas, TX, USA, pages 5382-5388.
  14. Kelly, J. and Knottenbelt, W. (2015). The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes. Sci. Data, (2):150007.
  15. Kim, H., Marwah, M., Arlitt, M., Lyon, G., and Han, J. (2011). Unsupervised Disaggregation of Low Frequency Power Measurements. In Proceedings of the 2011 SIAM, pages 747-758.
  16. Kolter, Z. and Johnson, M. J. (2011). REDD: A public data set for energy disaggregation research. In Proc. of SustKDD.
  17. Lachut, D., Banerjee, N., and Rollins, S. (2014). Predictability of energy use in homes. In IGCC, pages 1-10.
  18. Lee, S., Ryu, G., Chon, Y., Ha, R., and Cha, H. (2013). Automatic standby power management using usage profiling and prediction.IEEE Transactions on HumanMachine Systems, 43(6):535-546.
  19. Makonin, S., Ellert, B., Bajic, I. V., and Popowich, F. (2016). Electricity, water, and natural gas consumption of a residential house in Canada from 2012 to 2014. Scientific Data, 3(160037):1-12.
  20. Makonin, S. and Popowich, F. (2015). Nonintrusive load monitoring (NILM) performance evaluation. Energy Efficiency , 8(4):809-814.
  21. Mohsenian-Rad, A.-H. and Leon-Garcia, A. (2010). Optimal residential load control with price prediction in realtime electricity pricing environments. IEEE Trans. Smart Grid, 1(2):120-133.
  22. Monacchi, A., Egarter, D., Elmenreich, W., D'Alessandro, S., and Tonello, A. M. (2014). GREEND: An energy consumption dataset of households in Italy and Austria. In IEEE Int. Conf. SmartGridComm, pages 511-516.
  23. Parzen, E. (1962). On estimation of a probability density function and mode. The annals of mathematical statistics, 33(3):1065-1076.
  24. Pecan Street Inc. (2014). Dataport. https://dataport.pecanstreet.org/.
  25. Powers, D. M. W. (2011). Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation. Journal of Machine Learning Technologies, 2(1):37-63.
  26. S.a. (2005a). Benefits of demand response in electricity markets and recommendations for archiving them. U.S. Department of Energy.
  27. S.a. (2005b). Demand response program evaluation - Final report. Quantum Consulting Inc. and Summit Blue Consulting, LLC Working Group 2 Measurement and Evaluation Committee, California Edison Company.
  28. Schleich, J. and Klobasa, M. (2013). How much shift in demand? Findings from a field experiment in Germany. In Lindström, T., editor, Rethink, renew, restart. ECEEE 2013 Summer Study. Proc., pages 1919-1925. European Council for an Energy-Efficient Economy.
  29. Stephen, B., Galloway, S., and Burt, G. (2014). Self-learning load characteristic models for smart appliances. IEEE Transactions on Smart Grid, 5(5):2432-2439.
  30. Truong, N. C., McInerney, J., Tran-Thanh, L., Costanza, E., and Ramchurn, S. D. (2013a). Forecasting multiappliance usage for smart home energy management. In Proc. of the Twenty-Third Int. Joint Conf. on Artificial Intelligence (IJCAI), pages 2908-2914. AAAI.
  31. Truong, N. C., Tran-Thanh, L., Costanza, E., and Ramchurn, S. D. (2013b). Towards appliance usage prediction for home energy management. In Proc. of the Fourth Intern. Conf. on Future Energy Systems, e-Energy 7813, pages 287-288, New York, NY, USA. ACM.
  32. Uttama Nambi, A. S., Reyes Lua, A., and Prasad, V. R. (2015). LocED: Location-aware Energy Disaggregation Framework. In Proc. of the 2nd ACM Int. Conf. on Embedded Systems for Energy-Efficient Built Environments, BuildSys, pages 45-54, New York, USA.
  33. Valverde-Albacete, F. J. and Peláez-Moreno, C. (2014). 100% Classification Accuracy Considered Harmful: The Normalized Information Transfer Factor Explains the Accuracy Paradox. PLOS ONE, 9(1):1-10.
  34. Viswanath, S. K., Yuen, C., Tushar, W., Li, W. T., Wen, C. K., Hu, K., Chen, C., and Liu, X. (2016). System Design of Internet-of-Things for Residential Smart Grid. IEEE Wireless Communications, 23(5):90-98.
  35. Zhu, X. and Davidson, I. (2007). Knowledge Discovery and Data Mining: Challenges and Realities. Information Science Reference, Hershey, New York.
Download


Paper Citation


in Harvard Style

Wenninger M., Schmidt J. and Goeller T. (2017). Appliance Usage Prediction for the Smart Home with an Application to Energy Demand Side Management - And Why Accuracy is not a Good Performance Metric for this Problem . In Proceedings of the 6th International Conference on Smart Cities and Green ICT Systems - Volume 1: SMARTGREENS, ISBN 978-989-758-241-7, pages 143-150. DOI: 10.5220/0006264401430150


in Bibtex Style

@conference{smartgreens17,
author={Marc Wenninger and Jochen Schmidt and Toni Goeller},
title={Appliance Usage Prediction for the Smart Home with an Application to Energy Demand Side Management - And Why Accuracy is not a Good Performance Metric for this Problem},
booktitle={Proceedings of the 6th International Conference on Smart Cities and Green ICT Systems - Volume 1: SMARTGREENS,},
year={2017},
pages={143-150},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006264401430150},
isbn={978-989-758-241-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Smart Cities and Green ICT Systems - Volume 1: SMARTGREENS,
TI - Appliance Usage Prediction for the Smart Home with an Application to Energy Demand Side Management - And Why Accuracy is not a Good Performance Metric for this Problem
SN - 978-989-758-241-7
AU - Wenninger M.
AU - Schmidt J.
AU - Goeller T.
PY - 2017
SP - 143
EP - 150
DO - 10.5220/0006264401430150