Towards Comparability in Non-Intrusive Load Monitoring: On Data and\n Performance Evaluation

Non-Intrusive Load Monitoring (NILM) comprises of a set of techniques that\nprovide insights into the energy consumption of households and industrial\nfacilities. Latest contributions show significant improvements in terms of\naccuracy and generalisation abilities. Despite all progress made concerning\ndisaggregation techniques, performance evaluation and comparability remains an\nopen research question. The lack of standardisation and consensus on evaluation\nprocedures makes reproducibility and comparability extremely difficult. In this\npaper, we draw attention to comparability in NILM with a focus on highlighting\nthe considerable differences amongst common energy datasets used to test the\nperformance of algorithms. We divide discussion on comparability into data\naspects, performance metrics, and give a close view on evaluation processes.\nDetailed information on pre-processing as well as data cleaning methods, the\nimportance of unified performance reporting, and the need for complexity\nmeasures in load disaggregation are found to be the most urgent issues in\nNILM-related research. In addition, our evaluation suggests that datasets\nshould be chosen carefully. We conclude by formulating suggestions for future\nwork to enhance comparability.\n