Towards smart energy systems: application of kernel machine regression for medium term electricity load forecasting
© Alamaniotis et al. 2016
Received: 1 April 2015
Accepted: 4 January 2016
Published: 20 January 2016
Integration of energy systems with information technologies has facilitated the realization of smart energy systems that utilize information to optimize system operation. To that end, crucial in optimizing energy system operation is the accurate, ahead-of-time forecasting of load demand. In particular, load forecasting allows planning of system expansion, and decision making for enhancing system safety and reliability. In this paper, the application of two types of kernel machines for medium term load forecasting (MTLF) is presented and their performance is recorded based on a set of historical electricity load demand data. The two kernel machine models and more specifically Gaussian process regression (GPR) and relevance vector regression (RVR) are utilized for making predictions over future load demand. Both models, i.e., GPR and RVR, are equipped with a Gaussian kernel and are tested on daily predictions for a 30-day-ahead horizon taken from the New England Area. Furthermore, their performance is compared to the ARMA(2,2) model with respect to mean average percentage error and squared correlation coefficient. Results demonstrate the superiority of RVR over the other forecasting models in performing MTLF.
KeywordsRelevance vector regression Gaussian process regression Medium term load forecasting Smart energy systems
Limitations in current power infrastructure together with world-wide concerns, like climate change and economic stability are the driving factors to ongoing research efforts for developing a new generation of smart energy systems (Fainti et al. 2014). Realization of smart energy systems is greatly accommodated by coupling information technologies with power systems. In particular, the advent of internet and advancements in communication technologies inspired the notion of an Energy Internet (Alamaniotis et al. 2011a, b), in which information networks interact with power generation, transmission, and distribution systems aiming at optimizing power system operation.
Smart energy systems utilize information to overcome the significant constraints of the current power grid infrastructure (Tsoukalas and Gao 2008). The limited delivery capacity and the lack of large scale energy storage may lead to grid destabilization causing distribution failures with high financial impact to grid participants. For instance, (i) load demand beyond delivery capacity results in financially expensive system failures and blackouts (Alamaniotis et al. 2014b), and (ii) the amount of excess generated energy that cannot be stored is wasted since the generation does not closely follow the demand (Gao et al. 2003).
Electricity load forecasting has been recognized as a key issue in implementing smart energy systems (Alamaniotis et al. 2014a, b). Load forecasting may be used by all smart grid participants aiming at reaching their goals. For example, consumers utilize load forecasting for consumption planning and scheduling while grid operators for safe and secure electricity delivery. Depending on the forecasting time horizon, load forecasting may be identified as very short term (VSTLF) ranging from some minutes to an hour (Alamaniotis et al. 2012), short term (STLF) (Alamaniotis et al. 2011a, b) ranging from an hour to a week, medium term (MTLF) ranging from a week to a year (Ghiassi et al. 2006), and long term load forecasting (LTLF) for longer than a year ahead of time predictions (Kandil et al. 2002).
The current manuscript focuses on medium term load forecasting. MTLF is an efficient tool for implementing smart energy systems since it promotes optimal expansion planning by considering climate changes, maintenance scheduling, fuel purchase negotiating (for instance for nuclear power plants), component replacing or repairing, and maximizing utilization of renewable resources such as wind power. Furthermore, it is expected to play a crucial role in developing price directed energy markets in which entities will participate via intelligent meters (Gatsis and Giannakis 2012) and require forecasting tools to develop their electricity purchase strategies.
Though the number of proposed approaches for performing MTLF is limited, there are ongoing efforts for developing more sophisticated and advanced tools that satisfy the demands imposed by the advent of the “big data” era. The proposed approaches make use of tools coming from statistics and artificial intelligence fields. A dynamic artificial neural network is proposed in (Ghiassi et al. 2006), and a radial basis function neural network in (Xia et al. 2010), while combination of neural networks with expert systems in (Kim et al. 1995). Other methods employed adaptive neural networks (Tsekouras et al. 2006), particle swarm optimization (Rengcun et al. 2008), and singular value decomposition (Abu-Shikhah and Elkarmi 2011). Nonlinear multivariable regression for MTLF is presented in (Tsekouras et al. 2007), while a combination of linear and non-linear regression for MTLF is introduced in (Abu-Shikhah et al. 2011), and Gaussian processes for a year ahead monthly load forecasting in (Alamaniotis et al. 2014a). Furthermore, a support vector machine based approach for MTLF is discussed in (Bozic and Stojanovic 2011), while a hybrid methodology comprised of autoregressive integrated moving average (ARIMA) and artificial neural network is introduced and tested in (El Desouky and Elkateb 2000). The above methodologies, though effective, come at a cost of high prediction uncertainty. In addition they lack the necessary flexibility to update their predictions since they are unable to capture nonlinear load dynamics.
In this paper intelligent regression models for MTLF are examined. The proposed models make use of machine learning tools and more specifically of kernel machines (Scholkopf and Smola 2001). In particular, relevance vector regression (Tipping 2001) and Gaussian process regression (Rasmussen 2006) are utilized for making predictions for longer than a week ahead of time horizon. Generally speaking, kernel machines are nonlinear methods that inherently make use of semi-positive definite matrices in order to make predictions (Hoffman et al. 2008). They are able of detecting the kind of dependencies that dominate the load properties by formulating the feature space in terms of kernels. Formulation of feature space by kernels is the advantage of kernel machines as opposed to the rest load forecasting methods mentioned earlier; it allows the modeler to control the forecasting process by selecting the kernel form, and promotes model flexibility by offering a high variety of kernels (Alamaniotis et al. 2015). For instance, kernel regression facilitates selection of a kernel that models particular data properties, for example stationarity, in contrast to artificial neural networks that require not only selection of neuron activation functions but also network architecture (Tsoukalas and Uhrig 1997). Assessment of the forecasting performance is done using the mean average percentage error (MAPE) and squared correlation coefficient (R 2), while the testing datasets are comprised of the daily demand for a 30-day-ahead horizon.
The roadmap of the paper is as follows: in the next two sections a brief presentation on kernel machines is provided and the proposed methodology is presented. Medium term load forecasting results are given in the “Results” section, while the last section concludes and summarizes the main points of the paper.
Beyond the widely known kernels, new valid kernels may be created by composition of two, or more, valid kernels by applying the operations of addition and/or multiplication (Rasmussen 2006). The selection of an appropriate kernel function is a main design choice that must generally be made by the designer according to the specifications of the problem at hand.
Gaussian process regression
Relevance vector regression
Maximization of the marginal likelihood in Eq. (23) with an appropriate iterative method allows evaluation of its parameters. Therefore, the computed optimal values for α and σ2 are equal to α* and (σ2)* respectively. Some of the elements of the vector α* are driven to infinity and thus the posterior distribution of their weights is normal with both mean and variance being equal to zero. As a result, the corresponding kernel functions have no contribution in prediction making driving the output to depend exclusively on the non-zero weighted kernels. The inputs associated with non-zero weighted kernels are called relevance vectors.
Medium-term-load-forecasting using kernel machine regression
Gaussian process regression model equipped with a Gaussian kernel, and
Relevance vector regression model equipped with a Gaussian kernel.
In our study, we aim at making daily predictions for a 30-day-ahead horizon. Thus, the goal is to predict the load demand for every day in the next 30 days (overall 30 predicted values). To that end, we have our forecasters making predictions on a monthly basis (January–December) and therefore our study falls within the purpose of MTLF.
We apply the presented forecasters to medium term load forecasting for electricity demand load data obtained from the New England ISO (last accessed in 2015) for the period January 2004–August 2011. In particular, we analyze historical load datasets that represent the daily load demand in one of the hubs of the New England ISO Area. Taking into consideration the historical data at our disposal, the forecasters are applied to forecasting demand from January 2007 to August 2011.
Figure 5 exhibits that RVR forecaster provides more accurate daily predictions for a month-ahead-horizon (i.e., 30-day ahead horizon) with respect to MAPE. In particular RVR gives the best performance for all months but November, where ARMA is the best forecaster. GPR gives the worst performance for all months in 2007 except for August. In 2008 data, Fig. 6 exhibits RVR as the best performing forecaster in all tested months except for August, where it is slightly outperformed by GPR. ARMA(2,2) performance is better than GPR and worse than RVR in the majority of the cases, with the exception of June and August 2008; for the latter months the ARMA forecasts are the least accurate among all forecasters.
In Fig. 7, we observe that RVR once more provides the best performance in the majority of the cases for year 2009—with the exception of February, July and September. For the same time interval (i.e., 2009), GPR provides the worst performance among three forecasters with a few exceptions. Furthermore, results for year 2010 presented in Fig. 8 drive to similar conclusions as earlier: RVR is the best forecaster in the majority of the cases (in 10 out of 12), GPR the worst in most of them, while ARMA is the worst in two cases (January and October) and the best in other two (February and June). Additionally, in Fig. 9 provides the MAPE results for the first 8 months of year 2011: RVR clearly outperforms the other two forecasters in all cases, GPR provides the least accurate predictions in February, March, April, June, July and August, and ARMA is the least accurate for January and May.
Average per year MAPE obtained by GPR, RVR and ARMA forecasters
Average per year squared correlation coefficient (R2) obtained by GPR, RVR and ARMA forecasters
Squared correlation coefficient (R2)
Therefore, we observe that depending on the selected model kernel machine may provide high accurate MLTF, as taken by RVR, or may provide low accuracy, as is the case with GPR.
The application of two types of kernel machines for medium-term load forecasting has been presented in this paper. The kernel machines studied are GPR and RVR whose performance is tested on actual historic data collected at the New England Area on a daily basis up to a month, with the tested time period being from January 2007 to August 2011. In addition, both forecasters are also compared to the ARMA(2,2) statistical tool that has been widely used in time series forecasting.
Obtained results show the superiority of RVR over the other two tested methods with respect to MAPE and R 2. On a monthly comparison RVR provided the best accuracy in the majority of the cases while it is by far the best forecaster on a yearly based comparison. However, it should be emphasized that the kernel machines are equipped with a Gaussian kernel, which is the only kernel being tested in the current work; testing of other kernel functions is left for future work.
In addition, the promising method of core vector regression (Li and Liu 2010) will also be examined either as an independent forecaster or in combination with RVR and GP. Combination of kernel machines exhibits high potency for providing highly accurate medium term load predictions.
MA designed the study, developed the codes for the machine learning algorithms in Matlab, analyzed and interpreted the results and drafted the manuscript. DB designed and created the training and testing datasets, developed the statistical ARMA code in Matlab, and was involved in revising the manuscript for technical and intellectual content. LHT conceived the study and participated in its coordination and helped to draft the manuscript. All authors read and approved the final manuscript.
This work has been supported in part by the US National Science Foundation under Grant No. 1462393 and through the project “Hephaestus” under the auspices of “ARISTEIA” sponsored by the Hellenic General Secretariat for Research and Technology under the Action of Operational Program Education and Lifelong Learning co-funded by the European Social Fund and National Resources.
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Abu-Shikhah N, Elkarmi F (2011) Medium-term electric load forecasting using singular value decomposition. Energy 36(7):4259–4271View ArticleGoogle Scholar
- Abu-Shikhah N, Elkarmi F, Aloquili OM (2011) Medium-term load forecasting using multivariable linear and non-linear regression. Smart Grid Renew Energy 2:126–135View ArticleGoogle Scholar
- Alamaniotis M, Gao R, Tsoukalas LH (2011) Towards an energy internet: a game-theoretic approach to price-directed energy utilization. In: Energy-efficient computing and networking, pp 3–11Google Scholar
- Alamaniotis M, Ikonomopoulos A, Tsoukalas LH (2011) A Pareto optimization approach of a Gaussian process ensemble for short-term load forecasting. In: Proceedings of the 16th IEEE international conference on intelligent system application to power systems (ISAP), pp 1–6Google Scholar
- Alamaniotis M, Ikonomopoulos A, Tsoukalas LH (2012) Evolutionary multiobjective optimization of kernel-based very short-term load forecasting. IEEE Trans Power Syst 27(3):1477–1484View ArticleGoogle Scholar
- Alamaniotis M, Chatzidakis S, Tsoukalas LH (2014) Monthly load forecasting using gaussian process regression. In: Proceedings of the 9th Mediterranean conference on power generation, transmission, distribution, and energy conversion: MEDPOWER 2014, pp 1–7Google Scholar
- Alamaniotis M, Tsoukalas LH, Bourbakis N (2014) Virtual cost approach: electricity consumption scheduling for smart grids/cities in price-directed electricity markets. In: Proceedings of the 5th international conference on information, intelligence, systems and applications, IISA 2014, pp 38–43Google Scholar
- Alamaniotis M, Bargiotas D, Bourbakis N, Tsoukalas LH (2015) Genetic optimal regression of relevance vector machines for electricity price forecasting in smart grids. IEEE Trans Smart Grid 6(6):2997–3005View ArticleGoogle Scholar
- Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkGoogle Scholar
- Bozic M, Stojanovic M (2011) Application of SVM methods for mid-term load forecasting. Serbian J Electr Eng 8(1):73–83View ArticleGoogle Scholar
- El Desouky AA, Elkateb MM (2000) Hybrid adaptive techniques for electric-load forecast using ANN and ARIMA. IET Proc Gener Transm Distrib 147(4):213–217View ArticleGoogle Scholar
- Fainti R, Nasiakou A, Tsoukalas E, Vavalis M (2014) Design and early simulations of next generation intelligent energy systems. Int J Monit Surveill Technol Res 2(2):58–82View ArticleGoogle Scholar
- Gao R, Wang X, Bougaev A, Schooley DC, Tsoukalas LH (2003) Short-term elasticities via intelligent tools for modern power systems. In: Proceedings of 2003 medpower conference, pp 1–6Google Scholar
- Gatsis N, Giannakis GB (2012) Residential load control: distributed scheduling and convergence with lost AMI messages. IEEE Trans Smart Grid 3(2):770–786View ArticleGoogle Scholar
- Ghiassi MD, Zimbra DK, Saidane H (2006) Medium term system load forecasting with a dynamic artificial neural network model. Electr Power Syst Res 76(5):302–316View ArticleGoogle Scholar
- Hoffman T, Scholkopf B, Smola AJ (2008) Kernel methods in machine learning. Ann Stat 33(3):1171–1220View ArticleGoogle Scholar
- Huang SJ, Shih KR (2003) Short-term load forecasting via ARMA model identification including non-Gaussian process considerations. IEEE Trans Power Syst 18(2):673–679View ArticleGoogle Scholar
- ISO New England Historical Data web page. http://www.iso-ne.com. Accessed in March 2015
- Kandil MS, El-Debeiky SM, Hasanien NE (2002) Long-term load forecasting for fast developing utility using a knowledge-based expert system. IEEE Trans Power Syst 17(2):491–496View ArticleGoogle Scholar
- Kim KH, Park JK, Hwang KJ, Kim SH (1995) Implementation of hybrid short-term load forecasting system using artificial neural networks and fuzzy expert systems. IEEE Trans Power Syst 10(3):1534–1539View ArticleGoogle Scholar
- Li Y, Liu, K (2010) Core vector regression with particle swarm optimization algorithm in short term load forecasting. In: Proceedings of the IEEE second international conference on computer modeling and simulation (ICCMS’10), pp 325–329Google Scholar
- Mackay DJC (1998) Introduction to Gaussian processes. In: Bishop CM (ed) Neural networks and machine learning. Springer, BerlinGoogle Scholar
- Rasmussen CE (2006) Gaussian processes for machine learning. MIT Press, CambridgeGoogle Scholar
- Rengcun F, Jianzhong Z, Yongchuan Z, Qing-qing LI, Li LIU (2008) Application of particle swarm optimization based nonlinear grey Bernoulli model in medium-and long-term load forecasting. Power Syst Technol (Beijing) 32(12):60Google Scholar
- Scholkopf B, Smola AJ (2001) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, CambridgeGoogle Scholar
- Tipping ME (2001) Sparse Bayesian learning and the relevance vector machine. J Mach Learn Res 1:211–244Google Scholar
- Tsekouras GJ, Hatziargyriou ND, Dialynas EN (2006) An optimized adaptive neural network for annual midterm energy forecasting. IEEE Trans Power Syst 21(1):385–391View ArticleGoogle Scholar
- Tsekouras GJ, Dialynas EN, Hatziargyriou ND, Kavatza S (2007) A non-linear multivariable regression model for midterm energy forecasting of power systems. Electr Power Syst Res 77(12):1560–1568View ArticleGoogle Scholar
- Tsoukalas LH, Gao R (2008) From smart grids to an energy internet: assumptions, architectures and requirements. In: Proceedings of the third international conference on electric utility deregulation and restructuring and power technologies, pp 94–98Google Scholar
- Tsoukalas LH, Uhrig R (1997) Fuzzy and neural approaches in engineering. Wiley Interscience, New YorkGoogle Scholar
- Williams CKI (2002) Gaussian processes. In: Arbib MA (ed) Handbook of brain theory and neural networks, 2nd edn. MIT Press, Cambridge, pp 466–470Google Scholar
- Xia C, Wang J, McMenemy K (2010) Short, medium and long term load forecasting model and virtual load forecaster based on radial basis function neural networks. Int J Electr Power Energy Syst 32(7):743–750View ArticleGoogle Scholar