Probabilistic forecasting is becoming increasingly important for a wide range of applications,especially for en-ergy systems such as forecasting wind power production.A need for proper evaluation of probabilistic fore...Probabilistic forecasting is becoming increasingly important for a wide range of applications,especially for en-ergy systems such as forecasting wind power production.A need for proper evaluation of probabilistic forecasts follows naturally with this,because evaluation is the key to improving the forecasts.Although plenty of excellent reviews and research papers on probabilistic forecast evaluation already exist,we find that there is a need for an introduction with some practical application.In particular,many forecast scenarios in energy systems are inher-ently multivariate,and while univariate evaluation methods are well understood and documented,only limited and scattered work has been done on their multivariate counterparts.This paper therefore contains a review of a selected set of probabilistic forecast evaluation methods,primarily scoring rules,as well as practical sections that explain how these methods can be calculated and estimated.In three case studies featuring simple autore-gressive models,stochastic differential equations and real wind power data,we implement,apply and discuss the logarithmic score,the continuous ranked probability score and the variogram score for forecasting problems of varying dimension.Finally,the advantages and disadvantages of the three scoring rules are highlighted,and this provides a significant step towards deciding on an evaluation method for a given multivariate forecast scenario including forecast scenarios relevant for energy systems.展开更多
文摘Probabilistic forecasting is becoming increasingly important for a wide range of applications,especially for en-ergy systems such as forecasting wind power production.A need for proper evaluation of probabilistic forecasts follows naturally with this,because evaluation is the key to improving the forecasts.Although plenty of excellent reviews and research papers on probabilistic forecast evaluation already exist,we find that there is a need for an introduction with some practical application.In particular,many forecast scenarios in energy systems are inher-ently multivariate,and while univariate evaluation methods are well understood and documented,only limited and scattered work has been done on their multivariate counterparts.This paper therefore contains a review of a selected set of probabilistic forecast evaluation methods,primarily scoring rules,as well as practical sections that explain how these methods can be calculated and estimated.In three case studies featuring simple autore-gressive models,stochastic differential equations and real wind power data,we implement,apply and discuss the logarithmic score,the continuous ranked probability score and the variogram score for forecasting problems of varying dimension.Finally,the advantages and disadvantages of the three scoring rules are highlighted,and this provides a significant step towards deciding on an evaluation method for a given multivariate forecast scenario including forecast scenarios relevant for energy systems.
