Skill scores can be used to compare deterministic (also known as single-valued or point) forecasts made using different models at different locations and time periods. To compute the skill score, a reference forecasting method is needed. Nonetheless, there is no consensus on the choice of reference method. In this paper, three classes of commonly used references methods, namely, climatology, persistence, and their linear combination, are studied in a day-ahead solar forecasting scenario. Day-ahead global solar irradiance forecasts with an hourly resolution are generated using research-grade data from 32 sites around the globe, over a period of 1 year, in an operational manner. To avoid exaggerating the skill scores, it is generally agreed that the most accurate naïve forecasting method should be chosen as the standard of reference. In this regard, the optimal convex combination of climatology and persistence is highly recommended to be used as the standard of reference for day-ahead solar forecasting.

1.
A. H.
Murphy
and
R. L.
Winkler
, “
A general framework for forecast verification
,”
Mon. Weather Rev.
115
,
1330
1338
(
1987
).
2.
D.
Yang
,
J.
Kleissl
,
C. A.
Gueymard
,
H. T.
Pedro
, and
C. F.
Coimbra
, “
History and trends in solar irradiance and PV power forecasting: A preliminary assessment and review using text mining
,”
Sol. Energy
168
,
60
101
(
2018
).
3.
D.
Yang
, “
A guideline to solar forecasting research practice: Reproducible, operational, probabilistic or physically-based, ensemble, and skill (ROPES)
,”
J. Renewable Sustainable Energy
11
,
022701
(
2019
).
4.
J.
Zhang
,
A.
Florita
,
B.-M.
Hodge
,
S.
Lu
,
H. F.
Hamann
,
V.
Banunarayanan
, and
A. M.
Brockway
, “
A suite of metrics for assessing the performance of solar power forecasting
,”
Sol. Energy
111
,
157
175
(
2015
).
5.
L.
Vallance
,
B.
Charbonnier
,
N.
Paul
,
S.
Dubost
, and
P.
Blanc
, “
Towards a standardized procedure to assess solar forecast accuracy: A new ramp and time alignment metric
,”
Sol. Energy
150
,
408
422
(
2017
).
6.
R.
Marquez
and
C. F. M.
Coimbra
, “
Proposed metric for evaluation of solar forecasting models
,”
J. Sol. Energy Eng.
135
,
011016
(
2013
).
7.
R. C.
Hilborn
, “
Sea gulls, butterflies, and grasshoppers: A brief history of the butterfly effect in nonlinear dynamics
,”
Am. J. Phys.
72
,
425
427
(
2004
).
8.
G.
Williams
,
Chaos Theory Tamed
(
Taylor & Francis
,
1997
).
9.
C.
Persson
,
P.
Bacher
,
T.
Shiga
, and
H.
Madsen
, “
Multi-site solar power forecasting using gradient boosted regression trees
,”
Sol. Energy
150
,
423
436
(
2017
).
10.
M.
David
,
F.
Ramahatana
,
P.
Trombe
, and
P.
Lauret
, “
Probabilistic forecasting of the solar irradiance with recursive ARMA and GARCH models
,”
Sol. Energy
133
,
55
72
(
2016
).
11.
Y. V.
Makarov
,
P. V.
Etingov
,
J.
Ma
,
Z.
Huang
, and
K.
Subbarao
, “
Incorporating uncertainty of wind power generation forecast into power system operation, dispatch, and unit commitment procedures
,”
IEEE Trans. Sustainable Energy
2
,
433
442
(
2011
).
12.
D.
Yang
,
E.
Wu
, and
J.
Kleissl
, “
Operational solar forecasting for the real-time market
,”
Int. J. Forecasting
(in press).
13.
A. H.
Murphy
, “
Skill scores based on the mean square error and their relationships to the correlation coefficient
,”
Mon. Weather Rev.
116
,
2417
2424
(
1988
).
14.
R.
Perez
,
E.
Lorenz
,
S.
Pelland
,
M.
Beauharnois
,
G. V.
Knowe
,
K.
Hemker
,
D.
Heinemann
,
J.
Remund
,
S. C.
Müller
,
W.
Traunmüller
,
G.
Steinmauer
,
D.
Pozo
,
J. A.
Ruiz-Arias
,
V.
Lara-Fanego
,
L.
Ramirez-Santigosa
,
M.
Gaston-Romero
, and
L. M.
Pomares
, “
Comparison of numerical weather prediction solar irradiance forecasts in the US, Canada and Europe
,”
Sol. Energy
94
,
305
326
(
2013
).
15.
H. G.
Beyer
,
J.
Polo Martinez
,
M.
Suri
,
J. L.
Torres
,
E.
Lorenz
,
S. C.
Müller
,
C.
Hoyer-Klick
, and
P.
Ineichen
, “
Benchmarking of radiation products
,”
Technical Report No. 038665
, Mesor Report D.1.1.3 (
2009
).
16.
A. H.
Murphy
, “
Climatology, persistence, and their linear combination as standards of reference in skill scores
,”
Weather Forecasting
7
,
692
698
(
1992
).
17.
Y.
Eissa
,
S. N.
Beegum
,
I.
Gherboudj
,
N.
Chaouch
,
J. A.
Sudairi
,
R. K.
Jones
,
N. A.
Dobayan
, and
H.
Ghedira
, “
Prediction of the day-ahead clear-sky downwelling surface solar irradiances using the REST2 model and WRF-CHIMERE simulations over the Arabian Peninsula
,”
Sol. Energy
162
,
36
44
(
2018
).
18.
X.
Qing
and
Y.
Niu
, “
Hourly day-ahead solar irradiance prediction using weather forecasts by LSTM
,”
Energy
148
,
461
468
(
2018
).
19.
G. M.
Yagli
,
D.
Yang
, and
D.
Srinivasan
, “
Reconciling solar forecasts: Sequential reconciliation
,”
Sol. Energy
179
,
391
397
(
2019
).
20.
D.
Yang
and
Z.
Dong
, “
Operational photovoltaics power forecasting using seasonal time series ensemble
,”
Sol. Energy
166
,
529
541
(
2018
).
21.
A.
Driemel
,
J.
Augustine
,
K.
Behrens
,
S.
Colle
,
C.
Cox
,
E.
Cuevas-Agulló
,
F. M.
Denn
,
T.
Duprat
,
M.
Fukuda
,
H.
Grobe
,
M.
Haeffelin
,
G.
Hodges
,
N.
Hyett
,
O.
Ijima
,
A.
Kallis
,
W.
Knap
,
V.
Kustov
,
C. N.
Long
,
D.
Longenecker
,
A.
Lupi
,
M.
Maturilli
,
M.
Mimouni
,
L.
Ntsangwane
,
H.
Ogihara
,
X.
Olano
,
M.
Olefs
,
M.
Omori
,
L.
Passamani
,
E. B.
Pereira
,
H.
Schmithüsen
,
S.
Schumacher
,
R.
Sieger
,
J.
Tamlyn
,
R.
Vogt
,
L.
Vuilleumier
,
X.
Xia
,
A.
Ohmura
, and
G.
König-Langlo
, “
Baseline Surface Radiation Network (BSRN): structure and data description (1992–2017)
,”
Earth Syst. Sci. Data
10
,
1491
1501
(
2018
).
22.
A.
Roesch
,
M.
Wild
,
A.
Ohmura
,
E. G.
Dutton
,
C. N.
Long
, and
T.
Zhang
, “
Assessment of BSRN radiation records for the computation of monthly means
,”
Atmos. Meas. Tech.
4
,
339
354
(
2011
).
23.
D.
Yang
,
G. M.
Yagli
, and
H.
Quan
, “
Quality control for solar irradiance data
,” in
2018 IEEE Innovative Smart Grid Technologies - Asia (ISGT Asia)
(
2018
), pp.
208
213
.
24.
D.
Yang
, “
SolarData package update v1.1: R functions for easy access of Baseline Surface Radiation Network (BSRN)
,”
Sol. Energy
188
,
970
975
(
2019
).
25.
D.
Yang
, “
SolarData: An R package for easy access of publicly available solar datasets
,”
Sol. Energy
171
,
A3
A12
(
2018
).
26.
M.
Lefèvre
,
A.
Oumbe
,
P.
Blanc
,
B.
Espinar
,
B.
Gschwind
,
Z.
Qu
,
L.
Wald
,
M.
Schroedter-Homscheidt
,
C.
Hoyer-Klick
,
A.
Arola
,
A.
Benedetti
,
J. W.
Kaiser
, and
J.-J.
Morcrette
, “
McClear: A new model estimating downwelling solar radiation at ground level in clear-sky conditions
,”
Atmos. Meas. Tech.
6
,
2403
2418
(
2013
).
27.
X.
Sun
,
J. M.
Bright
,
C. A.
Gueymard
,
B.
Acord
,
P.
Wang
, and
N. A.
Engerer
, “
Worldwide performance assessment of 75 global clear-sky irradiance models using principal component analysis
,”
Renewable Sustainable Energy Rev.
111
,
550
570
(
2019
).
28.
L.
Lundstrom
,
camsRad: Client for CAMS Radiation Service
, R package version 0.3.0. (
2016
).
29.
A. H.
Murphy
, “
What is a good forecast? An essay on the nature of goodness in weather forecasting
,”
Weather Forecasting
8
,
281
293
(
1993
).
30.
D.
Yang
and
R.
Perez
, “
Can we gauge forecasts using satellite-derived solar irradiance?
,”
J. Renewable Sustainable Energy
11
,
023704
(
2019
).
31.
D.
Yang
, “
Post-processing of NWP forecasts using ground or satellite-derived data through kernel conditional density estimation
,”
J. Renewable Sustainable Energy
11
,
026101
(
2019
).
32.
A. H.
Murphy
,
B. G.
Brown
, and
Y.-S.
Chen
, “
Diagnostic verification of temperature forecasts
,”
Weather Forecasting
4
,
485
501
(
1989
).

Supplementary Material

You do not currently have access to this content.