Consistency, in a narrow sense, denotes the alignment between the forecast-optimization strategy and the verification directive. The current recommended deterministic solar forecast verification practice is to report the skill score based on root mean square error (RMSE), which would violate the notion of consistency if the forecasts are optimized under another strategy such as minimizing the mean absolute error (MAE). This paper overcomes such difficulty by proposing a so-called “potential RMSE skill score,” which depends only on (1) the cross-correlation between forecasts and observations and (2) the autocorrelation of observations. While greatly simplifying the calculation, the new skill score does not discriminate inconsistent forecasts as much, e.g., even MAE-optimized forecasts can attain a high RMSE skill score.
REFERENCES
1.
A. H.
Murphy
and
R. L.
Winkler
, “
A general framework for forecast verification
,” Mon. Weather Rev.
115
(7
), 1330
–1338
(1987
).2.
D.
Yang
,
S.
Alessandrini
,
J.
Antonanzas
,
F.
Antonanzas-Torres
,
V.
Badescu
,
H. G.
Beyer
,
R.
Blaga
,
J.
Boland
,
J. M.
Bright
,
C. F. M.
Coimbra
,
M.
David
,
Â.
Frimane
,
C. A.
Gueymard
,
T.
Hong
,
M. J.
Kay
,
S.
Killinger
,
J.
Kleissl
,
P.
Lauret
,
E.
Lorenz
,
D.
van der Meer
,
M.
Paulescu
,
R.
Perez
,
O.
Perpiñán-Lamigueiro
,
I. M.
Peters
,
G.
Reikard
,
D.
Renné
,
Y.-M.
Saint-Drenan
,
Y.
Shuai
,
R.
Urraca
,
H.
Verbois
,
F.
Vignola
,
C.
Voyant
, and
J.
Zhang
, “
Verification of deterministic solar forecasts
,” Sol. Energy
210
, 20
–37
(2020
).3.
T.
Gneiting
, “
Making and evaluating point forecasts
,” J. Am. Stat. Assoc.
106
(494
), 746
–762
(2011
).4.
R. J.
Hyndman
and
A. B.
Koehler
, “
Another look at measures of forecast accuracy
,” Int. J. Forecasting
22
(4
), 679
–688
(2006
).5.
M.
Déqué
, Forecast Verification: A Practitioner's Guide in Atmospheric Science
(
Wiley
, 2011
), pp. 77
–94
.6.
A. H.
Murphy
, “
Skill scores based on the mean square error and their relationships to the correlation coefficient
,” Mon. Weather Rev.
116
(12
), 2417
–2424
(1988
).7.
F. M.
Lopes
,
R.
Conceição
,
H. G.
Silva
,
R.
Salgado
, and
M.
Collares-Pereira
, “
Improved ECMWF forecasts of direct normal irradiance: A tool for better operational strategies in concentrating solar power plants
,” Renewable Energy
163
, 755
–771
(2021
).8.
M.
Paulescu
,
N.
Stefu
,
C.
Dughir
,
A.
Sabadus
,
D.
Calinoiu
, and
V.
Badescu
, “
A simple but accurate two-state model for nowcasting PV power
,” Renewable Energy
195
, 322
–330
(2022
).9.
D.
Anagnostos
,
T.
Schmidt
,
S.
Cavadias
,
D.
Soudris
,
J.
Poortmans
, and
F.
Catthoor
, “
A method for detailed, short-term energy yield forecasting of photovoltaic installations
,” Renewable Energy
130
, 122
–129
(2019
).10.
B.
Sommer
,
P.
Pinson
,
J. W.
Messner
, and
D.
Obst
, “
Online distributed learning in wind power forecasting
,” Int. J. Forecasting
37
(1
), 205
–223
(2021
).11.
A. H.
Murphy
, “
What is a good forecast? An essay on the nature of goodness in weather forecasting
,” Weather Forecasting
8
(2
), 281
–293
(1993
).12.
M. J.
Mayer
and
D.
Yang
, “
Calibration of deterministic NWP forecasts and its impact on verification
,” Int. J. Forecasting
39
(2
), 981
–991
(2023
).13.
S.
Kolassa
, “
Why the ‘best’ point forecast depends on the error or accuracy measure
,” Int. J. Forecasting
36
(1
), 208
–211
(2020
).14.
M. J.
Mayer
and
G.
Gróf
, “
Extensive comparison of physical models for photovoltaic power forecasting
,” Appl. Energy
283
, 116239
(2021
).15.
A. H.
Murphy
and
E. S.
Epstein
, “
Skill scores and correlation coefficients in model verification
,” Mon. Weather Rev.
117
(3
), 572
–582
(1989
).16.
D.
Yang
,
W.
Wang
,
J. M.
Bright
,
C.
Voyant
,
G.
Notton
,
G.
Zhang
, and
C.
Lyu
, “
Verifying operational intra-day solar forecasts from ECMWF and NOAA
,” Sol. Energy
236
, 743
–755
(2022
).17.
D.
Yang
and
J.
Kleissl
, “
Summarizing ensemble NWP forecasts for grid operators: Consistency, elicitability, and economic value
,” Int. J. Forecasting
39
, 1640
–1654
(2022
).18.
M. J.
Mayer
, “
Benefits of physical and machine learning hybridization for photovoltaic power forecasting
,” Renewable Sustainable Energy Rev.
168
, 112772
(2022
).19.
T.
Gneiting
and
A. E.
Raftery
, “
Strictly proper scoring rules, prediction, and estimation
,” J. Am. Stat. Assoc.
102
(477
), 359
–378
(2007
).20.
M. J.
Mayer
and
D.
Yang
, “
Pairing ensemble numerical weather prediction with ensemble physical model chain for probabilistic photovoltaic power forecasting
,” Renewable Sustainable Energy Rev.
175
, 113171
(2023
).21.
E.
Wheatcroft
, “
Interpreting the skill score form of forecast performance metrics
,” Int. J. Forecasting
35
(2
), 573
–579
(2019
).22.
D.
Yang
, “
Making reference solar forecasts with climatology, persistence, and their optimal convex combination
,” Sol. Energy
193
, 981
–985
(2019
).23.
A. H.
Murphy
, “
Climatology, persistence, and their linear combination as standards of reference in skill scores
,” Weather Forecasting
7
(4
), 692
–698
(1992
).24.
T. S.
Nielsen
,
A.
Joensen
,
H.
Madsen
,
L.
Landberg
, and
G.
Giebel
, “
A new reference for wind power forecasting
,” Wind Energy
1
(1
), 29
–34
(1998
).25.
H.
Morf
, “
A validation frame for deterministic solar irradiance forecasts
,” Renewable Energy
180
, 1210
–1221
(2021
).© 2024 Author(s). Published under an exclusive license by AIP Publishing.
2024
Author(s)
You do not currently have access to this content.
