Index of Metrics, Statistical Techniques and Data Processing Tools Included in scores#
Continuous#
Name (Alphabetical order) |
API |
Tutorial |
Reference(s) |
|---|---|---|---|
Additive Bias (Mean Error) |
Mean Error (WWRP/WGNE Joint Working Group on Forecast Verification Research) |
||
Consistent Expectile Score |
|||
Consistent Huber Score |
|||
Consistent Quantile Score |
|||
Flip-Flop Index |
|||
|
|||
|
|||
Isotonic Fit, see Isotonic Regression |
— |
— |
— |
Isotonic Regression (Isotonic Fit, Reliability Diagram) |
de Leeuw et al. (2009); Dimitriadis et al. (2020); Jordan et al. (2020), version 2 |
||
Mean Absolute Error (MAE) |
|||
Mean Elementary Score, see Murphy Score |
— |
— |
— |
Mean Error (Additive Bias) |
Mean Error (WWRP/WGNE Joint Working Group on Forecast Verification Research) |
||
Mean Squared Error (MSE) |
|||
Multiplicative Bias |
Multiplicative bias (WWRP/WGNE Joint Working Group on Forecast Verification Research) |
||
Murphy Score (Mean Elementary Score) |
|||
|
|||
|
Ehm et al. (2016) - Corollary 2 (p.521); Taggart (2022) - Corollary 5.6 |
||
Pearson’s Correlation Coefficient |
|||
Pinball Loss, see Quantile Loss |
— |
— |
— |
Quantile Loss (Quantile Score, Pinball Loss) |
|||
Quantile Score, see Quantile Loss |
— |
— |
— |
Reliability Diagram, see Isotonic Regression |
— |
— |
— |
Root Mean Squared Error (RMSE) |
Probability#
Name (Alphabetical order) |
API |
Tutorial |
Reference(s) |
|---|---|---|---|
Brier Score |
|||
Continuous Ranked Probability Score (CRPS) for Cumulative Distribution Functions (CDFs) |
|||
|
— |
— |
|
|
— |
||
|
|||
|
— |
— |
|
Continuous Ranked Probability Score (CRPS) for Ensembles |
Ferro (2014); Gneiting And Raftery (2007); Zamo and Naveau (2018) |
||
Isotonic Fit, see Isotonic Regression |
— |
— |
— |
Isotonic Regression (Isotonic Fit, Reliability Diagram) |
de Leeuw et al. (2009); Dimitriadis et al. (2020); Jordan et al. (2020), version 2 |
||
Mean Elementary Score, see Murphy Score |
— |
— |
— |
Murphy Score (Mean Elementary Score) |
|||
|
|||
|
Ehm et al. (2016) - Corollary 2 (p.521); Taggart (2022) - Corollary 5.6 |
||
Receiver (Relative) Operating Characteristic (ROC) |
Fawcett and Niculescu-Mizil (2007); Gneiting and Vogel (2022); Hand (2009); Hand and Anagnostopoulos (2013)); Hand and Anagnostopoulos (2023); Pesce et al. (2010) |
||
Reliability Diagram, see Isotonic Regression |
— |
— |
— |
Categorical#
Spatial#
Name (Alphabetical order) |
API |
Tutorial |
Reference(s) |
|---|---|---|---|
Fractions Skill Score (FSS) |
|||
|
|||
|
— |
||
|
Statistical Tests#
Name (Alphabetical order) |
API |
Tutorial |
Reference(s) |
|---|---|---|---|
Diebold Mariano (with the Harvey et al. 1997 and the Hering and Genton 2011 modifications) |
Diebold and Mariano (1995); Harvey et al. (1997); Hering and Genton (2011) |
Processing (tools for preparing data)#
Name (Alphabetical order) |
API |
Utilised For |
|---|---|---|
Binary Discretise |
Receiver (Relative) Operating Characteristic (ROC) |
|
Binary Discretise Proportion |
Flip-Flop Index |
|
Broadcast and Match Not-a-Number (NaN) |
Murphy Score (Mean Elementary Score) |
|
Comparative Discretise |
Receiver (Relative) Operating Characteristic (ROC) |
|
Cumulative Distribution Functions (CDFs) |
||
|
Continuous Ranked Probability Score (CRPS) for CDFs; CRPS CDF Brier Decomposition |
|
|
Adjust Forecast for CRPS |
|
|
Adjust Forecast for CRPS |
|
|
CRPS for CDFs; CRPS CDF Brier Decomposition |
|
|
CRPS for CDFs |
|
|
CRPS for CDFs; CRPS CDF Brier Decomposition |
|
|
Adjust Forecast for CRPS; CRPS CDF Brier Decomposition; CRPS for CDFs |
|
|
CRPS for CDFs; CRPS CDF Brier Decomposition |
|
Isotonic Fit, see Isotonic Regression |
— |
— |
Isotonic Regression (Isotonic Fit, Reliability Diagram) |
See “Isotonic Regression (Isotonic Fit, Reliability Diagram)” entries in Continuous and Probability |
|
Proportion Exceeding |
Flip-Flop Index |
|
Reliability Diagram, see Isotonic Regression |
— |
— |