Distribution differences
The -m diff and -m ratio metrics can be used to compare the observed and forecasted aggregation statistics. Any aggregator (i.e. mean, median, min, max, std, variance, etc) can be used.
The bias metric (-m bias) shows the aggregated difference between forecast and observations: aggregator(forecast - observation). The statistic difference metric (-m diff) shows the difference in aggregated statistics: aggregator(forecast) - aggregator(observation). When the aggregator is the mean, the two metrics give the same result. However, other aggregators give different results. For example, the following shows the standard deviation of the bias on the left and the difference of the standard deviation on the right:
verif IFS.nc MEPS.nc -m bias -agg std
verif IFS.nc MEPS.nc -m diff -agg std
The left figure shows that MEPS has lower standard deviation of forecast bias. The right figure shows that the MEPS has forecast variability that is closer to the observed whereas iFS has too little variability.
-m ratio shows the ratio of aggregated statistics, that is aggregator(fcst) / aggregator(obs). To see if the forecasts have the same variance as the observations, use:
verif IFS.nc MEPS.nc -m ratio -agg variance
The frequency diagram (-m freq) shows how often each value is observed and forecasted. -r sets the edges of the intervals to compute frequencies for and -b sets the interval type. For precipitation, a good combination is:
verif IFS.nc MEPS.nc -m freq -r 0,0.00001,1:10 -b =withinsince this creates the bins [0,0.00001), [0.00001,1), [1,2), and so forth. The first bin shows the non-occurrence of precipitation.
