In our last post, we took a close look at how Vāyuh.ai's forecasting model, leveraging both Physics and AI, was able to successfully predict the onset of the 2021 Pacific Northwest Heat Wave more than 2 weeks in advance. That's good for one heat wave, but how about heat waves across the years? For a forecasting model to be useful, we're looking for robustness - accuracy across extreme events in the aggregate.
This is particularly important because not every heat (or cold) wave looks the same. They can occur at different times of the year, with different impacts. A heat wave in the winter may, for example, not have much impact on human health. However, it may have an impact on agricultural output. Meanwhile, heat waves in the summer can have dramatic impacts on human health and infrastructure. While climate change increasingly drives heat waves in both scenarios, the atmospheric conditions contributing to a specific event will be context-dependent. A good forecasting model needs to be able to pick up on this.
How does Vāyuh.ai's forecast perform across many extreme events? We took a look at the most extreme events, and evaluated our forecast's performance in these scenarios, comparing it to the physics-based forecasts CFSv2. CFSv2 is a forecast from NOOA and is available daily for longer lead times. We evaluated the forecast quality with a quantity called the Anomaly Centered Correlation (ACC). This value is effectively a correlation (therefore, ranging from -1 to 1) that captures how well a forecast predicts a temperature beyond what history would predict. Therefore, it's a better baseline to use because it forces us to do better than the historical prediction on a given day.
We took a look at the ACC for Vāyuh's forecast and compared it to that of CFSv2's performance as a function of the top percentile of extreme events (Figure 1). For example, the leftmost points in Figure 1 are model performance for the top 1% of extreme temperature events from 2012-2022. For these events, the Vāyuh.ai model outperforms CFSv2 in forecast quality by nearly 10%. These are scenarios such as the 2021 Pacific Northwest heatwave - a 10% improvement for such extreme events translates into a much better forecast.
We can extend this analysis to different subsets of the dataset, considering the top 2%, 3%, etc. extreme events. No matter what portion of extreme events we examine, we see that Vāyuh.ai outperforms the CFSv2 forecast. In the aggregate, Vāyuh.ai's Physics + AI based forecast will capture extreme events better than physics-based models alone.
An analysis of Vāyuh.ai's medium-term weather forecast. In blue is the CFSv2 forecast and in orange is the Vāyuh.ai forecast. On the horizontal axis, we consider the fractions of extremes concerned. On the vertical axis, the anomaly-centred correlation, or a measure of how well the forecasts perform compared to ground truth.
Prior knowledge of such events is hugely useful to a variety of stakeholders in
Agriculture - disruption of harvesting, and planting cycles can dramatically be affected by temperature anomalies. Prior information can greatly help plan for such eventualities. Already, we are seeing trends of farmers harvesting before the sun is up.
Energy - one of the biggest impacts of temperature anomalies is the increased demand for power. This affects asset managers (renewable, gas, and other sources), customers, and traders. Prior knowledge of such events can greatly help with planning a supply purchase strategy.
Supply Chains - a big impact on supply chains happens due to temperature anomalies. Specifically, perishable items such as fisheries, produce, and more.