Big-Data Techniques to Improve Weather Forecasting (NERC EAO Doctoral Training Partnership)

All weather forecasts are wrong, but some can be useful. How to determine which forecasts are useful and which are not is a challenge in modern numerical weather prediction. Constructing useful forecasting models and diagnosing model output is ripe for fresh perspectives.

Because of their small-scales, high-impact weather events such as convective storms and their associated impacts (tornadoes, heavy rain, hail, convective winds, and lightning) cannot be forecast at this time by global forecasting models. As such indirect means are needed to understand how to forecast such events from the models.

The European Centre for Medium-Range Weather Forecasts in Reading, UK, produces a number of different forecast products every day as part of their operational suite. One product is the global ensemble forecasting system. Ensemble modelling takes advantage of the uncertainty in initial conditions and model formulation to produce 51 possible forecasts. Work is needed on how to best use this model output in operational forecasting environments and to understand its strengths and weaknesses, particularly for high-impact weather phenomena such as thunderstorms or windstorms associated with intense cool-season extratropical cyclones.

The approaches will include the following.

1. Generative adversarial networks are implemented by a system of two “competing” neural networks in a zero-sum game framework. One network is trained to generate pseudo-data (e.g., a hypothetical map of sea level pressure) and a second (the discriminator) is trained to detect whether data it is presented are real. The first network improves its ability to produce realistic data and the second improves its ability to detect false data. Applications have focused on the network producing the pseudo-data but opportunities to use the discriminator in meteorology may also exist: (a) another method for assessing the performance of numerical weather prediction model forecasts, perhaps especially for convection-allowing models – in this instance, the generating model would be the numerical weather prediction model rather than a neural network; (b) as in (a), but applied to climate model simulations in order to assess their performance in producing realistic future sensible weather scenarios.

2. Simulated multi-species coevolution within an evolutionary programming context is a means to produce more diverse and skillful deterministic and probabilistic forecasts of severe weather and heavy rainfall.

3. Meteorological analogs based on Deep Learning. Given the proficiency of convolutional neural networks in detecting patterns, this technique may be an ideal way to characterize multiscale meteorological flows. This follows the notion of breaking down intractable problems into tractable pieces. If multiscale flows are broken down into component parts, which pieces flow predictably from one type to the next, and which appear to behave with more randomness? Additionally, such pattern identification might reveal another way of quantifying the weather–climate interface, that is, to what extent are the frequencies of flow anomaly patterns so-identified changing?

The project would be well suited to a student with a quantitative background in meteorology, atmospheric science, physics, mathematics, statistics, computer science, engineering, or a related field. This project is ideal for students who want to be challenged and to develop a range of skills including theory, observations and modeling. Students should be comfortable and confident in computer programming.

The student has much say in the direction of this project. The successful candidate will be expected to spend several weeks each year at ECMWF under co-supervision with the research and forecasting teams. Under the supervision of the expert project team, the student working on this cross-disciplinary project will gain a wide breath of training in meteorology, data-mining, machine-learning, and artificial-intelligence approaches. This project provides an excellent opportunity to work with the best operational forecasting model in the world on cutting-edge science questions at the limit of our predictability. Upon graduation with a PhD, the successful candidate will have gained useful work experience at ECMWF, developing skills that can be used in a wide range of careers afterward. Work in this topic will prepare the student for work in the exciting discipline of weather and climate forecasting, informatics, the financial sector, or other big-data disciplines that would contribute to the data-driven economy.