The challenge of dealing effectively with large amounts of data is becoming increasingly important across all disciplines, as evidenced by the growing number of initiatives from government and industry tackling data analytics in, for example, healthcare, the digital economy and future cities. In particular, over recent years, variational data assimilation (VDA) methods have become increasingly important tools for analysis and prediction of complex physical phenomena in various fields of science and technology. They typically combine complex mathematical models and a priori statistical information about the underlying problem with observation data collected using instrumental sensors. In most practical cases, such as numerical weather prediction, the vast amount of data involved means that the data structures required are simply too large to be explicitly assembled, stored and manipulated at the finest representation level. Furthermore, the systems typically suffer from significant ill-conditioning arising from the background/forecast error covariance matrix, a problem which gets worse as the density and accuracy of assimilated observations is increased. Additional preconditioning (matrix preprocessing) methods are therefore required to make these problems computationally feasible, typically requiring specialist knowledge of the complex interlinked properties of the underlying VDA system.
The aim of this research network is to bring together experts with diverse experiences in various aspects of preconditioning VDA problems to share expertise, identify key challenges and work together to begin to tackle some of the core issues in this area. In today’s world, the increasing availability of large datasets is leading to a fast growing range of data-driven applications which means that there is a pressing need to face the associated computational challenges. The ultimate goal is to develop a theoretical framework underpinning a range of preconditioners for different VDA problems that can in time be translated from traditional physics-based problems such as weather forecasting to modern applications in areas such as wearable technology and future cities.
This is a two-year project (October 2023 to September 2025) partially funded by an INI network support grant and EPSRC (Ref: EP/V521929/1).
