Overview of the ELLIS unit Jena
Artificial intelligence and machine learning (AIML) are playing an increasingly important role for modeling and understanding the diverse and interlinked dynamic processes on Earth. Importantly, not only Earth system science benefits from machine learning, but complex Earth system problems can inspire fundamental research in the AIML domain. As in other scientific domains the major challenges relate to interpretability and the integration of machine learning and domain knowledge.
The goal of the ELLIS Unit Jena is thus to combine fundamental development in machine learning (Figure below, rows) with applied challenges concerning spatio-temporal dynamics in the Earth system (Figure below, columns) for a better understanding of the Earth system and its components. An important aspect here is the integration of knowledge into machine learning methods as appropriate assumptions – this can be qualitative knowledge about causal relationships („causal modeling“) or quantitative knowledge about functional relationships, which can be „cast“ into physical, chemical or biophysiological formulas („hybrid modeling“). Causal inference provides a fruitful ground for ML theory and methods, especially for applications in geosciences.
Causal models are more interpretable by construction and their combination with machine learning constitute a major current research avenue.
A promising approach is to apply causal modeling as basis for explanations, for example, to identify features used for the decision or counterfactuals based on knockoffs.
Complementarily, hybrid modeling aims at combining mechanistic models with machine-learning approaches, allowing for physically more consistent predictions and – scientifically more importantly – non-parametric inference of latent variables.
Overall, we identify at least five major challenges and avenues for the successful development and application of machine learning approaches in the geosciences:
Improving predictive accuracy is important but insufficient. Certainly, interpretability and understanding are crucial in this arena, including visualization of the results for analysis by humans. Interpretability has been identified as a potential weakness of deep neural networks, and achieving it is a current focus in deep learning. The field is still far from achieving self-explanatory models, and from causal discovery from observational data. Yet, we should note that, given their complexity, also modern Earth system models are in practice often not easily traceable back to their assumptions, limiting their interpretability as well.
2. Physical consistency
Machine learning models can fit observations very well, but predictions may be physically inconsistent or implausible, e.g. owing to extrapolation or observational biases. Integration of domain knowledge and achievement of physical consistency by teaching models about the governing physical rules of the Earth system can provide very strong theoretical constraints on top of the observational ones.
3. Complex and uncertain data
New machine learning methods are needed to cope with complex statistics, multiple outputs, different noise sources and high dimensional spaces. New network topologies that not only exploit local neighborhood (even at different scales), but also long-range relationships (e.g., for teleconnections) are urgently needed, but the exact cause-effect topologies are even not clear a priori. Modeling of uncertainties will be helped with Bayesian/probabilistic approaches.
4. Limited labels
Unsupervised density modeling and semi-supervised learning, which combine the few labeled samples with the wealth of unlabeled observations, will help in Earth system modeling.
5. Computational demand
There is a huge technical challenge regarding the high computational cost of current geoscience problems.