Predictive modeling of microbial data with interaction effects

Abstract Microbial interactions are of fundamental importance for the functioning and the maintenance of microbial communities. Deciphering these interactions from (time-series) observational data or controlled lab experiments remains a formidable challenge due to their context-dependent nature, such as, e.g., (a)biotic factors, host characteristics, and overall community composition. Complementary to the classical ecological view, recent research advocates an empirical “community-function landscape” framework where an outcome of interest, e.g., a community function, is learned via statistical regression models that include pairwise or higher-order statistical species interaction effects. Here, we adopt the latter viewpoint and present penalized quadratic interaction models that can accommodate all common microbial data types, including microbial presence-absence data, relative (or compositional) abundance data from microbiome surveys, and quantitative (absolute abundance) microbiome data. We propose novel interaction models for compositional data and bring modern statistical techniques such as hierarchical interaction constraints and stability-based model selection to the microbial realm. To illustrate our framework’s versatility, we consider prediction tasks across a wide range of microbial datasets and ecosystems, including butyrate production in model communities in designed experiments and environmental covariate prediction from marine microbiome data. We show improved predictive performance of these interaction models and assess their limits in the presence of extreme data sparsity. On a large-scale gut microbiome cohort data, we identify interaction models that can accurately predict the abundance of antimicrobial resistance genes, enabling novel biological hypotheses about microbial community composition and antimicrobial resistance. Author Summary Microbes live in complex communities where interactions between species shape their function and stability. Understanding these interactions is crucial for predicting how microbial communities respond to environmental changes, medical treatments, or shifts in their host organisms. However, identifying these relationships is challenging because they depend on many factors, including the surrounding environment and community composition. In this study, we introduce a new statistical modeling approach to uncover microbial interactions from different types of data, including presence-absence patterns, relative abundance from microbiome surveys, and absolute abundance measurements. Our method builds on modern statistical techniques to improve accuracy and reliability, even when data are sparse or noisy. We demonstrate the power of our approach by applying it to diverse microbial datasets, from marine ecosystems to gut microbiomes. In one case, we successfully predicted antimicrobial resistance gene abundance based on microbial interactions, opening new avenues for understanding how resistance spreads in microbial communities. By advancing statistical tools for microbiome research, our work provides a new way to explore the hidden relationships between microbes, with potential applications in medicine, environmental science, and biotechnology.