Forecasting insect abundance using time series embedding and machine learning

Implementing insect monitoring systems provides an excellent opportunity to create accurate interventions for insect control. However, selecting the appropriate time for an intervention is still an open question due to the inherent difficulty of implementing on-site monitoring in real-time. A possible solution to enhance decision-making is to apply forecasting methods to predict insect abundance. However, another layer of complexity is added when other covariates are considered in the forecasting, such as climate time series collected along the monitoring system. Multiple combinations of climate time series and their lags can be used to build a forecasting method. Therefore, we propose a new approach to address this problem by combining statistics, machine learning, and time series embedding. We used two datasets containing a time series of aphids and climate data collected weekly in two municipalities in Southern Brazil for eight years. We conduct a simulation study based on a probabilistic autoregressive model with exogenous time series based on Poisson and negative binomial distributions to evaluate the performance of our approach. We pre-processed the data using our newly proposed approach and more straightforward approaches commonly used to train learning algorithms. We evaluate the performance of the selected algorithms by looking at the Pearson correlation and Root Mean Squared Error obtained using one-step-ahead forecasting. Based on Random Forests, Lasso-regularised linear regression, and LightGBM regression algorithms, we showed the feasibility of our novel approach, which yields competitive forecasts while automatically selecting insect abundances, climate time series and their lags to aid forecasting. • We propose a new approach to forecast insect abundance by combining statistics, machine learning, and time series embedding. • We illustrate the approach using simulated and real time series of aphid counts in wheat crops in Southern Brazil, including climate covariates. • The approach yielded a competitive performance while automatically selecting the variables used for a prediction, such as insect abundance, climate covariates and their lags.