SegmentAnyTree: A sensor and platform agnostic deep learning model for tree segmentation using laser scanning data

This study focuses on advancing individual tree crown (ITC) segmentation in lidar data, developing a sensor- and platform-agnostic deep learning model transferable across a spectrum of dense laser scanning datasets from drone (ULS), to terrestrial (TLS), and mobile (MLS) laser scanning data. In a field where transferability across different data characteristics has been a longstanding challenge, this research marks a step towards versatile, efficient, and comprehensive 3D forest scene analysis. Central to this study is model performance evaluation based on platform type (ULS vs. MLS) and data density. This involved five distinct scenarios, each integrating different combinations of input training data, including ULS, MLS, and their augmented versions through random subsampling, to assess the model's transferability to varying resolutions and efficacy across different canopy layers. The core of the model, inspired by the PointGroup architecture, is a 3D convolutional neural network (CNN) with dedicated prediction heads for semantic and instance segmentation. The model underwent comprehensive validation on publicly available, machine learning-ready point cloud datasets. Additional analyses assessed model adaptability to different resolutions and performance across canopy layers. Our results reveal that point cloud random subsampling is an effective augmentation strategy and improves model performance and transferability. The model trained using the most aggressive augmentation, including point clouds as sparse as 10 points m⁻², showed best performance and was found to be transferable to sparse lidar data and boosts detection and segmentation of codominant and dominated trees. Notably, the model showed consistent performance for point clouds with densities >50 points m⁻² but exhibited a drop in performance at the sparsest level (10 points m⁻²), mainly due to increased omission rates. Benchmarking against current state-of-the-art methods revealed boosts of up to 20% in the detection rates, indicating the model's superior performance on multiple open benchmark datasets. Further, our experiments also set new performance baselines for the other public datasets. The comparison highlights the model's superior segmentation skill, mainly due to better detection and segmentation of understory trees below the canopy, with reduced computational demands compared to other recent methods. In conclusion, the present study demonstrates that it is indeed feasible to train a sensor-agnostic model that can handle diverse laser scanning data, going beyond current sensor-specific methodologies. Further, our study sets a new baseline for tree segmentation, especially in complex forest structures. By advancing the state-of-the-art in forest lidar analysis, our work also lays the foundation for future innovations in ecological modeling and forest management.