An automatic approach for tree species detection and profile estimation of urban street trees using deep learning and Google street view images

Tree species and canopy structural profile (‘tree profile’) are among the most critical environmental factors in determining urban ecosystem services such as climate and air quality control from urban trees. To accurately characterize a tree profile, the tree diameter, height, crown width, and height to the lowest live branch must be all measured, which is an expensive and time-consuming procedure. Recent advances in artificial intelligence aids to efficiently and accurately measure the aforementioned tree profile parameters. This can be particularly helpful if spatially extensive and accurate street-level images provided by Google (‘streetview’) or Kakao (‘roadview’) are utilized. We focused on street trees in Seoul, the capital city of South Korea, and suggested a novel approach to create a tree profile and inventory based on deep learning algorithms. We classified urban tree species using the YOLO (You Only Look Once), one of the most popular deep learning object detection algorithms, which provides an uncomplicated method of creating datasets with custom classes. We further utilized semantic segmentation algorithm and graphical analysis to estimate tree profile parameters by determining the relative location of the interface of tree and ground surface. We evaluated the performance of the model by comparing the estimated tree heights, diameters, and locations from the model with the field measurements as ground truth. The results are promising and demonstrate the potential of the method for creating urban street tree profile inventory. In terms of tree species classification, the method showed the mean average precision (mAP) of 0.564. When we used the ideal tree images, the method also reported the normalized root mean squared error (NRMSE) for the tree height, diameter at breast height (DBH), and distances from the camera to the trees as 0.24, 0.44, and 0.41.