Deep learning-based pilot-free channel estimation of UAV-FSO system using variational auto-encoder

Reliable channel estimation is critical for achieving high-speed and energy-efficient communication in Unmanned Aerial Vehicle-Free Space Optical (UAV-FSO) systems, particularly under dynamic impairments such as atmospheric turbulence (AT) and pointing errors (PEs). This paper proposes a pilot-free channel estimation framework based on a Variational Autoencoder (VAE). The system employs Intensity Modulation/Direct Detection (IM/DD) with M -ary one-hot encoded symbols ( M = 16 ). The VAE encodes noisy received signals into a 128-dimensional latent space and reconstructs the transmitted data, effectively learning the joint effects of AT, PEs, and AWGN. Unlike prior works that primarily consider boresight or Gaussian-jitter PEs, this study explicitly incorporates a Nakagami-modeled PE distribution, capturing UAV-induced beam misalignment under mobility, vibration, and turbulence coupling. Simulation results show that the proposed VAE significantly outperforms conventional estimators (LS, MMSE, LMMSE) and deep learning baselines (AE, DNN, CNN) across various turbulence strengths. Under strong turbulence and PEs, the VAE attains nearly two-fold lower MSE compared to CNN and DNN. In addition, evaluation on real turbulence-impaired datasets further validates robustness and generalization. The proposed pilot-free scheme delivers accurate channel estimation, reduced BER, and improved spectral efficiency, making it suitable for real-time adaptive UAV-FSO communication.