Robustness of Classical Machine Learning and Deep Learning Models for Automatic Modulation Recognition Under Unseen SNR Conditions

Automatic Modulation Recognition (AMR) is a key enabling function in intelligent wireless receivers, particularly in dynamic and non-cooperative environments. In practice, mismatches between training and operating conditions can degrade AMR performance, especially when the receiver encounters SNR levels that were not observed during model development. This paper evaluates the performance of six AMR classifiers under unseen SNR conditions using the RadioML2018.01A dataset. The study considers four modulation classes, BPSK, QPSK, 64QAM, and FM, and compares four classical Machine Learning (ML) models (linear SVM, KNN, AdaBoost, and Gradient Boosting) with two Deep Learning (DL) models (1D-CNN and LSTM). For the classical ML models, 15 engineered features are extracted, including amplitude-based and higher-order cumulant-based descriptors, whereas the DL models operate directly on normalized I/Q samples. The models are trained on a selected range of SNR values and evaluated on previously unseen SNR levels to assess their generalization capability. Experimental results show that the DL models achieve the highest overall accuracy under unseen-SNR conditions, reaching $81.25 \%$. In contrast, the performance of classical ML models, particularly SVM, KNN, and Gradient Boosting, improves substantially when cumulantbased features are incorporated, although they remain below the DL models overall. These findings provide insight into the relative performance of feature-based classical ML and DL AMR methods in realistic wireless environments.