Machine Learning-based Automatic Modulation Classification for 5G-Advanced and 6G Waveforms: Robust Identification Under Realistic Channel Impairments
DOI:
https://doi.org/10.26636/jtit.2026.3.2595Keywords:
5G-Advanced/6G, automatic modulation classification, cognitive radio, compact feature selection, dynamic spectrum access, machine learning, realistic channel impairmentsAbstract
Automatic modulation classification (AMC) for 5G-Advanced and 6G networks must blindly identify waveforms from received signals under realistic channel impairments, enabling cognitive radio dynamic spectrum access and interference avoidance. No prior work has simultaneously applied machine learning to classify all eight leading waveforms (UFMC, GFDM, FBMC, NOMA, OFDM-IM, OTFS, ODDM, and AFDM) under realistic channel impairments, nor quantified the minimum feature set for resource-constrained deployment.
We present a framework that (i) extracts a 38-dimensional feature vector that includes three novel channel-aware characteristics (amplitude fading variance, phase discontinuity, and frequency drift); (ii) benchmarks nine machine learning classifiers, including an FC-MLP deep learning baseline and five feature selection methods, on 201600 signals across twelve channel conditions (nine custom plus three 3GPP TDL profiles) and seven SNR levels, with leakage-free feature selection; and (iii) identifies a compact 10-feature subset validated with Bonferroni-corrected McNemar tests and Wilson confidence intervals.
FC-MLP achieves 99.09% accuracy; ensemble-boost (99.04%) and random forest (99.02%) are statistically equivalent. The 10-feature random forest reaches a score of 98.90% within 0.12 pp of the full feature baseline at a cost that is 74% lower and with a 0.071 ms inference per block. The five-fold cross-validation confirms stability (98.54%, Wilson 95% CI: 98.49%, 98.59%). Per channel accuracy ranges from 98.87% (Rayleigh) to 99.98% (AWGN/Rician); 3GPP TDL-A/B/C profiles confirm transferability to 5G NR. The three channel-aware features yield up to 3.1% gain under double-selective fading and an average overall improvement.
Downloads
References
[1] O.A. Dobre, A. Abdi, Y. Bar-Ness, and W. Su, "Survey of Automatic Modulation Classification Techniques: Classical Approaches and New Trends", IET Communications, vol. 1, pp. 137-156, 2007. DOI: https://doi.org/10.1049/iet-com:20050176
View in Google Scholar
[2] P.C. Sahu et al., "Cloud-enabled Automatic Modulation Classification Using Deep Feature Fusion and Moth-flame Optimized Elm Approach", Scientific Reports, vol. 16, art. no. 1061, 2026. DOI: https://doi.org/10.1038/s41598-025-30753-4
View in Google Scholar
[3] S. Chen and J. Zhao, "The Requirements, Challenges, and Technologies for 5G of Terrestrial Mobile Telecommunication", IEEE Communications Magazine, vol. 52, pp. 36-43, 2014. DOI: https://doi.org/10.1109/MCOM.2014.6815891
View in Google Scholar
[4] P. Li, J. Fan, and J. Wu, "Exploring the Key Technologies and Applications of 6G Wireless Communication Network", iScience, vol. 28, art. no. 112281, 2025. DOI: https://doi.org/10.1016/j.isci.2025.112281
View in Google Scholar
[5] W. Saad, M. Bennis, and M. Chen, "A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems", IEEE Network, vol. 34, pp. 134-142, 2020. DOI: https://doi.org/10.1109/MNET.001.1900287
View in Google Scholar
[6] W. Jiang, B. Han, M.A. Habibi, and H.D. Schotten, "The Road Towards 6G: A Comprehensive Survey", IEEE Open Journal of the Communications Society, vol. 2, pp. 334-366, 2021. DOI: https://doi.org/10.1109/OJCOMS.2021.3057679
View in Google Scholar
[7] L. Zhang et al., "Subband Filtered Multi-carrier Systems for Multi-service Wireless Communications", IEEE Transactions on Wireless Communications, vol. 16, pp. 1893-1907, 2017. DOI: https://doi.org/10.1109/TWC.2017.2656904
View in Google Scholar
[8] R. Hadani et al., "Orthogonal Time Frequency Space Modulation", IEEE Wireless Communications and Networking Conference (WCNC), San Francisco, USA, 2017. DOI: https://doi.org/10.1109/WCNC.2017.7925924
View in Google Scholar
[9] M. Wen, X. Cheng, and L. Yang, Index Modulation for 5G Wireless Communications, Springer, 164 p., 2017. DOI: https://doi.org/10.1007/978-3-319-51355-3
View in Google Scholar
[10] H.B. Chikha, A. Alaerjan, and R. Jabeur, "Automatic Classification of 5G Waveform-modulated Signals Using Deep Residual Networks", Sensors, vol. 25, art. no. 4682, 2025. DOI: https://doi.org/10.3390/s25154682
View in Google Scholar
[11] 3GPP, "Study on Channel Model for Frequencies from 0.5 to 100 GHz", Technical Report TR 38.901, 3rd Generation Partnership Project, 2022.
View in Google Scholar
[12] W.C. Jakes (ed.), Microwave Mobile Communications, Wiley-IEEE Press, 656 p., 1994 (ISBN: 9780780310698). DOI: https://doi.org/10.1109/9780470545287
View in Google Scholar
[13] N. Michailow et al., "Generalized Frequency Division Multiplexing for 5th Generation Cellular Networks", IEEE Transactions on Communications, vol. 62, pp. 3045-3061, 2014. DOI: https://doi.org/10.1109/TCOMM.2014.2345566
View in Google Scholar
[14] M. Matthe, N. Michailow, I. Gaspar, and G. Fettweis, "Influence of Pulse Shaping on Bit Error Rate Performance and out of Band Radiation of Generalized Frequency Division Multiplexing", 2014 IEEE International Conference on Communications Workshops (ICC), Sydney, Australia, 2014. DOI: https://doi.org/10.1109/ICCW.2014.6881170
View in Google Scholar
[15] M. Bellanger et al., "FBMC Physical Layer: A Primer", PHYDYAS, 2010.
View in Google Scholar
[16] M. Besseghier, A.B. Djebbar, and E. Kofidis, "Joint CFO and Highly Frequency Selective Channel Estimation in FBMC/OQAM Systems", Digital Signal Processing, vol. 128, art. no. 103629, 2022. DOI: https://doi.org/10.1016/j.dsp.2022.103629
View in Google Scholar
[17] Y. Liu et al., "Nonorthogonal Multiple Access for 5G and Beyond", Proceedings of the IEEE, vol. 105, pp. 2347-2381, 2017. DOI: https://doi.org/10.1109/JPROC.2017.2768666
View in Google Scholar
[18] Z. Ding et al., "Application of Non-orthogonal Multiple Access in LTE and 5G Networks", IEEE Communications Magazine, vol. 55, pp. 185-191, 2017. DOI: https://doi.org/10.1109/MCOM.2017.1500657CM
View in Google Scholar
[19] M. Besseghier, A. Zouggaret, S. Ghouali, and A.B. Djebbar, "Performance Degradation Analysis of Downlink NOMA Systems Under Carrier Frequency Offsets", IEEE Transactions on Consumer Electronics, vol. 71, pp. 2508-2516, 2025. DOI: https://doi.org/10.1109/TCE.2025.3571676
View in Google Scholar
[20] E. Basar, U. Aygolu, E. Panayirci, and H.V. Poor, "Orthogonal Frequency Division Multiplexing with Index Modulation", IEEE Transactions on Signal Processing, vol. 61, pp. 5536-5549, 2013. DOI: https://doi.org/10.1109/TSP.2013.2279771
View in Google Scholar
[21] E. Basar et al., "Index Modulation Techniques for Next-generation Wireless Networks", IEEE Access, vol. 5, pp. 16693-16746, 2017. DOI: https://doi.org/10.1109/ACCESS.2017.2737528
View in Google Scholar
[22] M. Besseghier, S. Ghouali, and A.B. Djebbar, "Optimized Greedy Detection for OFDM-IM Systems", IEEE Communications Letters, vol. 27, pp. 2034-2037, 2023. DOI: https://doi.org/10.1109/LCOMM.2023.3283032
View in Google Scholar
[23] P. Raviteja, K.T. Phan, Y. Hong, and E. Viterbo, "Interference Cancellation and Iterative Detection for Orthogonal Time Frequency Space Modulation", IEEE Transactions on Wireless Communications, vol. 17, pp. 6501-6515, 2018. DOI: https://doi.org/10.1109/TWC.2018.2860011
View in Google Scholar
[24] G.D. Surabhi, R.M. Augustine, and A. Chockalingam, "On the Diversity of Uncoded OTFS Modulation in Doubly-dispersive Channels", IEEE Transactions on Wireless Communications, vol. 18, pp. 3049-3063, 2019. DOI: https://doi.org/10.1109/TWC.2019.2909205
View in Google Scholar
[25] G.D. Surabhi and A. Chockalingam, "Low-complexity Linear Equalization for OTFS Modulation", IEEE Communications Letters, vol. 24, pp. 330-334, 2020. DOI: https://doi.org/10.1109/LCOMM.2019.2956709
View in Google Scholar
[26] Z. Wei et al., "Orthogonal Time-frequency Space Modulation: A Promising Next-generation Waveform", IEEE Wireless Communications, vol. 28, pp. 136-144, 2021. DOI: https://doi.org/10.1109/MWC.001.2000408
View in Google Scholar
[27] A. Bemani, N. Ksairi, and M. Kountouris, "Affine Frequency Division Multiplexing for Next Generation Wireless Communications", IEEE Transactions on Wireless Communications, vol. 22, pp. 8214-8229, 2023. DOI: https://doi.org/10.1109/TWC.2023.3260906
View in Google Scholar
[28] A. Bouguila, A. Khlifi, and M. Yahia, "Performance Analysis of OFDM, OTFS, and AFDM Modulation Over Doubly Selective Channels", 2025 IEEE International Multi-Conference on Smart Systems & Green Process (IMC-SSGP), Hammamet, Tunisia, 2025,. DOI: https://doi.org/10.1109/IMC-SSGP67001.2025.11473997
View in Google Scholar
[29] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, 800 p., 2016.
View in Google Scholar
[30] Q. McNemar, "Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages", Psychometrika, vol. 12, pp. 153-157, 1947. DOI: https://doi.org/10.1007/BF02295996
View in Google Scholar
[31] E.B. Wilson, "Probable Inference, the Law of Succession, and Statistical Inference", Journal of the American Statistical Association, vol. 22, pp. 209-212, 1927. DOI: https://doi.org/10.1080/01621459.1927.10502953
View in Google Scholar
[32] A. Swami and B.M. Sadler, "Hierarchical Digital Modulation Classification Using Cumulants", IEEE Transactions on Communications, vol. 48, pp. 416-429, 2000. DOI: https://doi.org/10.1109/26.837045
View in Google Scholar
[33] Z. Zhang et al., "Automatic Modulation Classification Using CNN-LSTM Based Dual-stream Structure", IEEE Transactions on Vehicular Technology, vol. 69, pp. 13521-13531, 2020. DOI: https://doi.org/10.1109/TVT.2020.3030018
View in Google Scholar
[34] F. Meng, P. Chen, L. Wu, and X. Wang, "Automatic Modulation Classification: A Deep Learning Enabled Approach", IEEE Transactions on Vehicular Technology, vol. 67, pp. 10760-10772, 2018. DOI: https://doi.org/10.1109/TVT.2018.2868698
View in Google Scholar
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Abdelkader Horch, Mokhtar Besseghier, Samir Ghouali, Abderrahmane Louni

This work is licensed under a Creative Commons Attribution 4.0 International License.