Performance Analysis of CNN-based Autoencoders Under Generalized Fading Channels

The emergence of \ac{DL} in wireless communications has revolutionized the design of transceivers, enabling end-to-end learning-based systems such as autoencoders. Unlike traditional communication systems that rely on separately optimized blocks for encoding, modulation, and decoding, an autoencoder jointly optimizes the transmitter and receiver, leading to potentially more efficient and adaptive communication. However, the performance of such architectures under practical fading conditions remains an open research question. This paper investigates the \ac{BLER} performance of a \ac{CNN}-based autoencoder under a generalized fading model. Specifically, we adopt the \(\kappa\)-\(\mu\) fading model due to its flexibility in representing various real-world fading scenarios. We also analyze classical modulation schemes' analytical \ac{BLER} under the same fading conditions to establish a meaningful comparison. Through simulations, we demonstrate that the \ac{CNN}-based autoencoder efficiently adapts to different fading environments while maintaining robust performance, showcasing its potential as a viable alternative to conventional communication system designs.