Fast Federated Reinforcement Learning-Based Load Balancing in Cell-Free Dynamic Scenarios

The next-generation of mobile communications (6G) introduces the concept of cell-free (CF) architectures, where multiple distributed access points (APs) cooperate to jointly serve users, while still facing several challenges to be addressed. This paper evaluates AP congestion during high-density events and user equipment (UE) association by APs in uplink scenarios. The proposed FRL-M (Federated Reinforcement Learning Modified) scheme, combined with a new UE balancing technique across APs, increases Jain's fairness index and enhances fairness in resource distribution without degrading spectral efficiency (SE). The maximum improvement in Jain's fairness index reaches 91.18%, from LSF [1] to FRL-M, increasing from 0.395 to 0.755. In addition, FRL-M stabilizes approximately \(math 2.8\times10^3\) steps earlier than standard Q-learning, reducing the fronthaul bottleneck and AP overload while accelerating learning convergence. These results demonstrate the potential of federated reinforcement learning-based strategies, combined with balanced UE allocation, to speed up convergence and enable fair resource management in Cell-Free massive Multiple-Input Multiple-Output (CFmMIMO) systems.