Editorial: Emerging optimization, learning and signal processing for next-generation wireless communications and networking

1 Introduction

The rapid evolution of wireless networks—driven by surging device connectivity, increasingly heterogeneous application requirements, and growing spectrum scarcity—has brought a synergistic intersection of optimization, machine learning, and signal processing to the center of communications research. As the field advances toward sixth-generation (6G) systems and beyond, these disciplines are proving essential not only for improving network performance, but for rethinking how wireless systems are designed, managed, and adapted in real time.

This special issue in Frontiers in Signal Processing brings together a selection of contributions that explore this intersection from a range of perspectives and methodological angles. The articles collectively reflect a growing consensus that the most effective approaches combine the rigor of model-based signal processing with the adaptability of modern machine and reinforcement learning; such a perspective runs throughout this issue and points toward further directions for future research.

We are grateful to the authors for their thoughtful contributions, to the reviewers for their constructive evaluations, and to the editorial team at Frontiers in Signal Processing for their support throughout this process. We hope that this collection proves a useful and interesting resource for researchers working in (re)shaping the next-generation of wireless communications and networking systems.

2 Contributions in this issue

2.1 Convergence analysis of hyperparameter-free MCC-based channel-estimation for mmWave MIMO systems

Bhatia et al. address the convergence analysis of the hyperparameter-free maximum correntropy criterion (MCC) technique for millimeter wave multiple-input multiple-output (MIMO) channel estimation in the presence of impulsive noise. This technique allows for asymptotic convergence to the squared-error performance of MCC with the ideal kernel width, which is nonetheless sensitive to the tuning of hyperparameters. The study analyzes the steady-state behavior of the hyperparameter-free MCC technique to characterize its error performance while a tight bound on the iterative step size is also developed to ensure statistically robustness for mean-square stability under impulse noise environment. Extensive simulation results validate the accuracy and effectiveness of the analysis and also establish the hyperparameter-free MCC method as a robust and practical channel estimator for MIMO systems subject to impulsive noise.

2.2 MI based beamforming optimization framework for integrated sensing and communication

Pu et al. consider the problem of beamforming optimization in integrated sensing and communication systems that have emerged as a key enabler for the next-generation of mobile communications. Different from traditional formulations that optimize various distinctive performance metrics and fall short of scalability and universality, the study leverages mutual information to develop a unified optimization framework. Specifically, they decompose the system mutual information into sensing and communication components for joint waveform optimization and performance balancing in integrated sensing and communication systems. Two important models, i.e., the time-division model and the general model, are considered and are all subject to the formulation of mutual information maximization under power constraints. Efficient gradient ascent and minimization-majorization algorithms have been developed, respectively, and shown to outperform classical algorithms with better sensing-communication performance tradeoffs.

2.3 Intelligent path selection algorithm for tactical communication networks enhanced by link state awareness

Shen et al. study the emerging communication challenges in next-generation tactical networks, which are a critical part of the modern battlefield. In particular, the authors consider the problem of path selection and routing in highly dynamic networks. To address the limitations of optimization and heuristic routing approaches that arise in such scenarios, the authors propose the use of recurrent graph reinforcement learning (RL) to design path selection strategies that leverage both the spatial and temporal relationships in link states. This algorithm takes advantage of modern deep learning architectures to improve existing deep learning solutions to large scale path planning. The authors demonstrate superior network performance in representative tactical communication scenarios.

2.4 Spectral Momentum Integration: hybrid optimization of frequency and time domain gradients

Huang et al. tackle the classic challenge of algorithm performance in non-convex optimization, but with a novel approach of leveraging both and frequency and time domains in gradient calculation in a method title Spectral Momentum Integration (SMI). While not directly related to communication networks, this paper addresses the relevant challenges in optimization deep neural networks through incorporation of signal processing theory. SMI combines time-domain gradients with their most important frequency domain components to reduce noise variance and improve both inference and convergence times. Empirical results show improved performance of this approach in a language processing use-case, and suggests promise in considering new ways of incorporating signal processing techniques into numerical optimization theory and practice.

2.5 Reinforcement learning, rule-based, or Generative AI: a comparison of model-free Wi-Fi slicing approaches

Rosales and Cavalcanti consider the optimization of emerging technologies in the next-generation of Wi-Fi 6 standards that are a key to state of the art quality of service (QoS) guarantees. Specifically, the authors consider approaches for the challenge of network slicing by comparing the performance of three representative techniques: deep constrained reinforcement learning, heuristic methods, and generative AI. Such comparisons are indeed critical in assessing relative success of various methodologies that are often proposed in network performance optimization problems. Results demonstrate both the performance gains of state augmentation in constrained reinforcement learning as well as the strong performance of LLMs in designing well performing policies in the industry standard ns-3 simulation of Wi-Fi six networks.

2.6 Energy efficiency and system complexity analysis of CNN based hybrid precoding for cell-free massive MIMO under terahertz communication

Abose et al. consider the challenges in cell-free massive MIMO in the emerging terahertz communication (THz) in wireless communications. The authors study the use of convolutional neural networks (CNNs) in hybrid precoding for massive MIMO and its benefits in terms of energy efficiency, adaptability, and robustness to contamination. Its application in the THz band makes the problem particularly challenging due to the fast varying nature of the channel. The application of CNNs in this however domain allow for fast inference and adaptability. Simulations demonstrate the strong performance of the deep learning approach for hybrid precoding.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited and reviewed by: Yonghui Li, The University of Sydney, Australia