Reinforcement Learning for xURLLC

This Research Topic invites contributions on Reinforcement Learning (RL) for extreme Ultra Reliable and Low Latency Communications (xURLLC).URLLC aim at transmitting data packets with a very low packet loss probability within a very short time delay compared to traditional wireless communications. Targeted use cases can be found e.g. in industry automation, health, transportations or electrical grid management. URLLC have been introduced in 5G as one of its three main families of use cases (together with enhanced Mobile Broadband eMBB and massive Machine Type Communications mMTC) but is also envisioned as a cornerstone service of 6G. In 6G, extreme URLLC (xURLLC) target reliability levels as low as 1-10-7 or 1-10-9 with latencies of less than 1ms. xURLLC raise several challenges in terms of reliability and latency (coding, HARQ, diversity), resource allocation (medium access, scheduling, coexistence with eMBB/mMTC), mobility, scalability and spectrum efficiency, energy consumption and energy efficiency, security and privacy, but also environmental impact. Several 6G enabling technologies like Reconfigurable Intelligent Surfaces (RIS), Non-Orthogonal Multiple Access (NOMA), Mobile Edge Computing (MEC), Integrated Sensing and Communication (ISAC), Unmanned Aerial Vehicles (UAV), evolutions of Multiple Input Multiple Output (MIMO, massive MIMO, cell-free massive MIMO, holographic MIMO), evolutions of medium access (modern random access, grant-free access, Rate Splitting Multiple Access, RSMA) could help achieving the stringent objectives of xURLLC. Specific scenarios in industry automation, health, transportation, vehicle networks, smart grids could require dedicated solutions. In this context, Reinforcement Learning (RL) appears as a powerful theoretical framework to meet xURLLC challenges. The basic paradigm of RL lies in an agent interacting with its environment by sequentially taking actions and obtaining rewards from the environment. Collected data allows the agent to train a policy with the objective to maximize the sum of rewards over a certain horizon. Depending on the scenario, xURLLC pose several RL-specific challenges, e.g., non-stationary environments, multiple agents, reward shaping and credit assignment, scalability, partially observable states, large action spaces, short training and inference delay budgets, computational cost, sim-to-real gap, etc. RL research is very active around subjects like Multi-Agent RL (MARL), offline RL and decision transformers, Multi-Objective RL (MORL), Federated RL (FedRL), safe RL, Transfer Learning, that deserve to be studied to meet xURLLC requirements. Practical implementations and real-world validations are also required to confirm the interest to apply RL to xURLLC. We thus welcome works that bridge theoretical RL advances with deployable xURLLC solutions spanning various use cases like industry automation, healthcare, immersive services, transportation or smart grids. Topic editors: Marceau Coupechoux, Alessandro Chiumento, Philippe GODLEWSKI Source: https://www.frontiersin.org/research-topics/83915/reinforcement-learning-for-xurllc