A Q-learning-based intelligent handover framework for Li-Fi/RF-enabled VANET and FANET networks using IEEE 802.11p

1 Introduction

The rapid advancement of intelligent transportation systems (ITS) and unmanned aerial technologies has significantly increased the demand for reliable, low-latency, and high-throughput wireless communication. VANETs and FANETs have emerged as key enabling technologies for supporting safety-critical, real-time, and data-intensive applications, including collision avoidance, traffic management, autonomous driving, disaster monitoring, surveillance, and aerial data relaying (Hartenstein and Laberteaux, 2008; Dressler et al., 2019). Despite their potential, both VANETs and FANETs face significant challenges arising from high mobility, dynamic network topologies, fluctuating link quality, and frequent handovers, all of which can adversely affect Quality of Service (QoS). In VANETs, vehicles communicate through Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) links. High vehicle speeds and constrained road topologies often result in short-lived communication links, leading to frequent handovers between roadside units (RSUs) and access points (Gerla et al., 2014). These handovers are further complicated by varying traffic density, interference, and signal fading, particularly in urban environments. Since many vehicular applications are safety-critical, even minor communication disruptions can lead to significant consequences (Bekmezci et al., 2013).

FANETs introduce additional complexity because unmanned aerial vehicles (UAVs) operate in highly dynamic three-dimensional environments. UAV mobility patterns are often unpredictable, causing rapid topology changes and intermittent connectivity (Zeng et al., 2016). Furthermore, UAVs are constrained by limited onboard battery resources, making energy-efficient communication and handover management essential for prolonging network lifetime and mission success (Fotouhi et al., 2019). Traditional handover mechanisms in both VANETs and FANETs primarily rely on static thresholds based on parameters such as Received Signal Strength (RSS), Signal-to-Noise Ratio (SNR), and hysteresis margins (Abboud et al., 2016). Although computationally efficient, such approaches lack adaptability to rapidly changing network conditions and frequently suffer from unnecessary handovers, increased signaling overhead, ping-pong effects, and higher latency (Chen et al., 2016).

Recent advancements have introduced more sophisticated handover solutions. Software-Defined Networking (SDN)-based approaches improve flexibility through centralized control but often encounter scalability and control-plane overhead issues in highly mobile environments (Sepulcre et al., 2011). Blockchain-based solutions improve security and trust but introduce additional computational and communication delays that may hinder real-time operation (Benslimane et al., 2011). Similarly, optimization and metaheuristic approaches such as genetic algorithms and swarm intelligence provide intelligent decision-making capabilities but may incur high computational complexity and limited adaptability to rapidly changing conditions (Bitam et al., 2015).

Machine learning and reinforcement learning (RL) techniques have recently emerged as promising alternatives. In particular, Q-learning has demonstrated the ability to learn adaptive handover policies without requiring prior knowledge of network dynamics (Motlagh et al., 2017; Abolhasan et al., 2004; Saini et al., 2021). Meanwhile, Light Fidelity (Li-Fi) has gained attention as a complementary communication technology for vehicular environments due to its high bandwidth, low interference, and enhanced security characteristics (Naser et al., 2026; Kharbouche et al., 2025). In FANETs, energy-aware communication and learning-based mobility management techniques have also shown promising results (David and Berndt, 2018), while deep reinforcement learning (DRL) approaches such as deep Q-networks (DQN) have demonstrated improved decision-making capabilities in complex environments (Bennis et al., 2018).

1.1 Research gap and motivation

Although considerable research has been conducted on intelligent handover management, several limitations remain. First, most existing Q-learning-based solutions focus exclusively on VANET environments and do not address the unique mobility and energy constraints associated with FANETs. Second, existing VANET and FANET handover solutions are generally designed independently, resulting in fragmented approaches that cannot be directly applied across heterogeneous vehicular and aerial networks. Furthermore, while Li-Fi has demonstrated significant potential for enhancing throughput and reducing communication latency, its dependence on line-of-sight conditions requires intelligent vertical handover mechanisms capable of dynamically switching between Li-Fi and RF communication technologies. Existing handover approaches rarely consider such hybrid communication environments.

Although DRL techniques can handle large and continuous state spaces, their computational and memory requirements make them challenging to deploy on resource-constrained vehicular and UAV platforms. Consequently, there remains a need for a lightweight, adaptive, and computationally efficient handover framework capable of operating across both VANET and FANET environments while supporting heterogeneous communication technologies. Motivated by these challenges, this work proposes a unified Q-learning-based Intelligent Handover Framework (IHF) that enables adaptive and context-aware handover management in both vehicular and aerial networks.

1.2 Main contributions

The major contributions of this paper are summarized as follows:

• A unified Q-learning-based intelligent handover framework is proposed for both VANET and FANET environments, enabling adaptive handover decision-making within a common reinforcement learning architecture.
• A multi-parameter decision mechanism is developed that jointly considers RSS, network occupancy, mobility, traffic load, network congestion, and residual energy to improve handover reliability and reduce unnecessary handovers.
• A hybrid communication framework is introduced that supports intelligent vertical handovers between Li-Fi and RF technologies in VANETs while enabling energy-aware multi-link operations in FANETs.
• A state normalization and discretization strategy is employed to support efficient tabular Q-learning implementation while maintaining scalability across heterogeneous network environments.
• Extensive simulation-based evaluation using OMNeT++ integrated with SUMO demonstrates significant improvements in handover success rate, throughput, packet delivery ratio (PDR), end-to-end delay, handover latency, energy consumption, and network lifetime compared with conventional RSS-based, heuristic, SDN-based, and existing learning-based approaches.

The remainder of this paper is organized as follows. Section 2 presents a comprehensive review of related work on handover management in VANETs and FANETs. Section 3 describes the problem formulation and objectives of the work. Section 4 details the proposed intelligent handover framework and simulation methodology. Section 5 presents the results and performance analysis. Finally, Section 6 concludes the paper and outlines future research directions.

2 Literature review

Handover management remains a critical challenge in next-generation wireless networks, particularly with the emergence of 5G and beyond systems. Early research primarily focused on traditional signal-based and modular handover frameworks. Kulkarni and Bavarva (2023) provided a detailed survey on various handover technologies in 5G networks using modular handover modules, highlighting their flexibility in supporting diverse requirements. However, these modular designs often suffer from limited adaptability in highly dynamic environments.

Traditional threshold-driven approaches, which rely on predefined RSS and network conditions, frequently result in unnecessary handovers, ping-pong effects, and increased signaling overhead, especially in dense and high-mobility scenarios. To mitigate frequent handovers, Mbulwa et al. (2025) proposed an optimized handover candidate cell list technique in 5G and beyond networks, achieving significant reductions in handover frequency and failures. Despite these improvements, such methods still depend largely on static criteria and lack real-time adaptability to rapidly changing conditions.

With the growing complexity of wireless environments, machine learning techniques have been widely explored. Gandhi and Narmawala (2024) presented a comprehensive survey on machine learning in opportunistic networks, underscoring the potential of adaptive learning approaches for dynamic scenarios. In vehicular ad hoc networks (VANETs), reinforcement learning has shown promise for intelligent decision-making. Kim et al. (2024) developed a reinforcement learning-based resource allocation scheme that enhances data transmission efficiency in VANETs. Savitha and Chinnasamy (2023) introduced an efficient VANET handover scheme using software-defined SDN (SSDN) integrated with the media-independent handover (MIH) framework, demonstrating improved handover performance and reduced latency through centralized control.

The proliferation of UAVs has led to the development of Flying Ad Hoc Networks (FANETs), which introduce additional challenges such as three-dimensional mobility, frequent topology changes, and strict energy constraints. Alshaibani et al. (2022) investigated mobility management of UAVs in ultra-dense heterogeneous networks and emphasized the instability of communication links under high mobility. Shayea et al. (2022) surveyed handover management for drones in future mobile networks and concluded that conventional mechanisms are inadequate, often resulting in high link failure rates and packet loss.

To address mobility issues in multi-UAV systems, reinforcement learning-based solutions have gained traction. Bai et al. (2023) surveyed reinforcement learning approaches toward autonomous multi-UAV wireless networks, highlighting improvements in connectivity, adaptability, and network lifetime. Rahmani et al. (2024) proposed a Q-learning-based routing approach with cluster formation (QRCF) for FANETs. Zhang et al. (2024) introduced an anticipatory Q-learning-based routing protocol (AQR-FANET) that significantly reduces delay and improves throughput. Gu et al. (2024) developed an environment-aware adaptive reinforcement learning-based routing scheme for VANETs, achieving higher packet delivery ratios under varying traffic conditions.

Metaheuristic and hybrid techniques have also been explored for handover optimization. Oladosu et al. (2023) presented an intelligent metaheuristic-based handover algorithm for VANETs. For broader public safety applications, Kumbhar et al. (2017) surveyed legacy and emerging technologies in public safety communications. Additionally, Duo et al. (2020) proposed an SDN-based handover scheme for cellular/IEEE 802.11p hybrid vehicular networks, demonstrating better adaptability in dynamic conditions. However, centralized SDN architectures can suffer from scalability limitations and control-plane latency in large-scale deployments. Tandon et al. (2024) introduced the D-BLAC dual blockchain-based decentralized architecture for authentication and communication in VANETs, enhancing security and trust, albeit with added computational overhead.

DRL has recently emerged to handle complex state spaces more effectively. Meer et al. (2025) proposed an explainable AI framework using deep Q-networks (DQN) for UAV mobility management, aiming to minimize unnecessary handovers while improving decision transparency. Sophia Mary and Pradeep Kumar (2025) modeled a dual-battery FANET system using multi-agent DRL (MADRL) for energy optimization. Neelima et al. (2026) developed the DRLO-VANET framework, a DRL-based offloading scheme that achieves low-latency and energy-efficient task execution in VANETs. Ramesh et al. (2025) integrated deep reinforcement transfer learning with explainable AI to enhance ITS in VANETs, reducing retraining overhead and improving adaptability.

The integration of heterogeneous communication technologies has further expanded handover research. Besjedica et al. (2023) surveyed the evolution of hybrid LiFi–WiFi networks. Ji et al. (2024) discussed the design, performance, and challenges of hybrid RF-VLC vehicular communication networks, noting substantial gains in data rate and reliability. Chen and Li (2024) proposed a hybrid handover scheme for vehicular VLC/RF communication networks, showing improved performance particularly under line-of-sight conditions. However, VLC links remain highly sensitive to blockages and mobility, requiring intelligent adaptation strategies. Jahangeer et al. (2024) proposed a hybrid Cat-Swarm Optimization (CSO)-based vertical handoff mechanism for VANETs. Their approach demonstrated improved network selection accuracy and reduced handover failures. However, the optimization process introduces additional computational complexity and lacks online learning capabilities for rapidly changing vehicular environments. Naskath (2023) combined Cat Swarm Optimization and TOPSIS for multicriteria network selection in VANETs. The proposed approach improved handover decision accuracy by considering multiple QoS parameters. Nevertheless, the scheme relies on optimization-based decision-making and does not continuously learn from changing network conditions. The Table 1 below summarize the different handover approaches addressed in the literature review.

Table 1. Comparative analysis of existing handover management approaches.

Despite the significant progress reflected in the above works, several critical gaps remain. Most studies address VANETs or FANETs independently rather than providing a unified framework for heterogeneous ground–aerial networks. Few works jointly consider hybrid RF/VLC communication alongside energy-aware decision-making under dynamic mobility and traffic conditions. Moreover, many reinforcement learning and DRL approaches, while effective, either lack scalability or impose high computational demands on resource-constrained UAV platforms.

Therefore, there is a pressing need for a unified, lightweight, and adaptive intelligent handover framework that seamlessly operates across VANET and FANET environments, intelligently manages hybrid RF/VLC interfaces, and incorporates energy awareness. The proposed Q-learning-based intelligent handover mechanism addresses these limitations by jointly considering mobility patterns, traffic dynamics, heterogeneous link quality, and residual energy in the decision process.

3 Problem formulation and objectives

Handover management in heterogeneous VANET–FANET environments is a complex optimization problem characterized by highly dynamic network conditions, multi-dimensional mobility, and stringent energy constraints. Existing approaches are reactive, limited to single-network domains, or computationally intensive, making them unsuitable for real-time deployment in large-scale, high-mobility scenarios. Therefore, an adaptive and lightweight decision-making framework is required to jointly optimize communication reliability, latency, and energy efficiency under continuously changing network states. The integration of VANETs and FANETs enables cooperative networking, where UAVs can function as aerial relays or temporary access points to enhance coverage and reliability. In such a heterogeneous setup, nodes may perform both horizontal handovers and vertical handovers.

To formally model this process, each node is treated as an intelligent agent interacting with the environment over discrete time steps. At each decision epoch, the agent observes the current state s_t, selects an action a_t, and receives a reward R_t reflecting communication performance and energy efficiency. The environment then transitions to a new state s_{t + 1}, and the process continues iteratively. The objective of the handover decision problem is to learn an optimal policy that maximizes the long-term cumulative reward. This formulation captures the inherent trade-off between maintaining stable connectivity and avoiding unnecessary handovers. Frequent handovers may improve instantaneous link quality but increase delay and signaling overhead, while delayed handovers may degrade throughput and reliability. In FANETs, this trade-off is further influenced by energy constraints, as excessive handovers reduce UAV operational lifetime (Munjal et al., 2024).

The problem aims to maximize network performance metrics such as throughput and packet delivery ratio while minimizing delay, handover failures, and energy consumption. Due to the stochastic and highly dynamic nature of the environment, traditional optimization techniques are insufficient, particularly in the absence of accurate mobility and channel models. To address these challenges, the problem is modeled as a Markov Decision Process (MDP) (Qazi, 2024) defined by the tuple (S, A, P, R, γ), where S represents the state space, A denotes the action space, P is the state transition probability, R is the reward function and γ ∈ [0,1] is the discount factor. At each time step t, the agent observes a state as in Equation 1.Where, SS_t = Signal strength, NO_t = Network occupancy, VM_ot = Mobility, TL_t = Traffic load, E_t = Residual energy (for UAVs). The action space is defined as: A = { a 0 , a ₁ , a ₂ , … , a n }where a₀ refers to Stay connected and aᵢ refers to Handover to candidate network i.

st = { S S t , N O t , V M ot , T L t , E t } (1)

Although VANET and FANET nodes exhibit different mobility regimes, the proposed framework employs a common state representation by normalizing each continuous variable relative to its domain-specific operating range (e.g., node mobility is expressed as a fraction of the maximum vehicle speed in VANET or maximum UAV speed in FANET) before discretization. This normalization allows a structurally identical Q-table to be reused across both environments while preserving the distinct dynamics of each domain within the discretized state bins.

Given this formulation, reinforcement learning techniques can be applied to learn optimal handover policies through interaction with the environment. Among these, Q-learning is particularly suitable due to its model-free nature, low computational complexity, and ability to operate in real-time on resource-constrained nodes (Zhong et al., 2024).

4 Proposed intelligent handover framework

The proposed framework introduces an intelligent handover framework (IHF) protocol designed for heterogeneous VANET and FANET environments. The system enables each mobile node, including vehicles and UAVs, to act as an autonomous agent capable of making adaptive handover decisions based on real-time network conditions.

The framework integrates multi-interface communication (Li-Fi, Wi-Fi, RF) with reinforcement learning to dynamically select the optimal communication link. Unlike conventional threshold-based or rule-based approaches, the proposed model continuously learns from the environment and improves decision-making over time. Each node periodically observes its environment, evaluates available candidate networks, and selects an optimal action, i.e., stay or handover to maximize long-term communication performance and energy efficiency.

The handover decision process is formulated using MDP, at each discrete time step, the agent observes the current state of the network, selects an action according to its policy, and receives a reward that reflects the quality of the decision. The objective is to learn an optimal policyΠ ∗ that maximizes the expected cumulative discounted reward over time as in Equation 2.

Π ∗ = arg max Π E [ ∑ t = 0 ∞ γ t R t ] (2)

Although Equation 2 represents the standard infinite-horizon Q-learning formulation, practical implementation was performed over finite simulation episodes of 1,000 s duration. Therefore, cumulative rewards were computed over the finite decision horizon observed during each simulation run.

The reward function is defined as in Equation 3where, PDR_t, Th_t, Delay_t, HF_t, and EC_t represent the Packet Delivery Ratio, Throughput, End-to-end delay, Handover failures, and Energy consumption, respectively. α, β, γ₁, γ₂, and γ₃ are the corresponding weighting factors.

R t = α ⋅ PD R t + β ⋅ T h t − γ ₁ ⋅ Dela y t − γ ₂ ⋅ H F t − γ ₃ ⋅ E C t (3)

The weighting coefficients used in Equation 3 were selected through empirical tuning during preliminary simulation experiments. The final values were chosen to balance handover reliability, throughput performance, delay minimization, and energy efficiency while preventing any individual metric from dominating the reward signal. Specifically, the coefficients were set to α = 0.30, β = 0.25, γ₁ = 0.20, γ₂ = 0.15, and γ₃ = 0.10.

Prior to reward computation, all performance metrics were normalized to the range [0,1] to ensure comparability among variables with different units and magnitudes. Positive reward terms, such as packet delivery ratio and throughput, were maximized, whereas delay, handover failures, and energy consumption were incorporated as penalty terms. The selected weighting scheme prioritizes handover reliability and packet delivery while maintaining balanced consideration of network performance and energy efficiency.

4.1 System architecture

The proposed architecture is composed of several interconnected functional modules that collectively enable intelligent and adaptive handover decisions in heterogeneous VANETs and FANETs as shown in Figure 1. The process begins with the sensing module, which continuously gathers real-time network and node parameters such as signal strength, network occupancy, mobility patterns, traffic load, and residual energy levels. These raw inputs are then processed by the state representation module, which encodes the collected information into a structured and discrete state vector suitable for machine learning. This state is fed into the decision engine, implemented as a Q-learning agent, which uses an adaptive Q-table to evaluate possible actions and select the most optimal handover decision based on learned experience and current network conditions. Once a decision is made, the handover execution module carries out the appropriate action, whether it is a horizontal handover or a vertical handover. Finally, the reward evaluation module assesses the effectiveness of the action by computing a reward based on key QoS metrics, including throughput, PDR and energy consumption. This reward is then used to update the Q-learning model, enabling continuous learning and improvement in decision-making over time.

Table 2 below compares the proposed Intelligent Handover Framework (IHF) with representative reinforcement learning-based handover approaches reported in the literature. Traditional threshold-based schemes such as RSS-based handover rely on predefined signal strength limits and are unable to adapt to dynamic network conditions. Existing tabular Q-learning approaches, including Zhong et al. (2024) and AQR-FANET (Zhang et al., 2024), improve decision-making through experience-driven learning but are generally designed for specific network scenarios and do not explicitly support both VANET and FANET environments within a common framework. DRL approaches such as DQN (Meer et al., 2025) address the scalability limitations of tabular Q-learning by approximating the action-value function using deep neural networks. However, these methods require additional computational resources, replay memory mechanisms, and longer training times, which may limit their applicability in resource-constrained vehicular and UAV platforms.

Table 2. Comparison of reinforcement learning-based handover approaches.

The proposed IHF adopts a lightweight tabular Q-learning architecture combined with state normalization and discretization techniques, enabling operation across both VANET and FANET environments while maintaining low computational complexity. Although DRL-based methods may offer advantages for very large state spaces, the proposed framework demonstrates that significant improvements in throughput, packet delivery ratio, handover latency, energy consumption, and network lifetime can be achieved without the overhead associated with deep neural network training. This balance between performance and implementation complexity makes the proposed framework suitable for real-time deployment in heterogeneous vehicular and aerial communication environments.

The proposed energy model considers communication-related energy consumption associated with transmission, reception, and handover operations. UAV propulsion energy was not included because the objective of this work is to evaluate communication-aware handover performance. UAVs were assumed to operate in quasi-hovering mode during communication intervals. Although propulsion energy represents a significant component of total UAV power consumption, incorporating detailed flight dynamics is beyond the scope of the present study and remains a direction for future work.

4.2 IHF strategy

The proposed method employs model-free Q-learning, where each node iteratively updates its knowledge through continuous interaction with the environment. At each decision epoch, the agent observes the current state s_t and selects an action at using an ε-greedy policy, balancing exploration and exploitation. The selected action, which corresponds to either maintaining the current connection or performing a handover, is then executed. Following this, the agent receives a reward R_t that reflects the quality of the decision in terms of communication performance and energy efficiency. Based on the observed reward and the resulting next state, the Q-values are updated to improve future decision-making.

The Q-value update rule is given by Equation 4:Where, α is the learning rate and γ is the discount factor.

Q ( s t , a t ) = Q ( s t , a t ) + α [ R t + γ max a Q ( s { t + 1 } , a ) − Q ( s t , a t ) ] (4)

The proposed Q-learning agent employs an ε-greedy exploration strategy to balance exploration and exploitation during the learning process. The learning rate was set to α = 0.10, allowing gradual updates of Q-values based on newly observed rewards, while the discount factor was fixed at γ = 0.90 to provide appropriate consideration of future rewards during decision making. The value α = 0.1 provided the best trade-off between convergence speed and stability and was therefore adopted in all experiments. Exploration was initially encouraged by setting ε = 1.0, enabling random action selection during the early learning stages. To facilitate convergence toward an optimal policy, ε was gradually reduced using an exponential decay schedule with a decay factor of 0.995 per episode until reaching a minimum value of ε = 0.05. These parameter values were determined through preliminary convergence experiments and were found to provide stable learning behavior, fast convergence, and consistent handover performance across both VANET and FANET scenarios. The proposed strategy performs handover decisions through a periodic monitoring and learning process. Candidate networks are discovered through periodic beacon broadcasts transmitted by neighboring Roadside Units (RSUs), access points, Li-Fi transmitters, and UAV communication nodes. These beacon messages contain information regarding signal quality, network occupancy, available bandwidth, and network identification.

At each decision epoch, the serving node collects relevant state information from the communication environment. The state vector consists of SS, NO, Mo, TL, and E in FANET scenarios. Signal quality measurements are obtained from PHY-layer statistics such as RSS and SNR, while network occupancy and traffic load information are obtained from MAC-layer monitoring mechanisms. Mobility information is derived from the current velocity and position updates provided by SUMO (VANET) and UAV mobility models (FANET). The handover decision process operates at fixed decision intervals of 1 s. During each interval, the current state is evaluated and provided as input to the Q-learning agent. Based on the learned Q-values and the ε-greedy action-selection policy, the agent selects either to remain connected to the current network or initiate a handover to one of the available candidate networks.

A handover is executed when the selected action differs from the currently associated network and the target network satisfies the minimum connectivity requirements. Upon successful handover completion, the resulting network performance metrics are observed and used to calculate the reward value. The Q-table is then updated according to the Q-learning update rule, enabling continuous adaptation to changing network conditions. This periodic decision-making process allows the proposed framework to dynamically adapt to variations in mobility, traffic conditions, network congestion, link quality, and energy availability, thereby improving handover reliability and reducing unnecessary handover events.

4.2.1 State space discretization and scalability analysis

The proposed Q-learning framework utilizes five state variables, and each continuous state variable is discretized into a fixed number of bins: signal strength (SS) into 5 levels, network occupancy (NO) into 4 levels, mobility (Mo) into 5 levels, traffic load (TL) into 4 levels, and residual energy (E, FANET only) into 5 levels. This discretization yields a state space of |S| = 5 × 4 × 5 × 4 = 400 states for VANET (where energy is not applicable) and |S| = 5 × 4 × 5 × 4 × 5 = 2,000 states for FANET. With |A| = 4 candidate actions (stay connected plus up to three handover targets), the resulting Q-table contains at most 1,600 entries for VANET and 8,000 entries for FANET. These table sizes remain computationally tractable and require only modest memory resources, making them suitable for real-time deployment in vehicular and aerial nodes. The Q-table size depends on the discretized state-action space rather than the total number of network nodes. Consequently, increasing network density from 20 to 100 nodes does not increase the size of an individual node’s Q-table. Each node maintains an independent Q-table of fixed size, and scalability is therefore primarily influenced by convergence time rather than memory growth.

To evaluate scalability, simulations were conducted under varying network densities ranging from 20 to 100 nodes in both VANET and FANET environments. The results indicate that the proposed framework maintains stable performance with increasing network density, achieving consistent throughput, packet delivery ratio, and handover success rate while incurring only a marginal increase in convergence time. Since each node performs local Q-table updates independently, memory consumption remains constant regardless of network size. Furthermore, Q-table lookup and update operations involve simple table access procedures, resulting in negligible computational overhead and enabling real-time handover decision-making even in dense network scenarios.

However, finer-grained discretization would significantly increase the state space. For example, increasing each variable to ten discrete levels would result in substantially larger Q-tables, potentially affecting convergence speed, storage requirements, and learning efficiency. This limitation motivates future research on function approximation, state aggregation, and DRL techniques capable of handling continuous or high-dimensional state spaces more efficiently.

4.3 Energy-aware multi-mode operation for FANET

To enhance UAV network lifetime, the proposed framework incorporates a multi-mode energy management mechanism in which UAV nodes dynamically operate in different energy states, namely active, sleep, hibernate, and wake-up modes. In the active mode, the UAV performs full communication and data transmission, whereas in the sleep mode, its activity is reduced to conserve energy. The hibernate mode represents a minimal energy consumption state with limited functionality, and the wake-up mode facilitates the transition back to active operation when required. The selection of these operating modes is performed adaptively based on real-time conditions, including residual energy levels, traffic demand, and overall network status. This dynamic mode switching significantly reduces unnecessary energy consumption and prolongs the operational lifetime of UAV nodes without compromising network performance as shown in Equation 5–8:

Let the total energy consumption of a UAV over a time horizon T be defined as:

E total = ∑ t = 1 T P m ( t ) Δt (5)

Where, P_*m(t)* denotes the power consumption corresponding to the operational mode m(t) ∈ {A, S, H, W} at time t; A, S, H, W represent Active, Sleep, Hibernate, and Wake-up modes, respectively and Δt is the duration of each time slot. The power consumption levels satisfy: P_a > P_W > P_s > P_H. HereP_a, P_s, P_H, and P_W denote power in active, sleep, hibernate, and wake-up modes, respectively.

The residual energy of a UAV at time t is updated as:where, Eᵗ represents the remaining battery energy at time t. The operational mode is selected based on system conditionswhere, Eᵗ: residual energy, TL^t: traffic load, NO^t: network occupancy. To incorporate energy efficiency into decision-making, an energy penalty term is included in the reward function and it is represented as follows:Where, λ is the energy weighting factor, E_max is the maximum battery capacity. The objective is to minimize energy consumption, min ∑ t = 1 T P m ( t ) while maintaining performance subject to Eᵗ > E_min, ∀t.

E t = E t − 1 − P m ( t ) Δt (6)

m ( t ) = f ( E t , , T L t , , N O t ) (7)

R t ( E ) = − λ P m ( t ) E max (8)

The Active Mode power consumption was set to PA = 2.5 W, representing normal communication, routing, and handover operations. The Sleep Mode power consumption was set to PS = 0.8 W to model reduced activity periods. The Hibernate Mode power consumption was set to PH = 0.2 W to represent deep energy-saving operation during extended inactivity. The Wake-Up Mode power consumption was set to PW = 0.5 W, accounting for the energy required to transition from low-power states back to Active Mode. These values were adopted to evaluate the effectiveness of the proposed energy-aware handover mechanism in FANET environments and the process is explained in Algorithm [1]. Algorithm 1: Proposed IHF scheme.Input:State space S Action space A Learning rate α Discount factor γ Exploration rate ε Output: Initialize Q(s, a) arbitrarily for all states s ∈ S and actions a ∈ A For each episode do: Initialize state s While termination condition not reached do: Generate random number r ∈ [0,1] If r < ε then Select random action a (Exploration) Else Select action a = argmax Q(s, a) (Exploitation) End If Execute action a Observe nextstate s' and reward R Update Q-value: Q(s,a) ← Q(s,a) + α [R + γ max Q(s',a') − Q(s,a)] Update state: s ← s' End While End For Return optimal policy π* = argmax Q(s,a)

4.4 Unified state representation and mobility normalization

Although VANETs and FANETs operate under different mobility conditions, the proposed framework employs a common state representation through normalization of all continuous state variables prior to discretization. This enables the same Q-learning structure to be applied across both environments while preserving their domain-specific characteristics. For the mobility parameter, raw speed values are transformed into dimensionless normalized values according to: Mv = V_vehicle/V_{vehicle, max} for VANETs and Mu = V_uav/V_{uav, max} for FANETs, where V_{vehicle, max} represents the maximum vehicle speed (120 km/h) and V_{uav, max} represents the maximum UAV speed (110 m/s). Consequently, both mobility variables are mapped into the interval: 0 ≤ M ≤ 1.

The normalized mobility values are subsequently discretized into three mobility states: low mobility: 0 ≤ M < 0.33, medium mobility: 0.33 ≤ M < 0.66, high mobility: 0.66 ≤ M ≤ 1. By transforming mobility into a dimensionless quantity before discretization, VANET and FANET nodes can share an identical state representation despite operating in fundamentally different physical environments. Similar normalization is applied to signal strength, network occupancy, traffic load, and residual energy parameters. Therefore, the proposed framework utilizes a common Q-table structure while preserving environment-specific dynamics through parameter normalization.

4.5 Workflow of proposed system

The overall workflow of the proposed Q-learning-based intelligent handover framework begins with the initialization of network nodes, including both vehicles in VANETs and UAVs in FANETs. Once initialized, the system continuously collects real-time network parameters such as RSS, network occupancy, node mobility or velocity, traffic load, and residual energy. These parameters are then processed to construct an appropriate state representation, which serves as the input for the learning model. Based on the current state, the Q-learning decision policy typically implemented using a α-greedy strategy which selects the most suitable action, such as maintaining the current connection, performing a horizontal handover within the same technology, or executing a vertical handover across different communication technologies. After the action is selected, the handover decision is executed in the network. The system then evaluates the outcome by calculating a reward using key QoS metrics, including throughput, PDR, delay, handover overhead, and energy consumption. This reward, along with the observed next state, is used to update the Q-table, enabling the model to learn from its experience. The entire process is repeated iteratively, allowing the system to continuously adapt and converge toward an optimal handover policy over time as represented in Figure 2.

4.6 Simulation setup and performance metrics

To evaluate the effectiveness of the proposed IHF, extensive simulations were conducted using OMNeT++ integrated with the SUMO tool. This integration provides a high-fidelity co-simulation environment, enabling realistic vehicular mobility modeling through SUMO and accurate protocol stack simulation including hybrid Li-Fi and RF communication through OMNeT++ using the INET framework. Each simulation scenario was executed 30 times using independent random seeds. The reported performance values correspond to the arithmetic mean across all runs. The associated 95% confidence intervals were computed using the standard confidence interval formulation: CI = x̄ ± 1.96(s/√n), where x̄ denotes the sample mean, s represents the sample standard deviation, and n = 30 denotes the number of simulations runs. Due to space limitations and figure readability considerations, confidence interval values are reported within the text rather than being explicitly displayed in the figures. For FANET scenarios, the same simulation platform was extended with custom three-dimensional mobility models to capture aerial network dynamics. Two primary simulation scenarios were considered:

4.6.1 VANET scenario

Vehicles move along realistic urban and highway road topologies generated by SUMO. Vehicle speeds range from 10 to 120 km/h, representing typical traffic conditions. Each vehicle is equipped with dual communication interfaces supporting Li-Fi for high-bandwidth line-of-sight communication and IEEE 802.11p-based RF communication for broader coverage. Roadside Units (RSUs) are strategically deployed along road segments to enable both horizontal and vertical handovers, ensuring continuous connectivity. The Li-Fi interface is modeled as a short-range line-of-sight (LoS) communication technology with higher data rates than conventional RF links. In the present study, Li-Fi connectivity is assumed to be available whenever a vehicle remains within the communication range of a Li-Fi-enabled RSU and a direct LoS path exists between the transmitter and receiver. To simplify the simulation environment, explicit modeling of dynamic blockage events, photodetector field-of-view (FoV) constraints, and detailed optical channel impairments was not incorporated. Instead, Li-Fi availability was represented through communication range and LoS conditions, while RF connectivity was maintained as a complementary fallback mechanism whenever Li-Fi connectivity became unavailable.

Although this simplified model does not capture all practical VLC deployment challenges, it enables evaluation of the proposed handover decision framework under heterogeneous Li-Fi/RF communication environments while maintaining manageable simulation complexity.

4.6.2 FANET scenario

A fleet of UAVs operates in a three-dimensional environment using custom mobility models based on random waypoint and Gauss–Markov processes. UAV velocities vary between 10 m/s and 110 m/s across different altitudes, reflecting realistic aerial mobility patterns. Each UAV is constrained by limited battery capacity, and the energy consumption model incorporates communication overhead, mobility dynamics, and handover operations.

The proposed framework is implemented at the network layer, where each vehicle and UAV functions as an autonomous learning agent. Each simulation is executed for a sufficiently long duration to ensure convergence of the learning process.

To ensure statistical reliability, all results were conducted on independent simulation runs with different random seeds, and 95% confidence intervals are computed. Furthermore, the framework is evaluated under varying network scales for 20–100 nodes, multiple traffic conditions ranging from low, medium, and high, and diverse mobility patterns to assess scalability and robustness. The simulation parameters are summarized in Table 3 and the simulation screenshot as captured in real time is shown in Figure 3.

Table 3. Simulation parameters for VANET and FANET scenarios.

4.6.3 Performance metrics

The performance of the proposed framework is evaluated using a comprehensive set of metrics that capture both communication efficiency and resource utilization: Handover Success Rate (%): Ratio of successful handovers to the total number of handover attempts, indicating the reliability of the handover mechanism, Average Throughput (Mbps): Total rate of successfully delivered data, reflecting overall network capacity.(PDR, %): Ratio of packets successfully received to packets transmitted, representing communication reliability. End-to-End Delay (ms): Average time required for a packet to travel from source to destination, including transmission, propagation, queuing, and processing delays. Handover Latency (ms): Duration of communication interruption during a handover event, measuring the seamlessness of connectivity transitions. Energy Consumption per Packet (J): Average energy consumed to successfully deliver a packet, particularly relevant for energy-constrained FANET scenarios. Network Lifetime (minutes): Time until the first UAV exhausts its battery, indicating network sustainability.

These metrics collectively provide a comprehensive evaluation of both QoS and energy efficiency in heterogeneous VANET and FANET environments. The proposed IHF is compared against conventional RSS-based handover methods (Munjal et al., 2024), heuristic multi-criteria decision-making approaches (Zhong et al., 2024), Software-Defined Networking (SDN)-assisted schemes using reinforcement learning-based techniques (Adil et al., 2025) under identical simulation conditions to ensure fair and meaningful performance comparison.

5 Results and analysis

This section presents a comprehensive performance evaluation of the proposed IHF approach against three established baseline approaches: the RSS-based conventional handover scheme of Munjal et al. (2024), the heuristic multi-criteria reinforcement learning-based method of Zhong et al. (2024), and the Software-Defined Networking (SDN)-assisted scheme proposed by Adil et al. (2025). All experiments were conducted under identical simulation conditions over 30 independent runs, with results reported as mean values accompanied by 95% confidence intervals. For all performance metrics, the observed confidence intervals were relatively narrow across the 30 independent simulation runs, indicating low variability and good repeatability of the results. Therefore, the mean values presented in Figures 4–10 are representative of the overall system behavior. The corresponding 95% confidence intervals are reported in the discussion of each performance metric. The evaluation covers seven key performance dimensions: throughput, end-to-end delay, PDR, handover latency, energy efficiency, network lifetime, handover success rate, and reinforcement learning convergence behavior.

5.1 Throughput analysis

Figure 4 compares the average throughput across VANET and FANET scenarios. The proposed IHF consistently achieves the highest throughput in all configurations. In RF-only VANET, the IHF attains 12.7 Mbps (95% CI: 12.3–13.1 Mbps), outperforming the RSS-based method by approximately 51% and the best baseline by 16.5%. In the hybrid Li-Fi + RF scenario, throughput increases to 15.8 Mbps, demonstrating the effectiveness of leveraging high-bandwidth optical links when available. Similarly, in FANET environments, the IHF achieves 10.6 Mbps, representing a significant improvement over all baseline methods. These gains are primarily attributed to the framework’s context-aware decision-making, which dynamically selects the most suitable communication link based on real-time network conditions. Overall, the results confirm that the proposed approach enhances spectral efficiency and link utilization across heterogeneous networks.

5.2 End-to-end delay analysis

The end-to-end delay results presented in Figure 5 which indicate that the IHF significantly reduces communication latency across all scenarios. In urban VANET environments, delay is reduced to 92 ms compared to 148 ms in conventional methods, representing a reduction of nearly 38%. In highway scenarios, delay decreases to 75 ms, while in FANET environments, the IHF achieves 108 ms, improving performance by over 40% relative to RSS-based approaches. These improvements stem from proactive handover decisions, where the framework anticipates link degradation and initiates handover before disconnection occurs. This minimizes buffering delays and re-association overhead, making the proposed approach suitable for latency-sensitive applications such as real-time traffic management and UAV coordination.

5.3 Packet delivery ratio (PDR) analysis

Figure 6 presents the PDR under varying traffic loads. The proposed IHF consistently achieves the highest delivery ratio across all network types and traffic conditions. Under low traffic, the IHF achieves up to 99.1% PDR (95% CI: 98.8–99.4%) in hybrid VANET scenarios. More importantly, under high traffic conditions, the degradation in performance is significantly lower compared to baseline methods. For instance, in RF-only VANET, the PDR drops from 98.3 to 91.9%, whereas the RSS-based method shows a much sharper decline. This robustness is due to the framework’s ability to consider traffic load and network congestion during decision-making, enabling efficient load balancing and reducing packet loss. Similar trends are observed in FANET scenarios, confirming the adaptability of the proposed approach under dynamic and high-density conditions.

5.4 Handover latency analysis

Figure 7 compares the handover latency across different methods. The proposed IHF achieves the lowest latency, with 41 ms (95% CI: 39–43 ms) in VANET and 54 ms in FANET environments. This represents a reduction of up to 50% compared to conventional approaches. The improvement is mainly due to predictive handover mechanisms and pre-selection of candidate nodes, which reduce signaling delays and eliminate unnecessary handover procedures. Lower handover latency directly contributes to seamless connectivity and improved user experience, particularly in high-mobility environments

5.5 Energy efficiency and network lifetime

Energy efficiency results for FANET scenarios are presented in Figures 8, 9. The proposed IHF consumes only 0.48 J per packet, significantly lower than all baseline methods. This reduction in energy consumption translates into a substantial increase in network lifetime, which reaches 168 min compared to 96 min for conventional approaches. The improvement of up to 75% demonstrates the effectiveness of incorporating energy-aware decision-making and multi-mode operation in UAV networks. By minimizing unnecessary handovers and optimizing communication processes, the framework ensures prolonged operation of energy-constrained UAV nodes.

5.6 Handover success rate analysis

The handover success rate is a critical metric for evaluating the reliability and robustness of mobility management, particularly in highly dynamic environments such as VANETs and FANETs. Table 4 presents the handover success rates under low, medium, and high mobility conditions for all considered approaches. The proposed IHF consistently achieves the highest success rates across all scenarios. Under low mobility conditions, the success rate exceeds 97% in VANET and 96% in FANET environments, indicating stable connectivity when topology changes are limited. As mobility increases, all methods experience performance degradation due to rapid link variations; however, the decline is significantly less pronounced in the proposed framework.

Table 4. Handover success rate (%) for different network types under varying mobility conditions.

In RF-only VANET under high mobility, the IHF maintains a success rate of 91.0%, compared to 75.8% for the RSS-based method and 84.3% for the reinforcement learning-based baseline. A similar trend is observed in hybrid Li-Fi + RF VANET scenarios, where the IHF achieves 93.8% under high mobility, benefiting from the availability of multiple communication interfaces and intelligent link selection. In FANET environments, which are inherently more challenging due to three-dimensional mobility and frequent topology changes, the proposed method achieves a success rate of 88.9% under high mobility. This represents a substantial improvement over conventional approaches, where success rates drop below 80%. The improved performance of the IHF can be attributed to its ability to incorporate multiple decision parameters, including signal strength, node velocity, traffic conditions, and residual energy, enabling more informed and timely handover decisions. Furthermore, the reinforcement learning-based decision mechanism continuously adapts to changing network conditions, allowing the system to anticipate link degradation and avoid unnecessary or failed handovers. This reduces the occurrence of ping-pong effects and connection drops, which are common in traditional threshold-based approaches.

5.7 Convergence analysis

As it be seen from above Figure 10 which illustrates the convergence behavior of the proposed IHF algorithm in both VANET and FANET scenarios. The average cumulative reward exhibits a steep increase during the initial exploration phase of first 200 episodes, followed by a gradual stabilization phase. The reward converges to a near-optimal value after approximately 500 episodes in both environments, with only marginal improvements observed beyond this point.

This rapid convergence is particularly noteworthy given the highly dynamic nature of VANET and FANET environments. The proposed IHF approach demonstrates excellent computational efficiency, requiring significantly fewer training episodes compared to DRL methods that often demand thousands of episodes and substantial computational resources. The similar convergence patterns observed in both VANET with hybrid Li-Fi/RF and FANET with energy constraints scenarios validate the robustness and generalization of the unified framework. The fast learning rate can be attributed to the carefully designed state space, action set, and multi-objective reward function, which effectively guide the agent toward optimal handover policies. Furthermore, the use of an ε-greedy exploration strategy with decaying ε ensures sufficient exploration in early stages while promoting exploitation of learned policies in later stages. These characteristics make the proposed IHF framework highly suitable for real-time deployment on resource-constrained vehicular and UAV platforms, where computational power and memory are limited.

5.8 Ablation study of shared state representation

Table 5 presents an ablation analysis evaluating the impact of the proposed common-state representation. The VANET-only and FANET-only configurations were trained independently, whereas the proposed framework employs a shared state abstraction across both environments. The results indicate that the common-state framework achieves the highest handover success rate and throughput while requiring fewer training episodes to converge. Specifically, convergence is achieved approximately 17–22% faster than the environment-specific implementations. This improvement can be attributed to the normalized state representation, which allows similar mobility and connectivity patterns observed in one environment to contribute to policy learning in the other environment. These findings support the use of a common Q-table structure and demonstrate that normalization-based state abstraction enables effective operation across heterogeneous VANET and FANET scenarios without requiring separate learning architectures.

Table 5. Ablation analysis of the proposed framework.

5.9 Comparison with advanced reinforcement learning approaches

Recent studies have demonstrated the potential of DRL techniques for mobility management and handover optimization in highly dynamic wireless networks. In particular, the DQN-based handover framework presented in Meer et al. (2025) employs deep neural networks to approximate the action-value function, enabling operation in large and continuous state spaces. Similarly, the AQR-FANET approach proposed in Zhang et al. (2024) utilizes an anticipatory Q-learning mechanism to improve routing and connectivity management in FANET environments. Compared with these approaches, the proposed Intelligent Handover Framework (IHF) adopts a lightweight tabular Q-learning architecture. While DRL methods can potentially achieve improved policy optimization in large-scale environments, they require neural network training, replay memory, parameter tuning, and significantly higher computational resources. Such requirements may increase implementation complexity for resource-constrained vehicular nodes and UAV platforms.

The proposed framework focuses on achieving a balance between learning performance and computational efficiency. Through state normalization and discretization, the framework maintains a manageable state-action space while achieving substantial improvements in throughput, packet delivery ratio, handover latency, energy consumption, and network lifetime. Furthermore, the proposed approach converges within a limited number of training episodes and can be executed using modest computational resources. Although a direct implementation-based comparison with DQN (Meer et al., 2025) and AQR-FANET (Zhang et al., 2024) is beyond the scope of the present study, these methods represent important benchmarks for future investigation. Future work will therefore extend the proposed framework using DRL techniques to evaluate the trade-off between performance gains and computational complexity in large-scale VANET and FANET environments (Table 6).

Table 6. Comparison of reinforcement learning-based handover approaches.

A two-tailed Student’s t-test was performed on throughput, PDR, delay, and handover success rate results obtained from the independent simulation runs. The observed improvements were statistically significant at the 95% confidence level (p < 0.05) (Table 7).

Table 7. p values for the performance metrics.

6 Conclusion

This paper proposed IHF strategy for heterogeneous VANET and FANET environments. By modeling the handover process as a Markov Decision Process (MDP), the proposed framework enables adaptive and context-aware handover decisions based on multiple network parameters, including signal quality, mobility, traffic load, network occupancy, and residual energy. The simulation results demonstrate that the proposed approach outperforms conventional threshold-based and existing learning-based methods in terms of handover success rate, throughput, packet delivery ratio, end-to-end delay, handover latency, and energy efficiency. The integration of hybrid Li-Fi/RF communication improves connectivity reliability in VANETs, while the energy-aware decision mechanism enhances network lifetime and communication stability in FANET environments. Furthermore, the state-space discretization and scalability analysis demonstrate that the proposed tabular Q-learning framework remains computationally feasible for network deployments containing up to 100 nodes, with fixed per-node memory requirements and low computational overhead. However, the use of tabular Q-learning may limit scalability when finer-grained state representations or larger continuous state spaces are considered.

Future research will therefore investigate advanced reinforcement learning techniques, including Deep Q-Networks (DQN), function approximation methods, state aggregation strategies, and transfer learning to further improve scalability while preserving handover decision quality in large-scale VANET and FANET deployments. In addition, real-world experimental validation and integration with emerging 5G/6G communication infrastructures will be explored to enhance the practical applicability of the proposed framework.

Generative AI statement

The author(s) declared that Generative AI was used in the creation of this manuscript. Grammarly tool was used in checking the grammars of the manuscript and no generative AI was used.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Abdenacer Naouri, Southwest Jiaotong University, China

Reviewed by: Karim Chabir, École Nationale d'Ingénieurs de Gabès, Tunisia

Naskath Jahangeer, National Engineering College, India