Hybrid indoor localization using Wi-Fi–magnetic LSTM fingerprinting and vision-based distance fusion with EKF and particle filtering

1 Introduction

Indoor localization is an important requirement for many modern applications, including smart buildings, healthcare monitoring, indoor navigation, asset tracking, robotics, and emergency response (Zafari et al., 2019; Roy and Chowdhury, 2021). While Global Navigation Satellite Systems do not have problems outdoors they work poorly padded between concrete walls because signals are incomplete, absorbed or nonexistent. Thus, for indoor positioning new strategies are needed that can work robustly in complicated indoor conditions (Zafari et al., 2019).

The fingerprinting method in indoor localization is relatively promising as it utilizes existing wireless infrastructure without the need for any additional hardware (Shang and Wang, 2022). With this approach, RSSI values are collected in known reference points and then the user position is estimated. Nevertheless, the Wi-Fi RSSI feature is sensitive to multipath propagation, signal variation, device orientation, obstructions and indoor environmental changes (Zafari et al., 2019; Shang and Wang, 2022). This issue can lead to two locations emitting similar signals although a considerable distance away, incurring positioning error.

Recent Wi-Fi-based indoor localization studies show that this research area remains active and practically important (Yoon et al., 2025), a real-world Wi-Fi RSSI dataset collected in university campus buildings, showing that reliable RSSI dataset construction remains an important challenge. Shen and Cui (2024) proposed a Wi-Fi fingerprint correction method based on distance metric learning to reduce the effect of RSSI fluctuation. Magsino et al. (2023) applied multiple Wi-Fi RSSI fingerprinting for commercial parking vacancy detection, demonstrating the use of Wi-Fi sensing in practical smart-environment applications. Zhu et al. (2023) proposed a neural-network-based Wi-Fi fingerprint localization method, showing the continued relevance of learning-based approaches for RSSI fingerprinting. These studies show that Wi-Fi RSSI sensing remains useful, but RSSI-based localization still faces challenges related to signal fluctuation, data collection effort, and ambiguity between nearby indoor positions.

In most cases, more information from different sources may enhance fingerprint-based localization performance. The magnetic-field measurements are helpful since indoor structures, electrical devices and metallic objects cause space-dependent perturbations of the magnetic field (Ouyang and Abed-Meraim, 2022).

Therefore, combining Wi-Fi RSSI with magnetometer readings can provide a richer fingerprint representation than Wi-Fi alone (Sun et al., 2023). Deep learning models can also help by learning nonlinear relationships between fingerprint features and physical coordinates (Kordi et al., 2024). In this work, an LSTM-based model is used to estimate the initial indoor position from Wi-Fi RSSI and magnetometer features (Chen et al., 2020; Wu et al., 2024).

Although radio and magnetic fingerprints provide useful global location information, they may still produce errors in areas with unstable signals or similar fingerprint patterns. Vision sensors can provide complementary geometric information. Cameras can detect a person and estimate the distance between the camera and the person. However, using a single camera to directly predict an indoor X, Y coordinate can be difficult because of perspective distortion, occlusion, shadows, and scale ambiguity. For this reason, the vision model in this study is used only to estimate camera-person distance, not to directly predict the coordinate (Ranftl et al., 2021).

This paper proposes a hybrid indoor localization framework that combines Wi-Fi–magnetic LSTM fingerprinting with vision-based distance estimation and probabilistic fusion. The LSTM model first predicts the initial indoor X, Y coordinate. Then, the vision model estimates the camera-person distance from CCTV images using person detection, depth-related features, geometric mapping, and residual correction. Finally, the LSTM coordinate and vision-derived distance are fused using an Extended Kalman Filter and a particle-filter-based method (Nurminen et al., 2013; Deng et al., 2015).

The main contribution of this work is the use of vision as an independent distance constraint to improve Wi-Fi–magnetic fingerprint localization. Instead of treating the camera as a full coordinate estimator, the proposed system uses the camera only for what it can estimate more reliably: distance. This allows the LSTM model to provide the global coordinate hypothesis, while the vision model provides a geometric correction constraint. The fusion methods then combine both sources to produce a more accurate final position.

The main contributions of this work are summarized as follows:

• A hybrid indoor localization framework is proposed that combines Wi-Fi RSSI, magnetometer readings, CCTV-based distance estimation, and probabilistic fusion for indoor position estimation.
• An LSTM-based Wi-Fi–magnetic fingerprinting model is developed to estimate the initial indoor X, Y coordinate using RSSI and magnetometer features.
• A vision-based distance-estimation stage is introduced, where CCTV images are used to estimate the camera-person distance rather than directly predicting the global coordinate.
• Two probabilistic fusion methods, EKF-based fusion and particle-filter-based fusion with EKF refinement, are applied to combine the LSTM coordinate estimate with the vision-derived distance constraint.
• The proposed framework is evaluated using a real indoor multimodal dataset collected in a corridor environment with Wi-Fi, magnetometer, and CCTV data.

The rest of this article is as follow, the section 2 presents and investigated the current state of the art, section 3 and 4 provides the details of the dataset and proposed approaches, respectively. Section 5 lists the design of the experiments and scenario. While, section 6 demonstrates the achieved experimental results and discussions. Finally, section 7 and 8 explain the main limitations and conclusions/future steps of the study, respectively.

2 Related work

Indoor localization has been widely studied using wireless, magnetic, visual, and sensor-fusion approaches. Wi-Fi fingerprinting is among the longest-established techniques as it does not require dedicated positioning hardware and leverages existing wireless infrastructure. Wi-Fi fingerprinting has been regarded as a promising localization technique for indoor localization by earlier review papers, however, it also highlights the impact of multipath propagation, signal fluctuation, device heterogeneity and environmental change on Wi–Fi fingerprint (Zafari et al., 2019; Shang and Wang, 2022).

These limitations have encouraged researchers to move from traditional matching methods toward machine learning and deep learning models that can learn nonlinear relationships between signal fingerprints and physical locations.

More recent Wi-Fi RSSI-based studies have addressed indoor localization from different perspectives, including dataset construction, fingerprint correction, application-specific sensing, and neural-network-based modeling. Yoon et al. (2025) presented CAMPUSRSSI, a real-world Wi-Fi RSSI dataset collected in university campus buildings, emphasizing the importance of practical RSSI measurements in real indoor environments. Shen and Cui (2024) proposed a distance metric learning-based Wi-Fi fingerprint correction method, showing that RSSI fluctuation can weaken the relationship between signal-space distance and physical distance. Magsino et al. (2023) applied multiple Wi-Fi RSSI fingerprinting to commercial parking vacancy detection, demonstrating that Wi-Fi sensing can support practical indoor applications beyond direct user positioning. Zhu et al. (2023) proposed a neural-network-based Wi-Fi fingerprint indoor localization method, showing the role of learning-based models in improving RSSI fingerprint localization. These studies confirm that Wi-Fi RSSI fingerprinting remains an active research direction, but they also show that RSSI-only approaches still require additional strategies to handle signal instability, environmental variation, and localization ambiguity.

Machine learning-based indoor localization has been investigated using classifiers and regressors such as k-nearest neighbors, support vector machines, random forests, artificial neural networks, and deep neural networks (Roy and Chowdhury, 2021; Kordi et al., 2024). Deep learning methods are particularly relevant because RSSI fingerprints are noisy, sparse, and high-dimensional. LSTM-based models have also been explored in Wi-Fi fingerprinting because recurrent architectures can learn complex patterns from fingerprint observations (Chen et al., 2020; Wu et al., 2024). However, many LSTM-based positioning studies focus either on sequential radio measurements or on radio-only fingerprinting. In contrast, the present work uses an LSTM model as the initial coordinate estimator and then improves the result through independent vision-based distance constraints. Therefore, the LSTM is not treated as the complete localization solution, but as the first stage in a wider multimodal fusion framework.

Several studies have shown that magnetic-field information can complement Wi-Fi RSSI in indoor positioning. Magnetic fingerprints are useful because indoor structural materials and electrical systems create location-dependent magnetic disturbances that can remain more stable than RSSI in some environments (Ouyang and Abed-Meraim, 2022). Sarcevic et al. (2023) proposed an indoor 2D positioning method for mobile robots using the fusion of RSSI and magnetometer fingerprints and showed that combining the two modalities can improve positioning performance, especially in areas where magnetic variance is informative. This supports the motivation of the present work, where Wi-Fi RSSI and magnetometer measurements are jointly used as input features for the LSTM fingerprinting model.

Vision-based localization has also received attention because cameras provide geometric and semantic information that is independent of wireless signal behavior. Human pose estimation methods can detect body keypoints and provide useful information about the position of a person in an image. A few YOLO-based pose estimation studies have achieved high-quality human keypoint detection in real-time, but significant problems such as occlusion, illumination variation and background noise still exist (Dong and Du, 2024). Monocular depth estimation also offers another modality for inferring scene depth in relation to one another from a single image (Sun et al., 2023), and DPT/MiDaS-based methods have been prolific within dense prediction and monocular depth estimation (Ranftl et al., 2021). Prior work on monocular depth estimation and object detection demonstrated that by combining detection with the prediction of depths, they could assist in estimating distance between camera and object (Masoumian et al., 2021). However, monocular vision suffers from scale ambiguity and camera-perspective dependence.

For this reason, the current study does not use the vision model to directly predict global X, Y coordinates. Instead, the vision model estimates only the camera-person distance, which is then used as a geometric constraint for correcting the LSTM position.

Sensor fusion methods are commonly used to improve indoor localization by combining complementary measurements. Kalman filtering and Extended Kalman Filtering have been applied to fuse Wi-Fi, inertial, landmark, and other sensor information in indoor environments (Deng et al., 2015). Sircoulomb and Chafouk introduced a constrained Kalman filter for Wi-Fi-based indoor localization and showed that adding motion or spatial constraints can improve localization accuracy (Sircoulomb and Chafouk, 2022). More recent work has also combined EKF with recurrent neural networks for multi-sensor localization, showing the value of integrating model-based estimation with data-driven learning (Huang et al., 2025). These studies support the use of EKF as a correction layer after an initial machine-learning prediction. In the present work, the EKF state is the two-dimensional position, initialized from the LSTM coordinate and updated using the vision-derived distance measurement.

Particle filters provide another probabilistic solution for nonlinear and multimodal localization problems. Unlike Kalman filters, particle filters approximate possible states using multiple samples, making them suitable when the measurement constraint produces more than one possible location. In indoor localization, particle filtering has been applied for tracking and sensor fusion, especially when uncertainty and nonlinear geometry are important (Nurminen et al., 2013; Cheng et al., 2022). This is directly relevant to the present study because a camera-person distance measurement defines a circle of possible positions around the camera rather than a single coordinate. The proposed particle-based method samples candidate positions on this distance circle, keeps particles consistent with the camera-facing direction, and then applies EKF correction to the selected particles.

Overall, existing studies support Wi-Fi fingerprinting, magnetic fusion, vision-based perception, EKF correction, and particle filtering as valuable components of indoor localization. However, most prior works focus on one modality or fuse sensors that directly provide coordinate estimates. The main gap addressed in this paper is the use of vision as a distance-only constraint to refine Wi-Fi–magnetic LSTM localization. This design allows the radio-magnetic model to provide a global coordinate hypothesis while the vision model provides an independent geometric correction signal. The proposed framework therefore combines the strengths of data-driven fingerprinting, vision-based distance estimation, and probabilistic fusion. Compared with these studies, the present work combines Wi-Fi RSSI and magnetometer fingerprinting with CCTV-based distance estimation and probabilistic fusion, where vision is used as an independent distance constraint rather than as a direct coordinate estimator.

3 Dataset and data preparation

3.1 Published multimodal indoor positioning dataset

To this end, we make use of a multimodal indoor positioning dataset that has been previously self-collected and published in Data in Brief (Abdalla et al., 2026). They consist of Wi-Fi RSSIs, embedded smartphone sensors data (including Magnetometer Sensor, Accelerometer Sensor, Gyroscope Sensor), and CCTV images in the indoor corridor environment. This section only summarizes the parts used for the present work since (Abdalla et al., 2026) contains all details of data set design, data collection procedure, hardware setup and file organization.

3.2 Indoor environment and coordinate grid

The data was collected in a 1 m resolution grid of an indoor coordinate system. Such that each location, which in fact the ground truth for localization would be represented absolutely by an X, Y coordinate. The locations for LSTM training and testing along with final estimated fused localization error are denoted by these coordinates.

The data were collected in the third-floor corridor of the Faculty of Engineering building at Koya University, Iraq. The indoor environment covers approximately 39 m × 49 m and was represented using a 1 m coordinate grid with 344 reference positions. The setup included 12 Wi-Fi access points distributed across the environment and 8 fixed CCTV cameras installed at corridor intersections for visual monitoring. At each reference position, Wi-Fi RSSI readings, magnetometer measurements, and other embedded sensor readings were collected using a smartphone, while CCTV images were recorded simultaneously from the surrounding cameras. This multimodal setup was designed to support fingerprint-based localization, vision-based distance estimation, and later fusion experiments.

Figure 1 illustrates the indoor environment and the experimental setup, including the coordinate grid, Wi-Fi access points, CCTV camera locations, and the multimodal data collection process.

3.3 Wi-Fi RSSI and embedded sensor measurements

This work involves Wi-Fi RSSI readings and magnetometer measurements for the LSTM-based localization model. The RSSI values show the wireless fingerprints of each coordinate, while the magnetometer readings supplement them with some spatial info from indoor magnetic-field fluctuations. The dataset also has measurements for accelerometers and gyroscopes but these features were not part of the final LSTM feature vector.

4 Proposed methodology

4.1 Overall system architecture

We propose a hybrid indoor localization framework, employing Wi-Fi–magnetic fingerprinting, vision-based distance estimation (VDE), and probabilistic fusion. The proposed system consists of four main components: (i) fingerprint localization based on LSTMs; (ii) distance estimation with vision; (iii) EKF fusion, and (iv) Particle-based candidate selection with EKF refinement. To do this, first the X, Y coordinate is predicted based on the Wi-Fi RSSI and magnetometer readings. Second, the distance from the camera to detected person is inferred from CCTV images. Finally, it fuses the LSTM coordinate and vision-derived distance to obtain a final position estimate. Overall architecture of the proposed framework: from offline data collection and model training stage to online localization and fusion stage as illustrated in Figure 2.

The step-by-step process of the proposed framework is as follows:

• Step 1: Multimodal data collection. Wi-Fi RSSI readings, magnetometer readings, and CCTV images are collected at known indoor reference positions.
• Step 2: Data preparation. Each X, Y coordinate is converted into a unique location label. The selected Wi-Fi RSSI and magnetometer features are normalized, and the dataset is divided into training and testing sets.
• Step 3: LSTM fingerprint localization. The Wi-Fi RSSI and magnetometer features are used as input to the LSTM model. The model estimates the initial indoor coordinate, denoted as (x LSTM , y LSTM).
• Step 4: Vision-based distance estimation. The CCTV image is processed to detect the person and estimate the camera-person distance, denoted as d vis.
• Step 5: Probabilistic fusion. The LSTM coordinate and vision-derived distance are fused using two methods: EKF-based fusion and particle-based candidate selection with EKF refinement.
• Step 6: Final coordinate output. The EKF method produces ( x EKF , y EKF ), while the particle-filter-based method produces (x PF , y PF). These fused coordinates represent the final estimated indoor position.

To better illustrate the fusion mechanism, we provide a simplified view of the two principal input branches used in our framework in Figure 3. The fingerprint branch produces the initial X, Y coordinate from Wi-Fi RSSI and magnetometer readings using the LSTM model. The vision branch produces the camera-person distance from the CCTV image using the vision model. These two outputs are then combined using EKF and particle-filter fusion to generate the final fused X, Y coordinate.

This design allows each modality to contribute according to its strength. Wi-Fi and magnetic fingerprints provide a global coordinate estimate, while the vision model provides an independent geometric distance constraint.

4.2 LSTM-based Wi-Fi and magnetic fingerprint localization

The LSTM model estimates the initial indoor X, Y coordinate using Wi-Fi RSSI and magnetometer readings. The RSSI values represent the wireless fingerprint of each coordinate, while the magnetometer readings add spatial information from indoor magnetic-field variations. Although accelerometer and gyroscope data are available in the dataset, the final LSTM input uses only Wi-Fi RSSI and magnetometer features.

The localization task is formulated as a multi-class coordinate classification problem. Each X, Y coordinate pair is converted into a unique label, and the model predicts the most likely coordinate class. The selected features are normalized using the training data only and reshaped into a single-step sequence. In the implemented model, each fingerprint sample is represented by a 15-dimensional feature vector consisting of 12 Wi-Fi RSSI values and 3 magnetometer readings. The Wi-Fi part contains the RSSI readings from the available access points, while the magnetic part contains the magnetometer readings along the X, Y, and Z axes. Therefore, each input fingerprint is arranged as:where RSSI 1​ to RSSI 12​ are the Wi-Fi signal-strength values, and Mx​, My​, and Mz​ are the magnetometer readings. In the code, each fingerprint is treated as an independent sample and reshaped into the LSTM input format ( N , 1 , 15 ) ,where N is the number of samples, 1 is the sequence length, and 15 is the number of selected features.

f = [ RSSI 1 , RSSI 2 , … , RSSI 12 , Mx , My , Mz ]

After reshaping, the fingerprint vector is passed through the LSTM model and the final SoftMax layer. The SoftMax layer produces a probability distribution over all coordinate classes, and the class with the highest probability is selected as the predicted location. Therefore, the LSTM is used as a learned fingerprint classifier rather than as a direct geometric coordinate estimator. The predicted class label is then converted back into its physical coordinate (x lstm​,y LSTM​), which is used as the initial position estimate for the fusion stage.

Although LSTM models are commonly used for temporal sequence modeling, the LSTM in this study is used as a fingerprint classifier. The single-step sequence does not represent user movement over time. Instead, it provides the required three-dimensional input format for the LSTM layer. In this structure, the LSTM learns nonlinear relationships among the Wi-Fi RSSI values and magnetometer readings and maps each fingerprint sample to one of the predefined coordinate classes. The SoftMax output layer produces the probability of each coordinate label, and the label with the highest probability is selected as the predicted location. During prediction, this label is converted back into its physical coordinate (x LSTM , y LSTM​), which is then used as the initial position estimate for the EKF and particle-filter-based fusion stages.

Therefore, the LSTM is not used to track continuous movement or model a temporal trajectory. Its role in the proposed framework is to provide the first coordinate estimate from the Wi-Fi–magnetic fingerprint. The later fusion stages then refine this estimate using the independent vision-derived distance constraint.

As shown in Figure 4, the LSTM architecture consists of a masking layer, an LSTM layer with 64 units, a dropout layer, a dense layer, and a final SoftMax output layer. The predicted class is then converted back into its physical coordinate, as expressed in Equation 1:

LSTM output = ( x LSTM , y LSTM ) (1)

Where x LSTMand y LSTMare the X-coordinate and Y-coordinate predicted by the LSTM model, respectively.

In this study, the LSTM is used as a fingerprint classifier rather than a temporal trajectory model. Therefore, the single-step sequence does not represent continuous movement over time; it only provides the three-dimensional input format required by the LSTM layer. Since each collected fingerprint is treated as an independent localization sample, no temporal sampling frequency is used in the LSTM sequence construction. The repeated measurements at each coordinate are treated as independent fingerprint samples and divided into training and testing sets.4.3 Vision-Based Distance Estimation.

The vision model estimates the distance between the CCTV camera and the detected person. It does not directly predict the global X, Y coordinate. Instead, it provides a camera-person distance measurement that is later used as a geometric constraint during fusion. The CCTV images were captured using eight fixed Dahua 5MP PoE cameras installed at corridor intersections. Each camera has a resolution of 2,592 × 1944 pixels, a 3.6 mm fixed lens, and a camera height of 3.2 m. These fixed camera parameters support the geometric distance-estimation process.

The vision pipeline detects the person in the CCTV image and estimates the ground-contact point using lower-body key points, especially the ankle region. If the ankle point is not reliable, the lower center of the bounding box is used. The detected image point is then combined with depth-related features and homograph-based geometry to estimate the camera-person distance. A residual correction model is also applied to improve the physics-based estimate. The vision equation is kept simple because the vision branch produces only one measurement: the camera-person distance. The CCTV image is first processed to detect the person. Then, the detected lower-body region or ground-contact point is used together with camera geometry, depth-related features, and residual correction to estimate the distance between the camera and the detected person. The vision model does not directly estimate the global indoor coordinate (x,y). Instead, it provides the distance value dvis, which is later used as an independent geometric constraint in the fusion stage. The final output of this stage is showed in Equation 2:where (d { vis }) is the estimated distance between the camera and the person.

Vision output = d { vis } (2)

To illustrate the distance-estimation output, Figure 5 shows example CCTV-based detection results, including the detected person and the estimated camera-person distance used in the fusion stage.

4.3 EKF-based fusion

The EKF fusion stage corrects the LSTM coordinate using the vision-estimated distance. The EKF state is defined as the two-dimensional position as expressed in Equation 3:where ( s) is the EKF state vector, and (x) and (y) represent the estimated indoor position coordinates.

s = [ x , y ] T (3)

The initial state is set using the LSTM prediction, as shown in Equation 4:where (s 0) is the initial EKF state, and ( x LSTM ) and (y _ LSTM )are the coordinates predicted by the LSTM model.

s 0 = [ x LSTM , y LSTM ] T (4)

Given the camera position (c = ( x c , y c , z c )), the predicted distance from any state (s = [ x , y ] T) is calculated as in Equation 5:where (x c) and (y c) are the camera position on the floor plane, and (z c) is the camera height. In Equation 5, h ( s ) is the nonlinear measurement function of the EKF. It converts the current estimated 2D position state s = [ x , y ] T into the expected camera-person distance. Therefore, h ( s ) calculates the predicted 3D distance between the camera and the estimated person position on the floor plane. This predicted distance is then compared with the vision-estimated distance d vis, and the difference is used by the EKF correction step to update the estimated position. The role of h ( s ) in the fusion process is also illustrated in the overall architecture Figure 2 in Section 4.1, where the LSTM coordinate and vision-derived distance are combined during the EKF fusion stage.

h ( s ) = { ( x − x c ) 2 + ( y − y c ) 2 + z c 2 } (5)

In the implemented EKF fusion code, the initial covariance was set to P 0 = 2.0 I, where I is the identity matrix. The observation-noise values were set to Rvis = 0.0005 for the vision-estimated distance and R LSTM = 0.15 for the LSTM-implied camera distance. Since the EKF was applied independently to each localization sample and not as a temporal tracking filter, no explicit motion model or process-noise-driven prediction step was used. Therefore, the EKF mainly works as a measurement-correction stage that refines the LSTM-initialized coordinate using the nonlinear camera-distance measurement.

Since the measurement function in Equation 5 is nonlinear, EKF linearization is used. The EKF correction step is calculated using the innovation, innovation covariance, Kalman gain, state correction, and covariance update, as shown in Equations 5a–5e:

Y k = Z k − h ( S k ) (5a)

S k = H k P k H k T + R k (5b)

K k = P k H k T S k − 1 (5c)

S k = S k + K k y k (5d)

P k = ( I − K k H k ) P k (5e)

where Y k is the innovation, Z k is the observed distance measurement, H k is the Jacobian matrix of the nonlinear measurement function, S k is the innovation covariance, K k is the Kalman gain, P k is the covariance matrix, and R k is the observation-noise covariance.

Since this measurement function is nonlinear, EKF linearization is used to update the position. The EKF uses the vision-estimated distance and the LSTM-implied camera distance to correct the initial coordinate. The final EKF output is expressed in Equation 6:where P EKF is the EKF-fused position, and X EKF and Y EKF are the corrected X and Y coordinates after fusion.

P EKF = ( x EKF , y EKF ) (6)

4.4 Particle-based candidate selection with EKF refinement

The particle-filter-based method is used because a single camera-person distance value can correspond to several possible positions around the camera. In this work, the particle-filter stage is used to select the most likely candidate positions, rather than as a fully recursive particle filter. EKF refinement is then applied to the selected particles to further correct their positions. This combination is useful because the particle stage reduces the ambiguity of the distance circle, while EKF refinement improves the selected positions using the nonlinear distance measurement. Since the vision distance is a slant distance, the floor radius is calculated as expressed in Equation 7:where (r) is the floor-plane radius, ( d { vis } ) is the vision-estimated camera-person distance, and (z c) is the camera height.

r = { d { vis } { 2 } − z c { 2 } } (7)

Then, 500 equally spaced particles are generated on the circle as expressed in Equation 8:

p i = ( x c + r cos θ i , y c + r sin θ i ) (8)

Where p i is the i − th particle, (x c , y c) is the camera floor position, r is the floor-plane radius, and θi is the particle angle.

The floor-plane radius is calculated from the vision-estimated camera-person distance as shown in Equation 7, where r is the floor-plane radius, d vis is the vision-estimated camera-person distance, and z c is the camera height.

Then, 500 equally spaced particles are generated on the vision-distance circle, as expressed in Equation 8. The number of particles was set to 500 to provide sufficient coverage of the possible candidate positions around the camera while keeping the computational cost practical. A smaller number of particles may not represent the valid region with enough resolution, while a much larger number increases runtime without necessarily improving the localization result significantly. Therefore, 500 particles were used as a balanced setting between localization accuracy and computational efficiency.

Figure 6 illustrates how the particles are generated on the vision-distance circle and filtered using the camera-facing direction.

The particles are filtered using the camera-facing direction and a strip constraint around the camera-facing line. Particles behind the camera are removed, and only front-facing particles inside the strip are retained. The strip width is adjusted according to the difference between the vision-estimated distance and the LSTM-implied distance, allowing the selected particle set to reflect both measurements. If no particle satisfies the constraint, the closest front-facing particle is selected as a fallback. Each selected particle is then corrected using the same EKF logic used in the EKF fusion stage.

Each particle state represents a candidate floor-plane position p i = ( x i , y i ) . The particles are initialized on the vision-distance circle with equal importance. Since the proposed method is applied independently to each localization sample and is not used as a fully recursive particle filter, no iterative particle-weight update or resampling step is applied. Instead, particles are selected geometrically using the camera-facing direction and strip constraint. Particles outside the valid region are rejected, while the selected particles are refined using EKF and averaged to obtain the final fused coordinate.

The final fused coordinate is calculated as the average of the EKF-corrected selected particles as expressed in Equation 9:where p PF = ( x PF , y PF ) is the final particle-filter-based fused coordinate, N s is the number of selected particles, and p i ˜ is the i-th EKF-refined selected particle. After the valid particles are selected, each one is refined using the EKF correction step. The corrected particles are then averaged to obtain the final fused coordinate.

p PF = 1 N s ∑ i = 1 N s p i ˜ (9)

This hybrid particle-EKF method reduces the ambiguity of the vision-distance circle and produces the final particle-filter-based coordinate as expressed in Equation 10:where ( x PF , y PF ) are the final fused X and Y coordinates, respectively.

p PF = ( x PF , y PF ) (10)

5 Experimental setup

The experiments were conducted to evaluate the proposed framework at four levels: LSTM-based coordinate prediction, vision-based distance estimation, EKF fusion, and particle-filter-based fusion. All experiments used the same indoor coordinate grid and the same multimodal dataset described in Section 3 Figure 1. For the LSTM experiment, Wi-Fi RSSI and magnetometer readings were used as input features. Each X, Y coordinate was converted into a unique class label, and the model was trained to predict the correct coordinate class. The repeated samples at each coordinate were divided using an 8:2 train-test split. Feature normalization was performed using the training set only. The model was trained using Adam optimizer, sparse categorical cross-entropy loss, 100 epochs, and a batch size of 8.

For the vision experiment, CCTV images were used to estimate the camera-person distance. The vision pipeline detected the person, estimated the ground-contact point, applied depth-related and geometric features, and used residual correction to improve the final distance estimate. The output of this model was a distance value, not a direct X, Y coordinate.

For the fusion experiments, only samples where both the LSTM coordinate and vision distance were available were used. In the EKF experiment, the LSTM-predicted coordinate was used as the initial state, and the vision-estimated distance was used as a nonlinear measurement constraint. The EKF produced a corrected X, Y coordinate.

In the particle-filter-based experiment, 500 particles were generated on the circle defined by the vision distance. The particles were filtered using the camera-facing direction and strip constraint. The selected particles were then corrected using EKF, and the final fused coordinate was calculated as the average of the corrected particles.

The LSTM model was evaluated using exact-coordinate accuracy and coordinate-wise RMSE. The vision model was evaluated using distance error. The EKF and particle-filter outputs were evaluated by comparing the fused X, Y coordinates with the true coordinates.

Table 1 summarizes the input, output, architecture, and dataset partitioning used for each implemented model. The LSTM model is the only trained neural-network localization model in the framework. The vision-based model produces a distance measurement from CCTV images, while the EKF and particle-filter-based methods are fusion stages applied to the shared subset where both LSTM and vision outputs are available.

Table 1. Summary of input, output, architecture, and dataset partitioning for the LSTM localization, vision-based distance estimation, EKF fusion, and particle-filter-based fusion models.

The performance of the proposed framework was evaluated using different metrics according to the output of each stage. For the LSTM-based fingerprint localization model, exact-coordinate accuracy was used to measure the percentage of test samples where the predicted coordinate exactly matched the true coordinate. The coordinate-wise root mean square error (RMSE) was also calculated to measure the localization error between the predicted and true X, Y positions.

The vision-based distance-estimation model was evaluated using the average absolute distance error and distance RMSE between the estimated camera-person distance and the true camera-person distance. For the EKF and particle-filter-based fusion methods, localization error was calculated as the Euclidean distance between the fused coordinate and the ground-truth coordinate. These metrics were used to compare the LSTM-only result, EKF-fused result, and particle-filter-fused result on the shared test subset.

The localization error is calculated using Equation 11:where (x i, y i) is the ground-truth coordinate and (x ˜ i, y ˜ i) is the predicted or fused coordinate for sample i.

e i = { ( x i − x i ̂ ) 2 + ( y i − y i ̂ ) 2 } (11)

The RMSE is calculated using Equation 12:

RMSE = 1 N ∑ i = 1 N e i 2 (12)

where N is the number of test samples.

6 Results and discussion

6.1 LSTM localization performance

The LSTM model was first evaluated as the initial fingerprint-based localization model. It used Wi-Fi RSSI and magnetometer readings to predict the indoor X, Y coordinate as a coordinate class. The main LSTM test set contained 688 samples. The model correctly predicted 555 samples and incorrectly predicted 133 samples, giving an exact-coordinate accuracy of 80.67%. Since exact-coordinate accuracy was used, a prediction was considered correct only when the predicted X, Y coordinate matched the true coordinate exactly.

The LSTM model achieved a coordinate-wise RMSE of 1.5559 on the main test set. On the camera-detected subset, the average RMSE was 1.68 m. On the 32-row shared vision-test subset, which was later used for fusion comparison, the LSTM RMSE was 1.58 m. These results are summarized in Table 2.

Table 2. LSTM localization performance.

Table 2 shows that the LSTM model provides a useful initial position estimate. The exact coordinate accuracy of 80.67% indicates that Wi-Fi RSSI and magnetometer readings can represent indoor locations effectively. However, the remaining RMSE shows that Wi-Fi–magnetic fingerprints alone can still be affected by RSSI fluctuation, magnetic variation, and similarity between nearby fingerprint patterns.

To provide a clearer spatial interpretation of the LSTM localization performance, the prediction results were also summarized at the coordinate level. Since two test samples were used for each indoor coordinate, each location was categorized as 2/2 correct, 1/2 correct, or 0/2 correct according to the number of correctly predicted samples. Figure 7 shows the spatial distribution of these categories over the indoor grid and helps identify locations where the model performs consistently well and locations where prediction errors occur more frequently.

As shown in Figure 7, most indoor coordinates were correctly predicted in both test samples, while some coordinates were correctly predicted once or not correctly predicted. This visualization supports the exact-coordinate accuracy result and provides a clearer view of the spatial distribution of LSTM localization performance across the collected indoor coordinates.

6.2 Vision distance-estimation performance

The vision model was evaluated only for camera-person distance estimation. It was not used to directly estimate X, Y coordinates. On the 32 vision-test samples, the model achieved an average distance error of 0.159 m and a distance RMSE of 0.218 m, as shown in Table 3.

Table 3. Vision distance-estimation performance.

Table 3 shows that the camera-based model can estimate camera-person distance with good accuracy. Although this distance does not directly provide the global X, Y coordinate, it provides a reliable geometric constraint. This supports its use in the fusion stage to correct the initial LSTM-based coordinate estimate.

6.3 EKF fusion performance

The EKF fusion model was evaluated on the same 32-row shared subset. The LSTM coordinate was used as the initial state, while the vision-estimated distance was used as a nonlinear measurement constraint. The EKF reduced the localization error to 0.70 m. Compared with the LSTM-only error of 1.58 m on the same subset, this shows that the vision-derived distance helped correct the original fingerprint-based prediction.

This result confirms that using the camera as a distance sensor is useful. Instead of forcing the vision model to predict X, Y coordinates directly, the EKF uses the distance measurement to adjust the LSTM coordinate according to camera geometry.

6.4 Particle-filter fusion performance

The particle-filter-based method achieved the best performance on the same 32-row subset. It generated candidate positions on the vision-distance circle, filtered them using the camera-facing direction and strip constraint, and then corrected the selected particles using EKF. The final position was calculated as the average of the corrected selected particles.

This method reduced the localization error to 0.40 m. The improvement over EKF shows that the particle method handled the ambiguity of the vision-distance circle more effectively before applying correction. Table 4 compares the LSTM-only, EKF-fused, and particle-filter-fused results on the shared test subset.

Table 4. Localization error comparison of the LSTM-only and fusion-based methods.

Table 4 shows that both fusion methods improved the LSTM-only result. EKF fusion reduced the localization error from 1.58 m to 0.70 m, while the particle-filter-based method achieved the lowest error of 0.40 m. This confirms that vision-derived distance information improves the initial Wi-Fi–magnetic fingerprint estimate, and that particle-based candidate selection provides a stronger correction than EKF alone.

Figure 8 visually compares the localization error of the LSTM-only, EKF-fused, and particle-filter-fused methods on the shared 32-row test subset.

Figure 8 further confirms the improvement obtained after fusion. The EKF method reduced the localization error by more than half compared with the LSTM-only result. The particle-filter-based method produced the lowest error because it first reduced the geometric ambiguity of the vision-distance measurement before applying EKF correction.

6.5 Comparative discussion

The results show a clear improvement from the original LSTM prediction to the fused outputs. The LSTM model achieved strong initial localization performance, with 80.67% exact-coordinate accuracy on the main test set. However, the remaining error shows that Wi-Fi–magnetic fingerprints alone are still affected by signal instability and location ambiguity.

The vision model provided an accurate distance estimate with an average error of 0.159 m. When this distance was combined with the LSTM coordinate, the EKF reduced the localization error to 0.70 m. The particle-filter-based method further reduced the error to 0.40 m by considering multiple possible positions on the vision-distance circle before correction.

To further compare the localization behavior of the different methods, Figure 9 presents a coordinate-level comparison of the ground truth, LSTM-only prediction, EKF-fused output, and particle-filter-fused output on the shared 32-row test subset. Since these samples do not represent one continuous walking trajectory, the points are ordered only for visual comparison. The figure shows that the LSTM-only estimates have larger deviations from the ground truth, while EKF fusion reduces these deviations. The particle-filter-based method provides the closest alignment with the true coordinates, confirming its stronger correction ability.

On the 32-row test subset common to all methods, Figure 10 shows the cumulative distribution function (CDF) of localization error for the LSTM-only, EKF-fused, and particle-filter fusing methods. Cumulative distribution functions of localization errors based on different localization methods and fusion strategies indicates that particle-filter based method achieves the most favorable error, EKF fusion produced better performance than LSTM-only model which yields the largest localization errors. This verifies that the new fusion methods are able to reduce both the average error and the localization error distribution over all shared test samples.

Table 5 summarizes the main methods used in the proposed framework based on their evaluation subset, output, and performance.

Table 5. Performance comparison of the proposed localization methods.

Table 6 compares the computational time of the EKF fusion and particle-filter-based fusion methods on the shared 32-row test subset. The EKF method required 12.2 ms in total, with an average runtime of 0.38 ms per sample. The particle-filter-based method required 79.8 ms in total, with an average runtime of 2.49 ms per sample, making it approximately 6.5 times slower than EKF. This increase is expected because the particle-filter-based method generates 500 particles, applies geometric filtering, refines selected particles using EKF, and averages the corrected particles. Therefore, EKF is computationally faster, while the particle-filter-based method requires more computation but provides better localization accuracy.

Table 6. Computational time comparison between EKF and particle-filter-based fusion on the shared 32-row subset.

In general, results support the main idea of the suggested framework. Fingerprinting: Wi-Fi and magnetometer fingerprints provide a global coordinate estimate, while distance estimation based on CCTV provides an independent geometric constraint; Probabilistic fusion of these two sources leads to great improvement in the indoor localization accuracy.

7 Limitations

Despite the encouraging results obtained using the proposed framework, several limitations must be noted. This dataset could be collected in only one indoor environment; thus the results are highly dependent on the Wi-Fi, magnetic and camera quality of this corridor. We need to test on buildings with different geometry and floor layouts to assess generalization.

This is because the fusion experiments were only conducted on samples that had both LSTM predictions and valid distances from camera-based distance estimation. Thus, a smaller joint vision-test fraction was used for the last EKF and particle-filter comparison. However, detection on a larger camera-visible test set would give us stronger evidence of the fusion results.

Third, the LSTM model only used Wi-Fi Received Signal Strength Indicator (RSSI) (Shang and Wang, 2022) and magnetometer readings as predictor. Accelerometer and gyroscope data are provided in the dataset but excluded from the final on the feature vector. Furthermore, they applied the LSTM as a single-time step fingerprint classifier and not as a temporal tracking model.

Lastly, the vision and fusion stages need reliable person detection, camera geometry, camera-facing direction and distance calibration. Because these final fused coordinates also potentially contain errors from pose detection, ground-contact estimation, and camera calibration. In future, our method should be evaluated with more environments, more camera-visible samples, multi-camera fusion; and temporal tracking.

8 Conclusion and future work

This paper proposed a hybrid indoor localization framework that integrates WiFi RSSI, magnetometer fingerprints, vision-based distance estimation and probabilistic fusion. The LSTM model was employed to predict the starting location of the indoor X, Y coordinate by utilizing RSSI gathered from Wi-Fi as well as magnetometer readings. The vision model then predicted the distance between the camera and person based on CCTV images. Ultimately, this distance was not used as a coordinate to be predicted directly, but as a geometric constraint that helped correcting the LSTM result using EKF and particle-filter-based fusions.

The experimental results indicated that the LSTM model performed an exact-coordinate accuracy of 80.67% and a root-mean-square error (RMSE) of 1.5559 on the main test set. It can be seen that the average distance error of vision model is 0.159 m, and after fusion, the localization error with EKF was significantly reduced to 0.70 m, and with particle-filter-based method it was, respectively, 0.40 m as ground truth. The results demonstrate that utilizing distance constraints from vision can substantially enhance localization in Wi-Fi–magnetic fingerprinting.

The primary contribution of this work is the use of distance estimation from CCTV as a additional constraint for fingerprint localization. This leaves the estimation of global coordinate to LSTM Model and enhances this using knowledge of geometric distance with vision model.

In the future, testing this framework in a broader spectrum of indoor environments, obtaining additional camera-visible samples and generalizing this system to multi-camera or temporally across trackers are main focuses. In the future (but most likely not in the next release) accelerometer and gyroscope readings may also be included to make localization more robust.

Generative AI statement

The author(s) declared that Generative AI was used in the creation of this manuscript. OpenAI ChatGPT was used to assist with language editing, manuscript organization, and drafting of explanatory text. All generated or edited content was reviewed and verified by the authors, who take full responsibility for the final manuscript. Generative AI was not used to generate, modify, or analyze the experimental data or reported results.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Laura Belli, University of Parma, Italy

Reviewed by: Elmer Magsino, University of the Fraser Valley, Canada

Xu Yang, Shaanxi Energy Institute, China