Object detection for autonomous driving environmental perception and its applications

1 Context and motivation

1.1 Background

With the rapid development of intelligent connected vehicle technology, the transition from L2++ assisted driving to L4 fully autonomous driving is accelerating. The environmental perception system is the “visual nerve” of autonomous vehicles, which perceives the external traffic conditions and avoids driving risks. It directly affects the safety of vehicles and traffic efficiency in terms of operational stability, detection accuracy and real-time response capability (Chen et al., 2025; Ruan et al., 2025). Object detection is a core component of environment perception. Tasks performed in object detection mainly include the accurate and rapid detection of the types, locations, motion states, and shape characteristics of various objects in the surrounding traffic environment. Object detection provides accurate data as input for downstream processes such as path planning and speed regulation. Current autonomous vehicle perception systems integrate sensors that include visual cameras, Light Detection and Ranging (LiDAR), millimeter-wave radar, and ultrasonic radar. Perception characteristics vary significantly among these sensor types. Visual cameras offer rich texture features, high accuracy in object classification, and low hardware costs; however, they are highly susceptible to variations in light intensity, rain, snow, fog, and occlusions. These sensors also exhibit weak 3D spatial localization capabilities (Zhang et al., 2025).

1.2 Related works

To address insufficient perception accuracy and poor adaptability to complex scenes in autonomous driving, Sumalatha I et al. investigated multi-sensor fusion schemes. They compared different fusion strategies and constructed an adaptive fusion model using deep learning algorithms. Experimental results demonstrated that this method improved object detection accuracy and computational efficiency. The method effectively handled harsh environments and data delays, providing a feasible optimization scheme for object detection technology in autonomous driving environment perception (Sumalatha et al., 2024). Zhu S researched autonomous driving object detection technology. In this analysis, the author discussed the advantages and disadvantages of mainstream sensors, such as cameras and millimeter-wave radars. The author discussed perception weaknesses caused by complex operational scenarios, adverse weather conditions, and cyberattacks. This research highlighted the shortcomings of the existing environment perception system and recommended future improvements to counter any interference and enhance security (Zhu, 2025). To meet the technical requirements of autonomous driving environment perception, Liu F et al. systematically reviewed mainstream methods for visual object detection, depth estimation, and simultaneous localization and mapping. They compared the advantages and disadvantages of visual perception schemes, including monocular and stereo vision. The analysis showed that deep learning significantly improves detection accuracy and environment adaptation (Liu F. et al., 2025). Vehicle localization errors in collaborative perception for autonomous driving cause point cloud misalignment and reduce detection performance. To address this, Zhang M et al. designed a multi-scale fusion collaborative perception model through the use of lightweight networks with hierarchical alignment methods. The application of the proposed method to publicly available datasets confirmed that this method mitigated the adverse effects of localization errors. The detection accuracy and perception capabilities were greatly improved by the design, which was computationally efficient and capable of real-time operation (Zhang M. et al., 2026).

Multi-sensor data fusion remains challenging, and 3D object detection accuracy is often insufficient. To solve these problems, Lu C developed different types of 3D perception algorithms for detection, segmentation, and cross-sensor calibrations. These methods are a combination of point cloud processing techniques and deep neural network models. In particular, they introduced a lightweight feature extraction method as well as a detection framework that is efficient for perception improvements in complex environments and the real-world application of the object detection technique (Lu, 2025). Similarly, Shi P et al. combined point cloud processing with deep learning networks to design multiple 3D detection, segmentation, and cross-sensor calibration algorithms. The investigation proposed a lightweight feature extraction model and a detection framework. This work improved perception capabilities in complex scenes, enhanced the perception system, and advanced the application of object detection technology (Shi et al., 2025). Autonomous driving fisheye cameras suffer from image distortion and limited perception. To address these limitations, Qian Y et al. studied the principles and classical models of fisheye imaging. The researchers collected real and virtual datasets, reviewed correction methodologies in object detection and semantic understanding, and analyzed technical development trends (Qian et al., 2023). Single-sensor-based autonomous driving is vulnerable to any interference, and the accuracy of object detection is relatively low. To overcome these limitations, Li W et al. employed multiple sensors, such as LiDARs and cameras, and established an environmental perception system using adaptive algorithms and fuzzy reasoning. According to vehicle experiments, the proposed approach for detecting objects achieved much smaller errors and much higher detection accuracy. The proposed approach reliably ensured autonomous driving security and improved the performance of autonomous driving perception systems (Li et al., 2025).

To address the insufficient depth estimation accuracy in autonomous driving environment perception, Bong EJ et al. proposed a dense depth map estimation method using camera LiDAR fusion, which sequentially completes preprocessing, weighted clustering, surface modeling, point cloud sampling, and semantic segmentation image processing. Multi-scenario experiments showed that this method improves the accuracy of mid-to long-distance depth estimation, with better performance in average absolute error, robustness to different detection models, and no significant distortion in 3D reconstructed objects (Bong and Kee, 2026). To address inaccurate depth estimation and semantic information loss in 3D detection under splashing conditions in autonomous driving environment perception, Zuo C J et al. constructed a dual space representation cross fusion framework and designed a gate controlled LiDAR supervised bird’s-eye view and multi attention cross fusion module to achieve cross modal information interaction. Tests on the nuScenes dataset have shown that this method effectively improves the aforementioned shortcomings, achieving detection metrics of 71.8 mAP and 74.2 NDS, and enhancing the overall performance of 3D object detection (Zuo et al., 2025). To address the complex and challenging task of detecting rural road scenes in autonomous driving environment perception, Wei F et al. constructed a spatial channel collaborative attention YOLO (you only look at it once) network for rural roads, integrating spatial attention and shared semantics, and introducing Ghost module to achieve network lightweighting. Through testing and verification on both public and private datasets, the network has improved the detection accuracy of YOLOv8, while also considering model lightweighting, making it suitable for object detection tasks on rural roads (Wei et al., 2025). To address the difficulties in extracting dangerous driving behavior features and handling imbalanced sample classification in autonomous driving environment perception, Ni W et al. proposed an improved residual network that integrates coordinate attention, scale invariant feature transformation features, and multi-scale feature pyramids, and designed a joint loss function. Experiments have shown that this model improves classification accuracy by 0.6%–4.7% compared to traditional networks, and further optimization of the model’s generalization ability and computational efficiency is needed in the future (Ni and Bai, 2025).

1.3 Objectives and highlights

In summary, mainstream object detection techniques for autonomous driving include single-modal object and multi-modal object. Single-modal object detection uses either visual information or LiDAR sensor data. The architecture of such models is relatively simple, and their inference speed is fast. However, its adaptability to working conditions is poor, and perception performance drops sharply in complex scenes. Multi-modal fusion detection couples data from multiple sensors to utilize complementary advantages. This approach compensates for the deficiencies of a single modality and represents the core development direction of high-level autonomous driving environment perception technology. This study proposes an object detection technology for autonomous driving environment perception. Specifically, this study develops a single-stage detection framework that utilizes an improved YOLOv12 image detection network and a deep cascade coupling of LiDAR point clouds. Furthermore, this study designs a sensor decision-level dynamic fusion algorithm based on scene awareness. The algorithm adaptively allocates decision weights to vision and LiDAR according to real-time conditions, including lighting, weather, and object distance. This design achieves low-level feature binding and synchronous inference of image texture features and point cloud 3D spatial features.

Most modules adopted in this study are mature existing technologies, including YOLOv12 backbone, Coordinate Attention, CIoU loss, voxel downsampling, Euclidean clustering and camera-LiDAR coordinate registration. The unique core innovation of this paper is the scene-adaptive decision-level dynamic weight fusion strategy: an illumination-distance dual attenuation weight adjustment mechanism is constructed, combined with a weighted heuristic Bayesian confidence fusion rule to realize real-time adaptive reliability allocation for visual and LiDAR sensors under complex driving scenes. The proposed system implements only late decision-level fusion of independent single-modal detection results, bypassing low-level deep feature cascade coupling between the image and point cloud branches.

The structural organization of this paper is as follows. Part 1 explains the research background, research significance, and research prospects of autonomous driving object detection technology. Part 2 focuses on the object detection process designed for autonomous driving environment perception in this study. This section represents the primary focus and innovation of this research. Part 3 describes the experimental data and results analysis based on the methodology presented in Part 2. Part 4 draws conclusions from the experimental results, explains current design limitations, and suggests directions for future research.

2 Materials and methods

2.1 Driving environmental perception and object detection based on improved YOLOv12

Current vision-based object detection algorithms rely on image pixel texture features to achieve object identification. In scenes with weak lighting, occlusion, or long distances, object features undergo blurring and degradation, leading to significant missed detections and false alarms. Single LiDAR point cloud detection algorithms possess precise 3D distance measurement and localization capabilities. However, the sparsity of point clouds leads to poor classification and recognition accuracy of small and fine-grained objects. In addition, the original point cloud data are of a huge volume and high redundancy. Feeding directly into the detection network causes a sharp increase in computational overhead, which does not meet the real-time perception and response requirements in autonomous driving (Liu C. et al., 2025; Himanshu et al., 2024). The three-layer architecture of the autonomous driving system is shown in Figure 1.

The three-layer architecture of the autonomous driving system is shown in Figure 1. The perception system acquires environment information through optical cameras, LiDAR, computer vision, and high-precision maps. The decision system combines real-time road conditions with path planning to generate control commands. The execution system relies on vehicle subsystems to complete drive-by-wire operations. These three components collaborate to constitute a complete closed-loop control workflow for autonomous driving (Xiao et al., 2024; Wang, 2025). To maintain the tradeoff between object detection performance, 3D localization accuracy, and real-time vehicle inference capabilities, this study proposes a detection architecture that uses an improved YOLOv12 deep learning network and point cloud processing. The YOLOv12 network is shown in Figure 2.

As shown in Figure 2, YOLOv12 adopts a single-stage end-to-end detection architecture. The backbone network extracts multi-scale image features and divides them into small-, medium-, and large-scale feature groups, which are then sent to the detection heads to achieve accurate recognition and localization of targets with different sizes (Wang et al., 2025a; Zhang T. et al., 2026).

This study selects YOLOv12 as the basic image detection backbone, which embeds native Area Attention to capture global region features. Nevertheless, Area Attention lacks independent horizontal and vertical spatial coordinate modeling capability, resulting in weak feature response for slender small targets (pedestrians, traffic cones) under occlusion and low-light conditions (Wang et al., 2025b). To remedy this defect, lightweight Coordinate Attention (CA) modules are hierarchically embedded after each stage of the YOLOv12 backbone. The two attention mechanisms form a complementary structure: Area Attention extracts coarse global regional context, while CA supplements precise pixel coordinate weighting along horizontal and vertical axes, eliminating mutual feature conflict and redundant computation. The channel weight allocation calculation of the CA mechanism is shown in Equation 1.

Z c = 1 H × W ∑ i = 1 H ∑ j = 1 W X c i , j (1)

In Equation 1, Z c represents the global pooling feature value of the c-th channel, and X c i , j represents the pixel feature value at coordinate i , j of the c-th channel feature map. On this basis, this study performs CA encoding optimization in both horizontal and vertical directions. The encoding calculation is shown in Equation 2.

Z c h h = 1 W ∑ j = 1 W X c h , j Z c w w = 1 H ∑ i = 1 H X c i , w (2)

In Equation 2, Z c h h represents the channel feature encoding value at position h in the height direction, and Z c w w represents the channel feature encoding value at position w in the width direction. This study aims to cut down on the YOLOv12 network’s regression errors and enhance bounding box fits. To do this, it swaps out the basic Intersection over Union (IoU) loss function for an improved Complete Intersection over Union (CIoU) version. The improved loss function yields superior performance because it accounts for bounding box overlap, center point distance, and aspect ratio consistency. The loss function is defined by Equation 3.

L C I o U = 1 − I o U + ρ 2 b , b g t c 2 + α v (3)

In Equation 3, ρ 2 b , b g t denotes the squared Euclidean distance between the centers of the predicted box and the ground-truth box, c denotes the diagonal length of the minimum enclosing rectangle, α denotes the weight balancing factor, and v is the aspect ratio similarity coefficient (Wu et al., 2026). The LiDAR branch adopts a rule-based geometric classification strategy without relying on additional trained neural network classifiers. After Euclidean clustering with 0.3 m distance threshold, four geometric features of each target cluster are extracted: vertical height, point cloud density, surface normal vector and average distance to the vehicle. Fixed threshold rules distinguish three object categories: 1) Vehicle: vertical height 1.2–2.5 m, point density >8 points/m³; 2)Pedestrian: vertical height 1.0–2.0 m, point density 3–8 points/m³; 3) Non-motor vehicle: vertical height 0.6–1.3 m, point density <5 points/m³. The clustering threshold of 0.3 m is selected after sensitivity testing of 0.1–0.5 m intervals; thresholds smaller than 0.2 m cause over-segmentation of connected vehicles, while thresholds larger than 0.4 m lead to under-segmentation of adjacent pedestrians. The 0.3 m threshold achieves the highest clustering accuracy of 94.7%. To process massive volumes of raw LiDAR data, including redundant ground points and distant noise, a pass-through filter is applied. It initially removes these unnecessary distant noise points. The filtering threshold constraint is calculated by Equation 4.

P v a l i d = p x , y , z | z min ≤ z ≤ z max , x 2 + y 2 ≤ R max 2 (4)

In Equation 4, P v a l i d represents the filtered valid point cloud set, z min and z max represent the threshold values of the effective driving elevation range, and R max represents the effective LiDAR detection distance threshold. The point cloud data is filtered and then downsampled using a voxel-based method, which reduces data dimensionality while preserving the 3D structure. The math for the voxel grid partition is shown in Equation 5.

V i d x = ⌊ x − x min s x ⌋ , ⌊ y − y min s y ⌋ , ⌊ z − z min s z ⌋ (5)

In Equation 5, V i d x represents the voxel grid index coordinate of the point cloud, x min, y min, and z min represent the minimum values of the point cloud coordinates, and s x, s y, and s z represent the 3D dimensions of the voxel grid (Yuan et al., 2023).

The complete LiDAR object detection pipeline of this study consists of four sequential steps: 1) Point cloud preprocessing: Pass-through filtering removes invalid ground and long-distance noise points via Equation 4, followed by voxel grid downsampling based on Equation 5 to reduce data redundancy while retaining 3D spatial contour information of targets; 2) Handcrafted geometric feature extraction: Calculate point cloud height, density, normal vector and distance features within each voxel grid to distinguish vehicles, pedestrians and non-motorized vehicles; 3) Point cloud target clustering: Euclidean distance clustering is adopted to segment independent object point clusters, with clustering threshold set to 0.3 m to avoid under-segmentation of adjacent traffic participants; 4) Matching confidence calculation: Compute LiDAR target matching confidence via Equation 8 according to the number of valid matched points within each cluster, outputting 3D bounding box coordinates and category matching confidence as the final LiDAR branch detection results. All point cloud processing modules are implemented based on the PCL library, without additional deep learning network for LiDAR branch, which reduces overall computational load of the multimodal perception system.

This study builds a spatiotemporal registration transformation model to unify LiDAR point clouds and visual images and match their data accurately. The coordinate transformation between the LiDAR and camera systems is determined using Equation 6.

μ = 1 λ K R P l i d a r + T (6)

In Equation 6, μ = u , v , 1 represents the homogeneous image pixel coordinate, K represents the camera intrinsic parameter matrix, R and T represent the rotation matrix and translation matrix between the radar and camera extrinsic parameters, respectively, and λ represents the scale normalization factor. The overall structure of the improved YOLOv12 network is shown in Figure 3.

In Figure 3, the network consists of four modules: the Input layer, the improved Backbone network, the Neck feature fusion layer, and the Head output layer. Lightweight CA modules are embedded throughout the network to adapt to autonomous driving image detection tasks.

2.2 Multidimensional object perception algorithm based on sensor decision-level fusion

Although both the improved YOLOv12-based visual detection and LiDAR-based single-modal detection have been enhanced, each still possesses limitations. Visual detection exhibits high classification accuracy but suffers from poor 3D localization, whereas LiDAR-based detection provides precise localization but lacks robust classification confidence. If a traditional fixed-weight decision fusion strategy is directly used, it cannot adapt to dynamic driving scenarios (Xi, 2024). This study uses single-modal detection outputs and adds a dynamic scene perception mechanism and a Bayesian probability optimization model. It builds a sensor decision-level adaptive fusion multidimensional perception algorithm. The method computes detection confidence from both sensors and a scene reliability factor, all in real time. The radar–camera spatiotemporal registration diagram is shown in Figure 4.

As shown in Figure 4, the hierarchical mapping relationship among LiDAR, camera, and image pixel coordinate systems is established. A rigid transformation in 3D space is achieved through extrinsic rotation R and translation T. Then, the camera intrinsic matrix K completes the projection from 3D space to 2D space. Finally, temporal synchronization aligns heterogeneous sensors in space and time, providing an accurate geometric constraint for multimodal object detection and feature fusion (Fu et al., 2024). First, this study defines the classification confidence of the improved YOLOv12 visual detection branch and the matching confidence of the LiDAR point cloud detection branch. These metrics provide the basis for fusion weight calculation. The visual detection confidence is calculated by Equation 7.

C c a m = S i g m o i d f c a m X i m g (7)

In Equation 7, C c a m represents the normalized confidence of the visual sensor, ranging from 0 to 1. f c a m X i m g represents the raw confidence score output from the improved YOLOv12 image feature inference. Sigmoid is the normalization activation function that maps the raw score into a standard confidence range (Zhang et al., 2024). The corresponding LiDAR point cloud matching confidence is calculated by Equation 8.

C l i d a r = N m a t c h N t o t a l (8)

In Equation 8, C l i d a r is the normalized confidence of LiDAR target matching. N m a t c h is the number of matched valid point cloud points. N t o t a l is the total number of sampled point cloud points in the target region. The confidence reflects LiDAR detection reliability through the matching ratio. This study presents adaptive fusion weight adjustment by adding a lighting intensity factor. It measures how illumination affects visual sensing performance in driving scenes. The illumination attenuation coefficient is calculated by Equation 9.

η l i g h t = e − k I − I s t d (9)

In Equation 9, η l i g h t is the illumination attenuation coefficient, k is a light sensitivity constant, I is the real-time illumination intensity, and I s t d is the standard optimal illumination level. As the illumination deviation increases, visual reliability decreases, leading to a corresponding decline in the attenuation coefficient. This study also introduces a distance factor to quantify how far-range targets affect point cloud sparsity. The distance attenuation coefficient is figured out using Equation 10.

η d i s = 1 − d d max η c a m , d = e − d 35 (10)

In Equation 10, η d i s is the distance attenuation coefficient, d is the distance between the target and the autonomous vehicle, and d max is the maximum effective LiDAR detection range. η c a m , d denotes the distance attenuation coefficient of visual feature pixels, which quantifies the gradual reduction of valid texture pixels as target distance increases. The raw weight of each sensor is jointly determined by illumination attenuation and distance attenuation: w v i s , r a w = η i l l · η c a m , d , w l i d , r a w = η d i s, The final normalized fusion weights are recalculated by: w v i s = w v i s , r a w w v i s , r a w + w l i d , r a w , w l i d = w l i d , r a w w v i s , r a w + w l i d , r a w.

As distance increases, point cloud sparsity increases and sensing reliability decreases. Based on the two attenuation coefficients, the adaptive fusion weights of visual and LiDAR sensors are calculated. The normalized weights are shown in Equation 11.

α = η l i g h t · C c a m η l i g h t · C c a m + η d i s · C l i d a r β = η d i s · C l i d a r η l i g h t · C c a m + η d i s · C l i d a r = 1 − α (11)

In Equation 11, α and β represent the dynamic fusion weights of vision and LiDAR, respectively. The weights adjust dynamically according to illumination and distance. Under strong light or nighttime conditions, the visual weight decreases while the LiDAR weight increases. For long-distance targets, the LiDAR weight decreases, while the visual classification weight increases. The attenuation coefficients in Equation 10 and Equation 11 only reflect the intrinsic sparsity decay law of LiDAR point clouds with distance. However, the final fusion weights are jointly determined by two coupled factors: distance attenuation coefficient of LiDAR and illumination attenuation coefficient of vision. At long distances above 40 m, the number of valid image texture pixels also decreases sharply, leading to a synchronous drop in visual classification reliability. Even if the LiDAR attenuation coefficient reduces with distance, the visual sensor’s overall reliability declines faster, so the normalized weight allocation result manifests as rising LiDAR weight within 0–70 m. To enhance robustness against interference, an enhanced Bayesian probability fusion model optimizes the final confidence output, thereby suppressing noise from individual sensor modalities. The fusion confidence is calculated by Equation 12 (Chen et al., 2024).

P f u s e = w v i s · P v i s + w l i d · P l i d · P v i s P v i s + P l i d (12)

This weighted confidence fusion rule is a heuristic improved Bayesian combination method based on conditional probability assumptions. This study assume that the classification confidence P v i s output by YOLOv12 represents the class conditional likelihood of visual features, and the LiDAR matching confidence P l i d reflects the spatial likelihood of target point cloud distribution. The dynamic adaptive weights w v i s, w l i d calculated from illumination and distance attenuation are introduced to quantify the reliability of each sensor’s likelihood. The cross-modal likelihood coupling term P v i s / P v i s + P l i d balances the mismatch of confidence distribution dimensions generated by different single-modal detection branches, and the final fused confidence P f u s e serves as the posterior probability of target existence under multi-sensor joint observation. This rule does not rely on strict prior probability distribution assumptions of targets, so it is defined as an enhanced heuristic Bayesian fusion model rather than a fully analytically derived Bayesian formula. Before the bounding box fusion calculated by Equation 13, a two-stage cross-modal target association matching mechanism is implemented to avoid mismatching heterogeneous targets: 1) Spatial overlap screening: Project all LiDAR 3D bounding boxes to the 2D image plane via the radar-camera extrinsic transformation matrix in Equation 6. Calculate the IoU between each projected LiDAR box and each visual detection box; only target pairs with IoU ≥0.3 are retained as candidate matching pairs; 2) Category consistency verification: Match the classification category label of visual detection and the geometric clustering category of LiDAR point clouds. Candidate pairs with inconsistent categories are eliminated to filter out cross-object mismatches. Only the target pairs passing both spatial IoU screening and category verification will enter the linear weighted fusion calculation in Equation 13. For unmatched single-modal targets, their original detection results are directly retained without fusion, which effectively avoids noise interference caused by mismatched targets. The linear weighted fusion strategy takes normalized dynamic sensor weights as coefficients, which adaptively reflects the reliability difference between vision and LiDAR under real-time driving scenes, so the fused bounding box can balance the classification advantage of images and the localization advantage of point clouds.

A fusion regression model for object bounding boxes and 3D coordinates is constructed. The fused 2D bounding box is calculated by Equation 13.

B f u s e d = α · B c a m + β · B l i d a r p r o j (13)

In Equation 13, B f u s e d is the fused 2D bounding box coordinate, B c a m is the original visual bounding box, and B l i d a r p r o j is the projected 2D bounding box from point cloud data. The fused 3D coordinate correction is calculated by Equation 14.

P f u s e d = α · P c a m r e p r o j + β · P l i d a r (14)

In Equation 14, P f u s e d represents the final corrected 3D coordinates, P c a m r e p r o j is the back-projected 3D coordinates from the image bounding boxes, and P l i d a r represents the original LiDAR 3D coordinates. Finally, this study defines a multidimensional perception evaluation metric. The back-projected 3D coordinates of the camera branch P c a m r e p r o j in Equation 14 are not solved solely using monocular 2D bounding boxes. During spatiotemporal synchronization, each image pixel is matched with the depth value of the aligned LiDAR point cloud, and the depth constraint of the target bounding box center is extracted to complete the reverse projection from 2D image coordinates to unified 3D world coordinates. The depth information used for camera branch back-projection comes from synchronous LiDAR raw data, which provides the unique depth constraint required for monocular 3D coordinate solving. The weighting coefficients in Equation 15 are fixed as ω 1 = 0.5, ω 2 = 0.5, indicating that target classification and 3D localization performance are equally important for autonomous driving environmental perception. The positioning accuracy percentage is normalized by the maximum allowable 3D localization error e max = 0.3 m specified by vehicle perception standards. The overall detection accuracy is calculated by Equation 15.

A c c t o t a l = ω 1 · A c c c l s + ω 2 · A c c l o c A c c l o c = 1 − e ¯ 3 D e max × 100 % (15)

In Equation 15, e ¯ 3 D denotes the average 3D center point error of all detected targets in meters. A c c t o t a l is the overall perception accuracy, A c c c l s and A c c l o c represent classification accuracy and 3D localization accuracy, and ω 1 and ω 2 are weighting factors for classification and localization balance. Overall, the workflow of the sensor decision-level dynamic fusion algorithm is shown in Figure 5.

In Figure 5, the algorithm adopts a dual-branch parallel detection structure. Vision and LiDAR independently perform object perception and output confidence scores. Then, illumination and distance conditions are used to compute attenuation factors in real time. Adaptive fusion weights are assigned to the two sensors. A Bayesian probability model helps make decisions, fixing 2D object shapes and 3D positions. This avoids the limitations of fixed-weight methods and fully leverages the classification capability of vision and the precise positioning capability of LiDAR. Consequently, this enhances detection accuracy and robustness under challenging driving conditions. A two-stage cross-modal target matching strategy is adopted, and the Hungarian bipartite graph matching algorithm is introduced to resolve one-to-many or many-to-one matching conflicts between camera boxes and LiDAR clusters: 1) Spatial screening: Project all LiDAR 3D bounding boxes to the 2D image plane via extrinsic transformation matrix, calculate IoU between each projected box and visual detection box, and retain candidate pairs with IoU ≥0.3; 2) Category consistency filtering: Eliminate candidate pairs with inconsistent classification labels; 3) Optimal assignment: Construct bipartite graph with IoU as matching weight, use Hungarian algorithm to assign each target a unique cross-modal matching pair, and discard unpaired single-modal targets. The IoU threshold of 0.3 is determined by sensitivity analysis of 0.1–0.5 m intervals; thresholds above 0.4 will filter out partially occluded matching targets, while thresholds below 0.2 introduce a large number of false cross-modal matches, and 0.3 achieves the maximum matching accuracy of 92.6%. For completely occluded targets that fail to pass matching screening, their original single-modal detection results are directly retained without fusion processing.

Taking the target distance d = 30 m and standard illumination I s t d = 5000 l x as a complete calculation example: The illumination attenuation coefficient η i l l = 0.785 under standard light; visual distance attenuation coefficient η c a m , d = e − 30 / 35 = 0.424; LiDAR distance attenuation coefficient η d i s = 1 − 30 / 70 = 0.571. The raw visual weight w v i s , r a w = 0.785 × 0.424 = 0.333, raw LiDAR weight w l i d , r a w = 0.571. Normalized weights are solved as w v i s = 0.368, w l i d = 0.632. As target distance increases, the effective texture pixels from the camera decrease exponentially, leading to a continuous reduction in the raw visual weight. When d = 70 m, the effective detection range of the LiDAR reaches its limit, causing its weight to drop to zero; thus, the perception task relies entirely on visual image features.

3 Results

3.1 Experimental environment and parameter settings

All experiments were jointly conducted based on the Karlsruhe Institute of Technology and Toyota Technological Institute Dataset (KITTI) autonomous driving standard public dataset and a self-built dataset from real-vehicle road tests. The self-built dataset is collected by an autonomous test vehicle equipped with a 16-line mechanical LiDAR (Velodyne VLP-16, 10 Hz sampling frequency, 360° horizontal field of view) and an industrial RGB camera (1920 × 1,080 resolution, 25 fps frame rate). Hardware time synchronization is realized through trigger signals with synchronization error less than 2 m. Data collection covers 12 continuous driving routes including urban morning peak, suburban road, night low light, light rain and haze weather, forming a total of 214 independent driving sequences. All targets are manually annotated with 2D bounding boxes, 3D coordinates and category labels following the KITTI annotation standard. To eliminate data leakage caused by adjacent similar frames, the dataset is split at the sequence level rather than random frame sampling: 70% of independent driving sequences are allocated to the training set, 20% to the validation set, and the remaining 10% to the test set.

The experimental environment and parameter configurations were shown in Table 1.

TABLE 1. Experimental environment and parameter configurations.

The latency indicators involved in the experiment are uniformly defined with clear measurement scope as follows: Single-modal LiDAR detection latency (35 m in Figure 6A): Contains full point cloud preprocessing (pass-through filtering + voxel downsampling, 6.25 m), geometric clustering, feature extraction and target matching confidence calculation, excluding image-related modules; Native YOLOv12 image detection latency: Only includes image preprocessing and network forward inference, without point cloud processing and fusion steps; Proposed multimodal fusion algorithm total latency (28 m): Covers all serial modules including image YOLO inference, full point cloud preprocessing (6.25 m), radar-camera spatiotemporal registration, cross-modal target association matching, dynamic weight calculation, Bayesian confidence fusion and bounding box post-processing.

The proposed dual-branch parallel execution architecture realizes simultaneous image inference and point cloud preprocessing, which offsets the independent time consumption of single-modal LiDAR detection, resulting in lower overall total latency than pure LiDAR single-modal detection.

All comparative and ablation experiments were repeated five times with different fixed random seeds (seed = 10, 20, 30, 40, 50). The reported performance metrics represented the average value of five repeated training runs, and the standard deviation of each indicator was provided in all tables to reflect result fluctuations. The final model for testing was selected based on the maximum mAP@0.5 achieved on the validation set during training.

Lightweight CA modules are embedded after the output feature layer of each of the four backbone stages of YOLOv12, with channel compression reduction ratio set to 16. The original Area Attention module is retained to extract global regional context, and the CA module supplements horizontal and vertical coordinate feature weighting. All training hyperparameters, data augmentation strategies and post-processing thresholds are listed in Table 2 for full reproducibility. The Coordinate Attention mechanism follows the original implementation proposed in (Guan et al., 2026), and YOLOv12 adopts the official open-source baseline network.

TABLE 2. Network parameter difference.

3.2 Analysis of comparative and ablation experiment results

Standard KITTI detection metrics including per-class 2D AP, BEV AP and 3D AP@0.5 are supplemented as core evaluation indicators. Small targets are uniformly defined as objects with pixel height less than 30 pixels in the image plane, consistent with KITTI official evaluation rules. Detection results are further grouped by target distance intervals of 0–20 m, 20–40 m and 40–70 m to analyze the distance-adaptive performance of each algorithm.

Figure 6 presents the results of the comparative and ablation experiments. As shown in Figure 6a, the proposed improved fusion algorithm achieved significant comprehensive performance advantages compared with various single-modal and traditional fusion algorithms. Core accuracy and real-time indicators reached an optimal balance. Compared with the baseline native YOLOv12 algorithm, the mean average precision (mAP@0.5) increased by 9.73% and the small-object recall rate grew by 16.42%, while the single-frame inference latency of the improved YOLOv12 backbone with CA modules rose by only 2.1 m compared with the native YOLOv12 baseline on an RTX 3090. The localization error was compressed to 0.08 m compared with the fixed weight fusion algorithm. The complete multimodal pipeline of the proposed method achieves an overall total latency of 28 m (including point cloud preprocessing, registration, matching and fusion), which meets the low-latency response requirement of real-time autonomous vehicle perception, and successfully addressed the core issues of unbalanced single-modal detection accuracy and poor scene adaptability of traditional fusion algorithms. The ablation experimental data shown in Figure 6b indicated that the three core innovative modules of this study were critical to performance improvement. Detection accuracy decreased significantly after the removal of any module. Among these components, the dynamic weight fusion module makes the most contribution to the overall perception performance. After removing this module, the comprehensive perception accuracy is improved by 5.83 percentage points (6.54% relative improvement) compared with fixed-weight fusion algorithm. Which demonstrated that the core of scene-adaptive weight adjustment is to enhance the perception robustness under complex working conditions. The CA module primarily improved the feature extraction capability for small and occluded objects. These results demonstrated that the small-object recall rate dropped by 8.52% upon the removal of this module.

Figure 7 presents a comparison between the improved YOLOv12 and the original YOLOv12, with a particular emphasis on the CA mechanism. Proportion of effective features: Ratio of pixels with activation value higher than threshold T_act = 0.4 in target bounding box regions; Proportion of background noise: Ratio of pixels with activation value higher than T_act = 0.4 outside all target bounding boxes; Feature response intensity: Average activation value of all effective feature pixels within target regions; Feature loss rate: Relative reduction of total effective feature pixels compared with native YOLOv12 under identical input conditions. As seen in Figures 7a,b, on average the ratio of effective features increased by more than 20% with the greatest increase in low-illumination and small-scale target conditions. Meanwhile, the ratio of background noise was remarkably reduced and the intensity of feature response was increased. To alleviate information loss during feature propagation, the associative coupling between spatial position information and channel-wise feature representations was strengthened, reducing the overall feature loss rate by approximately 14%. This suppressed irrelevant background interference and confirmed that the improved architecture effectively enhanced feature extraction quality under complex autonomous driving conditions.

Figure 8 shows the working effect of the passthrough filter on the LiDAR point cloud. From Figures 8a,b, it can be seen that the unprocessed point cloud contained a large amount of ground-plane returns and far-away noise points, which together accounted for a large proportion of the raw data. After filtering, these extraneous returns were systematically removed, leaving only the point cloud data relevant to the target and needed to drive perception tasks. The resulting spatial distribution of XYZ coordinates exhibited markedly improved regularity and compactness, with data redundancy reduced considerably and overall data efficiency enhanced.

3.3 Analysis of autonomous driving environmental perception and object detection results

Camera-LiDAR extrinsic calibration is completed using a 12 × 9 chessboard calibration target, with 500 groups of synchronous image-point cloud frame pairs collected for calibration parameter solving, and calibration parameters remain fixed in all subsequent test experiments. Quantitative registration evaluation metrics are defined as follows: Pixel projection deviation: Average Euclidean pixel distance between LiDAR projected box center and visual detection box center across all test targets; 3D matching error: Average Euclidean distance between LiDAR cluster 3D center and camera back-projected 3D center; Target structure retention rate: Ratio of reserved target contour point number after voxel downsampling to original raw point number; Cross-modal target overlap ratio: Average IoU between LiDAR projected 2D box and visual detection box of all matched targets. Figure 9 illustrates the downsampling by point cloud voxelization in comparison with radar-camera spatiotemporal registration. As shown in Figure 9a, the voxelization is indeed reducing the raw data. Each frame of points is reduced by more than 68%, which effectively removes the redundant points. The processing time for each frame dropped to 6.25 m, thereby reducing the computational load on the onboard system. The algorithm introduced only a minor contour error of 2.16 cm, while the overall structural retention rate of detected targets remained at 96.35%. As shown in Figure 9b, the spatiotemporal registration model applied extrinsic matrix calibration and scale normalization constraints, reducing the pixel projection deviation between radar and camera to 2.39 pixels and constraining 3D matching error to within 0.068 m. This largely eliminated spatial misalignment between heterogeneous sensors. Temporal synchronization error was significantly reduced, and cross-modal target overlap improved by 31.43%.

Figure 10 shows sensor dynamic weights, detection confidence of single-modality and fusion algorithms, and sensor adaptive weights. As shown in Figure 10a, visual sensor weight increased steadily as ambient illumination rose. Under strong backlight interference, visual weight dropped to 0.24 and the system automatically raised LiDAR weight to compensate for visual imaging degradation. Under standard natural lighting, the weights of the two sensor types approached equilibrium, ensuring effective simultaneous fusion of both modalities. In Figure 10b, single-modality algorithms clearly struggled with different scene conditions. Visual detection performance degraded severely under low-light and backlit conditions, whereas LiDAR performance remained highly stable. Yet, LiDAR’s overall limit was not that impressive either. The fixed-weight fusion did not help much either. The proposed dynamic decision fusion method, using scene-adaptive weight adjustment and Bayesian optimization, boosted average confidence by over 0.12 across all tested scenarios. This approach successfully compensated for the limitations of single-modal methods, rendering detection more reliable under challenging conditions.

The fixed hyperparameters adopted in all calculations are: maximum effective LiDAR detection range D max = 70 m, light sensitivity constant k l = 0.8, standard optimal illumination I s t d = 12000 lux. Figure 10c illustrates the variation trend of normalized sensor fusion weights with target distance under standard illumination. W6ithin 0–70 m, the LiDAR weight gradually declines as distance increases, while the visual weight rises monotonically. At the maximum effective detection distance of 70 m, LiDAR weight decreases to 0 and the visual sensor independently completes target detection. Intermediate calculation examples at 30 m and 60 m are supplemented in Table 3 to intuitively display the weight calculation process under different target distances.

TABLE 3. Intermediate weight calculation results (I_std = 5000Lx, C_cam = 0.8,d_max = 70 m).

Table 4 presents a quantitative comparison of multi-dimensional perception accuracy metrics. The proposed dynamic fusion method reduced the average 3D localization error to 0.068 m, while the classification accuracy reached 95.27%. The comprehensive perception accuracy was improved by 9.74 percentage points (a 12.7% relative improvement) compared with fixed-weight fusion. The following analysis mainly takes classification accuracy and absolute 3D localization error in meters as core evaluation indicators to avoid ambiguity caused by custom normalized percentage metrics.

TABLE 4. Quantitative comparison of multi-dimensional perception metrics.

Table 5 shows the Model Parameter&Inference Latency Ablation Comparison. The lightweight CA module only adds 0.72 M parameters and brings an extra 2.1 m single-frame inference delay, which maintains the real-time performance of the detection network.

TABLE 5. Model Parameter & inference latency ablation comparison.

The visual and LiDAR branches run simultaneously on independent GPU computing streams, so the main time consumption of the entire system is determined by the slower LiDAR branch (18.55 milliseconds on RTX3090). The results are shown in Table 6. All subsequent registration, matching and fusion modules are executed sequentially after both branches output detection results, and the accumulated serial time consumption is 6.0 m. The sum of the maximum parallel branch time and serial post-processing time matches the reported total latency of 28 m, ensuring logical consistency across the evaluation metrics. All latency results are the average value of 2000 random test frames with CUDA GPU synchronization enabled, and the Orin X edge platform results reflect the actual vehicle deployment performance.

TABLE 6. Step-by-step inference latency breakdown (average of 2000 test frames, GPU synchronization enabled).

All ablation experiments and comparative detection results were separately evaluated using the public KITTI dataset and the self-built road test dataset to distinguish the model performance under standard open data and real complex vehicle operating conditions. Supplementary cross-dataset generalization tests are conducted: the model is trained on the KITTI training set and directly evaluated on the self-built test set to verify the cross-scene generalization ability of the proposed algorithm. Table 7 lists the independent comprehensive accuracy results of each algorithm on two datasets.

TABLE 7. Comprehensive perception accuracy (%) on KITTI and self-built dataset.

To further verify the competitiveness of the proposed algorithm, comparative experiments with several recently published state-of-the-art multi-modal autonomous-driving detection approaches are supplemented, as shown in Table 8. All algorithms are tested under identical hardware (RTX3090) and the mixed KITTI + self-built test dataset. As illustrated in Table 8, compared with MVX-Net, PointPainting, BEVFusion-Lite and FusionYOLO, the proposed method achieves the highest mAP@0.5 and small-object recall, while maintaining the lowest inference latency. Different from feature-level fusion schemes such as BEVFusion-Lite which require heavy cross-modal feature encoding computation, this work adopts decision-level adaptive fusion combined with lightweight improved YOLOv12 backbone. It avoids massive feature-fusion overhead and realizes a favorable trade-off between detection accuracy and real-time performance for vehicle edge-computing scenarios.

TABLE 8. Comparison with recent SOTA multi-modal object detection methods.

The adopted Coordinate Attention (CA) module is computationally lightweight compared with other popular attention blocks. Compared with SE-Attention (channel-only attention) and CBAM (sequential channel-spatial attention), CA introduces horizontal-vertical coordinate decomposition without constructing large-dimensional spatial convolution kernels. Under our backbone configuration, CA brings 0.72 M extra parameters, and the additional FLOPs are 0.21 G. By contrast, CBAM would introduce 1.14 M parameters and 0.38 G FLOPs under the same feature dimension, while SE-Attention yields 0.48 M parameters but lacks explicit spatial coordinate modeling capability. Therefore, CA achieves a good balance: it provides horizontal-vertical spatial location awareness with moderate computational overhead, which fits the real-time constraint of autonomous-driving edge hardware.

4 Conclusion

This study addressed major challenges in autonomous driving under adverse environmental conditions. It tackled the poor adaptability of single-sensor object detection, the weak scene handling of traditional multi-modal fusion methods, and the struggle to balance detection accuracy with real-time needs, and it dealt with high rates of missed and false detections during extreme conditions. By combining improved YOLOv12 deep learning image detection with LiDAR point cloud preprocessing and registration, a dual-branch single-modality high-accuracy detection framework was constructed, and a sensor decision-level adaptive multi-dimensional object perception fusion algorithm was developed. Results showed that the spatiotemporal registration model reduced the pixel projection deviation between radar and camera to 2.39 pixels and constrained 3D matching error to within 0.068 m through extrinsic matrix calibration and scale normalization constraints, largely eliminating spatial misalignment between heterogeneous sensors. This approach significantly reduced the temporal synchronization error and increased the cross-modal target overlap by 31.43%. The dynamic adaptive multi-sensor fusion weight adjustment improved object detection and its ability to handle interference in all weather and scenes for self-driving cars. The model also featured a lightweight architecture and fast inference speed. The lightweight architecture and low inference latency meet the real-time computing constraints of vehicle edge hardware under the tested urban, night, light rain and haze scenes. The proposed method achieves reliable environmental perception under moderate complex weather, while severe snow and icy conditions remain to be further optimized before full large-scale engineering deployment. However, this study has several remaining limitations. Specifically, severe snow and icy conditions degrade both sensor modalities simultaneously. For the visual camera, snowflakes produce heavy foreground occlusion, image over-exposure and blurring, which significantly reduce valid texture features for classification. For LiDAR point clouds, falling snow generates massive noise point echoes; accumulated snow on road surfaces changes ground reflection characteristics and distorts target point-cluster segmentation results. In addition, ice and snow coverage on vehicle-mounted sensor lenses further reduces the effective signal-to-noise ratio for both modalities. Since our dynamic fusion strategy relies on valid feature outputs from both vision and LiDAR branches, simultaneous quality degradation of the two modalities will weaken the adaptive weight adjustment effect, leading to increased missed detection and false-positive rates under heavy snow-ice weather. Point cloud scattering interference under severe rain, snow, and icy conditions requires further optimization. The ability to predict and associate motion trajectories of dynamic targets at high speeds remains insufficient. Generalization to multi-class irregularly shaped traffic objects also needs improvement. The future work will address the design of advanced algorithms for point cloud optimization in bad weather conditions and the integration of a temporal motion prediction module for the target paths. Moreover, this study aims to develop a unified model to integrate object detection and trajectory prediction. The integration is expected to achieve major breakthroughs in the environmental perception technology for autonomous driving through continuous iteration.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Chengxi Zhang, Jiangnan University, China

Reviewed by: Tianya Wu, Nanchang University, China

Fadilla 'Atyka Nor Rashid, National University of Malaysia, Malaysia