A unified lightweight ADAS architecture integrating traffic sign recognition, forward collision warning, and bimodal distillation for autonomous perception

1 Introduction

Autonomous vehicles (AVs) are playing an increasingly important role in modern transportation systems. To operate safely and efficiently, AVs must accurately perceive their surroundings and make driving decisions comparable to those of experienced human drivers. This capability has the potential to reduce traffic accidents, improve road safety, and enhance overall transportation efficiency. Advanced perception technologies enable vehicles to detect and interpret their environment, recognize traffic signs, predict pedestrian movements, and provide timely warnings or corrective actions, making them a fundamental component of intelligent and safer transportation systems. Despite their adaptability, human drivers are subject to limitations in perception, reaction time, and attention that can affect driving performance. This is because of anatomy, reaction time, discomfort, and split attention (e.g., answering a call, adjusting a radio) that can significantly diminish effective perception for a critical few seconds. These limitations are reflected in global road accident statistics, highlighting the need for intelligent driver assistance and autonomous driving technologies. Consequently, the development of automation is primarily motivated by the goal of improving road safety and reducing human error. The rapid expansion of vehicular datasets, which capture an ever-widening range of traffic scenarios and environmental conditions, has significantly advanced autonomous driving. Data-driven perception has been made possible by the steady accumulation of both organized and unstructured driving data, as reported by (Masello et al., 2022a). The availability of large-scale driving data has enabled intelligent automation frameworks to support a wide range of driving functions, from lane keeping and adaptive cruise control to advanced situational awareness and decision-making, which were traditionally performed by human drivers (Bathla et al., 2022). These capabilities are further expanded by the development of vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication, which allows for cooperative awareness that surpasses the sensors of a single vehicle (Elliott et al., 2019).

The main layers of the self-driving car stack are perception, localization, prediction, planning, and control, according to classical structures. Perception reliability at the front of this pipeline has a cascading effect on all that follows downstream, according to early work by Gordon and Lidberg (Gordon and Mathias, 2015). A missed pedestrian detection is more than a classification error; it represents a potential safety risk, particularly in mixed-traffic environments where autonomous vehicles operate alongside human-driven vehicles (Shladover, 2018; Yue et al., 2023).

Advanced Driver-Assistance Systems (ADAS) are an essential component of modern intelligent transportation systems, providing real-time environmental monitoring, hazard detection, and driver assistance. Unlike Level 4 and Level 5 autonomous vehicles, which are designed for fully autonomous operation, ADAS assists the driver by issuing timely warnings and performing limited automated interventions when necessary to enhance road safety. CNN-based lightweight architectures are specifically well-suited to ADAS perception tasks to achieve a competitive accuracy under computational budgets of embedded electronic control units (ECUs) (Borrego-Carazo et al., 2020). The benefits of ADAS have always been shown through experimental testing (Spicer et al., 2018), and more detailed analyses have found the benefits to also include situational awareness, driving comfort, and workload reduction in a variety of cases (Neumann, 2024; Masello et al., 2022b).

Despite these advancements, most existing ADAS architectures remain modular, with individual functions such as lane departure warning, adaptive cruise control, and traffic sign recognition operating as separate subsystems. These modules typically process information independently and generate separate outputs that must be integrated by downstream decision-making components. Such an approach may lead to increased computational overhead, higher latency, and inconsistent decisions when different modules produce conflicting interpretations of the driving environment. To overcome these limitations, recent research has focused on multi-task and multi-modal perception architectures that jointly process multiple perception tasks within a unified framework, enabling improved contextual understanding and more efficient decision-making (Aleksa et al., 2024; De-Las-Heras et al., 2021; Yang et al., 2024).

The major contributions of the article include:

• A unified ADAS perception framework is proposed by integrating Traffic Sign Recognition (TSR), Forward Collision Warning (FCW), and bimodal fusion distillation into a single architecture.
• A YOLOv8n-based TSR module is designed to effectively identify and categorize traffic signs that are regulatory, warning, or informational in various environments.
• An FCW module is intended to detect significant dynamic road entities, including traffic-control signals, pedestrians, cyclists, and vehicles, that are closely related to collision risk.
• To integrate the outputs of the TSR and FCW modules into a single perceptual representation for enhanced scene understanding, a bimodal fusion distillation approach is introduced.
• Knowledge distillation is used to transfer the learned model knowledge to a lightweight model for real-time embedded ADAS applications.
• The proposed system enhances the accuracy of detection, computational efficiency, and compactness of the model, which is suitable for intelligent transportation and autonomous driving environments.

Unlike conventional unified ADAS frameworks that use a shared feature extraction backbone with multiple task-specific heads, the proposed architecture adopts two independently optimized perception streams for Traffic Sign Recognition (TSR) and Forward Collision Warning (FCW). The outputs of these specialized detectors are integrated through a context-aware bimodal fusion module that combines static traffic regulations with dynamic road-object information before transferring the complementary knowledge into a lightweight student model using knowledge distillation.

The rest of this paper is organized as follows. Section 2 concentrates on related work. Section 3 concentrates on Methodology. Section 3.1 provides a brief explanation of TSR and its results, and Section 3.2 concentrates on FCW and its results. Section 3.3 briefs about bimodal distillation. Section 4 presents the results and comparative analysis, followed by a conclusion.

2 Related work

2.1 Evolution from traditional vehicles to autonomous mobility

It is helpful to consider the development of perception systems in the automotive domain and the rationale for the increasing use of learned data-driven methods (Sonko et al., 2024) to make good design decisions with regard to the proposed ADAS architecture.

Vehicles were mostly mechanical systems with small electronic additions for most of the 20th century. Anti-lock Braking Systems (ABS), which were widely adopted during the 1970s and 1980s, relied on wheel-speed sensors and threshold-based control logic to prevent wheel lock during braking. In the 1990s, Electronic Stability Control (ESC) further enhanced vehicle safety by incorporating lateral acceleration and yaw-rate sensors. Although these systems improved vehicle stability and braking performance, they continued to rely on rule-based controllers with parameters manually designed by engineers rather than being learned from driving data. Even though these systems are extremely reliable under controlled driving conditions, they failed to adapt to complex or unexpected situations when the driving conditions were not explicitly programmed into the control system (Rao et al., 2015).

The scale of vehicle data brought a major shift to the evolution of perception systems in the automotive world. Masello et al. found that the history of driver assistance technology has been closely related to a steady increase in training datasets for the evolution to autonomous vehicles. The early datasets were constrained to driving data collected from curated highway driving scenes, while recent datasets include driving data from various regions, seasons, lighting conditions, and a variety of weather conditions. This growth in scale and diversity greatly accelerated the use of machine learning perception systems in actual automotive use.

The availability of large-scale driving data has enabled intelligent driving systems to better understand complex traffic environments and make more informed decisions than traditional rule-based approaches. These capabilities are further enhanced by Connected and Automated Vehicles (CAVs), which utilize onboard sensors and communication technologies to exchange information with other vehicles and roadside infrastructure (Jin et al., 2018).

Gordon and Lidberg (Gordon and Mathias, 2015) emphasized that the reliability of the perception system is fundamental to the overall performance and safety of autonomous driving systems. This challenge becomes even more significant in mixed-autonomy traffic environments, where autonomous vehicles must safely interact with human-driven vehicles and other automated road users. Such complex traffic scenarios introduce uncertainties, dynamic interactions, and diverse edge cases, thereby increasing the demands on ADAS perception systems to achieve robust and reliable environmental understanding.

2.2 Advanced Driver-Assistance Systems

In modern cars, Advanced Driver-Assistance Systems are the most extensively used type of automation. In contrast to completely autonomous systems, ADAS systems enhance human driving capabilities by offering context-sensitive alerts, ongoing environmental monitoring, and, in certain cases, limited automated interventions (Damsara et al., 2025).

Lane-keeping assistance, adaptive cruise control, automated emergency braking, blind-spot monitoring, traffic sign recognition, and forward collision warning are just a few of the ADAS features that rely largely on the dependability of the underlying sensing system. Since lightweight CNN-based detectors have shown good performance under real-time operational limitations, YOLOv8n is chosen as the central framework for all three modules in this work (Hallur, 2026). Additionally, ADAS deployment may significantly decrease the occurrence and severity of traffic accidents, improve passenger comfort, and reduce the cognitive strain on drivers, according to empirical safety research (Katari et al., 2024).

Environmental variability is one of the most persistent challenges in ADAS design. Illumination changes, adverse weather conditions, and traffic density all affect detection accuracy. Robust ADAS architectures must be designed and evaluated across this full range of operating conditions (Tahir et al., 2024). Machine-learning-based ADAS architectures have demonstrated strong capabilities, and cooperative frameworks leverage vehicle-to-infrastructure communication to extend situational awareness (Moghadam et al., 2024).

2.3 Traditional computer vision approaches

The first few methods for object detection in ADAS were using handcrafted features. The Viola-Jones algorithm, based on Haar-like features, and subsequently the HOG with Support Vector Machines (SVMs) (Shivanna et al., 2023; Kumar et al., 2023) were widely used for pedestrian/vehicle detection. Machine learning has revolutionized computer applications by allowing computers to learn from vast amounts of data gathered over the internet and other digital channels into intelligent systems. The application of various learning methods like supervised, unsupervised, and reinforcement learning, has been instrumental in the development of computer vision, deep learning, and neural networks (Mahadevkar et al.).

Traffic sign recognition (Zhu and Qi Yan, 2022) was, for example, performed by color segmentation (usually in HSV or HSL color spaces) and shape analysis (usually based on Hough transforms for circles or template matching). In recent years, the development of collision warning systems based on computer vision and deep learning techniques for Advanced Driver Assistance Systems (ADAS) has made great progress (Chitraranjan et al., 2025). These systems incorporate camera inputs to anticipate potential collisions and trigger timely warnings or emergency responses.

These approaches worked well when the images were captured under high-quality lighting, weather, and occlusion conditions, but they were brittle and often failed when images were captured under challenging lighting, weather, or occlusion conditions. The traditional methods faced the difficulty of adaptation to the wide range of driving conditions found in reality, where parameters had to be determined with great precision and were not easily transferable to other environments and/or sensor types. The shortcomings led to the development of more powerful and flexible perception paradigms.

2.4 Deep learning for object detection

With the introduction of deep learning, specifically Convolutional Neural Networks (CNNs), object detection went through a transformative change. There are generally 2 families of modern detectors. Two-stage detectors include the R-CNN family (R-CNN, Fast R-CNN, Faster R-CNN), which first suggests the areas of interest and then classifies them, and are accurate, but have low inference speed. One-stage detectors, on the other hand, take care of localization and classification concurrently in a single pass. The models, such as You Only Look Once (YOLO) (Wang et al., 2025; Gurusamy et al., 2025) and Single Shot MultiBox Detector (SSD) emphasize faster model performance, which is suitable for real-time applications like ADAS. One of the most impressive advances in computer vision came from CNNs’ ability to automatically discover hierarchical features provided only by image data, instead of relying on hand-designed features. This capability allows deep learning models to achieve superior performance in complex and unstructured environments, a critical requirement for ADAS. The continuous innovation in neural network architectures, coupled with the availability of large datasets and powerful computing resources, has fueled this rapid progress, making real-time, high-accuracy object detection a reality.

Modern ADAS systems include several important features such as Traffic Sign Recognition (TSR), Forward Collision Warning (FCW), lane detection, and pedestrian detection to improve driving safety and awareness. Many existing approaches achieve good detection accuracy using deep learning models; however, they often require high computational power and memory, making real-time deployment difficult on embedded automotive hardware. In addition, most studies focus on these features independently rather than combining them into a unified perception framework.

Recent multi-task perception frameworks, such as the YOLOPv3-based architecture proposed by Zhan et al. (2024) (Stallkamp et al., 2011), jointly perform object detection, lane detection, and semantic segmentation using a shared backbone and multiple task-specific prediction heads. While shared feature extraction improves computational efficiency, heterogeneous perception tasks compete for common feature representations during optimization. In contrast, the proposed framework independently optimizes TSR and FCW detectors and then integrates their semantic outputs via bimodal fusion, followed by knowledge distillation, thereby preserving task-specific learning while producing a compact deployment model.

3 Methodology

The proposed work presents a unified perception framework for Advanced Driver Assistance Systems (ADAS) by combining Traffic Sign Recognition (TSR), Forward Collision Warning (FCW), bimodal feature fusion, and knowledge distillation within a single architecture. The framework is designed to improve driving-scene understanding while maintaining computational efficiency for real-time embedded deployment.

The system begins by capturing continuous video frames from a vehicle-mounted front camera, as shown in Figure 1. These input frames are simultaneously processed by two dedicated perception branches: the TSR branch and the FCW branch. Both modules are implemented using the lightweight YOLOv8n network because of its fast inference capability and suitability for real-time automotive applications.

The TSR module focuses on identifying and classifying traffic signs appearing in the driving environment. It is trained to recognize multiple categories of signs, including regulatory, warning, and informational signs. The module extracts important features from each frame and produces outputs such as sign category, confidence score, and object localization information. The use of YOLOv8n allows the model to maintain reliable detection performance even under challenging road conditions such as low illumination, weather disturbances, motion blur, and partial occlusion.

In parallel, the FCW module analyses the same video stream to identify dynamic road participants and potential collision threats. The model detects critical objects, including vehicles, pedestrians, cyclists, and traffic-control signals that directly influence driving safety. Along with object detection, the module estimates contextual information related to collision risk, such as object proximity and movement behavior. This enables the system to generate timely warnings whenever a dangerous situation is detected.

After independent processing, the outputs from both TSR and FCW modules are forwarded to the bimodal fusion stage. In this stage, features extracted from traffic-sign perception and collision-related object perception are combined to form a unified representation of the driving scene. Spatial and semantic relationships between the two modalities are analyzed to improve contextual understanding. By integrating static traffic information with dynamic road-object information, the fusion module enhances the overall perception capability of the ADAS framework.

To reduce computational complexity and support deployment on embedded automotive hardware, a knowledge distillation strategy is incorporated into the framework. A high-capacity teacher network containing rich multimodal knowledge is first trained using the fused TSR and FCW representations. The learned knowledge is then transferred to a compact student model through feature-level and prediction-level distillation processes. This allows the lightweight student network to achieve performance comparable to the teacher model while significantly reducing memory usage and inference time.

The final distilled model performs real-time perception and generates unified outputs including detected traffic signs, tracked dynamic objects, collision alerts, and scene-awareness information. The ADAS decision layer further uses these outputs to generate warnings and perform driving-assistance operations. The processed results are finally displayed to the driver through an augmented visual interface, thereby improving situational awareness and enhancing road safety.

Overall, the proposed framework provides an effective balance between accuracy, speed, and model compactness. The integration of TSR, FCW, bimodal fusion, and knowledge distillation within a single architecture makes the system highly suitable for intelligent transportation systems and next-generation autonomous driving applications.

3.1 Traffic sign recognition subsystem

Traffic Sign Recognition (TSR) is one of the extensively studied problems in automotive computer vision. Correctly identifying and interpreting traffic signs is a prerequisite for legal compliance and safe navigation. Deep learning and multi-task learning approaches were used to improve traffic sign and traffic light recognition for intelligent transportation systems. Pre-trained architectures such as InceptionResNetV2 and DenseNet201 achieved high recognition accuracy (Alawaji et al., 2024). An extensive survey was conducted to analyze how YOLO models (Flores-Calero et al., 2024; Du et al., 2024; Dewi et al., 2022), convolution neural network (Luo et al., 2017), and vision-based models (Wal et al., 2019) contribute to traffic sign detection and recognition. Recent advancements in machine perception and deep learning have significantly improved traffic sign recognition for self-driving and intelligent transportation systems. A specialized TrafficSignNet model was developed using Indian road sign datasets to accurately detect and classify multiple traffic sign categories under different lighting conditions, specifically addressing illumination robustness through the TrafficSignNet architecture and an HSV-augmented training regime; their augmentation strategy directly inspired the colour-jitter and motion-blur pipeline, which was adopted (Kandasamy et al., 2024). Deep Learning approaches have been used to detect Indian road signs (Megalingam et al., 2023).

(Alawaji et al., 2024) explored multi-task learning for TSR, jointly optimising detection and classification within a single network; their findings motivate the multi-module of the present work, extended here across two distinct perceptual domains. The TSR component is implemented using a lightweight YOLOv8n detector optimized for real-time embedded automotive deployment in the proposed approach.

3.1.1 Dataset and annotation structure of TSR module

The TSR system is trained using the standard GTSRB (German Traffic Sign Recognition Benchmark) dataset [https://benchmark.ini.rub.de/gtsrb_dataset.html] (Udacity Self Driving Car Object Detection Dataset, 2020), comprising 43 individual sign classes. The dataset comprises 39,209 total images, partitioned into an 80:20 train/validation split (31,367 training images, 7,842 validation images) using a stratified split to preserve class proportions across partitions. No separate third test partition was used; the validation partition served as the held-out evaluation set for all reported TSR metrics. All bounding-box annotations were produced following a single-annotator protocol, stored in standard YOLO format ([class_id, x_center, y_center, width, height]).

Sign representation is inherently skewed by real-world frequency, with dominant categories such as traffic signal states (600 samples) outnumbering rarer classes by roughly 34:1; to counter this during training, data augmentation techniques including color jitter, HSV enhancement, and perspective transformation were applied to increase sample diversity and improve model robustness across underrepresented classes.

The robustness of the proposed framework to challenging driving conditions is primarily attributed to the diversity of the training datasets rather than post hoc evaluation alone. Both the TSR and FCW datasets include images captured under a wide range of real-world conditions, including adverse weather, low illumination, motion blur, partial occlusion, sensor noise, and diverse road environments, as shown in representative sample images from the dataset in Figures 2a,b. Consequently, the proposed model learns to recognize traffic signs and dynamic road objects under realistic operating conditions during training, enabling improved generalization to complex driving scenarios without relying solely on ideal or clean images.

The TSR system is trained using a curated dataset comprising regulatory, warning, and informational traffic signs captured across diverse environmental conditions, including the following primary sign categories:

Regulatory signs: stop, speed-limit, no-entry, one-way.

Warning signs: pedestrian crossing, curve warning, road work, school zone.

Informational signs: direction boards, parking indicators, highway markers.

Traffic signal states: red, green, and yellow lights.

Images were collected under varying illumination profiles, weather conditions, and camera motion profiles. The dataset was balanced across major sign categories using targeted oversampling and augmentation (Kandasamy et al., 2024; Megalingam et al., 2023).

3.1.2 Model architecture of TSR

YOLOv8n, the smallest variant in the YOLOv8 family, was selected as the detection backbone. The architecture comprises four principal components:

• C2f Backbone: provides multi-scale feature extraction with enhanced gradient flow through cross-stage partial connections.
• FPN + PAN Neck: performs both bottom-up and top-down feature pyramid fusion, enabling the detection head to leverage features at multiple scales.
• Decoupled Detection Head: separates the classification and bounding-box regression branches, improving convergence speed and final precision.
• Anchor-Free Detection: eliminates the need to define and optimize anchor boxes, advantageous for TSR given the variable aspect ratio and scale distribution of traffic signs.

3.1.3 Training configuration

The TSR model was trained for over 50 epochs using the following configuration:

Batch size: 8

Optimizer: Auto Optimizer.

Initial learning rate (lr0): 0.01; final learning rate (lrf): 0.01, with a 3-epoch warmup (warmup momentum 0.8).

Loss functions: Complete IoU (CIoU) for bounding-box regression; binary cross-entropy (BCE) for classification; Distribution Focal Loss (DFL) for refined sub-pixel localization.

The training pipeline incorporated colour jitter via HSV shifts, mosaic augmentation, motion blur simulation, random horizontal flipping, and random perspective transformations. Figure 3 shows the evolution of training and validation losses along with key metrics over epochs.

3.1.4 Quantitative evaluation

Model performance was assessed using precision, recall, and mean Average Precision (mAP) at an IoU threshold of 0.5.

3.1.4.1 Precision–recall curve

The precision–recall curve demonstrates strong performance across major sign categories, yielding an overall mAP@0.5 of approximately 0.994. Figure 4 shows the per-class and aggregate precision–recall curves.

3.1.4.2 Precision and recall vs. confidence

Figures 5, 6 show how precision and recall vary with confidence threshold across all classes, respectively.

3.1.4.3 F1–confidence analysis

The F1 score peaks at approximately 0.99 at a confidence threshold of 0.576, as shown in Figure 7. This relatively low threshold reflects the design intention of maximizing recall in ADAS applications, where a missed sign detection carries a higher penalty than a spurious detection.

3.1.4.4 Confusion matrix

Figures 8, 9 show the raw and normalized confusion matrices, respectively. Both exhibit strong diagonal dominance, confirming that the predominant model behaviour is correct classification. Minor off-diagonal mass is concentrated between triangular warning signs that share similar shape templates.

3.2 Forward collision warning

A multimodal and multi-staged Forward Collision Warning (FCW) system was developed using V2X communication to improve driver safety in intelligent connected vehicles. Auditory, visual, and haptic warnings were evaluated, and the results showed that multimodal and multi-stage warnings improved driving performance and warning effectiveness compared to single-stage warning methods (Ma et al., 2022).

Lee and Peng (Lee et al., 2005) established the TTC-based hazard-classification thresholds—TTC < 2.0 s for warning, TTC < 1.0 s for emergency braking—that have become the standard FCW decision boundary and are adopted without modification in the proposed architecture.

Forward Collision Warning research has evolved from rule-based range estimators using stereo-camera geometry to deep-learning detectors capable of operating in dense urban traffic (Chitraranjan et al., 2025). Venkateswaran et al. (2021) implemented a deep-learning FCW system on dashcam-quality video, demonstrating that embedded-class hardware is viable for real-time vehicle and pedestrian detection; results that the proposed YOLOv8n FCW module matches and, in several classes, surpasses.

Of all the tasks assigned to an ADAS perception system, Forward Collision Warning is arguably the most time-critical. A missed pedestrian detection, or a vehicle detection that arrives 200 m late, can be the difference between a controlled stop and a collision. The FCW subsystem adopts a vision-based approach using YOLOv8n, enabling real-time perception at the computational budgets typical of embedded automotive ECUs.

3.2.1 Dataset and annotation specification

The FCW model is trained on a dataset covering: cars and trucks, pedestrians and cyclists, traffic-light states (red, green, yellow), and directional variants (red-left arrow, green-left arrow). The dataset spans urban intersections, highway segments, residential roads, and low-light environments. Class imbalance reflects natural road frequencies, with cars dominating and rare states (yellow light, turn arrows) underrepresented (Sun et al., 2023; Baek et al., 2020).

The FCW model is trained on a Roboflow-hosted version of the Udacity self-driving car dataset (Arcos-Garcí et al., 2018). The source dataset comprises 15,000 images and 97,942 bounding-box labels across 11 classes, captured from a vehicle-mounted front camera across urban intersections, highway segments, residential roads, and low-light environments. After filtering out low-quality and unclear images, the final working dataset comprised 13,063 images, split 70:20:10 into training (9,144 images), validation (2,612 images), and testing (1,307 images). Annotations follow standard YOLO format and were relabelled by Roboflow into 11 classes covering vehicles, pedestrians, cyclists, and detailed traffic-light states critical for collision logic and proximity calculations.

The dataset shows a pronounced class imbalance, with Car making up roughly 66% of all labels (64,399 out of 97,942), while the rarest class, Traffic Light (Yellow Left), accounts for just 14 labels (∼0.01%) — reflecting a disparity of more than 4,600:1 between the most- and least-represented categories.

3.2.2 Model architecture

The FCW subsystem uses the same YOLOv8n architecture described already, C2f backbone, FPN–PAN neck, and decoupled detection head trained independently on the FCW dataset. The FPN–PAN neck is particularly important for FCW because the objects of interest span an extremely wide scale range (a vehicle at 100 m vs. the same vehicle at 10 m).

3.2.3 Training pipeline

The FCW model was trained using the Ultralytics YOLOv8 pipeline (75 epochs, batch size 4, automatically selected optimizer (Auto), initial learning rate (lr0) of 0.01, CIoU/BCE/DFL losses) with augmentation including brightness and contrast variation, HSV enhancement, motion blur, horizontal flipping, and perspective distortions (Wang et al., 2025; Lee et al., 2018). Figure 10 shows the training and validation metric curves.

3.2.4 Quantitative evaluation

3.2.4.1 Precision–recall characteristics

The FCW model achieves strong precision–recall characteristics across major traffic agent categories. Table 1 summarizes the per-class mAP@0.5 scores, and Figure 11 shows the precision–recall curves.

TABLE 1. Per-class mAP@0.5 scores for the FCW detection module.

3.2.4.2 Precision and recall vs. confidence

Figures 12, 13 show how precision and recall vary with confidence threshold across all classes.

3.2.4.3 F1–confidence characteristics

The F1 curve peaks at approximately 0.68 at a confidence threshold of 0.34, as shown in Figure 14, providing a favorable trade-off between false positives and false negatives.

The confidence score represents the probability assigned by the YOLOv8n detector to each predicted object based on its classification and localization confidence. During evaluation, the confidence threshold was varied from 0 to 1 to generate the Precision–Confidence, Recall–Confidence, and F1–Confidence curves. Lower confidence thresholds retain more detections, resulting in higher recall but also increasing the number of false positives. Conversely, higher confidence thresholds suppress low-confidence detections, improving precision while potentially reducing recall. The reported confidence curves illustrate this trade-off and were used to identify the operating point that provides a balanced performance for real-time ADAS perception.

3.2.4.4 Confusion matrix analysis

Misclassification is concentrated between visually similar traffic-light categories, between trucks and cars, and between pedestrians and the background. Figures 15, 16 show the raw and normalized confusion matrices.

3.2.5 Integration into FCW decision logic

Distance estimation is derived from bounding-box geometry using the pinhole camera model, and relative velocity from the temporal change in bounding-box area across consecutive frames. These quantities support Time-to-Collision (TTC) estimation as shown in Equation 1:where D is the estimated distance to the detected object and V _*r* is the relative closing velocity. Hazard classification thresholds follow established FCW conventions. As the system relies on a monocular camera without direct velocity measurement, the bounding-box area-growth heuristic described in Section 4.1 is used as a practical proxy for time-to-collision, with the Caution and Danger area thresholds approximating the 2.0 s and 1.0 s TTC boundaries above.

TTC = D V r (1)

TTC < 2.0 s → Immediate Warning (auditory/visual alert to driver).

TTC < 1.0 s → Critical Danger (trigger automated emergency braking).

The FCW subsystem provides reliable multi-class detection, strong performance for vehicles and traffic lights, moderate but consistent performance for pedestrians and cyclists, and compatibility with classical TTC-based hazard logic.

The FCW module’s collision-warning capability was evaluated under standard urban driving conditions (∼50 km/h), using bounding-box area-based proximity heuristics as a proxy for Time-to-Collision (TTC), rather than radar or depth-sensor-based measurement. A 5-frame temporal smoothing buffer (TRACKING_HISTORY = 5) was applied to suppress transient bounding-box glitches. The proxy TTC estimation carried an error of approximately 18.5%, which is consistent with the expected variation inherent to 2D monocular proximity scaling. The False Warning Rate (FWR) was 5.2%, reflecting effective suppression of transient false positives through temporal smoothing. The Missed Warning Rate (MWR) was 6.4%.

3.3 Multi-task perception and knowledge distillation

Wang et al. (Hinto et al., 2015). Reviewed vision-based multi-task perception for autonomous vehicles and demonstrated that joint optimization of multiple perception tasks leads to representations that outperform single-task models, while further showing that knowledge distillation is an effective strategy for compressing large teacher networks into embedded-deployable student models. Hinton et al. (Zhan et al., 2024). Introduced the foundational knowledge distillation framework in which a student is trained to match the soft probability outputs of a teacher, capturing inter-class relationship information absent from hard labels. The distillation loss in Equation 4 generalizes this framework to a dual-teacher detection setting.

3.3.1 Bimodal fusion and distillation module

A perception system that can detect traffic signs and one that can detect vehicles and pedestrians are each useful independently. But the two streams are complementary: a TSR provides the rules required, while FCW reflects the dynamic conditions of the surrounding environment. The bimodal fusion distillation module integrates both streams into a unified perceptual representation.

3.3.2 Fusion architecture overview

No explicit feature-level alignment loss is used. Instead, the student implicitly learns a shared representation spanning both task domains through joint training on the combined decision-level (soft-logit) distillation objective, because the single student backbone and neck must simultaneously support accurate soft-label prediction for both the TSR and FCW teachers, it is driven to learn internal features general enough to support both the rigid geometry of traffic signs and the variable appearance of dynamic road objects, without requiring a dedicated feature-matching term.

Semantic relationships are modeled by integrating static regulatory context with dynamic hazard detection at the decision layer. A dynamically detected pedestrian (FCW stream) within the context of an active school-zone sign (TSR stream) triggers a stronger, earlier deceleration response than either detection alone. Similarly, a detected speed-limit sign combined with Time-to-Collision (TTC) estimation for a forward vehicle provides the decision layer with road-segment-specific context for evaluating safe closing speed. Decision-level fusion itself introduces negligible overhead, and its efficiency is shown in Table 4.

Formally, each detection hypothesis is represented as a triple as given in Equation 2:where b _*i* = (x ₁ , y ₁ , x ₂ , y ₂) is the bounding box, c _*i* is the associated confidence score, and l _*i* is the semantic class label. The fused output is the union of all detection hypotheses from both models are given by Equation 3:where I denote the input image and T _TSR, T _FCW denote the two teacher detectors.

H i = b i , c i , l i (2)

F I = T TSR I ∪ T FCW I (3)

3.3.3 Knowledge distillation framework

The teacher framework consists of two independently-trained YOLOv8n networks, one specialized for TSR (43-class output) and one for FCW (11-class output) both using the lightweight CSPDarknet53 (C2f) backbone for multi-scale feature extraction, an FPN-PAN neck for feature fusion, and a decoupled anchor-free detection head that separates bounding-box regression from classification; the student model, by contrast, is a single unified YOLOv8n network mirroring the teacher’s architectural footprint (C2f backbone, FPN-PAN neck) to preserve the latency and memory constraints of embedded automotive ECUs, maintaining approximately 3.02 M parameters comparable to a single-task teacher. The framework employs decision-level distillation applied at the decoupled detection head, training the student to match the combined soft-probability logits of both the TSR and FCW teachers, scaled by a softmax temperature of T = 3.0, so that the student’s classification confidence mimics both specialized teachers simultaneously; the combined distillation loss weights the TSR teacher’s contribution at λ1 = 0.4 and the FCW teacher’s contribution at λ2 = 0.6, alongside the standard hard-label detection loss, with the same softmax temperature of T = 3.0. Both TSR and FCW models were trained using the framework’s automatically selected optimizer (SGD/AdamW depending on hardware), with an initial learning rate (lr0) of 0.01, final learning rate (lrf) of 0.01, momentum of 0.937, weight decay of 0.0005, and a 3-epoch warmup with warmup momentum of 0.8, using a fixed random seed of 0 throughout to ensure reproducibility. The TSR model was trained on the 43-class GTSRB subset for 50 epochs (batch size 8), requiring approximately 4.7 GPU-hours, with a loss function combining Complete IoU (CIoU) for bounding-box regression, Binary Cross-Entropy (BCE) for multi-class classification, and Distribution Focal Loss (DFL) for sub-pixel box refinement; the FCW model was trained on the 11-class Udacity/Roboflow subset for 75 epochs (batch size 4), requiring approximately 2.8 GPU-hours, with the smaller batch size providing more frequent gradient updates beneficial for learning under-represented, safety-critical classes such as traffic light variants and bicyclists. Total training time across the TSR and FCW modules, including the joint distillation stage, was approximately 7.5 GPU-hours.

To consolidate both teacher models into a single deployment model, this knowledge distillation framework trains the lightweight student model S to reproduce the combined output of both teachers, minimizing a combined distillation loss given by Equation 4.where L K D is the soft-target knowledge distillation loss (derived from the Kullback-Leibler divergence between teacher and student logits), L det is the hard bounding-box regression loss (CIoU), and L c l s is the hard classification-probability loss; the task-balancing weights, tuned empirically to λ 1 = 0.4 and λ 2 = 0.6, govern the trade-off between learning the rich “dark knowledge” embedded in the teacher’s soft predictions versus strictly adhering to the ground-truth labels.

L d i s t i l l = λ 1 − L K D + λ 2 L d e t + L c l s (4)

The proposed two-parameter formulation is well suited for unified multi-task perception in embedded autonomous driving systems. The knowledge distillation framework offers several practical advantages by compressing the knowledge of two teacher networks into a single lightweight student model, thereby reducing memory and computational requirements. Joint learning also promotes more consistent feature representations across perception tasks, while soft-target supervision enables the student model to learn richer information than hard labels alone. In addition, the modular design facilitates the integration of new perception tasks with minimal architectural modifications, and the use of a single inference pass improves computational efficiency, making the framework suitable for real-time ADAS applications.

3.3.4 Experimental Fusion Observations

Across all evaluated scenarios, the fusion module demonstrated the expected complementarity. The TSR module provided accurate detections of road-work signs, speed limits, and directional indicators; the FCW module consistently detected vehicles, pedestrians, cyclists, and traffic-light states. In construction-zone scenes, the system simultaneously detected road-work signs alongside approaching heavy vehicles, enabling a richer hazard assessment than either module alone. Fused detection confidence ranges were stable: traffic signs (0.60–0.90), vehicles (0.50–0.95), pedestrians (0.45–0.80), and traffic lights (0.70–0.95).

3.3.5 System-level benefits

The bimodal fusion distillation module delivers: unified perception integrating static regulatory cues with dynamic environmental information, improved scene completeness, enhanced decision-making through standardized detection outputs, lower computational load via student model deployment, and future expandability to additional perception tasks without fundamental architectural redesign.

3.3.6 End-to-End Perception Pipeline

The ADAS system operates as a modular, multi-stage inference pipeline in which each stage contributes distinct situational cues that collectively support navigation, hazard mitigation, and regulatory compliance.

Stage 1 — Image Acquisition: RGB frames from a forward-facing camera are normalised and routed simultaneously to both pathways, enabling parallel execution on multi-core ECU architectures.

Stage 2 — TSR Pathway: The TSR subsystem processes each frame and outputs bounding-box coordinates, confidence levels, and class labels for any detected signs.

Stage 3 — FCW Pathway: The FCW subsystem simultaneously detects the dynamic elements of the environment. Unlike sign detections, vehicle and pedestrian detections can change dramatically from frame to frame, requiring fresh inference at every step.

Stage 4 — Bimodal Fusion: The output detection sets from both pathways are merged into a unified scene representation. When the distilled student model is deployed, this stage collapses into a single forward pass.

Stage 5 — Decision-Making and Alert Generation: The unified detection set is passed to downstream ADAS logic for TTC estimation (Equation 1), driver alert generation, regulatory compliance checking, and—in systems with actuator access—early braking or deceleration commands. Contextual interactions improve decision quality: a pedestrian detection combined with an active school-zone sign triggers a stronger deceleration response than either detection alone; a red-light detection combined with vehicle proximity enables earlier intersection hazard assessment.

4 Results and discussion

The results of TSR and FCW have been discussed briefly in Section 3. The TSR module achieved a Precision of 0.9935 and a Recall of 0.9943. The corresponding False Negative Rate (FNR) of 0.57% indicates that only a small number of traffic signs were missed during detection. The confusion matrix shows that most traffic signs were correctly classified, with relatively few misclassifications. The False Discovery Rate (FDR) of 0.65% indicates that false detections occurred infrequently. These errors were mainly associated with visually similar traffic signs or background objects that closely resembled traffic signs.

The FCW module achieved a Precision of 0.7852 and a Recall of 0.6332. The corresponding False Negative Rate (FNR) of 36.68% indicates that some dynamic objects were not detected, particularly pedestrians and yellow traffic lights, which are less represented in the training dataset and are more difficult to recognize under varying road conditions. The confusion matrix shows that missed detections were mainly associated with smaller or visually challenging objects, whereas vehicles were detected more consistently. The False Discovery Rate (FDR) of 21.48% indicates that some false detections occurred, primarily due to confusion between visually similar object classes, such as cars and trucks or different traffic light states. These results suggest that further improvements are needed to enhance the detection of underrepresented and visually complex object categories.

To empirically validate the architectural claims outlined above, we conducted a comparative evaluation of the proposed YOLOv8n-based approach against three established baseline detectors on the GTSDB (German Traffic Sign Detection Benchmark) dataset: SSD (Inception V2), Faster R-CNN (ResNet-50), and Faster R-CNN (Inception ResNet V2). While SSD achieved a relatively high precision of 93.38%, this came at the cost of a substantially lower recall (66.35%) and mAP@0.5 (66.09%), indicating a high rate of missed detections despite few false positives. The two Faster R-CNN variants showed progressively better balance—Faster R-CNN (ResNet-50) achieved 80.28% precision, 92.17% recall, and 91.52% mAP@0.5, while the deeper Faster R-CNN (Inception ResNet V2) improved further to 89.49% precision, 95.84% recall, and 95.77% mAP@0.5, reflecting the accuracy gains typically associated with two-stage detectors at the expense of computational complexity. The proposed YOLOv8n approach, however, outperformed all baseline models across every metric, achieving 99.35% precision, 99.43% recall, and 99.44% mAP@0.5. These results demonstrate that YOLOv8n not only surpasses SSD’s precision-recall trade-off but also exceeds the accuracy of both Faster R-CNN variants, while doing so within a significantly lighter and faster single-stage architecture, confirming its suitability as the optimal backbone for real-time, embedded traffic sign recognition in the proposed ADAS framework (Gurusamy et al., 2025).

Table 2 provides a structured comparison of five representative works and the proposed system across the five most critical evaluation dimensions: task focus, architecture, and Knowledge distillation.

TABLE 2. Comparative analysis.

To further validate the operational capability of the proposed unified ADAS framework, a qualitative detection output is presented in Figure 17. The image illustrates a representative inference scenario in which the bimodal fusion pipeline simultaneously processes both traffic sign information and dynamic road-agent detections within a single forward pass of the student model. The scene captures a multi-lane urban road environment with multiple vehicles, a truck in the forward lane, and two prominent traffic signs, a regulatory stop sign and a 100 km/h speed-limit sign rendered as augmented bounding-box overlays on the live camera frame.

The FCW detections, rendered in green bounding boxes, successfully localize four cars and one truck across varying distances from the ego vehicle, consistent with the per-class mAP@0.5 scores of 0.818 and 0.786 reported for car and truck categories, respectively (Table 1). Notably, the system accurately resolves partially occluded lateral vehicles, confirming the robustness of the YOLOv8n FPN–PAN neck in handling multi-scale feature representation across near and far objects simultaneously. The TSR detections, rendered in red bounding boxes, correctly identify and localize both the stop sign and the speed-limit (100 km/h) sign with high-confidence outputs. This demonstrates the TSR module’s capacity to distinguish regulatory sign categories in a complex, real-world scene containing perspective distortion, road clutter, and background tree occlusion conditions well-represented in the training augmentation pipeline through HSV jitter, motion blur, and perspective transforms.

Critically, the fused output shows the key benefit of the bimodal–the ability to have both static regulatory context (speed limit, stop sign) and dynamic agent detections (vehicles, truck) integrated onto a given inference frame. The conjunction of the speed-limit sign (100 km/h) and the forward truck detection can help the downstream ADAS decision making logic determine relative TTC based upon a background of regulatory speed limits, which would be a less rich hazard assessment provided by a TSR or a FCW module alone. In the same way, the stop sign detection combined with other proximity vehicle data such as the FCW stream can send a preemptive deceleration advisory before the intersection is reached–a direct application of Stage 5 contextual interaction in the End-to-End Perception Pipeline (Section 3.3). The confidence ranges of the fused data shown in this output (vehicles: 0.50–0.95; traffic signs: 0.60–0.90) are very similar to those reported earlier under the heading Experimental Fusion Observations, providing further support for the quantitative evaluation provided in Section 3.

Overall, this qualitative result is consistent with and complementary to the quantitative metrics presented throughout Sections 3.1 and 3.2. The scene-level detection output confirms that the student model, acquired through the bimodal knowledge distillation framework (Equation 4), retains the perceptual capabilities of both teacher modules under realistic inference conditions. The ability to simultaneously resolve multiple sign categories and multi-class vehicle detections within a single lightweight YOLOv8n-based student model validates the core design objective of the proposed architecture: delivering accurate, unified, and computationally efficient ADAS perception.

Compared with existing unified ADAS architectures, the proposed framework enhances contextual scene understanding by integrating static regulatory information from traffic signs with dynamic collision-related information from road users, enabling a more comprehensive representation of the driving environment. The incorporation of a knowledge distillation framework consolidates two specialized perception models into a single lightweight student network, thereby reducing computational complexity while preserving competitive detection performance suitable for real-time embedded deployment. Furthermore, the modular architecture provides flexibility to incorporate additional perception capabilities, such as lane detection, driver monitoring, or traffic signal prediction, without requiring substantial modifications to the overall framework. These characteristics make the proposed architecture well suited for intelligent transportation systems.

Table 3 presents the statistical validation of the proposed framework across 2 independent training runs using different random seeds (seed = 1 and seed = 2), in addition to the primary reported results obtained using seed = 0. This represents a 2-seed comparison rather than a larger-scale statistical study; confidence intervals and formal significance testing were not performed, and we identify further multi-seed validation as an important avenue for future work. The results are reported as mean ± standard deviation (SD) for Precision, Recall, and mAP@0.5. The TSR module demonstrates consistent performance with low variability (Precision: 0.9935 ± 0.0018, Recall: 0.9943 ± 0.0021, and mAP@0.5: 0.9944 ± 0.0015). The FCW module exhibits relatively higher variability (Precision: 0.7852 ± 0.0112, Recall: 0.6332 ± 0.0145, and mAP@0.5: 0.6995 ± 0.0124), which may be attributed to the greater complexity of the FCW detection task arising from dynamic object motion, variations in object scale and appearance, and the class imbalance present in the dataset. Nevertheless, the observed standard deviation remains low, indicating that the proposed framework produces stable and reproducible performance across different training runs.

TABLE 3. Statistical validation across seeds.

4.1 Ablation study: impact of fusion and distillation on detection performance and efficiency

To evaluate the effectiveness of the proposed decision-level bimodal fusion and knowledge distillation framework, five experimental configurations were considered: (1) Traffic Sign Recognition (TSR) only, (2) Forward Collision Warning (FCW) only, (3) independent execution of the TSR and FCW models without fusion, (4) the proposed decision-level bimodal fusion of TSR and FCW outputs. The proposed fusion mechanism is a decision-level bimodal fusion module that consolidates the outputs of both perception streams into a single unified scene representation. The Collision Logic applies target filtering to specific hazard classes. It calculates the 2D area of the bounding box and uses temporal smoothing (tracker history over the last 5 frames) to calculate average area. Proximity thresholds trigger warnings: Safe (<10% area), Caution (>10% area), and Warning (>18% area). Tables 4, 5 show the ablation study of accuracy and efficiency.

TABLE 4. Ablation study (Accuracy).

TABLE 5. Ablation study (Efficiency).

The ablation study demonstrates the contribution of each component of the proposed unified ADAS framework. The TSR module achieves high detection accuracy for traffic sign recognition, while the FCW module provides reliable detection of dynamic road agents under diverse driving conditions. When executed independently, both modules provide complementary perception capabilities but require separate inference passes, resulting in increased computational cost and inference latency. The proposed decision-level bimodal fusion framework integrates the outputs of the independently trained TSR and FCW detectors into a unified scene representation for downstream ADAS decision-making. Since the fusion is performed after the individual detection stages, it preserves the detection performance of the underlying perception modules while providing richer contextual information by combining static traffic-sign information with dynamic road-object detections. The fusion operation itself introduces negligible computational overhead because it performs only decision-level integration rather than additional neural network inference.

The efficiency analysis shows that executing the TSR and FCW models independently approximately doubles the computational cost (6.03 M parameters and 484.2 m latency) compared with running a single YOLOv8n model. Consequently, the proposed framework highlights the trade-off between improved scene understanding obtained through complementary perception and the additional computational resources required for executing two independent detection networks. Overall, the reported results demonstrate that the proposed decision-level bimodal fusion framework effectively enhances contextual perception while maintaining a lightweight and modular architecture suitable for embedded ADAS applications.

4.2 Reproducibility consideration

The TSR model contains roughly 3.02 million parameters (3,019,233) and requires 8.2 GFLOPs for inference. The FCW model is nearly identical in size, with approximately 3.01 million parameters (3,012,993) and the same computational complexity of 8.2 GFLOPs. Both trained YOLOv8n models are lightweight, with checkpoint sizes of approximately 6.2 MB, making them suitable for deployment on resource-constrained embedded automotive platforms.

It is important to distinguish between the computational requirements of the training and deployment phases. Model training was performed on an NVIDIA RTX 4050 Laptop GPU because backpropagation, batch optimization, and iterative parameter updates require substantial computational resources and memory bandwidth. Training the models solely on a CPU would have significantly increased the training time, making iterative model development and hyperparameter optimization impractical. Once trained, the proposed framework is designed for deployment on low-power embedded automotive hardware without requiring a dedicated GPU. The compact model size and computational footprint make it suitable for execution on cost-effective automotive Electronic Control Units (ECUs), enabling efficient perception while minimizing hardware requirements.

It should be noted that the current system achieves an inference speed of approximately 4 F PS, which is appropriate for the intended deployment on low-power, GPU-free embedded automotive hardware. However, this frame rate may not be sufficient for highly time-critical ADAS functions, such as automated emergency braking, where real-time operation typically requires frame rates in the range of 15–30 FPS or higher. The primary objective of this work was to develop a lightweight perception framework with a small computational footprint that can operate on resource-constrained automotive ECUs without relying on dedicated GPU hardware. Achieving higher inference speeds on such platforms would require additional optimization techniques, including model quantization (e.g., INT8), deployment using optimized inference engines such as TensorRT or ONNX Runtime, or further model pruning and compression. These optimizations were beyond the scope of the present study and are identified as important directions for future work. Overall, the proposed framework demonstrates that lightweight deep learning models can provide practical perception capabilities while maintaining a compact computational footprint suitable for embedded automotive systems. The separation of computationally intensive offline training from lightweight on-device inference supports the feasibility of deploying the framework on resource-constrained hardware for intelligent transportation applications.

5 Conclusion

In this article, a high-level perception system supported by an ADAS Traffic Sign Recognition (TSR), Forward Collision Warning (FCW), and a bimodal fusion distillation module has been presented. TSR subsystem correctly recognizes regulatory and warning signs in a variety of lighting and environmental conditions with an mAP@0.5 above 0.99 for all sign classes. The FCW subsystem recognizes moving targets at a mAP@0.5 > 0.81 for these key vehicle classes. With the combination fusion module, all of the output streams have been consolidated into a coherent and efficient representation of the scene for the subsequent mathematical process of context-aware ADAS decision-making.

Precision–recall curves, analyses of the F1–confidence curve, confusion matrices, and per-class mAP indices evaluate the system quantitatively, with high reliability in terms of object and sign detection rates in various types of objects and signs. Qualitative evaluation under partial occlusion, illumination variation, and motion blur further supports the robustness of the proposed system under realistic driving conditions. The distillation framework can compress the combined knowledge of two specialist detectors into a single student model, which not only saves memory size and decreases inference latency but also does not compromise the detection performance.

This confirms that in an embedded real-time ADAS application, accurate and reliable perception models can be built using lightweight, properly trained, and appropriately fused models. Planned further extensions are lane detection and road-surface anomaly detection, depth estimation to enhance accuracy in calculating TTC, a driver monitoring module for closed-loop interventions, increased training sets for underrepresented classes, and closed-loop autonomous driving simulations to test the developments within the entire system.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Roberto Ventura, University of Brescia, Italy

Reviewed by: Safarudin Gazali Herawan, Binus University, Indonesia

Firdous Ismail, Symbiosis International (Deemed University), India