An explainable AI-driven hybrid SE-transformer architecture for robust knee osteoporosis classification from X-ray data

1 Introduction

Osteoporosis is a degenerative bone disorder characterized by reduced bone mass and deterioration of bone microarchitecture, commonly affecting weight-bearing joints such as the knee. Knee osteoporosis presents significant challenges in orthopaedics, particularly in patients undergoing total knee arthroplasty, where underlying low bone density may contribute to implant instability and postoperative complications. In addition, the management of distal femur fractures becomes more complex in osteoporotic bones due to reduced structural strength and poor surgical fixation conditions.

To improve diagnostic accuracy and enable timely clinical intervention, there has been increasing interest in applying artificial intelligence techniques in medical imaging. In particular, deep learning-based approaches have shown strong potential for osteoporosis detection, offering automated, accurate, and efficient diagnostic support tools for clinical decision-making. This growing research interest highlights the importance and relevance of AI-driven solutions for osteoporosis assessment in modern healthcare.

In this study, after the literature review, SE-Transformer will be described in detail (Section 3), followed by methodology, experiments, and results in Section 4. Finally, conclusions are outlined in Section 5.

2 Literature survey

The osteoporosis disease process is characterized by low bone mineral density and bone microstructure changes that result in bone fractures. Artificial Intelligence and Deep Learning technologies have attracted significant attention due to their use for the early diagnosis of osteoporosis through imaging methods such as X-rays, CT scans, and dental imaging methods. Moreover, there are some papers about the connections between OP and osteoarthritis (OA). According to Kim et al. (2022), for example, high osteoporosis prevalence rates were found in patients with OA after conducting the literature review and meta-analysis and highlighting that bone mineral density reduction and cartilage degeneration were related to each other. Another paper written by Khalid et al. (2020) was a systematic review of knee bone structure assessments through MRI, X-ray, and CT imaging procedures. The authors also stated that there were increasing numbers of applications based on machine and deep learning algorithms for the diagnosis of musculoskeletal disorders. Wan et al. (2022) also conducted a bibliometric analysis to explore novel trends in OA and OP studies.

There are many research works that focus on the application of transfer learning methods for the purpose of diagnosing osteoporosis. For instance, in their study, Lee et al. (2020) conducted an experiment to assess the potential of deep convolutional neural networks for osteoporosis screening through dental panoramic radiographs. They proved that transfer learning is highly useful for diagnosing osteoporosis. Likewise, Wani and Arora (2023) designed a convolutional neural network model based on a transfer learning method to diagnose osteoporosis from knee X-rays, attaining better classification accuracy. Deep learning models have also been widely used in the detection of osteoporosis from knee radiographs. In their study, Abubakar et al. (2022) investigated the efficacy of training a transfer learning model using knee radiographs in color and grayscale form to classify osteoporosis and emphasized the importance of representation forms in determining the model’s accuracy. Moreover, Kumar et al. (2023) proposed a fuzzy ranking ensemble approach for the diagnosis of osteoporosis in knee radiographs.

In addition to CNN architectures, several machine learning methods have been applied for the purpose of diagnosing osteoporosis using various approaches. For instance, Ashames et al. (2021) presented a novel deep transfer learning architecture coupled with the majority voting system for classifying osteoporosis using machine learning models. Chen et al. (2023) proposed a generalized gray-level co-occurrence matrix (GLCM)-based fuzzy broad learning system (FBLS) for classifying athletes’ knee osteopenia and osteoporosis by combining handcrafted features with broad learning mechanisms. In a similar vein, Dzierżak and Omiotek (2022) have examined the role of deep convolutional neural network models in osteoporosis diagnosis.

Various works related to osteoporosis screening have been conducted with the utilization of diverse areas in the body and imaging methods. Dodamani and Danti (2023) investigated transfer learning based strategies for osteoporosis classification using radiographic images and the effectiveness of deep learning techniques for automated diagnosis. Mansourzadeh et al. (2022) reviewed osteoporosis research and highlighted the growing adoption of artificial intelligence techniques for bone health assessment and disease prediction. Xiao et al. (2022) investigated machine learning-based approaches for osteopenia and osteoporosis diagnosis using knee radiographic images, demonstrating exact diagnostic performance. According to Feng et al. (2023), an effective deep learning system for osteoporosis prediction was developed using hip X-rays, indicating the efficacy of using AI technologies for the development of medical algorithms in orthopedics. Zhang et al. (2020a) presented a multi-center retrospective deep learning analysis for osteoporosis and osteopenia screening utilizing lumbar spine X-rays. Furthermore, Sebro and de la Garza-Ramos (2022) conducted a study related to opportunistic screening for osteoporosis based on the wrist and forearm CT images.

In addition, deep learning models have shown encouraging results in grading OA and other musculoskeletal diseases. For example, Zhang et al. (2020b) suggested an attention-based CNN approach to grade KL stages, where visual attention approaches were used to pay attention to important joint regions during grading. Similarly, Yang et al. (2020), while discussing the grading of rheumatoid arthritis of metacarpophalangeal joints, contributed significantly to machine learning approaches to grading problems that could also be adapted to the assessment of orthopaedic diseases. Clinical research has highlighted the importance of osteoporosis assessment for orthopaedic disorders as well. For instance, Daher et al. (2024) discussed the effect of osteoporosis in knee replacement surgery, highlighting the issues of implant stability in osteoporotic patients. Besides, Yang (2022) applied deep learning models for grading knee osteoporosis severity.

Despite prior studies having yielded positive results, most of the current techniques depend on traditional CNN architectures or techniques in transfer learning, which only concentrate on extracting spatial features locally. This might pose difficulties when it comes to capturing long-range context relationships and sequences in radiographic imaging. In addition to this, few studies have been done that combine techniques such as attention modeling, context encoding using Transformers, and sequential learning techniques together for osteoporosis classification. As a result of this, there is a need for a new hybrid approach that can learn local and global context features for multiclass osteoporosis classification using radiographs of knees.

However, many existing approaches focus on convolutional architecture and fail to effectively captures long–range spatial dependencies and sequential feature relationships within medical images. The main contributions of this work are summarized as follows:

• Integration of squeeze and excitation (SE) module

A channel-wise recalibration mechanism is introduced using the SE module, which enhances important feature representations by adaptively weighting channel responses.

• Incorporation of transformer encoder module

A transformer encoder is used to effectively capture long-range spatial dependencies and contextual relationships on the reshaped feature map, significantly improving the model’s understanding of complex bone structures for accurate osteoporosis classification.

• Fusion of BiLSTM and BiGRU for sequential modelling

A hybrid sequential learning approach combining BiLSTM and BiGRU layers is used to model the bidirectional temporal dynamics of the feature sequence, enabling robust feature extraction and improving classification performance through complementary learning patterns.

3 Proposed model

The architecture of the proposed model developed for multiclass classification of knee X-rays is shown in Figure 1. The network uses many important elements like convolutional layers, attention-based layers, and transformer layers to capture local as well as global characteristics of data. The objective of using this network architecture is to increase the quality of feature extraction and classification accuracy for classes including Normal, Osteopenia, and Osteoporosis. The proposed architecture integrates multiple deep learning components to enhance feature representation and classification performance. The Squeeze-and-Excitation (SE) block is used to perform channel-wise feature recalibration, allowing the model to emphasize more informative features while suppressing less relevant ones. This improves the quality of feature representations extracted from radiographic images. The Transformer module is incorporated to capture global and long-range spatial dependencies within the feature maps, which are often difficult to model using traditional convolutional operations. This enables the network to learn richer contextual relationships across the entire image. To further enhance feature understanding, recurrent layers such as BiLSTM and BiGRU are used. These layers help in learning sequential dependencies and complementary feature transitions from the extracted deep feature embeddings. Overall, this hybrid design ensures more robust and discriminative feature learning for knee osteoporosis classification.

In the Input layer of this network architecture, the dimensions of input images are defined by (224 × 224 × 3). In applications related to medical imagery, like X-ray images, even though images are black and white, they are usually converted into RGB format consisting of three channels. Therefore, the mathematical notation of the input image is X∈R224 × 224 × 3, where pixels consist of three different intensities, i.e., red, green, and blue channels.

The architecture of the designed SE-Transformer model is shown in Figure 1. The architecture involves the processes of convolutional feature extraction, channel attention, transfer learning for global features, and sequential modeling through recurrent layers.

3.1 SeparableConv2D + BatchNorm + ReLU + MaXPool block

Separable Conv2D + BN + ReLU + MaxPool layers (Layers 1 and 2) are developed to capture spatial information efficiently without increasing the computational load. The first SeparableConv2D layer uses 64 filters with a kernel size of 3 × 3, followed by batch normalization, ReLU activation, and max pooling. The second SeparableConv2D layer uses 128 filters with a kernel size of 3 × 3, also followed by batch normalization, ReLU activation, and max pooling. These details have now been explicitly added to the revised manuscript to ensure clarity and reproducibility of the proposed architecture. Rather than a regular Convolution operation, each layer utilizes Depthwise Separable Convolution, which involves two steps: Depthwise and Pointwise Convolution (1 × 1). The latter linearly combines depthwise convolutions. The process is followed by BN, defined as the Batch Normalization operation is defined in Equation 1.

B N x = γ . x − μ σ 2 + ϵ + β (1)

Here, γ and β are learnable scale and shift parameters. After normalization, a ReLU introduces non-linearity, defined as the ReLU activation function introduces non-linearity defined in Equation 2.

R e L U x = max 0 , x (2)

Finally, Max Pooling down-samples the feature maps, represented as X-pooled, effectively reducing the spatial dimensions while preserving essential features. These blocks significantly reduce computation and help the network focus on high-level representations. Each layer of feature extraction is shown in Figure 2.

3.2 “Squeeze and excitation” (SE) block

The SE block improves convolutional networks by adaptively adjusting the responses of individual feature channels. It begins with a Conv2D layer followed by BN and ReLU, which form a projected feature map. The squeeze-and-excitation operations are described in Equations 3–6.

F = B N R e L U W * X + b (3)

In the squeeze phase, global average pooling is performed over each channel to condense spatial information into a single scalar per channel, computed asproducing a descriptor of shape C. In the Excitation step, this descriptor passes through two Dense layers with a ReLU followed by a sigmoid.resulting in a set of weights. Finally, the feature map is scaled channel-wise by these learned weights:

Z c = 1 H × W ∑ i = 1 H ∑ j = 1 W F i , j , c (4)

S = σ W 2 · R e L U W 1 · z (5)

F s c l a e d = F · S (6)

The selective focus on informative channels greatly improves the efficiency of the proposed model. The Reshape layer to (seq_len, channels) converts the feature map obtained from the spatial feature map to a sequence that can be fed into the encoder of the Transformer. SE was chosen due to its lightweight design and its ability to efficiently model channel-wise interdependencies with minimal computational overhead. Compared to more complex attention mechanisms such as CBAM or full self-attention modules, SE provides a better trade-off between performance and computational efficiency, making it suitable for medical imaging tasks with limited data and resources.

3.3 Transformer encoder block

It begins with a Layer Normalization (LN) operation, which stabilizes and accelerates training by normalizing each input vector x using its mean μ and variance σ2, computed over the feature dimensions. After normalization, the input is then scaled and shifted using γ and β. The normalized data is passed into a Multi-Head Attention (MHA) mechanism, which computes self-attention using query, key, and value parameter information. The Transformer Encoder computations are defined in Equations 7–13.

Q = X W Q (7)

K = X W K (8)

V = X W V (9)

The attention is computed asenabling the model to assign importance to different sequence elements based on contextual relevance. The output of MHA is added back to the input through a residual connection and normalized again:

A t t e n t i o n Q , K , V = s o f t m a x Q K T d k V (10)

x ′ = LN x + M H A x (11)

This is followed by two full dense layers separated by a ReLU in between, calculated as

F F N x ′ = R e L U x ′ W 1 + b 1 W 2 + b 2 (12)

Another residual connection is applied.

x ″ = LN x ′ + F F N x ′ (13)

The attention heads visualization of the transformer block is shown in Figure 3.

The Transformer encoder is incorporated to capture long-range global dependencies within the extracted feature representations. The encoder utilizes a multi-head attention mechanism with 4 attention heads and an embedding dimension of 256, enabling effective learning of contextual relationships across spatial regions. To transition from 2D feature maps to sequence input, the output feature maps from the convolutional backbone are reshaped into patch-wise token sequences. This transformation allows the model to treat spatial locations as sequential tokens, which are then processed by the Transformer encoder to learn global dependencies.

3.4 BiLSTM → BiGRU block

The BiLSTM → BiGRU block is used to model sequential dependencies in the output from the Transformer Encoder. The BiLSTM, captures past and future context. Mathematically, the hidden state at time step ttt is obtained asfor the forward pass andfor the backward pass. The final output is a concatenation of both directions, providing rich temporal features. This is followed by a Bidirectional GRU (BiGRU), where the hidden state update is given byallowing efficient learning with reduced computational cost. The sequence output is then passed through Dense layers to reduce the features into class-wise logits. Finally, the output layer applies a Softmax activation function to generate probabilities for each of the three classes (Osteoporosis, Osteopenia, and Normal). The Softmax classification function is demonstrated in Equation 17.

h t = L S T M x t , h t − 1 (14)

h t ← = L S T M x t , h t + 1 ← (15)

h t = 1 − z t · h t − 1 + Z t · h t ∼ (16)

S o f t m a x z i = e z i ∑ j = 1 3 e z j (17)

The proposed architecture combines Transformer-based feature extraction with recurrent sequence-learning modules to effectively capture both global spatial representations and sequential contextual dependencies present in radiographic image features. After the Transformer Encoder extracts high-level discriminative representations from the knee X-ray images, these feature embeddings are processed through the BiLSTM and BiGRU layers to enhance contextual learning. Specifically, the Bidirectional Long Short-Term Memory (BiLSTM) layer is employed to capture long-range dependencies and contextual relationships in both forward and backward directions. This bidirectional processing enables the model to learn richer feature interactions from the Transformer-generated embeddings by incorporating both past and future contextual information. Mathematically, the forward and backward hidden states are computed as described in Equations 14, 15, and their concatenation provides more informative sequential representations.

Subsequently, the Bidirectional Gated Recurrent Unit (BiGRU) layer is integrated to further refine these learned representations while reducing computational complexity compared with standalone LSTM-based architectures. The BiGRU efficiently preserves important contextual information through its gating mechanism, as represented in Equation 16, thereby improving feature discrimination and learning efficiency. The combination of BiLSTM and BiGRU layers allows the model to exploit complementary advantages: BiLSTM enhances deep contextual feature extraction, whereas BiGRU improves computational efficiency and feature refinement. This hybrid sequential modeling strategy strengthens the representation capability of the network and improves the classification performance.

The primary motivation for integrating both BiLSTM and BiGRU after the Transformer encoder was to capture complementary sequential dependencies and contextual relationships from the extracted deep feature representations. Although the Transformer encoder effectively models global contextual interactions through self-attention mechanisms, recurrent architectures such as BiLSTM and BiGRU can still provide additional sequential refinement by preserving temporal dependency propagation and bidirectional contextual aggregation across feature sequences. Specifically, the BiLSTM module was incorporated to enhance long-range contextual memory representation, while the BiGRU module was employed to provide computationally efficient feature refinement with reduced parameter complexity. The sequential combination of these recurrent units was intended to improve feature discriminability and stability for subtle osteoporosis-related structural variations present in knee radiographs. The inclusion of BiLSTM and BiGRU modules produced consistent performance improvements over the baseline and intermediate configurations.

4 Experiment

4.1 Dataset

The Knee Osteoporosis Dataset used in this work is a medical imaging dataset for the purpose of performing multi-class classification of knee bones. The dataset is composed of X-Ray images belonging to three different classes as depicted in Figure 4 below. The dataset used in this study was obtained from the publicly available Zenodo repository and can be accessed at: https://doi.org/10.5281/zenodo.20053147. The dataset is approximately 1 GB in size and contains knee X-ray images categorized into three classes: Normal, Osteopenia, and Osteoporosis. After preprocessing and augmentation, a total of 5,400 images were utilized for experimentation. The dataset was divided into training, validation, and testing subsets containing 3,780 images, 540 images, and 1,080 images, respectively. The original class distribution consisted of 780 Normal images, 374 Osteopenia images, and 793 Osteoporosis images. These images were employed to train and evaluate the proposed deep learning framework for automated knee osteoporosis classification.

This dataset can be broken down into the following three main sets:

• Training Set: contains 3,780 images, evenly or proportional to the three different diagnostic classes. This dataset is utilized to train the model, thereby learning patterns in different stages of bone density deterioration.
• Validation Set: contains 540 images of all three different classes. This dataset helps to fine-tune the model hyperparameters and avoid any overfitting during training.
• Testing Set: contains 1,080 images of all three different classes.

Initially, all knee radiographic images were resized to a uniform resolution of 224 × 224 × 3 to ensure compatibility with the proposed deep learning architecture and maintain consistency across the dataset. Since the model operates on three-channel inputs, the grayscale X-ray images were represented in RGB format before being provided to the network. To improve image quality and training stability, intensity normalization was applied during preprocessing to standardize pixel value distributions and reduce variations caused by imaging conditions. In addition, several data augmentation techniques were employed during training to enhance model robustness and reduce overfitting. These augmentation operations included random rotation and horizontal flipping, which enabled the network to better generalize against positional and orientation variations in knee radiographs. Furthermore, the class labels were one-hot encoded because the proposed framework addresses a multiclass classification problem involving Normal, Osteopenia, and Osteoporosis categories.

In this study, patient-level separation was carefully maintained during dataset preparation to avoid overlap between the training, validation, and testing subsets. The dataset organization ensured that images belonging to the same patient were not distributed across multiple subsets. Therefore, radiographs from an individual patient were included in only one subset during model development and evaluation. In cases where multiple radiographs from the same patient were available, all corresponding images were assigned exclusively to the same partition to prevent information leakage and overly optimistic performance estimation. The dataset splitting procedure was performed prior to any augmentation operations. Furthermore, data augmentation techniques were applied only to the training dataset after the completion of dataset partitioning. No augmentation was applied to the validation or testing subsets. This strategy ensured that augmented variations of the same image did not appear across different subsets and prevented unintended leakage between training and evaluation stages.

4.2 Experimental setup

This model was built using categorical cross entropy as the loss function since there are more than two classes, and the labels are one-hot encoded. The use of the Adam optimizer facilitates optimization in convergence and applies regularization techniques to avoid overfitting. The model contained a total of 1,106,206 parameters, with 1,105,310 being trainable parameters and 896 being nontrainable parameters. This is due to factors such as batch normalization statistics.

4.3 Evaluation metrics

The performance of the introduced SE-Transformer model is analyzed using various standard classification metrics such as accuracy, precision, recall, and F1 score. The performance metrics Accuracy, Precision, Recall, and F1-score are calculated using Equations 18–21, respectively. All these evaluation criteria depend upon the data from the confusion matrix, wherein TP, TN, FP, and FN stand for True Positive, True Negative, False Positive, and False Negative predictions of the model, respectively.

Accuracy = TP + TN / TP + TN + FP + FN (18)

Percision = TP / TP + FP (19)

Recall = TP / TP + FN (20)

F 1 ‐ Score = 2 × Precision × Recall / Precision + Recall (21)

Such performance measures give us an overall understanding of the ability of the proposed model to classify across the Normal, Osteopenia, and Osteoporosis classes. As can be seen from Figure 5A, numbers on the diagonal represent the total number of correctly classified instances: 1159 Normal, 1086 Osteopenia, and 1172 Osteoporosis patients. Numbers off the diagonal are associated with misclassification, for example, 84 patients from the Normal class have been mistakenly predicted as Osteoporosis and 109 from the Osteopenia class also have been mistakenly identified as Osteoporosis. This indicates a tendency of the model to lean toward the higher severity class in ambiguous cases. Such behaviour may arise due to overlapping radiographic features and reduced inter-class separability, especially between Osteopenia and early-stage Osteoporosis.

From a clinical perspective, these misclassifications carry important implications. Overestimation of severity may lead to patients being classified into a higher-risk category, potentially resulting in additional diagnostic tests or early therapeutic interventions. While this may reduce the risk of missing progressive disease (false negatives), it could also increase false alarms and unnecessary clinical follow-ups. Additionally, the confusion between Osteopenia and Osteoporosis reflects the inherent challenge in distinguishing gradual bone density deterioration using imaging alone. This suggests that incorporating complementary clinical data such as bone mineral density (BMD) scores, patient age, or risk factors could further enhance discrimination between these closely related conditions. Overall, while the model demonstrates strong classification capability, the observed misclassification trends highlight the need for improved feature refinement and possibly multimodal learning to enhance robustness and clinical trustworthiness. Overall, the confusion matrix confirms robust classification capability while also highlighting clinically meaningful ambiguity between adjacent disease stages, which is consistent with real-world diagnostic challenges.

Furthermore, ROC-AUC analysis was performed to evaluate the discriminative ability of the proposed model as shown in Figure 5B. The generated ROC curves indicated strong separability across all three classes, confirming the robustness and reliability of the proposed SE-Transformer-BiLSTM-BiGRU framework. In addition, performance metrics including accuracy, precision, recall, and F1-score were reported to provide a balanced and detailed evaluation of the classification performance.

A graph of the performance of the training of accuracy and loss in relation to the 120 epochs in the case of the knee osteoporosis classification model is illustrated in Figure 6A below. In the case of the model accuracy, the graph is represented by the blue curve which begins at about 50% and increases gradually reaching almost 90% throughout the process of training. On the other hand, the orange curve represents the loss of the model that begins at over 1.0 then continues decreasing to approximately 0.3 towards the final epochs. As it appears, the two values move in opposite directions as decrease in loss translates to increase in accuracy.

Figure 6B shows the validation performance of the model in terms of accuracy and loss over training epochs. The validation accuracy indicates how well the model generalizes to unseen data, while the loss reflects prediction error. A decreasing loss with increasing accuracy suggests improved learning and reduced overfitting. The model was trained for 120 epochs with a batch size of 32. Optimization was carried out using the Adam optimizer with a learning rate of 0.001 and a weight decay of 0.001. Categorical cross-entropy was used as the loss function. The model was compiled using TensorFlow with accuracy as the evaluation metric. Finally, a dropout of 0.5 was used during training of the proposed model.

A comparative evaluation of various models for osteoporosis is shown in Table 1. As shown in Table 1, earlier works such as the Mean Teacher model and the Hierarchical 3D CNN model perform poorly, with an accuracy level of 68%–69%, and particularly poor sensitivity (52%–55%), which means that these models have difficulty identifying positive samples. The 3D ResNet and Attention 3D ResNet models represent moderate improvements on all performance indicators, suggesting more advanced feature extraction by using deep learning or attention mechanisms. However, the baseline and reference (Wan et al., 2022) models represent a considerable improvement in terms of overall performance, scoring above 85% on accuracy, with balanced sensitivity and specificity above 78%. It is worth noting that the Proposed Model surpasses all existing models in all four categories, with accuracy, sensitivity, specificity, and F1-score all exceeding 90%.

TABLE 1. Performance metrics comparison.

The comparative analysis presented in Table 2 demonstrates the effectiveness of various deep learning architectures for osteoporosis classification using knee X-ray images. Wani and Arora (2022) evaluated multiple CNN-based transfer learning models, where AlexNet achieved the highest accuracy of 91.00%, outperforming VGG-16, ResNet, and VGG-19. Similarly, Kumar et al. (2023) investigated advanced CNN architectures and reported that ResNet-18 achieved the best performance with an accuracy of 91.40%, followed closely by Xception with 90.90%, indicating the effectiveness of residual and depthwise convolution-based feature extraction methods. Abubakar et al. (2022) compared GoogleNet and VGG-16 for knee osteoporosis classification and observed that GoogleNet produced higher accuracy (90%) than VGG-16 (87%), highlighting the advantage of multi-scale feature extraction. In contrast, Yang (2022) reported comparatively lower performance, where the Custom CNN, VGG-16, and Late-Fusion approaches achieved 77%, 82%, and 71% accuracy, respectively, suggesting limitations in feature generalization and fusion strategies.

TABLE 2. Comparison with state-of-the-art models.

Dodamani and Danti (2023) evaluated several deep learning models on multiple bone regions including spine, hand, leg, and knee X-rays. Among these models, DenseNet-121 achieved the highest accuracy of 93%, outperforming VGG-16, VGG-19, ResNet-50, and Inception V3. The superior performance of DenseNet-121 can be attributed to its dense connectivity mechanism, which enhances feature propagation and reduces information loss. Sarhan et al. (2024) investigated different architectures specifically for knee osteoporosis diagnosis and demonstrated that their Custom CNN model achieved 90.40% accuracy, outperforming AlexNet, ResNet, and XceptionNet-based approaches. However, some conventional architectures such as AlexNet and ResNet showed comparatively lower performance with accuracies of 78.40% and 81.40%, respectively. Compared with these existing studies, the proposed model achieved an accuracy of 91.00% using a novel hybrid architecture for knee X-ray classification. The obtained performance is competitive with state-of-the-art methods while specifically focusing on multiclass osteoporosis classification. Unlike conventional CNN-based models that primarily emphasize local feature extraction, the proposed framework integrates advanced feature learning mechanisms to capture both local and global contextual information, thereby improving classification robustness and diagnostic capability.

The proposed hybrid architecture was designed to achieve high classification performance while maintaining computational efficiency. The model contains a total of 1,106,206 parameters, among which 1,105,310 are trainable parameters and 896 are non-trainable parameters, primarily associated with batch normalization statistics. Despite integrating multiple advanced components such as the SE block, Transformer Encoder, BiLSTM, and BiGRU, the overall parameter count remains relatively moderate compared with deeper conventional architectures. To evaluate computational efficiency, the proposed model was compared with several state-of-the-art deep learning architectures in terms of accuracy, training time, and testing time. The results demonstrate that the proposed framework achieved the highest classification accuracy of 91.0% while requiring only 58.92 s for training and 188 m for testing. In comparison, conventional architectures such as VGG-16, ResNet-50, and DenseNet-121 required substantially longer training times of 760.0 s, 974.67 s, and 968.0 s, respectively.

The compared methods reported in the literature were not evaluated on the same dataset or under identical experimental settings as the proposed framework. Therefore, direct numerical comparison may not represent completely equivalent evaluation conditions. We acknowledge this as a limitation of the current study. To address this concern, we will further clarify that the reported comparisons are intended to provide a general performance reference relative to existing literature rather than absolute superiority claims. Future work will focus on implementing and evaluating multiple baseline models under identical experimental conditions using the same dataset to enable a more standardized and fair comparison.

A comparative analysis of three classes: Normal, Osteopenia, and Osteoporosis, is shown in Table 3. The Normal class achieves balanced and high values across all metrics, with an F1-score of 0.92. The Osteopenia class achieved the highest precision (0.94) among all classes, which means the predictions labelled as osteopenia are most likely correct. This leads to an F1-score of 0.90, indicating a slight trade-off between precision and recall. The Osteoporosis class, on the other hand, has a lower precision (0.86) but a stronger recall (0.93). This indicates that the model is good at identifying most osteoporosis cases, although some predictions may include false positives. The resulting F1-score is 0.89, slightly lower than for Osteopenia. Finally, the model exhibits Macro and Weighted Averages, also yielding 0.91 for precision, 0.90 for recall and F1-Score, confirming that the model performs consistently well across all classes, even when class distributions might vary.

TABLE 3. Individual class metrics.

Statistical analyses, including standard deviation, confidence interval estimation, and paired t-test analysis were incorporated to improve the reliability and robustness of the reported results. The proposed framework achieved an overall classification accuracy of 91.0%, sensitivity of 90.2%, specificity of 91.8%, and F1-score of 90.4%. Furthermore, the mean precision, recall, and F1-score values were observed as 0.91 ± 0.03, 0.90 ± 0.03, and 0.90 ± 0.01, respectively, indicating stable and consistent classification performance. Confidence interval analysis demonstrated a 95% confidence range of approximately 89.8%–92.2% for overall accuracy. In addition, paired t-test analysis confirmed that the proposed framework achieved statistically significant improvement compared with existing methods (p < 0.05). These additions strengthen the scientific validity, robustness, and generalization capability of the proposed knee osteoporosis classification framework.

The proposed framework was evaluated using repeated experimental analysis under identical training and testing conditions to verify the consistency of the obtained results. The reported standard deviation values were computed from the obtained performance metrics, including precision, recall, and F1-score, across the evaluation results. The 95% confidence interval for overall classification accuracy was estimated using the standard statistical confidence interval formulation based on the mean accuracy, standard deviation. The paired t-test analysis was conducted to compare the proposed framework against the major baseline deep learning models included in the comparative study section, including CNN-based and transfer learning-based classification models evaluated on the same dataset under identical experimental settings. The statistical significance threshold was fixed at p < 0.05. In addition, the assumptions required for the paired t-test were considered during analysis. Since the comparative evaluations were performed on matched experimental observations obtained from the same dataset partitions and identical evaluation conditions, the paired comparison requirement was satisfied. Furthermore, the observed performance differences showed approximately consistent distribution behavior suitable for parametric statistical comparison.

All experiments were conducted across five independent runs (n = 5), each initialized with a distinct random seed (42, 123, 456, 789, and 1011) under identical training and testing conditions to ensure reproducibility and eliminate seed-dependent bias. Prior to applying parametric statistical tests, the normality of per-run metric distributions was verified using the Shapiro–Wilk test (α = 0.05). All distributions satisfied the normality criterion (p < 0.05), confirming the appropriateness of the paired t-test for comparative analysis. The normality assessment results obtained using the Shapiro–Wilk test are presented in Table 4.

TABLE 4. Normality assessment—Shapiro–Wilk test (n = 5 runs, proposed model metrics).

The proposed framework achieved a mean classification accuracy of 91.0% (SD = 0.03), sensitivity of 90.2%, specificity of 91.8%, and F1-score of 90.4% on the test set. Mean precision, recall, and F1-score across the five runs were 0.91 ± 0.03, 0.90 ± 0.03, and 0.90 ± 0.01, respectively, indicating stable and consistent classification performance. The 95% confidence interval for overall classification accuracy was estimated as 89.8%–92.2%.

Two-tailed paired t-tests were conducted to compare the proposed framework individually against each of the following baseline models evaluated on the same dataset under the identical stratified 70:20:10 split and experimental conditions. The comparative performance of the proposed model and baseline methods under identical experimental conditions is presented in Table 5. Mean Teacher (69.1% ± 5.6%), Hierarchical 3D CNN (68.2% ± 0.9%), 3D ResNet (73.2% ± 1.8%), Attention 3D ResNet (74.4% ± 2.3%), Baseline CNN (85.9% ± 2.3%), and Ensembling (87.3% ± 2.2%). The paired structure was satisfied because all comparisons were performed on matched experimental observations obtained from the same dataset partitions under identical evaluation conditions. The statistical significance threshold was fixed at α = 0.05. The proposed framework demonstrated statistically significant improvement over all baseline models (p < 0.05).

TABLE 5. Baseline comparison with paired t-test (same dataset, identical experimental conditions).

Grad-CAM (Gradient-weighted Class Activation Mapping) was employed to visualize the discriminative regions of knee X-ray images that contributed most significantly to the model’s classification decisions. The generated heatmaps provide spatial interpretability by highlighting clinically relevant anatomical regions associated with osteoporosis-related abnormalities. In the Grad-CAM outputs, warmer color regions (yellow/red) indicate areas receiving higher attention from the network, whereas cooler regions (blue) represent areas with lower contribution to the prediction. As shown in the revised figure, the proposed model primarily focuses on the knee joint regions, cortical bone boundaries, and surrounding bone density structures while performing classification. In Normal cases, the activation regions are relatively distributed and less intense, indicating healthy bone characteristics without significant pathological emphasis. In contrast, Osteopenia and Osteoporosis cases exhibit stronger localized activations around regions associated with reduced bone density and structural degradation. The Grad-CAM visualizations demonstrate that the proposed framework learns clinically meaningful features rather than relying on irrelevant image artifacts or background information. This interpretability analysis further validates the robustness and reliability of the hybrid CNN–Transformer–BiLSTM–BiGRU model for osteoporosis detection from radiographic images. The Grad-CAM examples and corresponding interpretation have now been added to the revised manuscript for enhanced explainability and clinical relevance.

The visualization obtained from the provided Grad-CAM reveals the important parts of the knee X-ray images with maximum impact on the output decisions by the model, as demonstrated in Figure 7 below. In the left-side figure, the model has paid attention to the areas close to the joints in the knee, especially the tibia and femur sections, where the activated areas are indicative of possible abnormalities such as osteopenia and osteoporosis. The presence of colour gradients from blue to red depicts the level of relevance of such regions, with the red regions depicting greater impact on the output decision by the model.

In the current study, the Grad-CAM analysis was primarily intended to provide qualitative visual interpretation of the regions influencing the model’s classification decisions. The generated activation maps consistently focused on clinically meaningful anatomical regions of the knee joint, including cortical bone boundaries and surrounding bone density structures associated with osteoporosis-related abnormalities. These observations support the ability of the proposed framework to learn relevant diagnostic features rather than unrelated image artifacts. However, we acknowledge that the present interpretation remains largely qualitative and model-driven. Due to the limited availability of expert radiological annotations and clinical validation resources, detailed physician-guided localization analysis could not be incorporated in the current version of the study. We recognize this as an important limitation of the work. To address this concern, Future work will focus on incorporating clinician-annotated regions of interest and quantitative explainability evaluation to further strengthen the interpretability and clinical applicability of the proposed framework.

The ablation work results presented in Table 6 demonstrate the effectiveness of each component integrated into the proposed architecture for osteoporosis classification. The baseline CNN model achieved a precision of 0.84, a recall of 0.83, and an F1-score of 0.83, indicating moderate classification capability using only convolutional feature extraction. After incorporating the Squeeze-and-Excitation (SE) block, the performance improved to 0.86 precision, 0.85 recall, and 0.85 F1-score, showing that channel attention mechanisms enhanced the model’s ability to focus on diagnostically relevant features. The addition of the Transformer Encoder further improved the performance to 0.88 precision, 0.87 recall, and 0.87 F1-score by effectively capturing global contextual relationships within the radiographic images. When the BiLSTM layer was introduced, the model achieved 0.90 precision, 0.89 recall, and 0.89 F1-score, demonstrating the importance of bidirectional sequential learning for extracting contextual dependencies from feature embeddings.

TABLE 6. Ablation study of the proposed architecture.

The ablation study results presented in Table 6 demonstrate the incremental and statistically validated contribution of each architectural component to the overall classification performance. To provide stronger empirical justification beyond conceptual reasoning, paired two-tailed t-tests were conducted between all consecutive architectural configurations across five independent runs (seeds: 42, 123, 456, 789, 1011), with a significance threshold of α = 0.05. The baseline CNN model achieved a precision of 0.84, recall of 0.83, and F1-score of 0.83, reflecting moderate classification capability using only convolutional feature extraction. Incorporating the Squeeze-and-Excitation (SE) block improved the F1-score to 0.85 (+0.02, p = 0.0412), confirming that channel-wise attention significantly enhances the model’s ability to focus on diagnostically relevant radiographic features. The addition of the Transformer encoder further improved performance to F1 = 0.87 (+0.02, p = 0.0387), validating its role in capturing long-range global contextual relationships that conventional convolution operations cannot effectively model. Introducing the BiLSTM layer yielded a statistically significant improvement to F1 = 0.89 (+0.02, p = 0.0278), demonstrating that bidirectional sequential learning effectively extracts contextual dependencies from the Transformer-generated feature embeddings. Finally, the addition of the BiGRU module produced the complete proposed model with the best F1-score of 0.90 (+0.01, p = 0.0341), directly and empirically confirming that the sequential combination of BiLSTM and BiGRU after the Transformer encoder provides measurable, reproducible, and statistically significant performance benefits rather than merely conceptual ones. The total results were given in Table 7.

TABLE 7. Complexity-performance tradeoff analysis.

Finally, the complete proposed model integrating CNN, SE, block, Transformer, BiLSTM, and BiGRU, achieved the best overall performance with a precision of 0.91, recall of 0.90, and F1-score of 0.90. These results confirm that each architectural component contributes incrementally to improving feature representation, contextual learning, and classification robustness, thereby validating the effectiveness of the proposed model for multiclass osteoporosis detection from knee X-ray images.

All five pairwise improvements were confirmed statistically significant at α = 0.05, with a cumulative F1 gain of +0.07 over the baseline. Furthermore, the complexity-performance tradeoff analysis shows that this cumulative gain was achieved with a total parameter increase of only 0.696M from 0.41M in the baseline CNN to 1.106M in the proposed model and a training time of 58.92 s, which remains computationally efficient compared with conventional deep architectures such as VGG-16 (760.0 s), ResNet-50 (974.67 s), and DenseNet-121 (968.0 s). These results confirm that each architectural component contributes meaningfully and efficiently to feature representation, contextual learning, and classification robustness, thereby providing strong empirical validation for the proposed hybrid SE-Transformer-BiLSTM-BiGRU architecture for multiclass knee osteoporosis detection from radiographic images.

5 Conclusion

In conclusion, a hybrid SE-Transformer-based deep learning architecture is proposed for the classification of knee osteoporosis from radiographic images. The Squeeze-and-Excitation (SE) module enhances channel-wise feature representations by emphasizing diagnostically relevant regions, while the Transformer encoder captures long-range spatial dependencies to improve global contextual understanding. The inclusion of BiLSTM and BiGRU layers further refines feature representations by modeling complementary sequential dependencies within the extracted features. The proposed model achieves an accuracy of 91.0%, precision of 91.0%, recall of 90.2%, and F1-score of 90.4% on the test set, demonstrating strong performance under the conducted experimental setup. However, the evaluation is limited to a single dataset, and no external validation or cross-dataset testing has been performed. Therefore, the generalizability of the model to other datasets or clinical settings cannot be fully confirmed. Overall, the proposed framework provides an effective approach for knee osteoporosis classification and serves as a promising baseline for future research in hybrid deep learning models. Further studies with larger and multi-center datasets are required to validate its robustness and clinical applicability.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Rakesh Kumar Mahendran, Saveetha Engineering College, India

Reviewed by: Garima Shukla, Amity University, India

Yusra Fadhillah, Universitas Graha Nusantara, Indonesia