An XGBoost–logistic regression hybrid stacking model for breast cancer outcome prediction

1 Introduction

Breast cancer is the most commonly diagnosed malignancy among women and remains one of the leading causes of cancer-related mortality worldwide [1]. In 2022, approximately 2.3 million new cases were reported, accounting for 11.6% of all cancer diagnoses and highlighting its continuing global health burden [2]. Although advances in screening, diagnosis, and treatment have improved patient outcomes, accurately predicting disease progression remains a major challenge in clinical oncology. Reliable prognostic models are essential for early risk stratification, treatment selection, and personalized clinical decision-making.

The disease burden is particularly severe in low- and middle-income countries, especially across Africa, where mortality remains disproportionately high relative to incidence [2]. Limited screening programmes, delayed diagnosis, and inadequate healthcare infrastructure contribute substantially to these poor outcomes. In 2022, the continent recorded approximately 198,300 new cases and 91,300 deaths [2], underscoring the need for robust data-driven prediction tools capable of supporting timely diagnosis and intervention in resource-constrained settings.

Traditional statistical models, including logistic regression and the Cox proportional hazards model, remain widely used because of their interpretability and strong theoretical foundations [3, 4]. Logistic regression estimates outcome probabilities through log-odds, whereas the Cox model is designed for survival analysis. However, both rely on assumptions such as linearity, proportional hazards, and additive effects that are often violated in complex biomedical data, limiting their flexibility and predictive capability [3].

Machine learning methods have emerged as effective alternatives by capturing non-linear relationships and complex interactions without requiring strict parametric assumptions. Among these, Extreme Gradient Boosting (XGBoost) has demonstrated excellent performance on structured clinical datasets because of its predictive accuracy and generalization ability [5]. The algorithm builds decision trees sequentially, allowing successive trees to correct errors from earlier iterations and improve overall prediction.

Despite these strengths, machine learning models are frequently criticized for their limited transparency, reducing confidence in clinical applications where explainability is essential for trust and accountability [6]. Explainable artificial intelligence (XAI) methods, including SHapley Additive exPlanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME), address this limitation by providing both global and instance-level explanations of model behavior [7, 8].

Comprehensive evaluation should also assess calibration using measures such as the Brier score and calibration curves [9, 10], together with clinical utility through Decision Curve Analysis (DCA) [11, 12]. However, these aspects remain insufficiently reported, limiting the translation of predictive models into routine clinical practice.

Although breast cancer prediction has advanced considerably, important methodological gaps remain in how statistical model selection, heterogeneous ensemble learning, and clinical evaluation are integrated within a single predictive framework. Hybrid and stacking approaches combining Logistic Regression with machine learning algorithms have previously been investigated; therefore, the present study does not claim novelty from the use of Logistic Regression, XGBoost, or stacking architecture individually. Instead, the methodological contribution lies in the structured integration of corrected Akaike Information Criterion (AICc)-guided predictor selection into the statistical component of a heterogeneous Logistic Regression–XGBoost stacking framework. In the proposed pipeline, AICc is used explicitly to identify a parsimonious subset of predictors before construction of the selected hybrid model, thereby linking information-theoretic statistical model selection with non-linear ensemble learning. Information-theoretic criteria such as AIC, AICc, and Bayesian Information Criterion (BIC) are traditionally used for statistical model selection [13, 14]; their explicit use as a predictor-selection stage preceding heterogeneous stacking for binary breast cancer outcome prediction constitutes the specific methodological feature investigated in this study.

The contribution is further distinguished by the evaluation framework applied to the resulting model. Rather than treating discrimination as the sole criterion of model performance, the proposed pipeline evaluates discrimination using ROC-AUC and PR-AUC, classification performance using F1-score, probabilistic calibration using the Brier score and Expected Calibration Error (ECE), clinical utility using Decision Curve Analysis (DCA), and prediction interpretability using SHAP and LIME. Consequently, the novelty claimed in this study is methodological and integrative rather than algorithmic: the study investigates whether information-theoretically guided predictor selection, heterogeneous Logistic Regression–XGBoost stacking, calibration assessment, explainability, and clinical utility analysis can be combined into a coherent framework for breast cancer outcome prediction.

Accordingly, the present study does not seek to establish that the proposed model is universally superior to previously published breast cancer prediction models. Instead, it evaluates whether the integration of these methodological components provides a more comprehensive framework for assessing predictive reliability. The framework therefore considers discrimination, class-imbalance-sensitive performance, calibration, clinical utility, and interpretability as complementary dimensions of model evaluation. This distinction is important because a model with marginally higher discrimination may not necessarily provide better calibrated probabilities, greater clinical net benefit, or more interpretable predictions.

This framework is therefore intended as an integrative methodological contribution rather than a claim of algorithmic novelty. The central hypothesis is that information-theoretically guided predictor selection can provide a more parsimonious statistical component, while heterogeneous stacking can preserve complementary non-linear information captured by XGBoost. Furthermore, evaluating the resulting model across discrimination, calibration, explainability, and clinical utility is expected to provide a more comprehensive assessment of predictive reliability than discrimination-based evaluation alone.

To overcome these limitations, this study presents a hybrid breast cancer prediction framework using the SEER dataset. The proposed approach combines information-theoretic model selection with ensemble learning by applying AICc to the logistic regression component before integrating Logistic Regression and XGBoost as heterogeneous base learners within a stacking architecture. This design seeks to improve predictive accuracy, model parsimony, calibration, and robustness while maintaining statistical interpretability.

Model transparency is enhanced using SHAP and LIME to provide complementary global and local explanations of predictions. Overall, the proposed framework unifies statistical model selection, ensemble learning, calibration-aware evaluation, and explainable artificial intelligence into a single pipeline, the resulting framework provides a unified, interpretable, and calibration-aware approach for breast cancer outcome prediction and clinical decision support.

2 Methodology

2.1 Study design

This study proposes a quantitative hybrid framework for breast cancer outcome prediction that integrates statistical learning and ensemble machine learning. Logistic Regression provides an interpretable statistical model, while XGBoost captures non-linear relationships and complex feature interactions.

The workflow comprises data pre-processing, AICc-guided variable selection for Logistic Regression, independent training of Logistic Regression and XGBoost models, stacking-based probability fusion, and model interpretation using SHAP and LIME. The complete framework is illustrated in Figure 1.

2.2 Data source

The study uses data from the Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute (NCI), a population-based cancer surveillance programme that collects cancer incidence and survival information from participating population-based cancer registries in the United States [15]. The present analysis therefore represents patients recorded within the SEER registry system rather than a single institutional cohort. The dataset used in this study contains demographic, clinical, pathological, and outcome variables, but does not include a patient-level geographic provenance variable suitable for reporting individual residential location.

The analytical dataset comprised 4,024 breast cancer patients and 14 variables, including demographic, clinical, pathological, and outcome information. The predictor variables consisted of four numerical variables (Age, Tumor Size, Regional Node Examined, and Reginol Node Positive) and nine categorical variables (Race, Marital Status, T Stage, N Stage, 6th Stage, Grade, A Stage, Estrogen Status, and Progesterone Status). The binary outcome variable was patient Status, indicating whether the patient was alive or deceased.

Breast cancer outcome prediction is formulated as a supervised binary classification problem, with the complete mathematical formulations presented in Equations 1–24.

D={(xi,yi)}i=1n, xi∈ℝp,yi∈{0,1},(1)

where x_*i* denotes the predictor vector for the ith patient, y_*i* = 0 denotes an alive patient, and y_*i* = 1 denotes a deceased patient. Thus, the positive class in the predictive modeling framework corresponds to non-survival (death).

2.3 Data pre-processing and cleaning

Data pre-processing was performed to ensure data quality, consistency, and reproducibility prior to model development. The initial dataset contained 4,024 breast cancer patients and 14 variables, comprising four numerical predictors (Age, Tumor Size, Regional Node Examined, and Reginol Node Positive), nine categorical predictors (Race, Marital Status, T Stage, N Stage, 6th Stage, Grade, A Stage, Estrogen Status, and Progesterone Status), and the binary outcome variable (Status).

The outcome variable was coded as a binary indicator, with 0 representing patients classified as alive and 1 representing patients classified as dead. Thus, the prediction task was defined as the prediction of non-survival (death).

Before model development, character strings representing missing observations (“?”) were converted to missing values. A complete-case assessment was then performed to determine the extent of missingness. No missing observations were identified in the final analytical dataset. Consequently, neither numerical nor categorical imputation was required, and all 4,024 patients were retained for analysis. This approach avoided introducing artificial values into the dataset and ensured that the observed distributions of the clinical variables were preserved.

For transparency, median imputation would have been appropriate for numerical variables such as Age, Tumor Size, Regional Node Examined, and Reginol Node Positive had missing observations been present, because the median is less sensitive to skewness and extreme observations than the arithmetic mean [16, 17]. For categorical variables, mode imputation would have been appropriate for Race, Marital Status, T Stage, N Stage, 6th Stage, Grade, A Stage, Estrogen Status, and Progesterone Status. However, because no missing values were observed, these imputation procedures were not applied in the present analysis. Avoiding unnecessary imputation also prevented potential distortion of category frequencies and numerical distributions.

Categorical predictors were subsequently converted into numerical representations using one-hot encoding with the first category omitted as the reference category. Numerical predictors were retained in their continuous form [15, 17–19]. For the Logistic Regression component and AICc-based feature selection, predictor variables were standardized to zero mean and unit variance to place continuous predictors on comparable scales and facilitate numerical optimization. The pre-processing transformations were fitted using the training data and subsequently applied to the test data to minimize the risk of information leakage.

To account for potentially complex relationships among predictors, second-order interaction terms were generated using polynomial transformation with interaction terms only. Constant and zero-variance features were subsequently removed. Infinite values generated during pre-processing were converted to missing values and replaced with zero where necessary after the feature-generation stage.

Finally, the dataset was divided into training and testing subsets using stratified random sampling with an 80:20 split and a fixed random seed of 42. This procedure was adopted to minimize information leakage and provide an unbiased internal assessment of model performance [17, 18]. Stratification preserved the proportion of outcome classes in both subsets, resulting in 3,219 patients in the training set and 805 patients in the test set. The training set contained 2,726 alive patients (84.68%) and 493 deceased patients (15.32%), whereas the test set contained 682 alive patients (84.72%) and 123 deceased patients (15.28%). The test set was completely withheld from model fitting and was used only for final performance evaluation. Importantly, this procedure constitutes internal validation because both the training and testing observations originated from the same SEER database. It should therefore not be interpreted as external validation across an independent population, institution, registry, or healthcare system.

2.4 Internal and external validation

Model validation was performed using an independent hold-out test set derived from the same SEER database. Following stratified random splitting, 80% of the observations (3,219 patients) were used for model development, while the remaining 20% (805 patients) were withheld for final evaluation. The held-out test set was not used for model fitting or AICc-guided feature selection and was reserved for final evaluation of the developed models. This procedure provided an internal assessment of model performance on previously unseen observations from the same source population.

However, external validation using an independent clinical cohort or a separate cancer registry was not performed in the present study. Consequently, the reported test-set performance should be interpreted as evidence of internal generalizability within the SEER population rather than evidence of transportability to other populations or healthcare settings. Differences in demographic characteristics, disease presentation, clinical practice, variable definitions, data collection procedures, and outcome distributions may affect model performance when the proposed framework is applied to external populations.

The absence of external validation represents an important limitation of the present framework. Nevertheless, the use of a pre-defined held-out test set, stratified sampling, and leakage-controlled pre-processing provides an independent internal assessment of predictive performance within the available dataset. Future studies should evaluate the proposed Hybrid (Selected) model using geographically, institutionally, or temporally independent cohorts to determine its generalizability and clinical transportability. Such external validation should assess not only discrimination but also calibration, clinical utility, and model interpretability to determine whether the proposed framework retains its performance under different population and healthcare conditions.

2.5 Predictive modeling framework

Breast cancer outcome prediction was performed using Logistic Regression (LR) and Extreme Gradient Boosting (XGBoost). LR was employed as an interpretable statistical classifier, whereas XGBoost was used to model non-linear relationships and complex feature interactions. Both models were subsequently incorporated as heterogeneous base learners within the proposed stacking framework.

The logistic regression model is defined as

log(pi1-pi)=β0+∑j=1pβjxij,(2)

where p_*i* is the predicted probability of the positive outcome (death) for the ith patient, β₀ is the intercept, β_*j* represents the regression coefficient associated with the jth predictor, and x_*ij* denotes the corresponding predictor value.

XGBoost constructs an additive ensemble of decision trees,

ŷi=∑k=1Kfk(xi),(3)

by minimizing the regularized objective function

L=∑il(yi,ŷi)+∑kΩ(fk),(4)

where l(y_*i*, ŷ_*i*) represents the differentiable loss function and Ω(f_*k*) controls the complexity of the individual trees. The regularization component is expressed as

Ω(f)=γT+12λ∑jwj2,(5)

where T denotes the number of terminal nodes, w_*j* represents the weight associated with the jth terminal node, and γ and λ are regularization parameters. The regularization term controls model complexity and helps reduce overfitting.

2.5.1 XGBoost hyperparameter optimization

Hyperparameter optimization was performed for the XGBoost classifier using RandomizedSearchCV implemented in the Scikit-learn framework. Randomized search was selected instead of exhaustive grid search because it permits efficient exploration of a relatively large hyperparameter space while reducing the computational burden associated with evaluating every possible parameter combination.

The optimization procedure was conducted exclusively on the training dataset. A stratified five-fold cross-validation scheme was employed to preserve the proportion of the two outcome classes within each validation fold. ROC-AUC was used as the optimization criterion because it provides a threshold-independent measure of discriminatory performance and is appropriate for the imbalanced outcome distribution observed in the present study.

The hyperparameters considered during optimization included the number of boosting trees (n_estimators), learning rate (learning_rate), maximum tree depth (max_depth), minimum child weight (min_child_weight), row subsampling ratio (subsample), feature subsampling ratio (colsample_bytree), and the regularization parameters γ, λ, and α. The corresponding candidate values and search space are presented in Table 1.

Table 1. Hyperparameter search space used for XGBoost optimization.

A fixed random seed of 42 was used to ensure reproducibility, and 50 randomly selected hyperparameter combinations were evaluated during the search. The configuration producing the highest mean cross-validated ROC-AUC was selected as the final XGBoost configuration. The independent test set was not used during hyperparameter optimization and was reserved exclusively for the final evaluation of model discrimination, calibration, classification performance, explainability, and clinical utility.

The optimized XGBoost classifier was subsequently incorporated with Logistic Regression as the heterogeneous base learners of the proposed stacking framework. This procedure ensured that the XGBoost component was systematically tuned rather than relying on default hyperparameter settings.

2.6 AICc-based model selection (logistic regression)

To obtain a parsimonious statistical component while retaining potentially informative relationships among predictors, an AICc-guided forward stepwise feature selection procedure was applied exclusively to the training data. The corrected Akaike Information Criterion (AICc) was selected because it provides a finite-sample correction to the Akaike Information Criterion (AIC), thereby providing an appropriate balance between model fit and model complexity when the number of candidate predictors is relatively large relative to the sample size [20–22].

Prior to feature selection, categorical predictors were transformed using one-hot encoding with the first category omitted as the reference category. Second-order interaction terms were subsequently generated using polynomial expansion with interaction_only=True. Consequently, the candidate feature space consisted of the encoded main-effect predictors and their pairwise interaction terms. Squared terms and higher-order interactions were not included. This procedure allowed the selection algorithm to evaluate potentially informative relationships between clinical predictors without introducing higher-order polynomial complexity.

Feature selection was performed exclusively using the training dataset, thereby preventing information from the independent test set from influencing the predictor-selection process. The candidate features were standardized prior to fitting the logistic regression models used during the selection procedure. At each iteration, each remaining candidate feature was added individually to the predictors already selected, and a logistic regression model was fitted. The candidate feature producing the lowest AICc was then retained if its inclusion resulted in an improvement over the current best AICc value. The procedure therefore followed a forward stepwise selection strategy.

The selection process was continued iteratively until no remaining candidate feature produced a further reduction in AICc. A maximum of 50 predictors was permitted; however, the procedure terminated earlier because the inclusion of additional candidate predictors no longer improved the AICc criterion. The final procedure therefore retained 26 predictors from the expanded candidate feature space.

For a candidate logistic regression model MS with predictor subset S⊆P, the corrected Akaike Information Criterion was calculated as

AICc=AIC+2k(k+1)n-k-1,(6)

where k denotes the number of estimated parameters in the fitted model, including the intercept, and n represents the number of observations in the training dataset. The Akaike Information Criterion is defined as

AIC=-2ℓ(β^S)+2k,(7)

where ℓ(β^S) is the maximized log-likelihood of the logistic regression model containing the predictor subset S. Lower AICc values indicate a preferred model because they represent a more favorable trade-off between goodness of fit and model complexity.

For the candidate models evaluated during forward selection, the preferred feature at each iteration was therefore determined according to

j*=argminj∈RAICc(MS∪{j}),(8)

where S represents the predictors already selected and R represents the remaining candidate predictors. A candidate feature was retained only when the resulting AICc was lower than the best AICc obtained at the preceding iteration. The procedure terminated when no remaining candidate predictor produced an improvement.

The resulting 26-predictor set contained both main effects and pairwise interaction terms. Among the retained predictors were Age, T Stage T4, and Regional Node Examined as main effects, together with interaction terms such as Age × Reginol Node Positive, Age × Tumor Size, Tumor Size × Estrogen Status Positive, and Regional Node Examined × T Stage T4. This indicates that the AICc procedure selected not only individual clinical predictors but also potentially informative relationships between clinical characteristics.

The selected predictors were subsequently used to construct the Logistic Regression component of the proposed Hybrid (Selected) model. Thus, the Hybrid (Selected) model differs from the Hybrid (Full) model through the use of the AICc-retained predictor set in the statistical component. Importantly, the independent test set was not used during feature selection, ensuring that the final evaluation was conducted on observations that remained unseen throughout the predictor-selection process.

The complete list and entry order of the 26 predictors retained by the AICc procedure are presented in Table 2. The table also distinguishes between main-effect predictors and pairwise interaction terms, providing transparency regarding the final predictor set used in the proposed Hybrid (Selected) model.

Table 2. Predictors retained by the AICc-guided forward stepwise selection procedure.

2.7 Hybrid stacking ensemble model

A stacking ensemble was developed using LR and XGBoost as Level-0 learners and Logistic Regression as the Level-1 meta-learner.

For each patient x_*i*, the base learners generate probability estimates

fLR(xi)=PLR(yi=1|xi),(9)

fXGB(xi)=PXGB(yi=1|xi).(10)

To minimize information leakage, meta-features were generated using K-fold cross-validation. The training data were partitioned into

D=⋃k=1KDk,(11)

where models were trained on

D-k=D\Dk,(12)

and used to predict the held-out fold. The resulting out-of-fold predictions form

zi=[fLR(xi)fXGB(xi)],(13)

which are combined into the meta-feature matrix

Z=[fLR(x1)fXGB(x1)fLR(x2)fXGB(x2)⋮⋮fLR(xn)fXGB(xn)].(14)

The meta-learner combines these probabilities to produce the final prediction,

P(yi=1|xi)=σ(α0+α1fLR(xi)+α2fXGB(xi)),(15)

where α₀ is the intercept, α₁ and α₂ are stacking coefficients, and

σ(z)=11+e-z(16)

is the logistic sigmoid function.

For a new observation, probabilities from the trained LR and XGBoost models are passed to the meta-learner to obtain the final prediction.

2.8 Statistical comparison of predictive performance

To assess the uncertainty surrounding the predictive performance estimates and to determine whether differences between competing models were statistically meaningful, confidence intervals were calculated for the principal performance metrics using non-parametric bootstrap resampling. The analysis was performed exclusively on the independent test set.

For each model, 2,000 bootstrap samples were generated by sampling patients with replacement from the test set while preserving the paired nature of the predictions. ROC-AUC, PR-AUC, accuracy, precision, recall, F1-score, and Brier score were recalculated for each bootstrap sample. The 2.5th and 97.5th percentiles of the resulting bootstrap distributions were used to construct 95% confidence intervals.

For ROC-AUC, pairwise comparisons between models were additionally assessed using DeLong's test for correlated receiver operating characteristic curves. Because all models were evaluated on the same independent test observations, the ROC curves were considered correlated. A two-sided significance level of α = 0.05 was adopted.

For PR-AUC, statistical uncertainty was assessed using bootstrap confidence intervals because the precision–recall curve is particularly informative in the presence of the observed class imbalance. Differences between models were interpreted in conjunction with their confidence intervals rather than solely on the basis of point estimates.

The statistical comparison was intended to determine whether observed differences in predictive performance were supported by sampling variability, rather than to establish that one model was universally superior to another. Consequently, small numerical differences in performance metrics were not interpreted as evidence of superiority unless supported by the corresponding statistical analysis.

2.9 Explainable AI framework

Model interpretability was achieved using SHapley Additive exPlanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME), which provide global and local explanations of model predictions, respectively.

SHAP decomposes each prediction into feature contributions based on cooperative game theory:

f(x)=ϕ0+∑j=1pϕj,(17)

where ϕ₀ is the baseline prediction and ϕ_*j* denotes the contribution of the jth feature.

LIME explains individual predictions by fitting a local surrogate model:

ξ(x)=argming∈GL(f,g,πx)+Ω(g),(18)

where g is the surrogate model, L(·) measures its fidelity to the original model, π_*x* defines the local neighborhood, and Ω(g) controls model complexity.

Together, SHAP and LIME provide complementary global and instance-level explanations to support clinical interpretation.

2.10 Model evaluation

Model performance was assessed in terms of discrimination, calibration, and clinical utility.

2.11 Software and computational environment

All statistical analyses, model development, evaluation, and visualization were performed using Python 3.12.13. The computational environment used NumPy 2.4.6, Pandas 2.3.3, Scikit-learn 1.6.1, XGBoost 3.2.0, SHAP 0.51.0, Statsmodels 0.14.6, Matplotlib 3.10.0, and Seaborn 0.13.2. LIME was also used for local model interpretability; however, the installed LIME package did not expose its version programmatically. These software versions are reported to facilitate reproducibility of the proposed modeling and evaluation framework.

2.11.1 Discrimination

Discrimination was evaluated using the Area Under the Receiver Operating Characteristic Curve (ROC-AUC) and the Area Under the Precision–Recall Curve (PR-AUC). ROC-AUC is defined as

AUCROC=P(ŷ1>ŷ0),(19)

where ŷ₁ and ŷ₀ are the predicted probabilities for the positive and negative classes, respectively.

Because the dataset is imbalanced, PR-AUC was also computed:

AUCPR=∫01Precision(r)dRecall(r).(20)

2.11.2 Calibration

Calibration was assessed using the Brier Score (BS) and Expected Calibration Error (ECE):

BS=1n∑i=1n(yi-pi)2,(21)

ECE=∑b=1B|Bb|n|acc(Bb)-conf(Bb)|,(22)

where B_*b* is the bth confidence bin, acc(B_*b*) is the observed accuracy, and conf(B_*b*) is the corresponding predicted confidence.

2.11.3 Clinical utility

Clinical utility was evaluated using Decision Curve Analysis (DCA), which quantifies the net clinical benefit of using a prediction model across a range of clinically relevant threshold probabilities [11, 12]. Net Benefit (NB) was calculated as

NB=TPn-FPn(pt1-pt),(23)

where TP and FP denote the numbers of true-positive and false-positive classifications at a given threshold probability, respectively, n is the total number of patients, and p_*t* is the threshold probability at which the expected benefit of identifying a positive case is considered to outweigh the potential harm associated with a false-positive classification.

The Net Benefit of the proposed Hybrid (Selected) model was compared with two reference strategies: treating all patients as positive (Treat All) and treating none of the patients as positive (Treat None). DCA was evaluated across threshold probabilities ranging from 0.01 to 0.49 using the independent test set. A higher Net Benefit indicates greater clinical utility at the corresponding threshold probability.

At selected threshold probabilities of 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, and 0.45, the Net Benefit of the Hybrid (Selected) model was compared directly with the corresponding Treat All and Treat None strategies. This analysis was performed to determine whether the proposed model provides additional clinical value beyond default treatment strategies across clinically relevant decision thresholds.

2.12 Clinical trust evaluation

To provide an overall assessment of predictive reliability, this study proposes a composite Trust Index (TI):

TI=14(AUC+PR-AUC+F1+(1-ECE)).(24)

The index combines four complementary dimensions of predictive performance: discrimination through ROC-AUC, minority-class discrimination through PR-AUC, threshold-dependent classification performance through the F1-score, and calibration through the Expected Calibration Error (ECE). The transformation (1−ECE) ensures that all components have the same direction of interpretation, such that higher values indicate better performance.

Equal weighting was adopted as an a priori neutral weighting scheme, with each component contributing 25% to the overall index. This choice was made because there was no external clinical or empirical basis available in this study for assigning greater importance to one performance dimension over another. Equal weighting therefore provides a transparent and reproducible baseline rather than implying that the four dimensions are intrinsically of equal clinical importance.

The Trust Index is intended as a comparative composite measure of model reliability rather than as a validated clinical utility score. Its purpose is to facilitate comparison of models that may exhibit different strengths across discrimination, minority-class performance, classification performance, and calibration.

To examine whether model conclusions were sensitive to the equal-weight assumption, a sensitivity analysis was performed using alternative weighting schemes. The resulting Trust Index values and model rankings were compared with those obtained under the primary equal-weight formulation.

3 Results

3.1 Study population and class distribution

The final analytical cohort consisted of 4,024 patients from the SEER breast cancer dataset. Of these, 3,408 (84.69%) patients were alive and 616 (15.31%) were deceased, corresponding to an outcome imbalance ratio of approximately 5.53:1. Following stratified partitioning, 3,219 patients (80.0%) were assigned to the training set and 805 patients (20.0%) to the independent test set. The training set contained 2,726 alive patients (84.68%) and 493 deceased patients (15.32%), whereas the test set contained 682 alive patients (84.72%) and 123 deceased patients (15.28%).

To mitigate the effect of class imbalance during model training, class weighting was applied to the minority deceased class. The class imbalance ratio was approximately 5.53:1, and a weight of 5.529 was assigned to the minority class. This approach increased the contribution of deceased patients during model optimization without altering the original class distribution or generating synthetic observations.

Table 3 summarizes the demographic and clinical characteristics of the study population. The mean age was 53.97 years (SD = 8.96), with patients ranging from 30 to 69 years. The largest age group was 50–59 years (34.54%), followed by 60–69 years (31.81%) and 40–49 years (27.93%). Most patients were recorded as White (84.82%) and married or in a common-law relationship (65.68%).

Table 3. Baseline demographic and clinical characteristics of the study population (N = 4, 024).

^aOther race includes American Indian/AK Native and Asian/Pacific Islander categories as recorded in the SEER dataset.

With respect to tumor characteristics, T2 was the most frequent tumor stage (44.38%), followed by T1 (39.84%). N1 was the pre-dominant nodal stage (67.89%). Grade II tumors constituted the largest proportion of the cohort (58.42%), while 97.71% of patients were classified as having regional disease. Estrogen receptor status was positive in 93.32% of patients, whereas progesterone receptor status was positive in 82.65%.

The mean tumor size was 30.47 mm (SD = 21.12), while the mean numbers of regional lymph nodes examined and positive regional lymph nodes were 14.36 (SD = 8.10) and 4.16 (SD = 5.11), respectively. The outcome distribution showed that 84.69% of patients were alive and 15.31% were recorded as dead. No missing values were detected in the final analytical dataset, and consequently no patients were excluded because of missing observations.

3.2 Comparative model performance

Table 4 summarizes the predictive performance of the evaluated models on the independent test set. Overall, the hybrid stacking approaches outperformed the standalone Logistic Regression and XGBoost models across most evaluation metrics.

Table 4. Performance comparison of candidate models on the independent test set.

The Hybrid (Selected) model achieved the best balance between discrimination and classification performance, recording the highest PR-AUC, F1-score, and recall. Although the Hybrid (Full) model produced a marginally higher ROC-AUC and specificity, its improvement over the selected model was minimal. Applying AICc-based feature selection improved the performance of logistic regression relative to the full model, indicating that a parsimonious predictor set enhanced predictive efficiency while reducing unnecessary model complexity.

3.3 Uncertainty estimation using Bootstrap confidence intervals

Table 5 summarizes the predictive performance of the evaluated models on the independent test set. To quantify the uncertainty associated with the observed performance estimates, non-parametric bootstrap resampling was used to obtain 95% confidence intervals for the principal evaluation metrics. This approach provides an empirical assessment of sampling variability and is particularly informative in the presence of class imbalance.

Table 5. Performance of the evaluated models on the independent test set with 95% bootstrap confidence intervals.

Values are point estimates followed by 95% bootstrap confidence intervals in brackets. ROC-AUC = receiver operating characteristic area under the curve; PR-AUC = precision–recall area under the curve; ECE = expected calibration error. Lower Brier scores and ECE values indicate better probabilistic calibration.

Overall, the hybrid stacking approaches demonstrated competitive or superior performance across the principal evaluation measures. The Hybrid (Selected) model achieved the highest F1-score (0.4078; 95% CI: 0.3427–0.4706), recall (0.5935; 95% CI: 0.5082–0.6807), and PR-AUC (0.3729; 95% CI: 0.2947–0.4570). It also produced a substantially lower Brier score (0.1168; 95% CI: 0.1010–0.1334), indicating improved probabilistic calibration. The Hybrid (Full) model achieved the highest ROC-AUC (0.7193; 95% CI: 0.6715–0.7671) and specificity (0.8314; 95% CI: 0.8026–0.8582), although the difference in ROC-AUC between the two hybrid models was small.

The selected Logistic Regression model also consistently outperformed its full-feature counterpart, with higher ROC-AUC, PR-AUC, F1-score, recall, and lower Brier score. This suggests that AICc-guided feature selection improved the predictive efficiency of the logistic model while reducing the number of predictors retained.

To quantify uncertainty in model performance, non-parametric bootstrap resampling was performed on the independent test set to obtain 95% confidence intervals for the principal evaluation metrics. This approach was particularly appropriate given the class imbalance in the outcome, as it provides empirical estimates of sampling variability without relying solely on large-sample parametric assumptions. The results show that the Hybrid (Selected) model achieved the highest F1-score (0.4078, 95% CI: 0.3427–0.4706), recall (0.5935, 95% CI: 0.5082–0.6807), and PR-AUC (0.3729, 95% CI: 0.2947–0.4570) among the evaluated models. It also produced a substantially lower Brier score (0.1168, 95% CI: 0.1010–0.1334), indicating improved probabilistic calibration. The Hybrid (Full) model had a marginally higher ROC-AUC (0.7193, 95% CI: 0.6715–0.7671) than the Hybrid (Selected) model (0.7176, 95% CI: 0.6690–0.7686), although their confidence intervals substantially overlap.

3.4 Receiver operating characteristic analysis

Figure 2 presents the ROC curves for all candidate models. Both hybrid models achieved the largest areas under the curve. The Hybrid (Full) model achieved the (0.7193), followed closely by the Hybrid (Selected) model (0.7176). However, the relatively small difference between the two hybrid approaches should be interpreted alongside their bootstrap confidence intervals, which substantially overlap. Therefore, ROC-AUC alone does not provide sufficient evidence to establish a meaningful superiority of one hybrid model over the other. Given the class imbalance in the outcome, the ROC analysis was considered together with PR-AUC, recall, F1-score, specificity, and calibration measures.

3.5 Precision–recall performance

Figure 3 illustrates the precision–recall performance of the evaluated models under class imbalance. PR-AUC provides informative measure in this setting because it focuses on the trade-off between precision and recall for the positive class. The Hybrid (Selected) model achieved the highest PR-AUC (0.3729; 95% CI: 0.2947–0.4570) indicating superior identification of minority-class observations, followed by Logistic (Selected) (0.3692; 95% CI: 0.2922–0.4555) and XGBoost (0.3626; 95% CI: 0.2893–0.4475). The Hybrid (Full) model recorded a PR-AUC of 0.3521 (95% CI: 0.2819–0.4414), while Logistic (Full) produced the lowest value (0.3043; 95% CI: 0.2368–0.3840), further demonstrating the benefit of AICc-guided selection.

3.6 Calibration performance

Calibration curves are presented in Figure 4. Both hybrid models exhibited substantially better agreement between predicted and observed probabilities than the standalone models. They achieved the lowest Brier scores and Expected Calibration Errors, indicating more reliable probability estimates. In contrast, Logistic Regression and XGBoost showed noticeably larger calibration errors, suggesting inferior probabilistic reliability.

3.7 Clinical trust evaluation

Table 6 summarizes the Trust Index for all evaluated models under the primary equal-weight formulation. The Hybrid (Selected) model achieved the highest Trust Index, followed closely by the Hybrid (Full) model. This result indicates that the Hybrid (Selected) model provides the most favorable overall balance across discrimination, minority-class performance, classification performance, and calibration under the proposed composite evaluation framework. The Trust Index should, however, be interpreted as a comparative composite measure rather than as an independently validated measure of clinical utility.

Table 6. Clinical trust evaluation of candidate models.

3.8 Weight stability analysis

Table 7 presents the stability of the stacking coefficients across five-fold cross-validation. In the Hybrid (Selected) model, Logistic Regression contributed more strongly than XGBoost, whereas the opposite pattern was observed in the Hybrid (Full) model. Stability scores remained consistently high across all folds, indicating reliable coefficient estimation and stable ensemble behavior.

Table 7. Weight stability analysis of the hybrid stacking models across five-fold cross-validation.

3.9 Statistical significance of ensemble components

As shown in Table 8, both Logistic Regression and XGBoost contributed significantly to the stacked ensemble (p < 0.001), confirming that each base learner provided complementary predictive information to the final hybrid model.

Table 8. Statistical significance of base learners in the stacked ensemble.

3.10 Global model interpretation using SHAP

Figure 5 presents the SHAP summary plot for the Hybrid (Selected) model. Tumor size, lymph node involvement, age, and hormone receptor status were identified as the most influential predictors. The consistency of these variables with established clinical knowledge supports the biological plausibility of the proposed model and enhances its interpretability.

3.11 Model disagreement entropy

Table 9 summarizes the disagreement entropy across the ensemble components, while Figure 6 illustrates its distribution. The moderate entropy values indicate that the base learners captured complementary predictive information without producing excessive disagreement, supporting the diversity required for effective ensemble learning.

Table 9. Model disagreement entropy across ensemble components.

The distribution of disagreement entropy is shown in Figure 6, while summary statistics are reported in Table 9.

3.12 Robustness analysis under feature perturbation

The robustness of the Hybrid (Selected) model was evaluated by progressively introducing feature noise (Table 10; Figure 7). Model performance remained largely unchanged under minimal perturbation but declined gradually as noise increased, with corresponding reductions in ROC-AUC and PR-AUC and increases in the Brier score. Despite this deterioration, the degradation was smooth rather than abrupt, indicating that the proposed model maintains reasonable predictive stability under moderate feature perturbations.

Table 10. Robustness assessment of the hybrid (selected) model under increasing levels of injected feature noise.

3.13 Local model explanation using LIME

LIME was used to explain individual predictions generated by the Hybrid (Selected) model. Figure 8 illustrates a representative patient-level explanation, showing the features that increased or decreased the predicted outcome probability. The locally important predictors were broadly consistent with the globally influential variables identified by SHAP, demonstrating agreement between global and local explanations and supporting the interpretability of the proposed framework.

The local explanation for a representative patient is illustrated in Figure 8.

3.14 Decision curve analysis

Table 11 and Figure 9 presents the Decision Curve Analysis for the Hybrid (Selected) model using the independent test set of 805 patients, comprising 123 positive cases and 682 negative cases. The observed prevalence of the positive outcome in the test set was 15.28%.

Table 11. Net benefit of the hybrid (selected) model compared with treat all and treat none strategies at selected threshold probabilities.

The Hybrid (Selected) model demonstrated positive Net Benefit across the entire evaluated threshold range from 0.01 to 0.49. At a threshold probability of 0.05, the model achieved a Net Benefit of 0.1102, compared with 0.1093 for the Treat All strategy and 0 for the Treat None strategy. At a threshold of 0.10, the corresponding Net Benefits were 0.0647, 0.0564, and 0, respectively. At a threshold probability of 0.15, the Hybrid (Selected) model achieved a Net Benefit of 0.0523, compared with 0.0026 for Treat All and 0 for Treat None.

The advantage of the Hybrid (Selected) model became more pronounced at higher threshold probabilities. At thresholds of 0.20, 0.25, 0.30, 0.35, 0.40, and 0.45, the model achieved Net Benefits of 0.0366, 0.0278, 0.0155, 0.0101, 0.0061, and 0.0024, respectively. In contrast, the Net Benefit of the Treat All strategy declined to −0.0578, −0.1260, −0.2119, −0.3022, −0.4070, and −0.5438 across the same thresholds, while Treat None maintained a Net Benefit of zero.

The maximum Net Benefit of the Hybrid (Selected) model was 0.1442 at a threshold probability of 0.01, which was equal to the Net Benefit of the Treat All strategy at this threshold. Thus, the principal clinical advantage of the Hybrid (Selected) model was observed above very low threshold probabilities, where it provided greater Net Benefit than both reference strategies. These findings indicate that the proposed model may provide useful clinical decision support when the threshold probability for intervention is sufficiently above the observed outcome prevalence.

4 Discussion

4.1 Novelty and methodological contribution

The principal contribution of this study is the integration of three complementary components within a single predictive framework: AICc-guided statistical model selection, heterogeneous stacking of Logistic Regression and XGBoost, and calibration-aware clinical evaluation. Unlike conventional pipelines in which feature selection is performed independently of model development, AICc was incorporated as a structural component of the modeling strategy. This enabled the construction of a parsimonious ensemble that balanced model complexity with predictive performance while maintaining interpretability.

The Hybrid (Selected) model represents a statistically guided ensemble in which predictor inclusion is determined by information-theoretic optimization rather than arbitrary feature reduction. In contrast, the Hybrid (Full) model serves as an unconstrained benchmark using all available predictors. Comparing these two configurations demonstrates the value of integrating statistical learning principles into modern ensemble methods for clinical prediction.

4.2 Principal findings

Both hybrid models consistently outperformed the standalone Logistic Regression and XGBoost models across most evaluation metrics. Although the Hybrid (Full) model achieved marginally higher discrimination, the Hybrid (Selected) model provided the most balanced performance by combining strong discrimination, improved classification balance, reliable calibration, and the highest Trust Index.

These findings indicate that superior clinical prediction should not be judged solely by discrimination metrics such as ROC-AUC but by a balanced assessment of discrimination, calibration, and classification performance.

4.3 Interpretation of AICc-guided model selection

The contribution of AICc extends beyond variable reduction. By selecting an information-theoretically optimal predictor subset, AICc reduced model complexity while preserving clinically relevant predictive information. This was reflected in the improved performance of the selected Logistic Regression model and its increased contribution within the hybrid ensemble.

The weight stability analysis further suggests that the selected predictor set provided consistent and stable estimates across cross-validation folds. These findings support the use of statistically constrained modeling as a complementary strategy for improving transparency, robustness, and generalization in ensemble learning.

4.4 Comparison with existing studies

Previous studies have demonstrated the potential of machine learning and statistical approaches for breast cancer prediction and prognosis using large-scale clinical and population-based datasets. However, direct quantitative comparison of predictive performance across studies should be interpreted cautiously because differences in study population, outcome definition, prediction horizon, predictor availability, class distribution, pre-processing procedures, and validation strategies can substantially affect the reported performance.

Several studies using SEER and other large-scale breast cancer datasets have investigated machine learning approaches for prognostic prediction. For example, recent SEER-based studies have employed algorithms including Random Forest, XGBoost, Support Vector Machines, and survival-oriented ensemble methods, with reported discrimination measures generally indicating that machine learning can provide competitive prognostic performance

However, a substantial proportion of these studies formulate the outcome as a time-to-event problem rather than as a binary survival-status classification problem.

This distinction is important when interpreting the results of the present study. The proposed framework formulates breast cancer outcome prediction as a binary classification task, where the outcome is defined as Alive or Dead. Accordingly, ROC-AUC, PR-AUC, F1-score, Brier score, expected calibration error (ECE), and decision-curve net benefit were used to evaluate the performance of the models. In contrast, survival-analysis studies generally evaluate models using measures such as the concordance index (C-index), integrated Brier score, or time-dependent AUC. Although these measures assess related aspects of predictive performance, they should not be interpreted as directly equivalent to the binary-classification ROC-AUC reported in the present study.

For example, a recent SEER-based study using the same 4,024-patient population investigated Cox proportional hazards, Random Survival Forest, DeepSurv, Random Forest, XGBoost, and Support Vector Machine approaches for survival prediction. The study reported survival-specific performance measures, including C-index and integrated Brier score. While these findings provide useful evidence that machine learning approaches can effectively model breast cancer prognosis, the survival modeling formulation differs from the binary outcome prediction investigated in the present study. Therefore, the numerical performance measures from that study are not used as direct benchmarks against the ROC-AUC and PR-AUC values of the proposed framework.

Within the present study, Logistic Regression achieved a ROC-AUC of 0.7055 and a PR-AUC of 0.3692, while XGBoost achieved ROC-AUC and PR-AUC values of 0.7175 and 0.3507, respectively. The Hybrid (Full) model achieved a ROC-AUC of 0.7193 and a PR-AUC of 0.3521. Most importantly, the proposed Hybrid (Selected) model achieved a ROC-AUC of 0.7176 and the highest PR-AUC of 0.3729 among the evaluated models.

Although the ROC-AUC of the Hybrid (Selected) model was marginally lower than that of the Hybrid (Full) model (0.7176 vs. 0.7193), the selected model achieved a higher PR-AUC (0.3729 vs. 0.3521). This finding is particularly relevant because the outcome distribution was imbalanced, with the minority outcome representing only 15.31% of the study population. Thus, the PR-AUC provides important complementary information regarding the model's ability to identify the minority outcome.

The proposed Hybrid (Selected) model also demonstrated improvements beyond discrimination alone. Compared with the standalone Logistic Regression model, the Hybrid (Selected) model increased ROC-AUC from 0.6605 to 0.7176 and PR-AUC from 0.3043 to 0.3729. In addition, the Brier score decreased from 0.2121 to 0.1168, while ECE decreased from 0.2607 to 0.0294, indicating substantially improved probabilistic calibration.

Furthermore, decision-curve analysis demonstrated that the Hybrid (Selected) model provided positive net benefit across the evaluated threshold range. The model therefore demonstrated utility not only in terms of discrimination but also in terms of probability calibration and potential clinical decision support.

Taken together, these findings suggest that the principal contribution of the proposed framework is not a claim of universally superior predictive accuracy compared with every previously published breast cancer model. Rather, its contribution lies in integrating AICc-guided feature selection, Logistic Regression, XGBoost, heterogeneous stacking, probability calibration, explainable artificial intelligence, and decision-curve analysis within a unified binary breast cancer outcome prediction framework. This multidimensional evaluation provides a more comprehensive assessment of model reliability and potential clinical utility than discrimination metrics alone.

4.5 Explainability and clinical interpretability

SHAP identified tumor size, lymph node involvement, age, and hormone receptor status as the principal contributors to model predictions, with several clinically meaningful interaction effects captured by the XGBoost component. These findings suggest that the hybrid framework successfully models complex relationships that cannot be represented adequately by conventional linear approaches.

LIME further demonstrated that individual patient predictions were driven by similar clinical variables identified through SHAP, providing consistency between global and local explanations. Together, these complementary XAI techniques improve model transparency while supporting clinician confidence in prediction interpretation. Nevertheless, these explanations describe predictive associations and should not be interpreted as evidence of causal relationships.

4.6 Clinical trust and utility

The Hybrid (Selected) model achieved the highest Trust Index, demonstrating the best overall balance between discrimination, calibration, and classification performance. Decision Curve Analysis further showed that the model consistently provided greater net benefit than both “Treat All” and “Treat None” strategies across clinically relevant decision thresholds. These findings support the potential application of the proposed framework for risk stratification and clinical decision support.

4.7 Robustness and stability

The robustness analysis demonstrated that predictive performance remained stable under small feature perturbations and declined gradually as noise increased. This behavior indicates good generalization capability and suggests that the proposed framework is resilient to moderate variability commonly encountered in clinical datasets.

4.8 Implications for clinical deployment

By integrating statistical model selection, ensemble learning, explainable artificial intelligence, and calibration-aware evaluation, the proposed framework generates accurate and clinically interpretable probability estimates. The Hybrid (Selected) model offers a practical balance between predictive performance, transparency, and computational efficiency, making it a suitable candidate for future clinical decision-support applications.

4.9 Limitations and future work

Several limitations should be acknowledged. First, the analysis was based on a single retrospective SEER cohort and therefore lacks external validation using an independent population. Although the dataset provides a relatively large and heterogeneous population for model development and the independent test set was strictly separated from the training data, internal testing cannot establish transportability to patients from different healthcare systems, geographical regions, registry periods, or clinical practice settings. Differences in patient characteristics, disease prevalence, diagnostic procedures, treatment patterns, and outcome definitions may affect model performance when applied to an external cohort.

Consequently, the reported performance estimates should be interpreted as evidence of internal predictive validity rather than definitive evidence of clinical generalizability. Future studies should evaluate the Hybrid (Selected) model using an independent external cohort, preferably from a different registry, institution, geographical population, or temporal period. External validation should assess discrimination, calibration, clinical utility, and model explanation stability using the same pre-specified evaluation framework. Prospective validation would subsequently be required before considering clinical deployment.

Third, the present analysis treats the outcome as a binary survival-status classification problem rather than a time-to-event outcome. Future work should therefore investigate extensions to survival modeling and time-dependent prediction while retaining the proposed principles of statistical feature selection, ensemble learning, calibration, explainability, and clinical utility assessment.

5 Conclusion

This study developed an AICc-guided hybrid stacking framework that integrates Logistic Regression and XGBoost for breast cancer outcome prediction using the SEER dataset. The proposed Hybrid (Selected) model achieved the best overall balance between discrimination, calibration, interpretability, and clinical utility, resulting in the highest Trust Index among the evaluated models.

Embedding AICc within the ensemble framework demonstrated that statistically guided predictor selection can improve model parsimony and stability while maintaining competitive predictive performance. Explainability analysis using SHAP and LIME further showed that the model's predictions were primarily influenced by clinically relevant factors, particularly tumor size, lymph node involvement, age, and hormone receptor status, thereby providing transparent global and patient-specific explanations that support clinical interpretation.

By integrating statistical model selection, heterogeneous ensemble learning, explainable artificial intelligence, calibration assessment, and Decision Curve Analysis within a single framework, the proposed approach provides an accurate, reliable, and clinically interpretable tool for breast cancer risk stratification. Future studies should focus on external validation, prospective clinical implementation, and temporal modeling to further establish its generalizability and clinical applicability.

Generative AI statement

The author(s) declared that Generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Lixin Shen, Syracuse University, United States

Reviewed by: Amos Kipkorir Langat, University of South Africa, Pretoria, South Africa

Giada Anastasi, National Research Council (CNR), Italy