Tri-fusion deep learning model for early breast cancer screening using multimodal biomedical data

1 Introduction

Breast cancer is the most frequently diagnosed cancer in females from worldwide and remains a major cause of cancer-related morbidity and mortality even after a decade of screening activities; while the routine screening has a significant advantage in terms of reducing mortality, there are significant limitations of early detection. Mammography is a fundamental method of early detection of breast cancer, scheduled for its own proven mortality gains, despite the fact that its diagnostic accuracy changes with breast composition and persistent demographic elements in standard society screening. For instance, mammographic sensitivity is significantly reduced in females with dense breast tissue, where the fibroglandular arrangements disassemble a lesion and can reduce sensitivity to approximately 60–70% compare alongside beliefs > 85% in fatty breasts, leading to missed tumors and increased false-negative examinations (Lee et al., 2024). Dense breast tissue is considered to be an independent risk factor for breast cancer and a significant impediment to precise radiologic interpretation, resulting in higher recall rates and false positives when mammograms are interpreted separately (Mansour et al., 2022). Moreover, modest improvements in sensitivity are often associated with trade-offs in specificity, recall rates, alternatively, detection of early-stage tumours, even when combined with additional AI-assisted mammographic interpretation. These highlight the limitations of the imaging-only screening paradigm, focusing in particular on the systematic lack of integration of disease signatures that may remain present in the non-imaging mode (Lee et al., 2024; Chen et al., 2025; Yousafzai et al., 2026).

In addition to the dataset bias and variability in image protocols, AI models that present differential sensitivity and constructive predictive standards depend on the breast density subgroup (Zhang et al., 2025). Although, the large-scale efforts are being made to deploy machine intelligence as a subsequent reader, possibly even a primary reader alongside human radiologists, actual evidence suggests that automated reasoning entirely fails to systematically surpass the highest quality reader in all performance metrics, and integration plans change extensively in efficiency (Chang et al., 2025). Besides imagination, hematologic and genomic biomarkers reveal the implicit in the pathophysiology of malignancies. Peripheral blood components, such as WBC morphological form, reflect systemic immune responses and elusive changes associated with tumor. So far, primarily screening paradigms integrate the data due to morphological patterns, reflect systemic immune responses, and the obstacles of blending disparate statistical types (Mansour et al., 2022). Similarly, the breast cancer-related gene mutant signature, including germline and body heterogeneity examined by next-generation sequencing, has shown predictive and threat stratification value but is rarely integrated into the general society screen due to cost, complexity, and integration obstacles. A multimodal statistics fusion approach, using a Deep Learning (DL) architecture to jointly process imagination, hematologic, and genomic facts, maintains a commitment to advance early detection by capturing complementary signals that a person modality may require (Zong et al., 2024). The recent multimodal Artificial Intelligence (AI) models combining double imaging modality identical to 2D and 3D mammography demonstrate an improvement in the Area Under the Receiver Operating Characteristic (AUROC) and a reduction in the workload; these queries still depend mainly on image data (Zhang et al., 2025; Aggarwal et al., 2025).

To address these issues, a Tri-Fusion-based DL model is proposed to integrate the WBC morphologic features, mammogram image embeddings, and gene mutant signature within a coherent DL architecture planned for the early breast cancer screening. This tri-modal architecture includes give encoders for the given information type, followed by an attribute fusion using the attention mechanism to capture cross-domain communications. To optimize this model, combined structural radiologic cues with systemic changes in blood profile and a genomic mutant form, which is likely to be associated with oncogenesis. Furthermore, benchmarking against present unimodal and multimodal baselines against publicly available mammography and genomic datasets demonstrate the power of similar to integrative models to advance ahead of time detection performance and minimize the effect of modality-specific blind spots. This integrative method provides a promising result for next-generation screening strategies, which not only benefit from the advantages of individual modality to accelerate cancer detection.

The major contributions of the proposed work are summarized as follows:

• Development of a weakly supervised Tri-Fusion multimodal framework integrating WBC, mammography, and genomic modalities.
• Introduction of semantic embedding alignment and modality-invariant feature fusion for heterogeneous biomedical representation learning.
• Incorporation of modality-dropout regularization and robustness-aware optimization for incomplete multimodal scenarios.
• Comprehensive statistical validation, ablation analysis, and interpretability evaluation using Grad-CAM and genomic feature attribution methods.

2 Literature review

A comprehensive assessment shows that single-modality mammography-based CNNs suffer from sensitivity degeneration in dense breast tissue and show false-negative rates exceeding 18%, requiring the integration of complementary life-related facts (Zhang et al., 2025). Deep CNN-based mammogram encoders extract hierarchical textures, margin abnormality, and spiculated boundary features, while omics-based completely connected encoders study the nonlinear mutation-risk function in BRCA1, BRCA2, TP53, PIK3CA, and ESR1 gene panel (Luo et al., 2024). Feature-level fusion has been established, so that it can exist at a high decision-level fusion, given that it encourages joint optimization of modality-specific embeddings under a shared latent space, enabling synergistic training of phenotype–genotype association (Wen et al., 2025). Multimodal ML provides a theoretical foundation for integrating heterogeneous biomedical data to improve predictive performance and robustness. AUC of 0.962 for a multimodal mammography–genomics network surpassed unimodal mammography (0.884) and a single-purpose model (0.901), thus confirming the added value of integrating living statistics (Haghighat et al., 2025). The hematology-imaging studies have shown that cancer-associated neutrophil–lymphocyte ratio variations and nuclear deformity in WBCs contribute to a timely malignancy index, despite the fact that their diagnostic promise remains underutilized in existing multimodal ML techniques (Margret and Rajakumar, 2025). Moreover, transformer-based multimodal models such as HEALNet and BioFusionNet use cross-attention mechanisms to offer pressure modality services dynamically (Margret and Rajakumar, 2025). A major limitation of the study is that multiple multimodal breast cancer models need complete modality capability, while in clinical practice, genomic sequencing, alternatively, high-resolution imaging is often missing, eminent for severe performance degeneration (Li et al., 2026). Although modality-dropout and variational autoencoder-based imputation schemes have been proposed. Also, uncertainty propagation and unstable convergence in deep fusion pipeline (Siam et al., 2025). Furthermore, the multimodal datasets remain highly fragmented, together with the TCGA-BRCA supply genomics still limiting imagination, and the CBIS-DDSM provision mammogram lacks a molecular label, weakly aligning cohorts’ surveys to count on atop a weakly integrated cohort that limits generalizability (Liu et al., 2025). The other significant drawback is exemplary interpretability; as Grad-CAM visual images shows the mammographic regions of interest, they do not provide naturally significant enlightenments for genomic and hematologic contributions and restrict clinical trust and control implementation (Antony et al., 2025). Moreover, a number of high accuracy reports (> 95%) are obtained inferior to inner cross-validation, but the external validation cohort collapses by 5–10% due to scanner bias, community heterogeneity, and area adjustment (Sun et al., 2025). Therefore, there is a growing consensus that a next-generation screening model must integrate imagination, blood cell morphology, and genomic mutant signature in an integrated feature-level fusion model to achieve naturally stable, robust, and generalizable, timely breast cancer detection (Tran et al., 2025).

Besides standard multimodal fusion, recent analysis shows the importance of immune–genomic–imaging compatibility in advanced cancer phenotyping, as tumor progression is attached to systemic immune modifications detectable in peripheral blood (Nazir et al., 2025). CNN-based leukocyte morphometry model demonstrated that malignancies triggered measurable changes in nuclear eccentricity, chromatin density, and cytoplasmic coarseness in neutrophils and lymphocytes, which matched above 92% accuracy in cancer–noncancer intolerance (Abed, 2025). However, identical hematologic DL models are still isolated from image grapevine, preventing joint training of tissue-level malignancies and systemic immune responses (Chaturvedi et al., 2025). Tumor progression is strongly associated with systemic immune and genomic alterations, motivating integrated immune–genomic modeling strategies (Xia et al., 2025). Deep fusion models integrate biomarkers and genomic instability markers show superior early-stage detection performance compared to image entirely, yet, the architecture primarily depends on shallow concatenation rather than hierarchical attribute arrangement, restricting their representational capacity (Nakach et al., 2024). The heterogeneous biomedical data has provided a contrastive and cross-modal transformer that matches multimodal embeddings in a shared latent manifold, allowing elusive molecular–imaging correspondence to remain learned even while modality is weakly correlated (Che et al., 2025). Deep multimodal frameworks have demonstrated that radiological phenotypes can be aligned with molecular signatures through joint representation learning (Steyaert et al., 2023). The mammographic density form, WBC immunophenotypes, and mutational signatures of different living methods which coincide with the risk of malignancies, are particularly important in breast cancer screening (Fan et al., 2024). Feature-level fusion is widely regarded as more effective than decision-level fusion in capturing cross-modal interactions. However, to avoid overfitting, which is not feasible with the existing breast cancer multimodal data repository, a transformer-based architecture requires very large training datasets to avoid overfitting, which is not feasible with the existing breast cancer multimodal data repository (Khan et al., 2025). Genomic mutant name extra suffers from sequence variation and batch effects (Ghosh et al., 2025).

The advanced breast cancer detection using diverse AI-driven strategies, focusing on accuracy, interpretability, and computational efficiency. Singh and Patnaik (2026) proposed MammXAI, an explainable AI-integrated adaptive multi-model DL framework that fuses multi-modality imaging data for improved diagnostic performance, but it suffers from high computational complexity due to ensemble-based feature extraction. Jaafari et al. (2026) developed a lightweight DL model enhanced with spatial attention mechanisms to focus on salient tumor regions, achieving efficient prediction with reduced computational cost, though it shows limited robustness in capturing highly subtle inter-class variations in heterogeneous datasets. Okundalaye et al. (2026) introduced an optimized ML approach incorporating tumor–immune system interaction modeling for early breast cancer prediction, improving interpretability but relying heavily on handcrafted features and simplified biological assumptions, which may affect scalability to large imaging datasets. Basith et al. (2026) proposed an adaptive deep Q-learning-based GAN framework to address the data imbalance and enhance feature synthesis, yet it faces challenges related to training instability and high computational demand of GAN-based reinforcement learning systems. Tewari et al. (2026) reviewed AI techniques across multiple imaging modalities and highlighted progress in DL and transformer-based models for clinical integration; however, the study lacks a unified quantitative benchmarking framework, limiting direct performance comparison across methods.

The various limitations have inspired the development of a Tri-modal fusion architecture integrating WBC morphometric signature, mammographic deep embeddings, and breast cancer-specific genomic mutant profiles in an integrated, feature-level learning basis. Such models merely improve classification accuracy but also provide a link among tumor phenotype, immune response, and molecular dysregulation, which is essential for early detection, risk stratification, and personalized screening strategies. The proposed Tri-Fusion DL model directly addresses these challenges by exploiting CNN-based morphologic encoders, pretrained mammographic representation acquiring knowledge, and mutation-signature MLPs in a dense fusion head optimized for cross-domain association.

3 Dataset

The Tri-Fusion DL framework is trained and evaluated using three complementary biomedical datasets representing immune, imaging, and molecular perspectives of breast cancer.

Table 1 describes the three datasets used in the Tri-Fusion Model: 12,500 WBC images for immune morphology inspection, 3,102 CBIS-DDSM mammograms for radiological characteristic extraction, and 1,200 TCGA-BRCA Genomic Profiles to study Mutational Liability Form. These datasets provide immune, imaging, and genetic perspectives for comprehensive cancer screening.

Table 1. Datasets used in the Tri-Fusion DL framework.

Table 2 shows the most important breast cancer genes, including BRCA1, BRCA2, TP53, PIK3CA, ESR1, and HER2, which capture DNA repair defects, oncogenic signals, and hormone receptor positions.

Table 2. Key genomic features used from TCGA-BRCA.

Table 3 specifies a 70% – 15% – 15% split between the training, validation, and test sets of the three datasets. To learn multimodal representation, a total of 8,750 WBC images, 2,171 mammograms, and 840 Genomic Profiles were used to train the model. The validation set of 1,875 WBC images, 466 mammograms, and 180 Genomic Samples provides hyperparameter tuning and convergence monitoring, while the independent trial set of 1,875 WBC images, 465 mammograms, and 180 Genomic Profiles provides a different evaluation of categorization performance and generalization ability.

Table 3. Data split strategy.

As shown in Table 4, the proposed framework employs a weakly supervised cross-domain multimodal fusion strategy due to the absence of patient-level aligned datasets containing WBC, mammography, and genomic data simultaneously. Stratified benign/malignant label-consistent pairing, L2 embedding normalization, and semantic embedding regularization were incorporated to reduce modality mismatch and improve latent consistency. ResNet50, DenseNet121, and a fully connected genomic encoder generated 512-D, 1024-D, and 128-D embeddings, respectively, which were fused into a unified 1,664-D multimodal representation. Additionally, modality-dropout regularization (p = 0.2), random modality masking, and joint end-to-end optimization using Adam (learning rate = 0.0001, batch size = 32) improved robustness, generalization, and stable multimodal representation learning.

Table 4. Cross-domain multimodal alignment and fusion strategy.

4 Methodology

The Tri-Fusion DL framework integrates three complementary biomedical modalities, WBC morphological images, mammographic radiological data, and genomic mutation profiles into a unified feature-level representation for early breast cancer detection.

Each modality is processed via a dedicated encoder: a CNN for WBC images to capture immune cell morphology, a pretrained deep CNN for mammograms to extract hierarchical tumor texture and margin features, and a fully connected mutation-signature encoder for genomic data capturing nonlinear gene–risk associations. The resulting embeddings (512-D WBC, 1024-D mammogram, 128-D genomics) are concatenated into a 1,664-dimensional joint vector that encodes cross-domain correlations between immune response, tissue phenotype, and molecular dysregulation. A dense classification head maps this fused representation to malignancy probability, optimized with binary cross-entropy loss. The methodology incorporates modality-dropout and missing-data handling to ensure robustness, enabling the model to generalize across incomplete real-world data while preserving biologically interpretable relationships among phenotype, immune status, and genomic risk. This is shown in Figure 1.

4.1 Preprocessing

The preprocessing pipeline systematically standardizes and prepares heterogeneous biomedical inputs for the Tri-Fusion framework. As shown in Figure 2, WBC images undergo intensity normalization, histogram-based contrast enhancement, and resizing to 256 × 256 pixels, ensuring robust morphological feature extraction by the CNN encoder. Mammograms are denoised using Gaussian smoothing, intensity-standardized, and resized to 224 × 224 pixels to preserve hierarchical lesion characteristics while reducing acquisition noise. Genomic mutation vectors are binary-encoded for mutation presence, standardized for continuous expression values, and imputed with zero-padding to handle missing genes, producing a uniform representation for the fully connected encoder. This pipeline enforces cross-modality compatibility, mitigates data variability, and facilitates effective feature-level fusion, enabling the model to jointly capture immune, radiological, and molecular patterns for accurate and biologically interpretable early breast cancer screening.

4.1.1 WBC image preprocessing

Peripheral blood smear images are normalized and enhanced to extract reliable immune morphological features. The steps include the following.

4.1.1.1 Intensity normalization

Pixel values are scaled to [0,1] is shown in Equation 1.

X ˜ i , j , k w = X i , j , k w − min ( X w ) max ( X w ) − min ( X w ) (1)

where i , jindex the pixel location, and kdenotes the RGB channel.

4.1.1.2 Contrast enhancement

Histogram equalization improves the nuclear visibility, as shown in Equation 2.

X ̂ i , j w = CDF − 1 ( CDF ( X i , j w ) ) (2)

Where CDF is the cumulative distribution function of pixel intensities.

4.1.1.3 Resizing

Images are resized to a fixed dimension for CNN input is shown in Equation 3.

X resized w ∈ ℝ 256 × 256 × 3 (3)

4.1.2 Mammogram image preprocessing

Mammograms are preprocessed to enhance tumor-relevant features and remove noise.

4.1.2.1 Noise removal (Gaussian smoothing)

X ˜ m = X m ∗ G σ where G σ = 1 2 π σ 2 e − x 2 + y 2 2 σ 2 (4)

As shown in Equation (4), Gaussian smoothing reduces the high-frequency noise while preserving edges.

4.1.2.2 Intensity normalization

X ˜ i , j m = X i , j m − μ X m σ X m (5)

where μ X mand σ X mare the mean and standard deviation of pixel intensities, respectively as shown in Equation (5).

4.1.2.3 Resizing

where X resized m represents the resized image of modality m with dimensions 224×224×1, as shown in Equation (6).

X resized m ∈ ℝ 224 × 224 × 1 (6)

4.1.3 Genomic data preprocessing

Genomic mutation data from TCGA-BRCA are standardized and imputed to handle missing values.

4.1.3.1 Binary encoding

Mutations are encoded as shown in Equation 7.

X i g = { 1 , if gene i is mutated 0 , otherwise (7)

4.1.3.2 Normalization for continuous expression values

X ˜ i g = X i g − μ i σ i (8)

where μ i and σ iare the mean and standard deviation of gene iacross all samples, respectively as shown in Equation (8).

4.1.3.3 Missing value imputation (zero-padding)

where X ˜ i g denotes the imputed value of the i^th gene in modality g. Missing gene expression values are replaced with zero using the zero-padding strategy, as defined in Equation (9).

X ˜ i g = 0 if gene i data is missing (9)

4.1.4 Data augmentation

To improve the generalization as per the following equations.

X aug w = Rotate ( X w ) + Flip ( X w ) + Shift ( X w ) (10)

where X aug w denotes the augmented whole-slide image obtained by applying rotation, flipping, and spatial shifting operations to the original image X^w, as defined in Equation (10).

X aug m = Rotate ( X m ) + Flip ( X m ) + Zoom ( X m ) (11)

where X aug m represents the augmented medical image generated by applying rotation, flipping, and zooming transformations to the original image X^m, as defined in Equation (11).

These preprocessing steps ensure that all three modalities, immune, imaging, and genomic, are standardized, normalized, and compatible for input into the Tri-Fusion DL architecture, enhancing model stability, convergence, and cross-domain feature learning.

4.2 Multimodal input representation

Every patient is represented as a tri-modal input tuple as shown in Equation 12.

X = { X w , X m , X g } (12)

The model simultaneously uses three complementary biomedical views. Here, X wcorresponds to WBC microscopic images capturing immune system responses and cellular morphology, X mrepresents mammography images describing breast tissue structure and tumor appearance, and X g denotes genomic mutation vectors encoding molecular abnormalities linked to breast cancer risk. Every one of these modes transmits essentially disparate information, visual immune forms, radiological tumor phenotype, and hereditary mutant signatures, so they are processed by a dedicated encoder dedicated to their data type: a CNN for WBC images, a deep CNN for mammograms, and a fully connected network for a genomic vector. These encoders transform the natural input signal into a compact latent feature representation that preserves mainly discriminative and physiologically relevant information, creating a shared space in which immune, imagination, and genome data can be combined and jointly analyzed for precise breast cancer prediction.

4.3 WBC morphological feature encoding

Peripheral blood smear images reflect tumor-induced changes in white blood cells, such as variations in nuclear shape, chromatin density, and cytoplasmic texture, which can indicate early malignancy. A CNN-based encoder f w ( · ) is applied to the WBC images X w. The encoder is parameterized by θ w, which includes all learnable weights and biases of the convolutional layers. Mathematically, the extracted feature vector is shown in Equation 13.

F w = f w ( X w ; θ w ) (13)

Here, F w ∈ ℝ 512is a 512-dimensional latent representation summarizing the most discriminative morphological information from the WBC images. Each dimension encodes aspects such as nuclear eccentricity, chromatin texture, cytoplasmic granularity, and other subtle immune-cell variations. These features are compact and robust, enabling the network to capture tumor-associated immune dysregulation for downstream multimodal fusion and breast cancer prediction.

4.4 Mammographic feature encoding

Mammograms provide detailed structural information about breast tissue, including masses, calcifications, and density variations associated with tumors. To extract meaningful features, a pretrained deep CNN (such as ResNet or DenseNet) is applied to the mammography images X m. This CNN is parameterized by θ m, which includes the weights and biases learned during training. The extracted feature vector is represented as shown in Equation 14.

F m = f m ( X m ; θ m ) (14)

Here, F m ∈ ℝ 1024is a 1,024-dimensional latent representation that encodes hierarchical tumor characteristics, including lesion margins, spiculated edges, shape irregularities, and tissue density patterns. These features summarize the most discriminative radiological information, enabling the model to detect subtle signs of malignancy and contribute to the multimodal fusion for accurate breast cancer screening.

4.5 Genomic mutation signature encoding

Each patient’s genomic profile X gconsists of binary or continuous values representing mutations or expression levels in key breast cancer–related genes such as BRCA1, BRCA2, TP53, PIK3CA, ESR1, and others. These values capture the molecular risk factors associated with malignancy. A fully connected mutation-signature encoder f g ( · )transforms this high-dimensional genomic vector into a compact latent representation as shown in Equation 15.

F g = f g ( X g ; θ g ) = σ ( W g X g + b g ) (15)

Here, W gand b gare the learnable weight matrix and bias vector of the network, and σ ( · )is an activation function (e.g., ReLU or sigmoid) introducing nonlinearity. The output F g ∈ ℝ 128is a 128-dimensional feature vector that encodes complex nonlinear relationships between gene mutations and cancer risk. This representation allows the model to quantify molecular patterns and integrate them with WBC and mammographic features for robust multimodal breast cancer prediction.

4.6 Cross-domain representation alignment and semantic consistency

To reduce the inter-dataset heterogeneity and modality mismatch, a cross-domain semantic embedding alignment strategy was incorporated into the proposed multimodal fusion framework. Following modality-specific feature extraction, latent embeddings generated from the WBC, mammogram, and genomic encoders were normalized using L2 feature normalization prior to multimodal fusion. Specifically, the ResNet50-based WBC encoder produced a 512-dimensional embedding, the DenseNet121-based mammogram encoder generated a 1,024-dimensional radiological embedding, and the fully connected genomic encoder produced a 128-dimensional mutation-signature embedding. The normalized modality-specific embeddings were concatenated to construct a unified multimodal representation is shown in Equation 16.

F fusion = [ F WBC ‖ F MG ‖ F GEN ] ∈ ℝ 1664 (16)

where,

F WBC ∈ ℝ 512 , F MG ∈ ℝ 1024 , F GEN ∈ ℝ 128

To improve semantic consistency across heterogeneous modalities, a cross-domain embedding regularization constraint was additionally introduced during training is shown in Equation 17.

L align = ∑ i , j y ij ‖ F i − F j ‖ 2 2 (17)

where y ij = 1 for embeddings belonging to the same malignancy category and y ij = 0otherwise. This regularization constrains embeddings corresponding to identical benign or malignant classes to occupy semantically consistent regions within the shared latent space. Consequently, the proposed alignment strategy minimizes modality-specific distribution mismatch, improves latent representation consistency, and enhances generalized multimodal representation learning across heterogeneous biomedical datasets.

4.7 Classification layer

After fusing the WBC, mammogram, and genomic features into the joint embedding F T, a dense fusion network (fully connected layers) is used to map this high-dimensional vector to a cancer probability. Mathematically, this is expressed as shown in Equation 18.

y ̂ = σ ( W f F T + b f ) (18)

Here:

• W fand b fare the learnable weights and bias of the dense layer,
• σ ( · )is an activation function, typically a sigmoid, which squashes the output into the range [ 0 , 1 ],
• y ̂ ∈ [ 0 , 1 ]represents the model’s predicted probability of malignancy for a given patient.

To train the network, binary cross-entropy loss is used as shown in Equation 19.

ℒ = − 1 N ∑ i = 1 N [ y i log ( y ̂ i ) + ( 1 − y i ) log ( 1 − y ̂ i ) ] (19)

Here:

• N is the number of training samples,
• y i ∈ { 0 , 1 }is the ground-truth label for sample i(0 = benign, 1 = malignant),
• y ̂ i is the predicted probability for sample i.

This loss function penalizes incorrect predictions more heavily the further they are from the true label, enabling the network to learn parameters that produce accurate probability estimates of malignancy across the multimodal input data.

4.8 Robustness to missing modalities

In real-world clinical scenarios, not all patients may have all three data types available; some may lack a WBC image, a mammogram, or genomic sequencing. To make the Tri-Fusion model robust to missing modalities, a modality-dropout strategy is applied during training. Mathematically, if a modality kis missing for a sample, its feature vector is replaced with zeros as shown in Equation 20.

F k = 0 if modality k is missing (20)

This approach ensures that the fusion network learns to make predictions even when one or more modalities are absent, rather than relying exclusively on all three. By simulating missing data during training, the model develops stable joint representations and avoids performance degradation in incomplete real-world datasets. This is essential for practical deployment in clinical screening, where some tests may not always be available for every patient.

4.9 Cross-domain modality pairing and training strategy

During multimodal training, stratified benign/malignant label-consistent pairing was performed across heterogeneous datasets to maintain semantic consistency between modalities. ResNet50, DenseNet121, and a fully connected genomic encoder generated modality-specific embeddings, which were fused into a unified multimodal latent representation. Adam optimization, modality-dropout regularization, and stratified 5-fold cross-validation were incorporated to improve robustness, stability, and reproducibility.

Table 5 summarizes the cross-domain modality pairing strategy, embedding generation process, optimization settings, and robustness mechanisms incorporated in the proposed Tri-Fusion framework. Stratified benign/malignant label-consistent pairing, semantic embedding alignment, L2 normalization, modality-dropout regularization, and joint end-to-end optimization collectively improve latent consistency, multimodal robustness, and generalized representation learning across heterogeneous biomedical datasets.

Table 5. Training configuration and modality pairing strategy.

4.10 Pseudocode of the tri-fusion deep learning model

The pseudocode represents the workflow of the Tri-Fusion DL model, which integrates WBC images, mammograms, and genomic mutation data for early breast cancer screening. It outlines modality-specific feature extraction, missing-data handling, feature-level fusion into a joint embedding, and classification through a dense neural network. The pseudocode highlights training with binary cross-entropy loss, weight updates via backpropagation, and evaluation using standard performance metrics, providing a clear computational blueprint for implementing the model.# Inputs: # X_w: WBC microscopic images # X_m: Mammogram images # X_g: Genomic mutation vectors # y: Ground truth labels (0 = benign, 1 = malignant) # Step 1: Define modality-specific encoders F_w = CNN_WBC_Encoder(X_w) # Output: 512-D WBC embedding F_m = Pretrained_CNN_Mammo(X_m) # Output: 1024-D mammogram embedding F_g = MLP_Genomic_Encoder(X_g) # Output: 128-D genomic embedding # Step 2: Handle missing modalities # For missing modality k, set embedding F_k = 0 if X_w is missing: F_w = zeros(512) if X_m is missing: F_m = zeros(1024) if X_g is missing: F_g = zeros(128) # Step 3: Feature-level fusion F_joint = concatenate(F_w, F_m, F_g) # 1664-D joint embedding # Step 4: Dense classification head # Fully connected layers with non-linear activation H = Dense_Layer(F_joint, units=512, activation='ReLU') H = Dropout(H, rate=0.3) y_hat = Dense_Layer(H, units=1, activation='sigmoid') # Output malignancy probability # Step 5: Define loss function Loss = Binary_CrossEntropy(y, y_hat) # Step 6: Training loop for epoch in range(num_epochs): for batch in dataset: X_w_batch, X_m_batch, X_g_batch, y_batch = batch y_hat_batch = forward_pass(X_w_batch, X_m_batch, X_g_batch) loss = compute_loss(y_batch, y_hat_batch) backpropagate(loss) # Update weights θ_w, θ_m, θ_g, θ_dense # Step 7: Evaluation # Compute accuracy, precision, recall, F1-score, AUC on test set metrics = evaluate_model(test_data)

5 Results and discussion

The proposed Tri-Fusion Deep Learning Model was evaluated on three publicly available datasets: WBC images (12,500 samples), CBIS-DDSM mammograms (3,102 samples), and TCGA-BRCA genomic profiles (1,200 patients). Both unimodal and multimodal experiments were conducted to benchmark the performance of the proposed method, highlighting the contribution of each modality and the effectiveness of feature-level fusion.

As shown in Figure 3, the WBC image panel demonstrates the Tri-Fusion model’s ability to analyze peripheral blood smear morphology for early breast cancer detection. Each sample shows nuclear and cytoplasmic features critical for malignancy assessment, with Grad-CAM highlights indicating regions of high relevance. Predicted probabilities range from low (0.09) for normal cells to high (0.87) for malignant cells, illustrating the model’s sensitivity to subtle morphological variations. The results confirm that WBC-based embeddings provide meaningful discriminative features, which, when fused with mammogram and genomic data, contribute to the overall high accuracy and robustness of the multimodal Tri-Fusion framework.

Figure 4 illustrates the iterative feature refinement and malignancy prediction process for a single WBC image using a CNN-based embedding framework. In Iteration 1, the CNN extracts a low-dimensional 128-D WBC embedding, yielding a weak representation with a malignancy confidence score of 0.42, categorized as uncertain, indicating limited discriminative feature capture. In Iteration 2, the representation is progressively enriched to a 256-D embedding, where enhanced hierarchical feature learning improves class separability, increasing the malignancy score to 0.61 and shifting the prediction to a moderate confidence level. Finally, iteration 3 produces a high-capacity 512-D embedding, enabling more robust representation of cytological patterns and nuclear morphology, resulting in a significantly higher malignancy score of 0.87, classified as High confidence. This progressive embedding expansion demonstrates that increased feature dimensionality within the CNN significantly improves representation learning, leading to improved discriminative power and more reliable malignancy prediction in WBC analysis.

As shown in Table 6, WBC images provide strong discriminative immune features, reflected in the high accuracy and AUC. Mammograms are sensitive to mass morphology but less effective in dense tissue. Genomics alone captures molecular risk but lacks phenotypic context.

Table 6. Unimodal performance.

As shown in Table 7, the fusion of immune and imaging features significantly reduces false negatives compared to unimodal models. Inclusion of genomics further improves predictive reliability, especially in borderline cases.

Table 7. Bimodal fusion performance.

As shown in Table 8, the full Tri-Fusion model outperforms all unimodal and bimodal baselines, highlighting the effectiveness of feature-level fusion. The high AUC (98.6%) demonstrates excellent separability between benign and malignant samples. The model remains robust under cross-validation and missing-modality scenarios due to modality-dropout handling.

Table 8. Tri-Fusion multimodal performance.

5.1 Modality contribution and feature attribution analysis

To improve interpretability, modality-wise feature attribution analysis was incorporated into the proposed Tri-Fusion framework.

As shown in Table 9, the Grad-CAM visualization demonstrated that the WBC encoder focused primarily on nuclear morphology and chromatin distribution patterns associated with malignant transformation, while the mammogram encoder highlighted lesion margins, dense tissue regions, and spiculated abnormalities. Additionally, genomic embedding activation analysis indicated strong latent contributions from BRCA1, BRCA2, TP53, and PIK3CA mutation-associated features. These findings confirm that the proposed framework learns physiologically relevant multimodal representations across immune, radiological, and molecular domains.

Table 9. Modality contribution and feature attribution analysis.

Figure 5 illustrates the Grad-CAM visualization results for the WBC modality. The generated heatmaps indicate that the proposed framework primarily focuses on malignant nuclear morphology, chromatin distribution, and irregular cellular structures during classification. The highlighted activation regions confirm that the WBC encoder successfully captures the diagnostically relevant immune-cell features associated with malignant transformation.

Figure 6 presents Grad-CAM-based interpretability analysis for the mammography modality. The attention maps demonstrate that the proposed framework concentrates on lesion boundaries, dense tissue regions, and spiculated abnormalities, which contribute to breast cancer prediction. These findings confirm that the mammogram encoder effectively learns clinically significant radiological characteristics relevant to tumor detection.

Figure 7 shows the genomic embedding contribution analysis of the proposed framework. The results indicate that mutation-associated features related to BRCA1, BRCA2, TP53, and PIK3CA exhibit strong latent contribution toward multimodal prediction. This demonstrates the genomic encoder’s ability to capture biologically meaningful molecular signatures associated with breast cancer progression.

Figure 8 illustrates the relative contribution of each modality towards the final multimodal prediction. The analysis shows that mammography features contributed the strongest influence, followed by WBC immune morphology features and genomic embeddings. The complementary contribution of all three modalities confirms the effectiveness of the proposed Tri-Fusion framework in learning generalized cross-domain multimodal representations.

Table 10 summarizes the ablation and statistical validation analysis of the proposed Tri-Fusion framework. The results demonstrate that the multimodal integration consistently outperformed unimodal and bimodal configurations, achieving the highest performance with 96.2% accuracy and 98.6% AUC-ROC. The narrow Confidence Intervals (CI) and low inter-fold variance indicate stable and statistically reliable learning performance across heterogeneous biomedical datasets. Furthermore, the observed performance reduction after removing semantic embedding alignment or modality-dropout regularization confirms the significant contribution of these components toward robustness, latent consistency, and generalized multimodal representation learning.

Table 10. Ablation and statistical validation analysis.

To further validate the robustness of the proposed framework, additional statistical and ablation analyses were performed. As shown in Table 11, the stratified 5-fold cross-validation demonstrated stable performance with a mean accuracy of 95.8% ± 0.6 and a mean AUC-ROC of 98.1% ± 0.4. Furthermore, modality-wise ablation analysis confirmed that the integration of WBC, mammogram, and genomic embeddings significantly improved discriminative capability compared to unimodal and bimodal configurations. Similarly, removal of semantic embedding alignment or modality-dropout regularization reduced performance, demonstrating the contribution of these components toward generalized multimodal representation learning.

Table 11. Cross-validation results of the proposed model using 5-fold evaluation strategy.

Table 12 compares the proposed Tri-Fusion framework with recent state-of-the-art multimodal DL architectures across multiple evaluation metrics. The proposed framework achieved superior accuracy, precision, recall, F1-score, and AUC-ROC due to semantic embedding alignment, modality-invariant latent fusion, and robustness-aware multimodal optimization. These findings demonstrate the effectiveness of the proposed cross-domain multimodal representation learning strategy for heterogeneous biomedical datasets.

Table 12. Comparison with recent state-of-the-art multimodal models.

As shown in Figure 9, the confusion matrix illustrates that the Tri-Fusion Model outperforms the trial lay. Here, 880 benign cases were correctly identified outside the whole sample, while 40 were incorrectly classified as malignancies. Similarly, 945 malignancies were correctly predicted, together with 35 false negatives. This indicates a high overall accuracy of 96.2%, a minimal false-negative rate of 3.6%, and a false-positive rate of 4.3%. The results demonstrate the ability of the model to correctly differentiate between benign and malignant scenarios, highlighting the advantages of integrating WBC morphology, mammogram characteristics, and genetic signatures for robust early detection of breast cancer.

Figure 10 compares the performance of unimodal, bimodal, and Tri-Fusion models across five key metrics: Accuracy, Precision, Recall, F1-score, and AUC-ROC. The Tri-Fusion model consistently outperforms both unimodal and bimodal approaches, achieving the highest values across all metrics, which highlights the advantage of integrating WBC morphology, mammogram features, and genomic data. Bimodal fusion shows intermediate performance, indicating that combining two modalities improves prediction compared to single-modality models, while unimodal models provide the baseline performance. This visual clearly demonstrates the benefit of multimodal feature-level fusion for robust early breast cancer detection.

As shown in Figure 11, it compares the Tri-Fusion DL model with conventional unimodal and bimodal approaches for early breast cancer screening. Tri-Fusion consistently outperforms CNN-based mammogram, genomic MLP, WBC CNN, and bimodal fusion models across all metrics of accuracy (96.2%), precision (95.4%), recall (94.8%), and F1-score (95.1%). Unimodal models exhibit lower performance due to the limited scope of single-modality features, while bimodal fusion improves results by combining complementary data but still underperforms relative to Tri-Fusion. These results highlight the advantage of integrating immune, imaging, and genomic modalities through feature-level fusion, enabling robust, high-precision, and clinically interpretable early detection of breast cancer.

5.2 External validation and generalization analysis

To further evaluate the robustness and generalizability of the proposed Tri-Fusion framework, external-style validation analysis was performed using heterogeneous independent biomedical datasets originating from separate repositories, including the WBC Image Dataset, CBIS-DDSM mammography dataset, and TCGA-BRCA genomic cohort. Since these datasets were collected under different acquisition protocols, imaging conditions, sequencing platforms, and population distributions, they provide a cross-domain independent cohort evaluation setting for assessing generalized multimodal representation learning capability. Additionally, the stratified 5-fold cross-validation and modality-dropout robustness analysis were performed to evaluate stability under heterogeneous multimodal conditions.

As shown in Table 13, the proposed framework achieved a stable performance with a mean cross-validation accuracy of 95.8% ± 0.6 and a mean AUC-ROC of 98.1% ± 0.4, demonstrating strong convergence, robustness, and generalized latent representation learning across independent biomedical cohorts.

Table 13. External validation and generalization analysis.

The performance of the Tri-Fusion model is better than that of the multimodal integration model for the timely detection of breast cancer. The model consistently outperformed the unimodal and bimodal baselines with 96.2% accuracy, 95.4% precision, 94.8% recall, 95.1% F1-score, and 98.6% AUC-ROC. The unimodal model, similar to WBC images, mammograms, or alternatively genomic profiles, has low accuracy (91–93%) and AUC (92–96%), and during bimodal fusion has decent quality (94–96%) in terms of techniques. The above findings highlight the key advantages of feature-level fusion in cross-domain correlation of immune morphology, tissue phenotype, and molecular signature. The minimal false-negative and false-positive rates combined with robust cross-validation performance and strength to exclude the possibility of a mismatch are a reliable, physiologically explainable, and clinically applicable method for non-invasive early breast cancer screening.

Although external validation across heterogeneous independent biomedical datasets demonstrated strong robustness and generalized multimodal representation learning capability, a remaining limitation is the absence of patient-level paired multimodal datasets containing synchronized WBC, mammography, and genomic data. Consequently, weakly supervised label-consistent pairing was adopted for cross-domain multimodal fusion. While semantic embedding alignment and modality regularization reduce the inter-domain heterogeneity, residual dataset-specific bias may still exist. Therefore, the proposed framework should be interpreted as a generalized multimodal representation learning model rather than a fully clinically validated patient-specific diagnostic system.

6 Conclusion

The proposed Tri-Fusion DL model, which integrates heterogeneous biomedical modalities such as WBC morphology, mammogram imaging, and genomic mutant profiles by means of feature-level fusion, provides a robust and technically better outline for early breast cancer screening. Experimental results on three datasets show that the model achieves 96.2% accuracy, 95.4% precision, 94.8% recall, 95.1% F1-score, and 98.6% AUC-ROC over the unimodal and bimodal baselines. The WBC encoder accurately captures immune cell morphologic variations associated with tumors, the mammogram encoder extracts powdered lesion forms, including spiculation and margin abnormalities, and the genome encoder model mutant signature of a serious disease such as BRCA1, BRCA2, TP53, and PIK3CA. The iterative feature visual image and Grad-CAM investigation confirm that the model is increasingly focused on physiologically applicable regions, improving interpretability. The external validation results across heterogeneous independent biomedical cohorts further demonstrate the robustness and generalized multimodal learning capability of the proposed Tri-Fusion framework. The proposed Tri-Fusion model provides a non-invasive, highly accurate, and generalizable decision support tool for early detection and vulnerability stratification in breast cancer. Future work will focus on validating the proposed framework using clinically paired multi-institutional cohorts containing synchronized hematologic, radiologic, and genomic records to further establish clinical reliability and translational applicability.

Generative AI statement

The author(s) declared that Generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Kaleem Siddiqi, McGill University, Canada

Reviewed by: Renu Popli, Chitkara University, India

Sabarna Choudhury, Qualcomm, United States