TF-Mamba-DPHNet: a time-frequency state-space and dual-path interaction framework with hybrid cross-scale feature calibration for speech enhancement

1 Introduction

Speech enhancement (SE) aims at improving the perceptual quality and intelligibility of speech signals degraded by environmental noise. It plays a critical role in communication systems, hearing assistive devices, automatic speech recognition, and real-time telecommunication applications. While early signal processing techniques, e.g., statistical filtering and spectral estimation, laid foundational groundwork, their reliance on stationary assumptions and handcrafted priors limits robustness under highly non-stationary conditions.

Recent advances in deep learning have significantly transformed SE into a data-driven non-linear mapping problem between noisy and clean speech representations (Xu et al., 2015). Convolutional neural networks (CNNs) and convolutional recurrent neural networks (CRNs) demonstrated strong capability in modeling local spectral structures and temporal dynamics (Tan and Wang, 2020; Zhao et al., 2018; Kolboek et al., 2016). Complex domain architectures, such as DCCRN (Hu et al., 2020), additionally improved phase-aware recognition, while time-domain models such as SEGAN (Pascual et al., 2017) and waveform-based frameworks (Defossez et al., 2020; Pandey and Wang, 2019) elevated perceptual naturalness.

To model long-range dependencies, dual-path structures such as DPCRN (Le et al., 2021) and DPRNN (Luo et al., 2020) are introduced that efficiently learn intra- and inter-segment contexts. More recently, transformer-based networks have leveraged global self-attention to capture long-term temporal correlations (Mamun and Hansen, 2024; Xu and Jianjun, 2022; Kim and Seo, 2021). Apart from structural modeling breakthroughs, metric-guided adversaries such as MetricGAN+ (Fu et al., 2021) and CMGAN (Cao et al., 2022) directly optimize perceptual metrics (PESQ and STOI). Although these frameworks boost objective scores, adversarial training could also result in instability and increased sensitivity to hyperparameters.

Multi-scale representation learning has become a fundamental of modern speech enhancement systems. By capturing features at different time scales and across multiple frequencies, multi-scale convolutional architectures and feature calibration learn to effectively combine detailed spectral knowledge with large-scale contextual information to increase perceptual quality and speech intelligibility (Xian et al., 2020; Xiang et al., 2021a; Shaik and Vanambathina, 2025). Time-frequency attention then selectively magnifies discriminative spectral-temporal patterns while suppressing background noise (Zhang et al., 2022a; Zhang et al., 2022b; Jannu and Vanambathina, 2023).

These developments inspire several attention-enhanced U-Net variants, including MASENet (Xiang et al., 2021a), SADNUNet (Xiang et al., 2021b), MANNER (Park et al., 2022), HATFANet (Zhang et al., 2024), ROSE-Net (Yu et al., 2024), and CNUNet (Hwang et al., 2025), that leverage various types of channel-wise, spatial, hybrid, and sequential attention blocks to strengthen the progressive fusion of local details and long-range information. They have consistently improved objective evaluation metrics (PESQ and STOI). Despite their effectiveness, most existing methods primarily rely on pairwise attention and conventional skip-connection strategies (either direct or nested), without explicitly modeling higher-order cross-channel dependencies and structured interactions within hierarchical feature representations. More recently, TFDense-GAN (Chen et al., 2025) introduced a time-frequency dense generative adversarial framework that combines dense feature extraction with adversarial learning to improve perceptual speech quality and intelligibility. Although the model achieves competitive enhancement performance, its dense network architecture increases model complexity and computational cost, limiting its suitability for lightweight speech enhancement applications. DMC-Net (Saleem et al., 2025) further explored dual multi-context feature aggregation to improve speech enhancement under challenging noise conditions by exploiting complementary contextual representations. However, the multiple context aggregation modules increase architectural complexity and parameter count.

Despite their effectiveness, self-attention mechanisms may not always be optimal for speech enhancement tasks. Global attention mechanisms in high-resolution time-frequency representations (TFRs) tend to overemphasize long-range contextual information and global speech patterns, and they may not retain enough fine-grained spectral details, losing the fine structure that is crucial for natural and intelligible speech. This limitation is particularly severe in low SNR scenarios where important parts of speech need to be accurately disentangled from the background noise.

To tackle these problems, recent work has examined state-space models (SSMs) (Ku et al., 2023; Gu et al., 2021) as an effective approach for Transformers. SSMs are particularly effective at modeling long sequential data while maintaining stable temporal representations. Among these models, Mamba (Gu and Tri, 2024), derived from the Structured State Space Sequence (S4) framework, introduces a selective input-processing mechanism together with a hardware-efficient design. As a result, it achieves performance comparable to or better than Transformer-based models across a wide range of applications, including speech processing, computer vision, and natural language understanding tasks (Li et al., 2024; Wang et al., 2024; Hatamizadeh and January 2025). Intriguingly, previous models (Chao et al., 2024) demonstrate that simply replacing the Conformer block in MP-SENet with Mamba substantially increases the performance and robustness of enhancement.

Building on these insights, the proposed model adapts Time-Frequency Mamba (TF-Mamba) modules (Wang et al., 2025) inside the U-Net architecture, enabling the network to learn both coarse and fine-grained representations across multiple resolutions, thereby enhancing the modelling of long-range dependencies and the restoration of spectral details through multi-scale feature fusion.

Selective SSMs provide two advantages over Transformer-based self-attention for speech enhancement: (i) they model structured long-range dependencies that are stable in temporal dynamics over long time-frequency sequences, and (ii) they continuously evolve the state, preserving spectral continuity and avoiding potential over-smoothing effects of global attention aggregation. These properties make selective SSMs especially suitable for speech signals, where temporal coherence and spectral consistency are essential for perceptual quality.

Motivated by these advantages, we integrate a Time-Frequency Mamba (TF-Mamba) module into the proposed speech enhancement framework. Unlike the original Mamba architecture proposed in (Gu and Tri, 2024), which employs a selective state-space model to process one-dimensional sequential data, the proposed TF-Mamba extends this paradigm to dual-axis time-frequency feature representations within the U-Net feature space. Specifically, TF-Mamba consists of separate Time-Mamba and Frequency-Mamba branches, where each branch performs bidirectional selective state-space modeling to capture long-range contextual dependencies along the corresponding temporal or frequency dimension. The outputs of the two branches are subsequently fused through residual addition and concatenation to obtain a unified time-frequency representation. Although the underlying selective state-space mechanism remains identical to the original Mamba, its extension to dual-axis processing enables more effective modeling of temporal dynamics and spectral structures while preserving fine-grained spectral continuity. Consequently, TF-Mamba is better suited for spectrogram-based speech enhancement than the original one-dimensional Mamba formulation.

In parallel, multi-scale representation learning has proven crucial for robust speech enhancement. Multi-scale convolutional and feature recalibration networks improve contextual aggregation and perceptual quality (Xian et al., 2020; Xiang et al., 2021a; Shaik and Vanambathina, 2025), while time-frequency attention mechanisms refine spectral-temporal feature weighting (Zhang et al., 2022a; Zhang et al., 2022b; Jannu and Vanambathina, 2023). However, most existing frameworks treat multi-scale extraction, global dependency modeling, and skip fusion as independent design components. Moreover, feature interactions are mostly limited to first-order convolutional aggregation or second-order pairwise attention, without any explicit higher-order cross-channel modeling.

Despite significant advances, current speech enhancement systems suffer from several drawbacks that degrade both intelligibility and perceptual quality. First, noise leakage through skip connections spoils the signal, thus reducing intelligibility. Second, insufficient modeling of long-range temporal and spectral dependencies limits the reconstruction of global speech structures, leading to degraded quality and intelligibility. Third, over‐smoothed spectral masks tend to suppress the essential harmonic components, which reduces naturalness and intelligibility. Finally, many systems are still sensitive to unseen noise conditions, resulting in unstable enhancement performance and inconsistent perceptual outcomes in different acoustic environments.

To overcome these limitations, we propose TF-Mamba-DPHNet, a unified encoder-decoder framework for single-channel speech enhancement. Unlike existing methods that primarily employ multi-scale feature extraction, state-space modeling, higher-order interaction, and feature calibration as independent components or directly combine them with limited task-specific adaptation, the proposed framework systematically redesigns the MSFE, DPH-TFA, and H-CS-FC modules while strategically integrating TF-Mamba to enable complementary local-global representation learning. Therefore, the primary methodological novelty of the proposed framework lies not in introducing entirely new standalone modules, but in their task-oriented redesign and coordinated integration within a unified architecture specifically tailored for single-channel speech enhancement. The proposed architecture combines multi-scale convolutional representation, state-space-based global modeling, structured higher-order interaction, and cross-scale calibration in a unified framework to effectively improve speech quality with efficient and stable performance. Experimental results on benchmark datasets show consistent advantages in perceptual quality (PESQ), intelligibility (STOI), and distortion reduction (SDR), especially in low-SNR and highly non-stationary noise scenarios.

The main contributions of this work are summarized as follows:

• The incorporation of MSFE blocks in the encoder and decoder allows for multi-scale feature extraction, effectively capturing both local and global speech characteristics.
• the introduction of an efficient global dependency modeling mechanism in the TF‐Mamba using a selective state‐space-based neural module
• A DPH-TFA module for structured dual-axis long-range modelling and H-CS-FC is introduced to perform cross-scale-guided (bottleneck and deeper features) feature recalibration for skip refinement, replacing direct concatenation and attention-based fusion.
• A structured Higher-Order Information Interaction (HOII) mechanism enabling cross-channel structured multiplicative modeling beyond pairwise attention.
• Performance improvements over state-of-the-art baselines in terms of PESQ, STOI, and SDR.

The remainder of this paper is organized as follows: Section 2 details the proposed model architecture and individual modules. Section 3 describes the datasets, experimental setup, ablation studies, and performance evaluation. Section 4 provides the overall discussion of the proposed models. Finally, Section 5 concludes the paper and outlines future research directions.

2 Detailed overview of the proposed model: (TF-mamba-DPHNet)

2.1 Proposed method

The model TF-Mamba-DPHNet architecture for single-channel speech enhancement is shown in Figure 1. The network is organized into four hierarchical encoder stages, a context modeling bottleneck, and four decoder stages, interconnected through H-CS-FC modules. The noisy input spectrogram I ∈ R T × F × 1 is initially processed by the encoder, consisting of four MSFE blocks, denoted as e 0 , e 1 , e 2 , e 3. Each MSFE block in an encoder is employed to extract both fine-grained and global contextual patterns (Yechuri and Vanambathina, 2023). Following each MSFE block, the TF‐Mamba module (Chao et al., 2024; Wang et al., 2025) refines the extracted features by modeling long‐range time‐frequency dependencies through a selective state‐space model mechanism. This increases the encoder’s capability to preserve temporal coherence and spectral continuity, resulting in better speech intelligibility and perceptual quality. By inserting TF‐Mamba after each MSFE stage, both short‐term and long‐term contextual cues are effectively encoded into the feature space. To reduce the spatial resolution progressively and enlarge the receptive field, a MaxPooling2D layer is used after each MSFE block, which allows obtaining compact latent representations while preserving key spectral-temporal features.

The compressed output from the last encoder block is propagated to the Dual-Path Higher-Order Information Interaction with Time-Frequency Attention (DPH-TFA) module, repeated four times. This module integrates time-frequency attention (TFA) (Zhang et al., 2022a) with higher-order information interaction (HOII) (Wei et al., 2023), effectively capturing long-range dependencies and cross-dimensional correlations in both temporal and spectral domains. The final bottleneck representation is denoted as h. The bottleneck output h is distributed to multiple H-CS-FC blocks (H-CS-F C 0 to H-CS-F C 3, which adaptively recalibrate encoder features before merging them with decoder pathways. In the early studies (Wang et al., 2024), each bottleneck-guided FC (feature calibration) unit performs channel-wise modulation to selectively emphasize speech-dominant components while suppressing residual noise artifacts before skip fusion. In the extended design, the conventional FC mechanism is further generalized into a hierarchical Hybrid Cross-Scale Feature Calibration (H-CS-FC) module. Unlike standard feature gating, H-CS-FC jointly exploits cross-scale contextual interactions and hierarchical encoder-decoder dependencies. Specifically, the bottleneck representation not only calibrates features within the same resolution level but also facilitates inter-scale information exchange, enabling more structured feature refinement across multiple temporal-spectral resolutions. This hybrid formulation enhances global-local consistency while preserving discriminative speech structures under harmful noise conditions.

The decoder mirrors the encoder with four hierarchical stages, d 0 , d 1 , d 2 , d 3. At each stage, an UpSampling2D operation is first applied to progressively restore the time-frequency domain resolution, ensuring alignment with the corresponding encoder features. The up-sampled features are fused with encoder representations using H-CS-FC, which mediates skip connections, enabling structured cross-scale feature refinement. After skip fusion, each decoder stage incorporates an MSFE block to refine localized spectral-temporal representations. Subsequently, a TF-Mamba module is applied to model long-range dependencies within the reconstructed feature space. While the encoder-side TF-Mamba modules focus on extracting globally coherent representations from noisy inputs, the decoder-side TF-Mamba modules operate on progressively resolved feature maps, enforcing global consistency during speech reconstruction. In particular, the decoder TF-Mamba modules facilitate: (i) reinforcement of long-range temporal continuity, (ii) Preservation of spectral harmonic structures, (iii) Correction of residual noise artifacts introduced during up-sampling, and (iv) Structured integration of cross-scale contextual information.

Placing TF-Mamba after each decoder MSFE block ensures that both local refinement and global dependency modeling are jointly optimized at each reconstruction stage, preventing fragmentation of speech components and over-smoothing effects arising during hierarchical decoding. Finally, a 1 × 1 separable convolution with linear activation provides the enhanced speech output.

Overall, the proposed TF-Mamba-DPHNet integrates multi-scale feature extraction, selective state-space modeling, higher-order interaction, and hierarchical cross-scale calibration within a unified encoder-decoder framework. MSFE blocks provide enriched local spectral-temporal representations, TF-Mamba modules ensure efficient global dependency modeling with linear complexity, DPH-TFA captures structured higher-order cross-dimensional interactions, and H-CS-FC provides for adaptive cross-scale feature refinement. Together, these components provide joint local-global modeling to remove residual noise while preserving fine spectral details and improving the continuity of harmonic speech, resulting in improved speech naturalness and intelligibility. Both TF-Mamba and DPH-TFA capture local and global dependencies but complement each other in our speech enhancement framework. The TF-Mamba module uses selective state-space modelling to capture long-range temporal-spectral correlations to disentangle speech and noise over long time-frequency (TF) sequences. Different from TF-Mamba, the DPH-TFA module emphasizes time-frequency domain interaction and higher-order local feature refinement, which helps to recover fine spectral details and transient cues that are usually lost in noisy environments. The joint integration of TF-Mamba and DPH-TFA modules provides the synergistic balance between global context awareness and localized spectral precision, thereby further improving the perceptual quality, intelligibility, and robustness of the enhanced speech signal.

2.2 Multi-scale feature extraction (MSFE) block

Convolutional encoder-decoder models and U-Net style models have been widely adopted to model spectral and temporal correlations of noisy speech (Hu et al., 2020; Zhang et al., 2024; Gong et al., 2019) for single-channel speech enhancement. However, the traditional convolutional layers are not very good at capturing multi-scale contextual dependencies, especially in extremely non-stationary noise environments. To alleviate this limitation, we propose a Multi-Scale Feature Extraction (MSFE) block based on a D3Net backbone to enable hierarchical and efficient contextual modeling.

The proposed MSFE block is based on depth-wise separable convolutions to enhance the computational efficiency while preserving the representational expressiveness, consistent with the efficient convolutional designs reported in (Chollet, 2017; Howard et al., 2017; Sandler et al., 2018). Separable convolution decouples spatial filtering and channel projection, which can not only reduce the redundancy of parameters but also improve the channel-wise modeling ability without sacrificing the coverage of receptive fields.

The proposed MSFE block employs depth-wise separable convolutions to address the computational complexity whilst preserving the ability to learn informative features. This design separates spatial filtering from channel-wise processing, decreases the number of parameters and computations without reducing the receptive field. As a result, the network can learn channel-specific information more effectively with lower computational cost (Chollet, 2017; Howard et al., 2017; Sandler et al., 2018).

Figure 2 shows the architecture, which is an encoder-decoder framework based on MSFE, enhanced with a D3Net bottleneck for hierarchical spectro-temporal modeling. The input feature map is first processed by depth-wise separable convolutions (S e p a r a b l e C o n v 2 D), followed by Layer Normalization (LN) (Ba et al., 2016) and a LeakyReLU activation to efficiently capture local patterns f 1 I while reducing parameter redundancy and improving training stability. The proposed MSFE block consists of five encoder layers e 1 m s to e 5 m s that sequentially apply SeparableConv2D, Layer Normalization (LN), and LeakyReLU, followed by temporal MaxPooling2D, producing hierarchical features E N l , E N l D o w n , , l = 1 , 2 , . . . , 5 . The deepest feature E N 5 D o w n is handled by a D3Net block with multi-dilated convolutions d ∈ 1 , 2 , 4, which allows to extract both short- and long-term dependencies while enabling efficient multi-scale feature propagation. The decoder reconstructs the signal through an architecture that corresponds to the design of the encoder, using the corresponding MSFE layers d 1 m s to d 5 m s. It recovers higher-resolution features through skip connections defined as U k C a t = C o n c a t U k , E N k D o w n, and by applying sub-pixel convolution for upsampling.

The upsampling operation is defined in Equation 1:where U 5 = B o u t. A residual fusion with the initial stem feature f 1 I guarantees that the final output maintains fine-grained spectro-temporal details while incorporating high-level contextual information.

U k = σ S e p a r a b l e C o n v 2 D U p S a m p l i n g U k + 1 , 3 , 3 (1)

The residual fusion is expressed in Equation 2:

O u t p u t = D e c o d e r o u t p u t + f 1 I (2)

The proposed MSFE block extracts information from different receptive fields by using multi-scale convolutions together with densely connected dilated convolutions. To facilitate stable training and learn features better, skip connections, Layer Normalization, and LeakyReLU are used. High-resolution feature maps are recovered using sub-pixel up-sampling with minimal reconstruction artifacts. The generated hierarchical representations are informative features for downstream modules such as dual-path interaction blocks, which further capture long-range dependencies. The proposed MSFE framework combines efficient convolutional operations with the hierarchical feature abstraction, context-aware dense modeling, and residual learning mechanisms to generate a multi-scale representation. Exploiting localized spectro-temporal patterns and long-range contextual dependencies jointly, the architecture strikes a balance between fine structural details and global semantic information. This integrated design enhances the representation robustness and generalization ability, especially for single-channel speech enhancement in non-stationary noise conditions. The proposed MSFE is derived from the conventional D3Net-based multi-scale feature extractor. Rather than directly adopting the original D3Net architecture, the standard convolutional layers are replaced with depth-wise separable convolutions to reduce parameter redundancy while preserving multi-scale receptive-field learning and improving the efficiency of hierarchical feature extraction.

2.3 D3Net (densely connected multi-dilated dense net)

The proposed framework adopts a Densely Connected Multi-Dilated Dense Network (D3Net) (Yechuri and Vanambathina, 2023) to improve the learning of contextual features while retaining the recovery of fine spectral details. D3Net combines dense connections and dilated convolutions to gather information across multiple scales and detect local and global context patterns. Compared with conventional convolutional layers (Gong et al., 2019; Song et al., 2022), dilated convolutions enlarge the receptive field without increasing the number of parameters or reducing feature-map resolution. This property makes them well-suited for speech enhancement tasks, where the network must simultaneously model short-term spectral structures and long-range contextual information. In addition, dense connections promote feature reuse and improve the flow of gradients across the network by concatenating outputs from previous layers (Takahashi and Mitsufuji, 2020). The structure of the proposed D3Net is shown in Figure 3. The operational flow of D3Net is as follows. First, the input feature map E N 5 D o w n (denoted as drop5) is processed through three parallel convolutional layers with dilation rates d ∈ 1 , 2 , 4, each using a 3 × 3 kernel.

The feature maps obtained are activated by LeakyReLU (σ) to ensure stable gradient propagation, as defined in Equation 3:The resulting feature maps are concatenated to form B ₁, as expressed in Equation 4:

D i = σ S e p a r a b l e C o n v 2 D E N 5 D o w n , d i l a t i o n = d i , i = 1 , 2 , 3 (3)

B 1 = C o n c a t D 1 , D 2 , D 3 (4)

A second stage of dilated convolutions produces the bottleneck output, as defined in Equation 5:The resulting second-stage feature maps are concatenated to produce the final bottleneck output, as expressed in Equation 6:where D i , and D i ′ are the D3Net dilated convolution outputs, and B 1 and B o u t are the bottleneck features. The first layer, which is a standard convolution with a dilation rate of 1 for local feature extraction. The second layer with the dilation rate of 2 observes a larger context region and captures larger temporal or spectral patterns. The third layer uses a dilation rate of 4 to further increase the receptive field to incorporate global information such as long-range dependencies in speech transitions. The multi-dilated structure is applied again to achieve deep hierarchical feature extraction. One of the key features of the D3Net is a dense connection mechanism, where the output of each convolution is gradually enriched with information from previous layers. This results in a highly connected architecture, which contains multi-scale features learned from different receptive fields.

D i ′ = σ S e p e r a b l e C o n v 2 D B 1 , d i l a t i o n = d i , i = 4 , 5 , 6 (5)

B o u t = C o n c a t D 4 , D 5 , D 6 (6)

D3Net exploits the dilated convolutions with dense connections to effectively capture different temporal and spectral patterns. This is especially useful for tasks such as segmentation, classification, and speech enhancement. D3Net uses densely connected multi-dilated convolutions for multi-scale contextual aggregation to preserve high-resolution representations.

2.4 Time-Frequency Mamba (TF-Mamba)

The Time-Frequency Mamba (TF-Mamba) module extends the structured state-space model (SSM) to a dual-axis formulation that processes features sequentially along both the time and frequency dimensions. This design is commonly known as a two-stage or bidirectional Mamba, as information flows through complementary temporal and spectral paths, enabling the network to exploit both forward and reverse contextual information in the time-frequency domain. As illustrated in Figure 4, TF-Mamba consists of time-Mamba and frequency-Mamba branches whose outputs are fused via residual addition and concatenation to form a unified representation. Each branch applies a linear projection, a 1-D convolution, and a SiLU activation (σ, followed by forward and backward Mamba units to capture bidirectional dependencies. Long-range modeling is handled through a selective SSM (Hatamizadeh and January 2025), which dynamically modulates salient features. The continuous-time SSM computes hidden-state transitions using learnable matrices, A, B, and C, expressed as The continuous-time selective state-space model is expressed in Equation 7:where x t ∈ R is the input, y t ∈ R is the output, and h t ∈ R N denotes the hidden state with N channels. Since continuous-time parameters Δ , A , B cannot be directly used in discrete sequences, a conversion into discrete counterparts (A ¯ , B ¯) is done, as described in (Wang et al., 2025).

h t ′ = A h t + B x t ; y t = C h t (7)

The bidirectional outputs of the time and frequency branches are normalized using RMSNorm, merged through element-wise addition, concatenated, and refined using a final convolutional layer. The integration of selective state-space analysis allows TF-Mamba to maintain efficient long-range dependency modeling while achieving strong global modeling capability, similar to Transformer architectures but with lower memory overhead. The integration of state-space selectivity with time-frequency dual processing enables the module to effectively model both the transient and stationary components of speech signals. Such selective control of temporal and spectral dependencies renders TF-Mamba very effective for speech enhancement, where fine discrimination between the target speech and background noise is critical. Overall, the TF-Mamba provides an elegant framework that unifies structured state-space learning, selective attention gating, and bidirectional time-frequency analysis to improve speech intelligibility and perceptual quality.

2.5 Dual-path HOII with time-frequency attention (DPH-TFA)

Transformer-based dual-path models are effective in capturing temporal and spectral dependencies (Dang et al., 2022); however, conventional MHSA (Rao et al., 2022) and localized alternatives, including Local Dense Synthesizer Attention (LDSA) (Wei et al., 2023; Xu et al., 2021), either suffer from high complexity and insufficiently model joint long-range cross-axis interactions, resulting in degraded harmonic preservation and residual noise artifacts. To address this problem, we propose a Dual-Path HOII module with combined Time-Frequency Attention (DPH-TFA). The proposed DPH-TFA is a task-oriented redesign of the conventional IIFC-Net (Wei et al., 2023) rather than a completely new interaction module. Specifically, the original Local Dense Synthesizer Attention (LDSA) mechanism is replaced with Time-Frequency Attention (TFA) to enable explicit temporal-frequency saliency modeling for speech enhancement while preserving the higher-order interaction capability of the original architecture. Unlike conventional multi-head self-attention mechanisms, HOII advances the dual-path structure to facilitate structured higher-order interactions between temporal and spectral features. HOII facilitates deeper cross-dimensional feature coupling and enhances residual noise suppression while preserving fine-grained harmonic structures. This leads to improved contextual modeling and significantly enhances overall quality. Furthermore, a Time-Frequency Attention (TFA) module is integrated within DPH-TFA to explicitly refine joint temporal-spectral saliency. Unlike LDSA, which focuses on localized dense synthesis, TFA (Zhang et al., 2022a) adaptively emphasizes speech-dominant regions across both axes while suppressing noise-dominant activations. This combined design enhances harmonic preservation, strengthens cross-domain dependency modelling, and maintains the speech quality and intelligibility suitable for robust single-channel speech enhancement.

Although both TF-Mamba and DPH-TFA are capable of modeling long-range dependencies, they are designed with fundamentally different representation-learning objectives. TF-Mamba progressively propagates long-range temporal-frequency contextual information through selective state-space modeling, thereby enhancing global contextual consistency during hierarchical feature extraction and reconstruction. Unlike TF-Mamba, which primarily performs contextual propagation, DPH-TFA performs higher-order semantic interaction in addition to long-range dependency modeling. The Higher-Order Information Interaction (HOII) mechanism captures complex semantic interactions between temporal and spectral representations, while the integrated Time-Frequency Attention (TFA) explicitly emphasizes speech-dominant time-frequency regions and suppresses noise-dominant activations. Consequently, TF-Mamba primarily performs global contextual representation learning, whereas DPH-TFA performs discriminative semantic interaction together with speech-oriented time-frequency feature enhancement. Their complementary representation-learning objectives justify the necessity of incorporating both modules within the proposed framework.

To determine an appropriate bottleneck depth, we evaluated the proposed DPH-TFA module using 2, 3, 4, and 5 sequential DPH-TFA blocks while keeping all other architectural components and training settings unchanged. Increasing the number of DPH-TFA blocks from two to four progressively refines the bottleneck feature representations through repeated higher-order information interaction, resulting in consistent improvements in speech quality, intelligibility, and distortion reduction. However, increasing the depth beyond four provides only marginal performance improvements while introducing additional model complexity. Therefore, four sequential DPH-TFA blocks were adopted in the proposed TF-Mamba-DPHNet because they provide the most favorable trade-off between enhancement performance and model complexity.

The DPH-TFA module, as illustrated in Figure 5a, captures long-range correlations along both temporal and frequency dimensions, integrating four sequential DPH-TFA blocks D P H i , i = 1 , . . . , 4 in the TF-Mamba-DPHNet. Each block contains a SeparableConv2D layer with PReLU, two HOINets, and Layer Normalization (LN) for stable training. The dual-path design processes the input first along the time axis and then along the frequency axis, which enables HOINet to model the dependencies across time and frequency while treating one axis as feature channels. This strategy, inspired by the dual-path recurrent networks (Luo et al., 2020) and the Dual-Path HOIIFormer (Wei et al., 2023), is able to efficiently capture intra-time and intra-frequency dependencies, expand the receptive field, and preserve fine-grained details through residual connections with no quadratic cost of self-attention.

The HOII module integrates within the dual-path bottleneck is an intentional architectural decision. The encoder stages (MSFE blocks) are designed to capture speech representations at multiple receptive fields, enabling the extraction of both fine-resolution spectro-temporal details and broader contextual dependencies. At this stage, feature representations are still spatially broad and semantically evolving. Applying higher-order interaction modelling too early may be suboptimal, as the representations have not yet accumulated sufficient contextual abstraction to effectively leverage nonlinear cross-channel dependencies. In addition, introducing strong nonlinear interactions at shallow layers may destabilize enhancement behaviour, particularly for low-energy speech components that are essential for intelligibility and subtle perceptual cues.

The HOII is placed inside the dual-path bottleneck; it processes compact and contextually enriched representations, where global dependencies across time and frequency have been progressively aggregated. The dual-path mechanism separates modelling along time-frequency dimensions, enabling structured dependency learning tailored to speech signals. Embedding the HOII module within this dual-path decomposition enables structured higher-order interactions across temporal and frequency dimensions. Such a design improves long-range temporal consistency and cross-frequency dependency modeling, which enables more stable speech enhancement under non-stationary noise conditions. The proposed structured interaction mechanism effectively reduces the residual artifacts, improves speech intelligibility in low-SNR scenarios, and improves overall perceptual quality. As the decoder is mainly devoted to resolution recovery and fine-detail reconstruction, placing HOII in the decoder, which primarily focuses on resolution recovery and fine-detail reconstruction, could disturb the delicate reconstruction process and introduce unnecessary complexity at the refinement stage. Therefore, positioning HOII within the dual-path bottleneck ensures effective global context modeling while maintaining stable and perceptually consistent speech enhancement performance.

The Higher-Order Interaction Network (HOINet) in Figure 5b is a sequential pipeline of HOII, time-frequency attention (TFA) module, bidirectional gated recurrent unit (Bi-GRU), dense projection, and residual Layer Normalization. HOII provides multiplicative and cross-channel interactions beyond pairwise attention. HOII enhances the local contextual features by gated feature mixing. The TFA module generates a lightweight 2-D attention map to highlight speech-dominant regions and suppress noise (Zhang et al., 2022a; Zhang et al., 2022b). The Bi-GRU models order-aware temporal patterns in both directions with linear complexity. The dense layer projects the features back to the original size, and residual addition preserves gradient flow. Compared with Conformer-U-Net architectures such as SE-Conformer (Kim and Seo, 2021), which integrate convolutional modules with multi-head self-attention, the proposed HOINet achieves a balanced integration of local and global contextual modeling while explicitly preserving speech quality and intelligibility. Although SE-Conformer benefits from global attention modeling, its reliance on pairwise self-attention may lead to over-smoothing of fine-grained harmonic components and insufficient preservation of low-energy speech cues under low-SNR conditions. Instead, HOINet improves the speech intelligibility and perceptual quality, especially under extremely non-stationary noise conditions, by enhancing the speech in the time-frequency domain, which is suitable for single-channel speech enhancement tasks.

2.6 Higher-order information interaction (HOII) block

Unlike conventional CNN- or CRN-based speech enhancement models that mainly learn local spectro-temporal correlations (Zhao et al., 2018; Pandey and Wang, 2019) and the RNN-based approaches that model sequential dependencies but suffer from gradient degradation and latency (Luo et al., 2020), the proposed High-Order Information Interaction (HOII) module explicitly models higher-order nonlinear dependencies beyond pairwise correlations. The HOII block overcomes the limitations of the dual-path transformer-based model (Dang et al., 2022) by introducing structured multiplicative and cross-channel interactions, which enable the network to learn richer representations that boost speech intelligibility and perceptual quality, while maintaining robustness against highly non-stationary noise.

The input feature map x ∈ R T × F × C is first projected by a SeparableConv2D layer to increase the channel dimension x 0 ∈ R T × F 2 × C 1 as shown in Figure 6, thus increasing the capacity of the representation. The expanded features are then divided into two branches: a main feature branch f 1 ∈ R T × F × C 1 and a reference (gating) branch r ∈ R * × C 1. The latter is processed by a depth-wise convolution (kernel size 7 × 7) and divided into n hierarchical sub-features g 1 , g 2 , . . , g n . A depth-wise convolution with a large receptive field is used in the reference branch to extract larger context information. Then it is divided into multiple hierarchical sub-features that act as progressive gating signals. These gating components recursively interact with the primary branch through multiplicative modulation, enabling structured refinement of higher-order features. Lightweight separable convolutions are applied at each stage to further refine the updated representation. Channel projection is applied to get the final output to maintain the dimensional consistency.

This hierarchical interaction strategy enables HOII to progressively learn to incorporate local detail and global context cues while suppressing the irrelevant activations. Compared to the usual convolutional or transformer-based modules, HOII improves nonlinear feature coupling without the quadratic attention complexity. Empirical evaluations show consistent improvements in PESQ and STOI over CNN, RNN and Transformer-based baselines, corroborating that HOII effectively extracts context-aware representations under non-stationary noise conditions.

2.6.1 Comparison with related higher-order modeling approaches

The distinction between the proposed HOII mechanism and related studies, including IIFC-Net (Wei et al., 2023) and DPHT-ANet (Saleem et al., 2024). Although higher-order interaction was explored in IIFC-Net (Wang et al., 2025), it was incorporated within a limited convolutional backbone and primarily coupled to its original calibration strategy. In the proposed framework, HOII is reformulated within a hierarchical MSFE-D3Net encoder-decoder architecture and implemented using depth-wise separable convolutions for parameter efficiency and improved spectral sensitivity. Rather than acting as an isolated enhancement unit, HOII is embedded within stacked dual-path time-frequency blocks at the U-Net bottleneck, allowing it to operate on multi-scale representations and contribute to a progressive local-to-global modeling process. HOII-related approach is DPHT-ANet (Saleem et al., 2024), which introduces high-order modeling through transformer-style recursive gated convolution; however, its interaction mechanism is realized mainly through global attention aggregation. It does not explicitly structure higher-order interaction as multiplicative cross-channel modulation, nor does it integrate it with cross-scale skip calibration inside a hierarchical encoder–decoder pathway. By contrast, the proposed design couples structured channel-wise multiplicative interaction with multi-scale encoding and adaptive skip refinement (H-CS-FC), enabling coordinated feature propagation across scales and improving reconstruction stability under non-stationary noise. Finally, the HOII module is termed a structured multiplicative cross-channel interaction mechanism because it introduces learnable element-wise multiplicative modulation across feature channels under constrained parameterization, enabling higher-order dependency modeling while strengthening harmonic consistency, reducing spectral distortion, and improving perceptual speech quality and intelligibility under adverse noise conditions.

2.7 Hybrid cross-scale feature calibration (H-CS-FC) mechanism

In conventional U-Net-based speech enhancement frameworks (Cao et al., 2022), skip connections directly concatenate encoder and decoder features to retain fine-grained spectro-temporal details. However, under low-SNR conditions, such direct transmission may inadvertently propagate noise-dominant, task-irrelevant components, thereby limiting perceptual quality and intelligibility (Yecchuri and Dayal Vanambathina, 2024). To mitigate this limitation, we introduce a lightweight bottleneck-guided FC module that performs bottleneck-guided feature modulation instead of explicit attention-based fusion. Specifically, a compact spatial-channel gating mask is generated from the semantically enriched bottleneck representation using separable and depth-wise convolutions, followed by Layer Normalization and sigmoid activation, as shown in Figure 7a. The encoder feature is then recalibrated through element-wise multiplication with this mask before skip propagation. The design allows for structured reliability filtering of encoder activations, while maintaining architectural simplicity and interpretability.

In contrast to the attention-driven skip refinement models (Xu and Jianjun, 2022; Xiang et al., 2021b), the proposed feature calibration module utilizes linear-complexity modulation without the construction of attention matrices, and explicitly focuses on suppressing the propagation of noise over skip pathways. In contrast to direct stacking, FC ensures structured semantic refinement and improves the quality of speech reconstruction.

To further strengthen hierarchical consistency, FC is extended to a hierarchical hybrid cross-scale feature calibration (H-CS-FC) mechanism, as shown in Figure 7b. While the original FC utilizes global bottleneck semantics for modulation, H-CS-FC additionally incorporates contextual guidance from the adjacent deeper encoder level. This design introduces structured cross-scale interaction while maintaining the lightweight modulation-based philosophy.

In hierarchical H-CS-FC, both the bottleneck feature and the deeper encoder feature are spatially aligned and channel-projected before fusion. Their element-wise summation yields a hybrid contextual representation that captures global semantics together with neighbouring hierarchical cues. A modulation mask derived from this hybrid representation recalibrates the target encoder feature before skip transmission. Unlike the conventional FC module, which performs feature calibration using only bottleneck semantic representations, the proposed H-CS-FC additionally incorporates contextual information from the adjacent deeper encoder stage to generate a complementary cross-scale calibration mask. Consequently, H-CS-FC serves as a refinement module that enhances hierarchical feature propagation during decoder reconstruction rather than functioning as the primary representation-learning component. By combining global bottleneck semantics with neighbouring hierarchical information through structured gated modulation rather than dense feature concatenation, H-CS-FC improves the consistency of skip-feature fusion while introducing only a marginal increase in model complexity. The effectiveness of this complementary refinement is further demonstrated in the ablation study, where H-CS-FC consistently improves the objective evaluation metrics over the conventional FC module.

2.8 Loss function

For robustness to outliers and stable optimization, the proposed framework uses the Huber loss (Hung et al., 2024) as the objective to train. The Huber loss incorporates the advantages of mean squared error (MSE) and mean absolute error (MAE) by using a quadratic penalty for small residuals and a linear penalty for large residuals. This maintains the good gradient properties of MSE for small prediction errors and limits the influence of large deviations that may be caused by severe noise corruptions or transient artifacts in speech signals.

The Huber loss is formally expressed as,

L δ ( Z , Z ^ ) = 1 2 Z − Z ^ 2 , f o r Z − Z ^ ≤ δ δ . ( Z − Z ^ − 1 2 δ , f o r Z − Z ^ > δ (8)

Where Z is the actual target value and Z ^ is the predicted value (Equation 8). δ is a threshold parameter that defines the transition between quadratic and linear behaviour. For residuals smaller than δ, the loss behaves exactly as MSE, giving smooth gradients and enabling accurate spectral reconstruction. For larger residuals, the loss becomes linear like MAE, which reduces the sensitivity to outliers and prevents excessively large gradients. This formulation improves robustness in highly non-stationary noise conditions in comparison with pure MSE and provides continuity of gradients in the neighborhood of zero in comparison with MAE improving the stability of convergence during training. The Huber loss provides a stable and balanced objective for supervised speech enhancement, which allows for robust spectral estimation and reduces the impact of extreme errors in predictions.

3 Results and discussion

3.1 Datasets

The robustness of the proposed model TF-Mamba-DPHNet with FC was examined on the Mozilla Common Voice Corpus 22.0 (CommonVoice. Mozilla, 2017) and the LibriSpeech dataset (Panayotov et al., 2015) under various noise conditions. To verify the architectural refinement under a consistent experimental setting, the experiments were performed on the LibriSpeech dataset (Panayotov et al., 2015) using the model TF-Mamba-DPHNet with hierarchical H-CS-FC. Furthermore, to comprehensively evaluate the generalization capability of the proposed framework and facilitate comparisons with recently published state-of-the-art speech enhancement methods, additional experiments were conducted on the widely used VoiceBank + DEMAND (VCTK) benchmark dataset (Veaux et al., 2013; Thiemann et al., 2013).

Common Voice comprises approximately 1.6 million utterances from 97,925 speakers. From this corpus, 95,000 utterances were randomly selected and partitioned into disjoint training (55,000), validation (20,000), and testing (20,000) subsets. The test set was obtained from the official Common Voice test partition, and speaker-level separation was strictly enforced to prevent possible data leakage between splits. This partitioning strategy allows for sufficient diversity for model learning and statistically reliable validation and test sets. We consider two evaluation conditions for evaluating the generalization capability: (i) seen noise, including street, restaurant, and babble from NOIZEUS (NOIZEUS, 2026; Ma et al., 2009); and (ii) unseen noise, including airport, exhibition, and train station. Performance was assessed using standard objective metrics, namely, perceptual evaluation of speech quality (PESQ) (Recommendation, 2001), short-time objective intelligibility (STOI) (Taal et al., 2011), and signal-to-distortion ratio (SDR) (Vincent et al., 2006).

The experiments were performed on LibriSpeech; the training set was the train-clean-100 subset (100 h) with 20,000 noise samples added from the DNS Challenge dataset (Du et al., 2024). We evaluated the dev-clean (1 k utterances) and test-clean (1.2 k utterances) subsets. Representative background noise conditions were chosen from the DEMAND corpus (Thiemann et al., 2013) (Street Cafe and Nature Park) and babble noise from NOISEX-92 (NOIZEUS). All signals were resampled to 16 kHz for consistency. This experimental setup includes various speech and noise conditions. To further validate the generalization capability of the proposed framework and facilitate comparison with recently published state-of-the-art methods, additional experiments were conducted on the VoiceBank + DEMAND (VCTK) benchmark dataset (Veaux et al., 2013; Thiemann et al., 2013). The clean speech recordings from the VCTK corpus were corrupted with environmental noises from the DEMAND database following the standard VoiceBank + DEMAND evaluation protocol. The training set contains 11,572 noisy-clean utterance pairs from 28 speakers, while the test set consists of 824 utterances from 2 unseen speakers. The proposed TF-Mamba-DPHNet was trained and evaluated using the same preprocessing configuration and objective evaluation metrics (PESQ, STOI, and SDR), enabling a fair comparison with recently published speech enhancement methods.

3.2 Experimental setup and baseline models

All speech recordings were resampled to 16 kHz. For feature extraction, each utterance was processed using a 512-point Hanning window with a hop size of 128 to generate time-frequency representations. The proposed model was compared against several baseline approaches, including DCCRN (Hu et al., 2020), MCGN (Xian et al., 2020), MASENet (Xiang et al., 2021a), SADNUNet (Xiang et al., 2021b), MANNER (Park et al., 2022), TFA-S-TCN (Jannu and Vanambathina, 2023), HATFANet (Zhang et al., 2024), ROSE-Net (Yu et al., 2024), and CNUNet (Hwang et al., 2025). To further strengthen the comparison with recent state-of-the-art methods, additional comparisons on the VoiceBank + DEMAND benchmark were conducted with TFDense-GAN (Chen et al., 2025) and DMC-Net (Saleem et al., 2025). The proposed network was optimized using the Adam optimizer (Kingma and Jimmy, 2014) with an initial learning rate of 1 × 10⁻³, a batch size of 16, and a maximum of 100 training epochs. Early stopping with a patience of 10 epochs was applied based on validation loss. For data synthesis, clean speech signals were mixed with 200 distinct noise types, with SNR levels uniformly sampled from −5 dB to +10 dB (−5, 0, +5, 10 dB). The same preprocessing configuration and evaluation metrics were adopted for the VoiceBank + DEMAND experiments to ensure a fair and consistent comparison across all benchmark datasets.

3.3 Ablation study of the TF-Mamba-DPHNet

The contribution of each architectural component to the overall enhancement performance using the Common Voice Corpus under seen noise conditions is summarized in Table 1. The U-Net (Macartney and Weyde, 2018) as a baseline yields PESQ, STOI, and SDR of 2.42, 84.10%, and 9.30 dB, respectively, and is used as a reference performance without any special modules.

TABLE 1. Ablation study of the TF-Mamba-DPHNet model.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

TABLE 2. Averaged PESQ results for all baseline systems evaluated under seen-noise conditions. The proposed TF-Mamba-DPHNet model is emphasized in bold italics to indicate superior performance.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

The addition of the MSFE block improves the PESQ from 2.42 to 2.79, the STOI from 84.10% to 88.00%, and the SDR from 9.30 dB to 10.55 dB, demonstrating that multi-scale feature extraction effectively enhances the representation of speech structures under noisy conditions. Introducing TF-Mamba into the MSFE-based U-Net further increases the performance to 3.06 PESQ, 90.45% STOI, and 12.18 dB SDR, confirming the effectiveness of progressive long-range temporal-frequency contextual refinement through selective state-space modeling. The complementary contribution of DPH-TFA is further verified through the ablation results. Employing DPH-TFA alone improves the performance to 3.31 PESQ, 91.73% STOI, and 12.64 dB SDR, demonstrating that higher-order semantic interaction together with speech-oriented Time-Frequency Attention (TFA) provides additional discriminative feature representations beyond contextual modeling. More importantly, simultaneously integrating TF-Mamba and DPH-TFA achieves 3.50 PESQ, 93.55% STOI, and 14.35 dB SDR, consistently outperforming the corresponding variants employing either module individually. These results confirm that TF-Mamba and DPH-TFA provide complementary contextual and semantic representations rather than merely increasing model capacity.

Furthermore, progressively increasing the number of DPH-TFA blocks from two to four yields consistent performance improvements, while replacing HOII with conventional multi-head self-attention results in lower performance, demonstrating the effectiveness of higher-order interaction learning over standard attention mechanisms. Increasing the number of DPH-TFA blocks beyond four provides only marginal improvement, indicating that four DPH-TFA blocks offer a favorable balance between representation capability and model complexity. Finally, incorporating the conventional FC module further improves the performance to 3.73 PESQ, 95.04% STOI, and 15.76 dB SDR, while replacing FC with the proposed H-CS-FC achieves the best overall performance of 3.79 PESQ, 95.16% STOI, and 15.85 dB SDR. Compared with the conventional FC module, H-CS-FC consistently improves all three objective evaluation metrics, increasing PESQ from 3.73 to 3.79, STOI from 95.04% to 95.16%, and SDR from 15.76 dB to 15.85 dB. Although these numerical improvements are relatively moderate, they are consistently observed across complementary speech quality, intelligibility, and distortion metrics. Since H-CS-FC is intended as a lightweight hierarchical feature refinement module rather than the principal representation-learning component, these consistent gains indicate that incorporating complementary cross-scale contextual information provides additional refinement during decoder reconstruction with only a negligible increase in model complexity. Despite integrating all proposed modules, the complete framework contains only 1.40 M trainable parameters. Overall, the ablation study demonstrates that each architectural component contributes incrementally, while the complementary integration of MSFE, TF-Mamba, DPH-TFA, FC, and H-CS-FC collectively improves speech quality, intelligibility, and distortion reduction.

3.4 Assessment of baseline performance under seen noise conditions

The proposed model is evaluated under trained noise environments (Restaurant, Babble, and Street) at −5, 0, 5, and 10 dB SNR. Tables 2–4 report averaged PESQ, STOI, and SDR results in comparison with established baselines.

TABLE 3. Average STOI values of all baseline systems under seen-noise conditions. The proposed TF-Mamba-DPHNet model is highlighted in bold italics to show its better performance.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

TABLE 4. Average SDR scores for all baseline systems evaluated on seen-noise conditions. In bold italics, we represent the proposed TF-Mamba-DPHNet model, which demonstrates better performance.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

TABLE 5. PESQ scores of baseline models evaluated under unseen noise conditions. The proposed TF-Mamba-DPHNet model is highlighted in bold italics.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

The unprocessed noisy speech (PESQ 1.61, STOI 64.69%, SDR 4.74 dB) results in the lowest performance, confirming the severity of distortion under adverse noise. DCCRN (Hu et al., 2020) adopts complex domain recurrent modeling to boost temporal consistency (PESQ 2.36) in learning-based methods, but does not have explicit multi-scale or attention mechanisms. Multi-scale convolutional refinement is proposed in MCGN (Xian et al., 2020) and MASENet (Xiang et al., 2021a), but mainly depends on the first-order feature aggregation without structured cross-feature coupling. SADNUNet (Xiang et al., 2021b) and CNUNet (Park et al., 2022) use nested skip connections to promote feature propagation. However, the dense structures may cause redundant feature transmission and residual noise propagation, which may affect the enhancement performance in severe noise situations. Attention-centric models such as MANNER (Park et al., 2022), TFA-S-TCN (Jannu and Vanambathina, 2023), and HATFANet (Zhang et al., 2024) enable contextual calibration but do not explicitly consider the multi-scale enrichment or higher-order feature interaction modeling. ROSE-Net (Yu et al., 2024) improves skip fusion by adding attention, but does not explicitly model structured cross-scale or higher-order coupling.

In contrast, the proposed TF-Mamba-DPHNet integrates: (i) MSFE-D3Net multi-scale enrichment for hierarchical representation learning, followed by TF-Mamba modules that employ selective state-space modeling to capture long-range time–frequency dependencies with linear complexity while preserving spectral continuity; (ii) stacked dual-path HOII-based bottleneck modeling for structured higher-order temporal–frequency interactions; and (iii) adaptive H-CS-FC-based skip calibration for hierarchical cross-scale feature alignment and controlled information fusion, enabling refined reconstruction and improved global consistency across decoding stages.

The unified local-context-global enhancement path empowered by TF-Mamba-based long-range time-frequency dependency modeling and hierarchical cross-scale skip calibration (H-CS-FC) achieves the best overall performance of PESQ 3.79, STOI 95.16% and SDR 15.85 dB, surpassing the strongest baseline (CNUNet) by +0.39 PESQ, +2.40% STOI and +3.04 dB SDR. Interestingly, the improvements are uniform for all SNRs, with higher gains at low SNR, which indicates better robustness and more efficient structured feature coupling. These results confirm that explicitly combining multi-scale enrichment, TF-Mamba-based global modeling, dual-path higher-order interaction learning, and calibrated hierarchical skip fusion provides measurable advantages over attention-only or dense-only enhancement strategies.

3.5 Comparison with baseline models under unseen noise conditions

The comparison of performance under unseen airport, exhibition, and train noise conditions are presented in Tables 5–7. The PESQ, STOI, and SDR of the conventional convolution and attention models, e.g., DCCRN (Hu et al., 2020), MCGN (Xian et al., 2020), and MASENet (Xiang et al., 2021a), are progressively improved, but their limited long-range spectral-temporal modeling ability leads to noticeable degradation at −5 dB SNR. Advanced architectures, e.g., SADNUNet (Xiang et al., 2021b), MANNER (Park et al., 2022), TFA-S-TCN (Jannu and Vanambathina, 2023), HATFANet (Zhang et al., 2024), ROSE-Net (Yu et al., 2024), and CNUNet (Hwang et al., 2025), with multi-scale and attention mechanisms, also improve robustness. Among these, CNUNet achieves the strongest baseline performance with average scores of PESQ 3.36, STOI 92.55%, and SDR 12.43 dB. Nevertheless, slight performance drops are still observed under extremely low-SNR conditions, indicating incomplete preservation of fine spectral continuity and weak speech components.

TABLE 6. STOI scores (%) of baseline models evaluated under unseen noise conditions. The proposed TF-Mamba-DPHNet model is highlighted in bold italics.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

TABLE 7. SDR scores (dB) of baseline models evaluated under unseen noise conditions. The proposed TF-Mamba-DPHNet model is highlighted in bold italics.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

In contrast, the proposed TF-Mamba-DPHNet (with the H-CS-FC proposed model) achieves the highest overall performance across all unseen conditions, obtaining average scores of PESQ 3.72, STOI 94.97%, and SDR 15.76 dB. Compared with CNUNet, this corresponds to improvements of +0.36 PESQ, +2.42% STOI, and +3.33 dB SDR. The improvements are especially significant at −5 dB SNR, which is indicative of an improvement in robustness in very adverse conditions. These consistent improvements validate that integrating multi-scale MSFE enrichment, TF-Mamba-based long-range time-frequency modeling, dual-path higher-order interaction learning, and H-CS-FC enables more structured spectral-temporal representation learning and superior generalization to unseen noise conditions.

3.6 Comparison of existing and proposed models trained on LibriSpeech dataset

The PESQ, STOI, and SDR results of all models at −5, 0, 5, and 10 dB SNR for all three seen noise conditions (Street Café, Babble, and Nature Park) are shown in Tables 8–10 using the LibriSpeech dataset. As expected, the performance on the unprocessed noisy mixture is the worst, with overall averages of 1.57 PESQ, 64.21% STOI and 4.62 dB SDR, indicating a significant degradation in perceived quality and intelligibility. Among the baseline systems, DCCRN (Hu et al., 2020) achieves better performance of 2.51 PESQ, 84.35% STOI and 10.25 dB SDR using complex-domain recurrent modeling. Multi-scale convolutional models, such as MCGN (Xian et al., 2020) and MASENet (Xiang et al., 2021a), and nested U-Net structures, such as SADNUNet (Xiang et al., 2021b) and CNUNet (Hwang et al., 2025), further improve the reconstruction quality, especially by enhancing cross-scale feature aggregation. CNUNet achieves the best baseline performance (3.47 PESQ, 92.85% STOI, and 12.96 dB SDR). Further gains are achieved from attention-augmented methods such as MANNER (Park et al., 2022), TFA-S-TCN (Jannu and Vanambathina, 2023), HATFANet (Zhang et al., 2024), and ROSE-Net (Yu et al., 2024), suggesting better contextual modelling. However, these methods mainly rely on attention or dense aggregation mechanisms, without explicitly modeling structured higher-order temporal-frequency interactions.

TABLE 8. PESQ scores of all baseline models under seen noise conditions using the LibriSpeech dataset. The proposed TF-Mamba-DPHNet model is highlighted in bold italics.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

TABLE 9. STOI scores of all baseline models under seen noise conditions using the LibriSpeech dataset. The proposed TF-Mamba-DPHNet model is highlighted in bold italics.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

TABLE 10. SDR scores of all baseline models under seen noise conditions using the LibriSpeech dataset. The proposed TF-Mamba-DPHNet model is highlighted in bold italics.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

On the other hand, the proposed TF-Mamba-DPHNet achieves the best performance in all metrics and SNR levels. The FC version gives 3.82 PESQ, 95.32% STOI, and 16.13 dB SDR. The performance is further improved to 3.87 PESQ, 95.56% STOI, and 16.41 dB SDR after adding hierarchical hybrid cross-scale feature calibration (H-CS-FC). Compared to the strongest baseline (CNUNet), the proposed model achieves improvements of +0.40 PESQ, +2.71% STOI, and +3.45 dB SDR. It is worth noting that the improvements are consistent across all SNR levels and are particularly prominent at −5 and 0 dB, indicating improved robustness under severe noise conditions. These results confirm that combining multi-scale feature enrichment, TF-Mamba-based long-range temporal-frequency modeling, dual-path higher-order interaction learning, and H-CS-FC calibrated skip fusion leads to structured representation learning and superior enhancement performance over convolutional, attention-only, and nested U-Net-based approaches.

3.7 Comparison of existing and proposed models trained on LibriSpeech dataset

The proposed TF-Mamba-DPHNet is compared with representative CNN-, attention-, and Transformer-based speech enhancement models in terms of objective performance and computational complexity in Table 11. In addition to PESQ, STOI, and SDR, the comparison includes the number of trainable parameters, multiply-accumulate operations (MACs), inference latency on the CPU platform, and memory consumption. MACs are reported as the primary computational complexity metric because they are widely adopted for deep neural network evaluation. Under the conventional definition, the corresponding FLOPs can be approximated as twice the number of MACs, allowing direct interpretation of the computational cost. The proposed TF-Mamba-DPHNet contains 1.40 M trainable parameters with a model memory footprint of 5.36 MB while achieving superior enhancement performance (PESQ of 3.87, STOI of 95.56%, and SDR of 16.41 dB). Although the integration of TF-Mamba and DPH-TFA modules increases the computational cost compared with the lightweight MSFE-only architecture, the overall model complexity remains substantially lower than many existing CNN- and Transformer-based methods while consistently providing better speech enhancement performance. These results demonstrate that the proposed architecture achieves an effective balance between enhancement quality and computational complexity.

TABLE 11. Comprehensive comparison of objective performance and computational efficiency of baseline speech enhancement models on LibriSpeech seen noise.

Bold values indicate the best performance, and italic values indicate the proposed TF-Mamba-DPHNet model.

3.8 Comparison with recent state-of-the-art methods on the VoiceBank + DEMAND benchmark

To further strengthen the comparative evaluation with recent speech enhancement approaches, additional experiments were conducted on the VoiceBank + DEMAND (VCTK) benchmark. Besides the representative methods already included in the original manuscript, SE-Conformer (Kim and Seo, 2021), a representative Transformer-based speech enhancement model, together with two recently published methods, TFDense-GAN (Chen et al., 2025), a Transformer-based speech enhancement framework, and DMC-Net (Saleem et al., 2025), were incorporated into the performance comparison. The evaluation was performed using the same objective metrics, namely, PESQ, STOI, and SDR.

Table 12 presents the comparison between the proposed TF-Mamba-DPHNet and recent speech enhancement methods on the VoiceBank + DEMAND benchmark. Compared with SE-Conformer (2021), HATFANet (2024), ROSE-Net (2024), CNUNet (2025), TFDense-GAN (2025), and DMC-Net (2026), the proposed model achieves the highest PESQ (3.81) and SDR (16.58 dB) while requiring only 1.40 M trainable parameters. Although TFDense-GAN reports a slightly higher STOI (97.00%), the proposed method achieves higher speech quality and distortion reduction with substantially lower model complexity than several existing methods. Furthermore, compared with the Transformer-based SE-Conformer, the proposed TF-Mamba-DPHNet improves both PESQ and STOI, demonstrating the effectiveness of combining selective state-space modeling with higher-order semantic interaction for single-channel speech enhancement. These results demonstrate the competitiveness of the proposed architecture on the widely adopted VoiceBank + DEMAND benchmark.

TABLE 12. Comparison of the proposed TF-Mamba-DPHNet with state-of-the-art speech enhancement methods on the VoiceBank + DEMAND (VCTK) benchmark.

“- “represents that the results are not provided in the associated papers.

3.9 Interpretability analysis of the proposed TF-Mamba-DPHNet

Figures 8–10 present representative speech waveform and spectrogram comparisons under different datasets and noise conditions. Figure 8 illustrates the enhancement results for Common Voice English corrupted with Babble noise at 0 dB SNR, Figure 9 presents the corresponding results at 10 dB SNR, and Figure 10 shows the enhancement performance on LibriSpeech-train corrupted with NOISEX-92 noise at 10 dB SNR. Across all evaluation scenarios, the enhanced speech exhibits effective background noise suppression while preserving harmonic structures, speech formants, and spectral continuity. These qualitative visualizations complement the quantitative evaluation by illustrating the speech enhancement capability of the proposed TF-Mamba-DPHNet under different datasets and noise conditions.

Figure 11 presents representative intermediate feature maps extracted from the proposed TF-Mamba-DPHNet to illustrate the progressive evolution of learned feature representations. Warm colors (yellow, orange, and red) indicate higher activation responses corresponding to informative speech-dominant regions, whereas cool colors (blue and cyan) represent lower activation responses associated with suppressed noise components. Feature Map 1 (Input + MSFE) illustrates the initial multi-scale speech representations extracted from the noisy input. Feature Map 2 (U-Net + MSFE + TF-Mamba) shows refined contextual representations after long-range temporal-frequency modeling. Feature Map 3 (U-Net + MSFE + TF-Mamba + DPH-TFA) demonstrates enhanced speech-dominant feature representations through higher-order interaction and time-frequency attention. Feature Map 4 (Proposed TF-Mamba-DPHNet with H-CS-FC) exhibits the final refined feature representations after hierarchical cross-scale feature calibration, showing improved preservation of speech structures and suppression of residual noise. Together, these visualizations illustrate the complementary roles of the proposed modules and demonstrate how TF-Mamba-DPHNet progressively transforms noisy speech into high-quality enhanced speech through successive feature refinement.

3.9.1 Qualitative spectrogram comparison and harmonic structure analysis

A new qualitative spectrogram comparison has been added as Figure 12 to illustrate the progressive enhancement process of the proposed TF-Mamba-DPHNet. The figure presents the noisy speech spectrogram, followed by the intermediate spectrogram representations generated by the MSFE, TF-Mamba, DPH-TFA, and H-CS-FC modules, and finally the final enhanced speech spectrogram. All results correspond to the same speech utterance from the Common Voice English corpus corrupted with NOISEX-92 Restaurant noise at 0 dB SNR. This visualization provides intuitive evidence of how each proposed module progressively contributes to speech enhancement.

As shown in Figure 12a, the input noisy speech contains noticeable background noise that affects the continuity of the speech spectral patterns. After processing by the MSFE module in Figure 12b, prominent speech spectral structures become more distinguishable while preserving important local time-frequency information. Incorporating TF-Mamba in Figure 12c further improves the continuity of the spectral patterns by modeling long-range temporal-frequency dependencies. Following this, the DPH-TFA module in Figure 12d enhances speech-dominant spectral regions while further suppressing residual noise through higher-order interaction and time-frequency attention. The H-CS-FC module shown in Figure 12e provides additional cross-scale feature calibration, resulting in more consistent spectral representations and reduced interference. Finally, the enhanced speech spectrogram in Figure 12f exhibits clearer speech spectral components, improved spectral continuity, better preservation of speech characteristics, and effective background noise reduction.

These qualitative observations are consistent with the corresponding improvements in PESQ, STOI, and SDR, providing visual evidence of the complementary contributions of the proposed MSFE, TF-Mamba, DPH-TFA, and H-CS-FC modules toward progressive speech enhancement.

4 Discussion

4.1 Effect of TF-Mamba

Experimental results demonstrate that the proposed TF-Mamba module greatly improves the speech enhancement performance under both seen and unseen noise conditions. Compared with traditional convolutional architectures, TF-Mamba uses selective state-space modeling to efficiently model long-range temporal and spectral dependencies with linear computational complexity. This capability is particularly beneficial in speech enhancement, where important speech components may be distributed across distant time-frequency regions. Thus, the long-term features of speech are better retained, while the effects of noise-dominant regions are reduced. The consistent improvements in PESQ, STOI and SDR show that TF-Mamba improves spectral continuity and enhances the reconstruction of weak speech components, especially in low-SNR conditions. The results indicate that selective state-space modeling is a more efficient way of encoding global speech context than conventional attention-based approaches.

4.2 Effect of dual-path higher-order information interaction (DPH-TFA)

The ablation study shows that the Dual-Path Higher-Order Information Interaction (DPH-TFA) module is important for performance improvement. Traditional self-attention mechanisms model the pairwise feature interactions but may not be able to capture complex nonlinear interactions common between speech and noise components. To address this limitation, the proposed DPH-TFA module employs higher-order information interaction modeling through HOII blocks along both temporal and frequency dimensions. DPH-TFA separately models temporal and frequency dependencies and then integrates them via dual-path interaction learning, allowing the network to capture structured correlations in multiple feature dimensions. The superior performance of DPH-TFA over conventional multi-head self-attention demonstrates that the higher-order interaction modeling results in more discriminative representations for speech enhancement. Moreover, the increasing DPH-TFA depth shows a progressive improvement, indicating that modeling hierarchical higher-order features is helpful for effective speech-noise separation and robust representation learning.

4.3 Effect of hybrid cross-scale feature calibration (H-CS-FC)

The performance of the proposed H-CS-FC mechanism is further improved by improving the flow of skip-connection information in the U-Net framework. Traditional skip connections directly propagate encoder features to decoder stages, which may inadvertently transfer noise-related activations together with useful speech information. Such direct feature transmission becomes particularly problematic under severe noise conditions.

To overcome this limitation, H-CS-FC performs bottleneck-guided and cross-scale feature calibration before skip propagation. H-CS-FC leverages both global semantic information from the bottleneck layer and contextual information from deeper encoder stages to produce adaptive modulation masks that selectively enhance speech-relevant features and suppress noise-dominated activations. This results in scale-cross consistency, which improves the feature alignment among different hierarchies and allows a more accurate speech reconstruction. The improvements of H-CS-FC over the baseline show the effectiveness of structured feature calibration as an alternative to computationally expensive attention-based skip refinement schemes.

5 Conclusion

This paper proposes TF-Mamba-DPHNet, an encoder-decoder framework for speech enhancement that integrates MSFE, TF-Mamba, DPH-TFA, and H-CS-FC within a unified architecture. Unlike conventional U-Net models that focus mainly on local spectral reconstruction and may propagate residual noise through skip connections, the proposed design jointly addresses multi-scale representation learning, long-range dependency modeling, and guided cross-scale fusion. The MSFE is a composite of several receptive fields that are useful for improving feature extraction and differentiating between speech and noise. The TF-Mamba module uses selective state-space modeling in the time-frequency domain to more efficiently capture global contextual dependencies with fine spectral resolution. Stacked DPH-TFA blocks at the bottleneck enhance the modeling of temporal and spectral interaction, strengthening the contextual consistency. The H-CS-FC mechanism recalibrates the skip-connected features before fusion, which alleviates the noise leakage and stabilizes the reconstruction of the decoder. Experiments on the Common Voice and LibriSpeech datasets show consistent improvements on PESQ, STOI, and SDR under all SNR levels, which verify the effectiveness of the combination of selective state-space modeling, structured dual-path interaction, and calibrated cross-scale fusion for robust speech enhancement.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Farook Sattar, University of Victoria, Canada

Reviewed by: Yuxuan Zhang, China University of Mining and Technology, China

ZiPeng Zhang, Technical University of Munich, Germany