Wavelet-based features to improve cross-forgery generalization in deepfake detection

1 Introduction

Deepfakes are a class of synthetic media generated through a variety of computer-vision and graphics techniques, nowadays largely driven by deep-learning methods. These technologies can produce highly realistic manipulated visual and audiovisual content, raising serious concerns due to their potential misuse in disinformation campaigns, privacy violations, identity fraud, and other malicious activities. Notable examples reported on the Web include the Jordan Peele and BuzzFeed Obama PSA (BuzzFeed and Peele, 2018) and an AI-driven CEO vishing scam that defrauded a company of US$243,000 (Stubley, 2019).

The rapid evolution of generative models, from autoencoders (Hinton and Salakhutdinov, 2006) and Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) to more recent diffusion models (Ho et al., 2020), has made manipulated content increasingly difficult to distinguish from authentic media. As a consequence, deepfake detection has become a central problem in multimedia forensics. Although many existing detectors achieve strong performance in controlled settings, their robustness often degrades when evaluated on unseen manipulation methods, different compression levels, or varying acquisition conditions. This limited cross-forgery generalization remains one of the main open challenges in the field.one of the main challenges in the field.

Most current detection approaches operate directly in the RGB domain, learning discriminative cues from spatial appearance alone. While effective in matched train/test scenarios, such methods may partially rely on manipulation-specific artifacts and therefore generalize poorly to previously unseen forgeries. In this context, frequency-based representations provide a useful complementary perspective, as image manipulation pipelines may introduce subtle inconsistencies that are less evident in the pixel domain. Among them, the Discrete Wavelet Transform (DWT) is particularly appealing because it provides a multi-scale decomposition of the image while preserving spatial locality, thus making it possible to separately analyze coarse structural information and directional high-frequency details.

Motivated by these considerations, in this work we investigate a wavelet-based representation for deepfake detection aimed at improving robustness across forgery types. More specifically, we apply a first-level DWT to each detected face crop and use the resulting sub-bands as input to a modified ResNet-50 architecture.

The proposed representation is intentionally simple: our aim is not to introduce a new heavy detector, but to provide a controlled empirical analysis of how a native wavelet-domain input changes the behavior of a standard convolutional backbone when compared with RGB inputs. The main contributions of this work are as follows:

• We study a lightweight DWT-based input formulation for deepfake detection, where first-level wavelet sub-bands are stacked and used as input to a standard convolutional backbone with only the first layer adapted to the number of input channels.
• We evaluate the representation under a cross-forgery protocol on FaceForensics++, considering both c23 and c40 compression settings and separating in-forgery from off-diagonal cross-forgery behavior.
• We complement the original TPR/TNR tables with aggregate metrics, paired confidence intervals, and sign-flip tests over train-target pairs, showing that the DWT representation mainly improves cross-forgery fake-class sensitivity and F1 rather than overall accuracy or AUC.
• We add an RGB-112 spatial-resolution control to address the fact that the native DWT input differs from RGB not only in representation and channel count, but also in spatial resolution.

2 Related work

Deepfake detection has attracted extensive research interest in recent years, leading to a large variety of approaches based on spatial, temporal, and frequency-domain cues (Afchar et al., 2018; Sabir et al., 2019; Guarnera et al., 2020). In parallel, several public benchmarks have driven progress in the field, among which FaceForensics++ (FF++), proposed by Rössler et al. (2019), remains one of the most widely adopted datasets for both image- and video-based evaluation, thanks to its variety of manipulation methods and compression settings.

Early deepfake detectors mainly focused on learning discriminative visual artifacts directly from the spatial domain. Representative examples include MesoNet (Afchar et al., 2018) and approaches based on convolutional traces (Guarnera et al., 2020). For video analysis, Guera and Delp (2018) and Sabir et al. (2019) showed that temporal modeling can further improve detection by exploiting inter-frame inconsistencies. More recently, Transformer-based video architectures have been proposed to better capture spatio-temporal dependencies, as in the joint spatial-temporal Transformer by Zhou et al. (2021) and the motion-aware formulation by Zi et al. (2023). Ciamarra et al. (2024) also investigated temporal surface anomalies, highlighting that temporal artifacts may provide useful forensic evidence complementary to purely spatial cues.

A second important research direction has focused on improving robustness and generalization to unseen manipulation methods and datasets. In this context, Cozzolino et al. (2019) introduced ForensicTransfer, showing that domain adaptation can help bridge the gap across forgery sources. Chai et al. (2020) investigated which intrinsic image properties make manipulations detectable, offering useful insights into what may generalize beyond a specific forgery family. Other efforts addressed this problem through architectural or training innovations: Liu et al. (2021) combined spatial and phase-spectrum information to improve generalization across forgery types, while Chen et al. (2022) employed self-supervised adversarial training to reduce reliance on manipulation-specific artifacts. Yang et al. (2024) further pursued cross-manipulation and cross-dataset robustness through latent-space augmentation.

This line of research has expanded significantly in 2025. Cui et al. (2025) proposed Forensics Adapter, a CLIP-based framework that explicitly adapts the backbone to face forgery traces by focusing on blending-boundary inconsistencies. Yang et al. (2025) introduced D³, which scales training from single-generator to multi-generator settings in order to learn more universal discrepancy patterns. Yan et al. (2025) addressed video generalization with a plug-and-play strategy based on video-level blending and spatio-temporal adapter tuning, designed to capture temporal artifacts shared across different deepfakes. Similarly, Nguyen et al. (2025) proposed a fine-grained spatio-temporal framework that explicitly models vulnerable spatial and temporal regions, while Cai et al. (2025) combined local patch guidance and global forgery diversification to improve robustness to unseen forgeries. Along a different but related direction, Sun et al. (2025) explored visual-linguistic forgery detection, showing that mask-guided text annotations can improve both interpretability and cross-domain generalization.

Alongside spatial and temporal modeling, a growing body of work has shown that frequency-domain information can provide complementary forensic evidence. Miao et al. (2023) introduced a high-frequency fine-grained transformer to emphasize spectral manipulation traces, while Zhao et al. (2023) proposed an interpretable spatial-temporal video transformer for more robust detection. Wolter et al. (2022) specifically motivated the use of wavelet representations, showing that wavelet packets can capture manipulation artifacts that are difficult to observe in the pixel domain alone.

Very recent works in 2025 reinforced this trend. Baru et al. (2025) proposed Wavelet-CLIP, which combines wavelet transforms with CLIP-based representations to improve cross-dataset generalization and robustness to unseen synthetic images. Dutta et al. (2025) introduced WaveDIF, a lightweight frequency-domain detector based on wavelet sub-band energies, showing that interpretable wavelet statistics can still provide competitive performance. These studies confirm that wavelet-based decompositions offer a promising compromise between spatial localization and frequency analysis, making them particularly attractive for exposing subtle manipulation artifacts.

Overall, the literature suggests that generalizable deepfake detection benefits from complementary cues rather than from a single image domain alone. Spatial methods are effective in capturing visible inconsistencies, temporal approaches help detect motion-related artifacts, and frequency-based representations can expose subtle traces that are less evident in RGB. Unlike recent wavelet or frequency-aware works that introduce specialized architectures, multi-stage designs, or pretrained vision-language components, this paper focuses on a narrower and more controlled question: what changes when the input to the same ResNet-50 detection backbone is replaced by a native first-level DWT representation? This framing clarifies that the contribution is not architectural complexity, but a systematic RGB-vs-DWT comparison under cross-forgery and cross-compression evaluation, including aggregate metrics and input-resolution controls.

3 Background on DWT

The Discrete Wavelet Transform (DWT) analyzes digital images by decomposing them into components associated with different spatial frequencies while preserving local spatial information. Unlike the Fourier transform (Bracewell, 1986), which represents a signal through globally supported sine and cosine functions, the DWT relies on localized basis functions called wavelets (Daubechies, 1992). Because wavelets are spatially localized and decay rapidly, they enable a joint analysis of structure and texture at different scales.

For images, the DWT is implemented by applying low-pass and high-pass filters along both horizontal and vertical directions. In this work, we use a one-level Haar wavelet decomposition. This produces four sub-bands for each channel: LL, LH, HL, and HH. The LL component, also referred to as the approximation band, captures the coarse low-frequency structure of the image. The remaining bands encode directional high-frequency information: LH emphasizes vertical details, HL emphasizes horizontal details, and HH captures diagonal details.

Although the LL band contains most of the image energy, manipulation traces may appear across all sub-bands. In particular, the high-frequency components are relevant for deepfake detection, since synthesis and blending operations often introduce subtle local inconsistencies, edge distortions, resampling traces, or unnatural textures that may be less evident in the RGB domain. The DWT is therefore attractive because it provides a compact representation in which both coarse image structure and directional detail information are explicitly separated.

As illustrated in Figure 1, the visual differences between authentic and manipulated samples may remain subtle in the RGB domain. However, the corresponding wavelet decompositions can expose local frequency discrepancies that are more informative for automatic analysis. As a qualitative example, the diagonal-detail band of manipulated samples may appear visually flatter than in the real case. While this is not intended as a universal rule, it suggests that wavelet-domain representations can reveal forensic cues that are difficult to perceive directly in the pixel domain.

4 DWT-based proposed approach

Our method transforms each detected face crop into a wavelet-domain representation before classification. The overall pipeline is illustrated in Figure 2.

Given an input video frame, we first extract the facial region using a pre-trained MTCNN face detector (Zhang et al., 2016). As also observed in prior deepfake detection works (Rössler et al., 2019), restricting the analysis to the face region helps the model focus on the most informative area, where manipulation artifacts are generally more evident. At the same time, an excessively tight crop may discard useful context around facial boundaries, blending regions, or hair contours. For this reason, after face detection, we enlarge the bounding box by a factor of 1.3.

Since the detected region is generally rectangular, we convert it into a square crop by extending the shorter side so as to match the longer one, without altering the facial aspect ratio. The resulting square crop is then resized to 224 × 224 pixels.

Once the face crop is obtained, we apply a one-level Haar Discrete Wavelet Transform (DWT) independently to each RGB channel. For each channel, the transform produces four sub-bands, namely LL, LH, HL, and HH. The LL band contains the low-frequency approximation of the image, whereas the remaining three bands encode directional high-frequency details corresponding to vertical, horizontal, and diagonal structures. In the context of deepfake detection, these high-frequency components are of particular interest because manipulation pipelines often introduce subtle local inconsistencies, unnatural textures, resampling artifacts, or blending traces that may be attenuated or less apparent in the original RGB representation.

A decomposition example for both real and manipulated samples from FaceForensics++ is reported in Figure 1. We restrict the analysis to the first decomposition level. Since a one-level DWT reduces the spatial resolution by a factor of two along each dimension, a 224 × 224 crop produces sub-bands of size 112 × 112. Applying deeper decompositions would further reduce the spatial support of the representation and could suppress fine-grained forensic traces that are useful for discrimination.

Instead of feeding the original RGB crop into the classifier, we use the first-level wavelet representation only. More precisely, each of the three color channels is decomposed into four sub-bands, yielding a total of 3 × 4 = 12 channels. These components are concatenated along the channel dimension to form the network input. Formally, given an RGB image I ∈ ℝ^*H*×*W*×3, the corresponding wavelet representation is defined in Equation 1.

I_DWT∈ℝH2×W2×12.(1)

In our setting, with H = W = 224, the resulting tensor has size 112 × 112 × 12.

It is worth noting that the proposed DWT representation differs from the RGB input not only in the number of channels, but also in spatial resolution. In particular, the classifier receives the wavelet coefficients at their native resolution, without upsampling them back to 224 × 224. Preliminary experiments with upsampled sub-bands did not provide meaningful improvements; therefore, we retain the native wavelet resolution.

To process this input, we adopt a modified ResNet-50 architecture in which only the first convolutional layer is adapted to accept 12 channels instead of the standard three-channel RGB input. All the remaining layers are left unchanged. This modification allows the network to directly learn discriminative patterns from the stacked wavelet sub-bands while preserving the structure and representational capacity of the original backbone. In this way, the model operates on a frequency-aware representation that explicitly encodes both coarse image content and directional high-frequency details. It is important to note that the first-level DWT is used in its critically sampled form: although each sub-band has spatial size 112 × 112, the four sub-bands obtained from each RGB channel preserve the same total number of scalar coefficients as the original 224 × 224 channel. Therefore, the DWT input tensor 112 × 112 × 12 has the same total coefficient count as the RGB tensor 224 × 224 × 3, namely 150, 528 values. The comparison with RGB should thus not be interpreted as a simple reduction of input information, but rather as a different organization of the same input crop into spatial-frequency components. The main difference between the two inputs is the representation itself: RGB provides spatial color intensities, whereas DWT provides approximation and directional detail coefficients arranged across a larger number of channels.

5 Experimental results

All experiments use the same video-level split for real and manipulated videos: 720 videos for training, 140 for validation, and 140 for testing for each manipulation method. Each binary detector is trained using pristine samples and one manipulation type, and is then evaluated both on the same manipulation type and on the remaining manipulation types. We refer to the former as diagonal or in-forgery evaluation and to the latter as off-diagonal or cross-forgery evaluation.

Face crops are obtained using MTCNN face detection. The detected bounding box is enlarged by a factor of 1.3, converted to a square crop while preserving the facial aspect ratio, and resized to 224 × 224 pixels. The RGB baseline receives this 224 × 224 × 3 crop directly. The DWT model applies a one-level Haar decomposition independently to the three RGB channels, producing four sub-bands per channel and therefore a native 112 × 112 × 12 input tensor. To better isolate the effect of spatial resolution, we additionally train an RGB-112 control baseline using the same RGB pipeline but resizing the input to 112 × 112.

All models use a ResNet-50 backbone and are trained with cross-entropy loss and the Adam optimizer. For the RGB-224 and RGB-112 baselines, we use the ImageNet-pretrained torchvision ResNet-50 implementation from the RGB training pipeline. The input image is resized to either 224 × 224 or 112 × 112, converted to a tensor, and normalized with mean [0.5, 0.5, 0.5] and standard deviation [0.5, 0.5, 0.5]. Both RGB baselines are trained for 20 epochs with batch size 64 and learning rate 10⁻⁴. The best checkpoint is selected according to validation accuracy, and the final checkpoint is also saved.

For the DWT model, the same ResNet-50 architecture is used, but the first convolutional layer is modified to accept the 12-channel wavelet input. Unlike the RGB baselines, the DWT model does not use ImageNet pretraining and is trained from scratch, since the native 12-channel wavelet input is not directly compatible with the standard 3-channel pretrained first layer.

Frames are sampled at a fixed cadence of one frame every 10 frames. The resulting frame-level samples are processed independently during training and evaluation. This protocol is used consistently for RGB-224, RGB-112, and DWT models so that the domain comparison is not affected by differences in frame selection.

In addition to TPR and TNR, we report accuracy, balanced accuracy, precision, F1 score, and AUC. Aggregate confidence intervals are computed by bootstrapping over train-target pairs. Paired comparisons report DWT-minus-RGB deltas and use paired sign-flip tests over train-target metric differences. These tests are pair-level tests over the cross-forgery matrix, not frame-level McNemar tests; the latter would require storing aligned per-sample predictions for both domains. Both models were trained on a single NVIDIA geforce RTX 2080 TI GPU.

Different neural network architectures have been tested providing substantially similar results; on this basis, hereafter, results obtained using the ResNet-50 backbone are presented. To analyze the performance of wavelet-based features, the case of RGB pixel domain is taken as baseline reference. Each model is trained on a single forgery type from FF++ and evaluated both on the same forgery (intra-forgery testing) and on the remaining forgery types (cross-forgery testing) to assess generalization. In Table 1, values of TPR (True Positive Rate) and TNR (True Negative Rate) are reported, where the case positive stands for fake, while the negative indicates real. According to this, TNR values are listed in just one column because they are the same for each specific row of the table. It can be observed that the pixel-based method always achieves superior values of TNR (second column) and performs better, in terms of TPR, on the diagonal of the table, that is for the in-forgery case (the only exception is the case for FaceShifter, bottom-right cell of the table).

Table 1. In-forgery and Cross-forgery detection accuracy between methods based on RGB (pixel) and DWT, respectively, on the left and on the right of each table entry; best values for each table entry are in bold.

FF++ dataset c23 (top) and c40 (bottom) compressed have been considered.

Table 1 reports the in-forgery and cross-forgery TPR/TNR comparison between RGB and DWT. The first observation is that RGB generally preserves higher TNR, that is, it is more conservative on pristine samples and produces fewer false positives. RGB also usually performs better on the diagonal, where training and testing use the same manipulation type. This indicates that RGB remains a strong representation when the test distribution is close to the training forgery distribution. The behavior changes in the off-diagonal cross-forgery setting. In several cross-forgery cells, DWT increases TPR, meaning that it detects a larger fraction of fake samples generated by manipulation methods unseen during training. However, this increase in fake-class sensitivity comes at the cost of lower TNR. Therefore, the DWT representation should not be interpreted as a uniform improvement over RGB. Rather, it shifts the operating point of the detector toward higher sensitivity to fake samples and lower specificity on real samples.

This distinction is important because cross-forgery generalization can be evaluated from different perspectives. If the priority is to maximize fake recall on unseen manipulations, DWT is advantageous in many off-diagonal cases. If the priority is to minimize false positives on real content or maximize in-forgery accuracy, RGB remains stronger. For this reason, the following subsections summarize the complete cross-forgery matrix using aggregate metrics, paired statistical tests, and an input-resolution control.

5.1 Aggregate RGB vs. DWT comparison

Table 2 provides an aggregate view of the pairwise evaluations reported in Table 1. For each compression level, we group the 5 × 5 train–test matrix into three evaluation regimes: all pairs, diagonal pairs where the model is tested on the same manipulation type used for training, and off-diagonal pairs where the model is tested on manipulation types unseen during training. These aggregates are not separate experiments, but summarize the same RGB and DWT evaluations in order to make the overall cross-forgery behavior easier to interpret. The aggregate results confirm that the main effect of DWT is not a consistent improvement in accuracy. On c23, off-diagonal accuracy changes only from 53.20% to 54.14%, while off-diagonal TPR increases from 8.83% to 17.32% and F1 from 15.37% to 24.81%. At the same time, TNR decreases from 97.57% to 90.97%. On c40, the same pattern is observed: off-diagonal TPR increases from 24.50% to 32.90% and F1 from 31.48% to 40.29%, while TNR decreases from 83.97% to 77.30%.

Table 2. Aggregate RGB and DWT performance on FF++ c23 and c40.

Values are percentages. Bold indicates the best value between RGB and DWT within the same compression setting and evaluation group. The diagonal group corresponds to matched train–test manipulation types; the off-diagonal group corresponds to cross-forgery evaluation. Balanced accuracy is omitted because, under the balanced real/fake evaluation protocol, it coincides with accuracy.

Overall, these results support a more nuanced conclusion: DWT improves fake-class sensitivity and F1 in the cross-forgery regime, but RGB remains stronger in specificity and in-forgery performance. AUC is not consistently improved by DWT, which further confirms that the advantage is not a blanket improvement in separability.

It is also interesting to investigate how the generalization capacity of the DWT based approach becomes even more evident when working with different compression levels like shown in Figure 3, which compares performance of models trained on the c23 compression level and tested on the c40 compressed dataset. The results tend to confirm the trend observed previously with some differences.

While the pixel-based approach (blue bars) maintains superior TNR values, it suffers of a significant drop in TPR in cross-forgery cases. In contrast, the DWT-based model (orange bars) consistently outperforms the pixel-based method in cross-forgery scenarios, always achieving higher TPR values across different manipulation techniques, this time also in the in-forgery cases. This behavior is particularly noticeable in cases such as Deepfake and FaceShifter, where the TPR improvements exceed 10%−15% compared to the pixel-domain method. This reinforces the hypothesis that the DWT representation captures more generalizable forensic traces that are less sensitive to specific forgery techniques or compression.

6 Conclusions

In this work, we investigated whether a wavelet-based representation can improve the generalization ability of deepfake detectors with respect to standard pixel-domain inputs. To this end, we transformed face crops into first-level DWT sub-bands and used the resulting representation as input to a modified ResNet-50, evaluating the approach in both cross-forgery and cross-compression settings.

The revised analysis shows that DWT does not provide a uniform improvement over RGB. Instead, it induces a sensitivity–specificity tradeoff. In off-diagonal cross-forgery evaluations, DWT increases fake-class TPR and F1 on both c23 and c40. However, RGB preserves higher TNR and generally stronger in-forgery performance. Accuracy and AUC do not show a consistent DWT advantage. These findings suggest that wavelet sub-bands expose forensic cues that are useful for detecting unseen fake classes, while also making the detector less conservative on pristine samples.

Overall, the proposed DWT formulation should be interpreted as a lightweight and complementary representation for cross-forgery detection, rather than as a universal replacement for RGB. It is particularly useful when fake-class recall under unseen manipulations is prioritized, while RGB remains preferable when false positives on real content are critical.

Future work will extend the evaluation beyond FaceForensics++ to additional benchmarks, such as Celeb-DF, DFDC, and DeeperForensics, as well as to newer generative pipelines including diffusion-based and recent identity-swap methods. Moreover, since the present evaluation is performed at the frame level, future work will investigate video-level aggregation strategies and temporal consistency cues. Finally, hybrid RGB-wavelet systems will be explored in order to preserve the strong specificity of RGB-based models while exploiting the cross-forgery sensitivity of wavelet-domain features.

Generative AI statement

The author(s) declared that Generative AI was used in the creation of this manuscript. Rephrasing some parts of the text.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Sandeep Kumar Mishra, Yale University, United States

Reviewed by: Dafni Rose, St. Joseph's Institute of Technology, India

Abu Salam, Universitas Dian Nuswantoro, Indonesia