Efficient video violence detection through segmented temporal sampling and CNN–transformer modeling

1 Introduction

The rise of digital cameras and video platforms has created a major challenge: the need for more automated surveillance systems to identify instances of violence in schools, public spaces, and other locations. Identifying these instances of violent behavior relies largely on operators who must simultaneously monitor multiple feeds, which can result in slow response times and, in some cases, failure to report such incidents (Huillcen et al., 2024). This need has driven, on the one hand, the development of deep learning models designed to recognize violent patterns more quickly and with greater scalability than manual inspection; and on the other hand, the deployment of these models without requiring disproportionate infrastructure (Negre et al., 2024).

In a computer system, identifying a situation of violence in a video remains a complex problem because it depends on the interaction between people, the context of the scene, and the temporal evolution of the actions—and not solely on the isolated appearance of a single frame (Huszár et al., 2023). Differences in lighting, scale, perspective, crowd density, and capture distance increase intraclass variability and reduce transferability between scenarios (Elzein et al., 2024). Furthermore, frame loss, blurring, or insufficient sampling can eliminate brief, yet decisive moments needed to recognize abrupt actions such as blows, shoves, and scuffles (Zeng et al., 2024). Processing extensive sequences of frames does not guarantee better results either, as many frames may be redundant, while excessive sampling may omit brief movements that are crucial for distinguishing a violent situation from normal activity (Azim et al., 2026).

Two-dimensional convolutional neural networks (CNNs) remain a benchmark in this context, due to their ability to extract poses, textures, objects, and other features. Tyagi et al. (2026) designed a variant of MobileNetV2 with 1.93 million parameters, achieving an accuracy score of 97% on RLVS and 94% on Hockey Fight, demonstrating its viability for resource-constrained hardware. Trinh et al. (2024) proposed a CNN-based feature approach that achieved 97.96% on Hockey Fight, although temporal modeling still relied on post-processing. Kang et al. (2021) proposed grouping three consecutive frames, complementing the backbone with motion saliency maps and temporal attention, and reported lower complexity than alternatives based on 3D CNNs.

Three-dimensional CNNs integrate appearance and motion using spatiotemporal filters, but their computational cost increases with the resolution and duration of the clip. Dündar et al. (2024) proposed a shallow 3D-CNN evaluated on a combination of four benchmarks and designed for edge devices, with results comparable to more complex models. Getsopon et al. (2025) combined 2D and 3D branches and reported 98.1% on Hockey Fight, 100% on Movie, and 96.32% on Violent Flows. Huang and Jiang (2025) achieved 97.5, 99.5, and 89.4% on Hockey Fight, Movies Fight, and RWF-2000, respectively. These scores demonstrate the ability of 3D convolutions to learn spatiotemporal information, although they are not directly comparable due to differences in datasets, partitions, and experimental protocols.

Recurrent CNN models separate spatial feature extraction from sequential learning and remain a common solution when motion must be explicitly represented. CNN-BiLSTM architecture achieved 99.27% on Hockey Fight, 100% on Movies Fight, and 98.64% on Violent Flows, while classifying one-second clips in 0.923 s (Halder and Chatterjee, 2020). El-Fiky et al. (2026) proposed a hybrid framework that integrates a lightweight CNN, ConvLSTM, and temporal attention. In experimental results across four datasets, they achieved up to 99.15% accuracy and 99.37% F1 score. In another study by Salman et al. (2026), focused on edge deployment, the authors implemented a lightweight CNN with a GRU to run the model on an embedded device, achieving an average accuracy of 96.9%, an F1 score of 96.3%, and a speed of 26 FPS on five public datasets and one of their own.

Optical flow and multimodal fusion aim to resolve ambiguity by incorporating motion, sound, or body representations. In their work, Mahmoud et al. (2025), combined GMFlow, RGB images, and ResNet3D with CBAM, achieving 97.5% accuracy on Hockey Fight, 94.0% on Crowd Violence, and an AUC of 0.963 on UBI-Fight. Along these lines, Mohammed et al. (2025) fused VGGish and Vision Transformer; when validated on the RLVS dataset, they achieved an accuracy and precision of approximately 97.13%. These strategies add to the body of evidence, but they require modules, synchronization, and modalities that are not always available in conventional cameras. Furthermore, the pre-computation of optical flow or the incorporation of audio adds stages that can increase latency, memory usage, and deployment complexity.

Vision Transformers (ViT) model global relationships using self-attention and avoid strictly sequential recurrence. Shagufta et al. (2022) evaluated a ViT on two datasets (RLVS and Hockey Fight), achieving up to 98% accuracy and outperforming other models such as ConvLSTM and VGG16-LSTM, which were run under the same protocol. Rendón-Segador et al. (2024) proposed CrimeNet, a ViT with an adaptive threshold sliding window model based on the Transformer architecture. This proposal was evaluated on 11 datasets, achieving near-perfect scores of 99 and 100% for AUCROC and AUCPR, respectively. Although these approaches expand the temporal scope of modeling, the cost of self-attention increases with the number of tokens, so long sequences can increase latency and memory consumption.

Hybrid architecture has been proposed in this context with the aim of preserving the local biases of CNNs and the long-term temporal dependencies of transformers. In their work, Chatterjee et al. (2025) combined the classic MobileNetV2 architecture with temporal-aware modeling for real-time video, while Natha et al. (2025) incorporated CNNs, BiLSTMs, and multi-head self-attention, which allowed them to increase temporal capacity at the cost of a more complex processing chain. Azim et al., 2026 proposed hybrid architecture aimed at reducing redundancy through unsupervised key-frame selection and CNN attention, achieving a median accuracy of 94.6% and an F1 score of 93.9% across five evaluated datasets.

Previous studies have achieved very high results and, in some cases, nearly perfect results. However, they still have limitations that prevent efficient and generalizable deployment. While 2D CNN architecture reduces computational cost, their temporal representation depends on frame batching or subsequent modules (Kang et al., 2021; Trinh et al., 2024; Tyagi et al., 2026). 3D CNNs capture both appearance and motion, but this comes at the cost of increased model complexity, as it requires more resources to process spatiotemporal volumes (Dündar et al., 2024; Getsopon et al., 2025; Huang and Jiang, 2025). Recurrent models, optical flow, and multimodal fusion improve temporal representation, but they incorporate sequential processing, additional modalities, or stages that increase both memory usage and latency (El-Fiky et al., 2026; Halder and Chatterjee, 2020; Mahmoud et al., 2025; Mohammed et al., 2025; Salman et al., 2026), while transformers and hybrid models see their computational cost increase with sequence length (Chatterjee et al., 2025; Natha et al., 2025; Rendón-Segador et al., 2024; Shagufta et al., 2022).

The available comparisons do not clearly show how much performance can be maintained when the number of processed frames decreases. Although some methods reduce redundancy through clustering, keyframe selection, or adaptive windows (Azim et al., 2026; Kang et al., 2021; Trinh et al., 2024), their evaluations use different protocols, partitions, and sequence lengths. This heterogeneity makes it difficult to separate the contribution of the architecture from the benefit derived from processing more temporal information. Furthermore, several studies do not document the seeds or specify whether partitioning was performed at the video level before extracting frames or segments, further hindering the ability to quantify experimental variability and verify the independence between the training, validation, and test partitions (Dündar et al., 2024; El-Fiky et al., 2026; Mahmoud et al., 2025; Shagufta et al., 2022; Tyagi et al., 2026). Therefore, there remains a need to compare different temporal assumptions under homogeneous training, validation, and test conditions.

To address these limitations, the contribution of this study is methodological and experimental rather than the introduction of a new convolutional or self-attention operator. Instead, it focuses on determining how much temporal information is needed for the detection of violent video situations under controlled and uncontrolled conditions. For this purpose, CNN-Transformer-based architecture is evaluated using segmented temporal samplings of 4, 8, and 16 frames. The CNN, based on EfficientNetB0, extracts spatial representations while preserving RGB channels, whereas the Transformer models temporal relationships between time points distributed throughout the video without the need for optical flow, audio, or recurrent modules. The protocol maintains independent partitions at the video level, ensuring that there is no data leakage. Multiple seeds are also established, and configurations are compared using predictive metrics, paired tests, robustness evaluation, and computational efficiency measurements. This design allows for the study of the effect of the temporal budget while keeping the architecture, partitions, and evaluation conditions constant. In this way, the aim is to provide experimental evidence on the trade-off between temporal information and computational efficiency, contributing to the design of viable video surveillance systems for scenarios in which memory, latency, and computational capacity constrain practical implementation.

2 Methodology

2.1 Materials

To validate the proposed framework, we use three datasets that, according to the literature reviewed, are the most widely studied for identifying instances of violence and for comparing different deep learning (DL) architecture approaches. These datasets are as follows:

• Real Life Violence Situations (RLVS), developed by Soliman et al. (2019), consists of 2,000 videos, of which 1,000 depict violent events and 1,000 depict nonviolent events. The videos include fights in various locations and settings, such as streets, prisons, and schools. Those classified as nonviolent show generic human activities, such as playing different sports and eating.
• Hockey Fight Videos contains 1,000 videos of fights between players in the United States National Hockey League, organized into 500 videos per category, and is available on Kaggle (n.d.).
• AIRTLab Violence Dataset. The dataset introduced by Bianculli et al., 2020 consists of 350 Full-HD video clips recorded at 30 fps, including 230 violent and 120 non-violent samples. Unlike datasets primarily focused on explicit fight scenes, its non-violent subset deliberately contains behaviors such as hugging, clapping, exulting, high-fives, and gesticulation, which may exhibit rapid body movements similar to violent interactions.

All experimental results were obtained on a computer equipped with an Intel Core Ultra i9 275HX processor, 32 GB of RAM, the 64-bit version of Windows 11, and an NVIDIA GeForce RTX 5080 graphics card. The implementation was developed using Python 3.12.7, PyTorch 2.11, Torchvision 0.26, OpenCV, NumPy, Pandas, and Scikit-learn. GPU acceleration and mixed-precision training were used when the device supported them.

2.2 Proposed CNN–transformer framework

The framework uses CNN components for spatial feature extraction and a Transformer encoder aimed at modeling temporal dependencies between a limited number of frames. Its main objective is to evaluate how different temporal budgets affect violence detection when the spatial extractor, temporal modeling strategy, and training conditions remain unchanged. As shown in Figure 1, this framework is organized into five functional stages, which are described and explained in the following subsections.

2.2.1 Temporal sampling and preprocessing

A video is a sequence of consecutive frames; processing the entire sequence increases the number of operations and resource consumption without guaranteeing a proportional improvement in performance (Azim et al., 2026; Mahmoud et al., 2025). This situation is particularly relevant for the detection of violent incidents, where frames with the most discriminative content may be concentrated in specific time intervals. For this reason, a segmented temporal sampling approach was adopted that distributes a fixed number of observations throughout the entire duration of the video. This strategy was designed to reduce redundancy and preserve global temporal coverage while keeping computational costs under control.

For each video consisting of N frames, the complete frame sequence is represented as shown in Equation 1:

v = { F 0 , F 1 , … , F N − 1 } (1)

The video is divided into T approximately equal time segments. The boundaries delimiting these segments are calculated according to Equation 2 as follows:

b j = jN T , j = 0 , 1 , … , T , (2)

where, b j represents the j-th temporal boundary used to delimit the segments, and ·corresponds to the floor function, used to obtain a valid integer index.

During training, a frame is randomly selected within each segment. In validation and testing, the selection is made deterministically using a relative offset within each interval. The main evaluation uses the midpoint (ρ = 0.5) of each segment, while the robustness analysis also considers early (ρ = 0.25) and late (ρ = 0.75) positions.

The selected sequence is defined in Equation 3 as:

v ¯ = { F k 0 , F k 1 , … , F k T − 1 } , T ∈ { 4 , 8 , 16 } (3)

where k j represents the selected index within segment j. The 4, 8 and 16 frames configurations share the same architectural components and differ only in the amount of temporal information provided to the model. This configuration allows us to analyze the effect of the temporal budget without introducing additional changes to the architecture.

Each frame is converted from BGR to RGB, resized to 224 × 224 pixels, and normalized using the statistics employed during pre-training on ImageNet. For a batch of B videos, the model’s input takes the form shown in Equation 4:

X B ∈ ℝ B × T × 3 × 224 × 224 (4)

During training, moderate and consistent transformations were applied to all frames of the same video. These included horizontal flipping, adjustments to brightness and contrast, and occasional Gaussian blurring. Applying these transformations together prevents artificial discontinuities between temporal positions.

2.2.2 Spatial feature extraction using EfficientNet-B0

The spatial extractor converts each sampled frame into a compact, high-level representation. In the proposed framework, EfficientNet-B0 was used as the backbone due to its balance between representational capacity and computational complexity, which allows for the extraction of discriminative features with a moderate number of parameters compared to other architectures (Tan and Le, 2019). However, this choice is not presented as a proven superiority over other backbones, since a comparison between different convolutional architectures was not part of the experimental protocol.

Therefore, to process a batch of B videos with T frames per video, the batch and time dimensions are combined before being fed into the backbone. The original EfficientNet-B0 classifier is removed, and the descriptor obtained after global spatial aggregation is retained. For each selected frame F k t, the feature extraction is expressed in Equation 5 as:

h t = f θ ( F k t ) , h t ϵ R 1280 , (5)

where f θ represents EfficientNet-B0 initialized with ImageNet weights and h t corresponds to the spatial descriptor of the frame at time step t. The descriptors of the T frames are organized to form sequences as shown in Equation 6:

H B ϵ R B × T × 1280 (6)

Each video is thus represented by T spatial vectors of 1280 dimensions. These descriptors encode visual information related to appearance, posture, interaction, and context of the scene, while their temporal dependencies are subsequently modeled using the Transformer encoder.

2.2.3 Feature projection and positional encoding

The 1280 -dimensional spatial descriptors are transformed into temporal tokens of dimension d = 256. The operation applied to each frame is defined in Equation 7 as:

z t = Dropout [ GELU ( LN ( Wp · ht + bp ) ) ] , z t ∈ R 256 (7)

The projection limits the Transformer’s cost and establishes a common representation for the three temporal configurations. The projected sequence is expressed in Equation 8 by:

Z = [ z 0 , z 1 , … , z T − 1 ] T ∈ R T × 256 (8)

Since self-attention alone does not incorporate the token order, a sinusoidal positional encoding is added (Vaswani et al., 2017). In compact form, the token supplied to the Transformer is calculated according to Equation 9 as:

u t = z t + p e t , U = Z + PE ∈ R T × 256 (9)

The encoding allows for the distinction of similar visual characteristics observed at different positions in the sequence. Its maximum length was set at 64 positions, thus covering 4, 8, and 16-frame configurations without modifying the modulus.

2.2.4 Temporal modeling using transformers

The projected representations, supplemented with positional information, are processed by a Transformer encoder responsible for modeling the relationships between the selected frames. The architecture uses two Transformer layers, four self-attention heads, a latent dimension of 256, a feed-forward network of 512 units, GELU activation, prenormalization, and a dropout of 0.20.

Each head operates on a 64-dimensional representation, obtained by dividing the latent dimension among the four heads. The self-attention mechanism compares all sampled frames and determines the contribution of each temporal position to the contextualization of the others. This operation is expressed compactly in Equation 10 as:

Attention ( Q , K , V ) = softmax ( Q K T d k ) V , d k = 64 (10)

where Q , K and V correspond to learnable projections of the input tokens. The attention matrix represents the relationships between the selected T frames, allowing each position to incorporate information from previous and subsequent segments without resorting to recurrent processing.

2.2.5 Temporal aggregation and classification

The Transformer encoder produces a contextualized representation for each selected frame. Since not all video segments contribute the same amount of information to distinguish between violent and non-violent situations, an attention-pooling mechanism is used to learn the relative importance of each temporal position. The resulting scores are normalized using softmax and used to calculate a weighted combination of the representations generated by the Transformer. The result is a 256-dimensional global vector that summarizes the most relevant temporal information from the video.

This mechanism allows for increased contribution from segments with more discriminatory content and reduces the influence of redundant or uninformative frames. Its contribution was evaluated through ablation in which attention pooling was replaced by temporal averaging, while keeping the other framework components constant.

Finally, the overall representation is normalized, regularized using dropout, and fed into a two-output linear layer. The probabilities corresponding to the non-violence and violence classes are obtained using softmax. The final configuration of the proposed framework is presented in Table 1.

Table 1. Main configuration of the proposed CNN-Transformer framework.

2.3 Experimental design and evaluation

The experimental design was carried out in two sequential stages. First, the temporal budget was evaluated using 4, 8, and 16 frames across three independent seeds (42, 123, and 2026), while keeping the dataset partitioning (60/20/20), preprocessing, training protocol, and model architecture constant. Based solely on validation performance, the best configuration was selected. Second, the main architectural ablations were evaluated using the selected configuration and the same three seeds. These ablations are summarized in Table 2.

Table 2. Variants evaluated in the ablation study.

The variants without Transformer, without positional coding, and with temporal averaging allowed for the evaluation of the contributions of temporal modeling, order information, and learned frame weighting, respectively. The variant with a frozen backbone allowed for the analysis of the contribution of EfficientNet-B0 progressive adaptation. After selecting the architectural configuration, the final training runs were repeated with seeds 42, 123, and 2026. The data partitioning remained fixed, while the module initialization, batch order, and random transformations varied between runs. The mean and standard deviation were calculated for each metric across the three seeds.

Training used AdamW with a batch size of 8 for up to 40 epochs, with learning rates of 1 × 10 − 4 for the temporal/classification layers and 1 × 10 − 5 for the EfficientNet-B0 backbone, weight decay of 1 × 10 − 4, cross-entropy loss with 0.02 label smoothing, cosine annealing, and gradient clipping at 0.5. Early stopping was applied after five epochs without validation improvement. The best checkpoint and decision threshold were selected exclusively on the validation set, using a composite score and MCC maximization, respectively. EfficientNet-B0 was progressively unfrozen during training, and the test set was reserved solely for final evaluation.

The evaluation was performed at the video level. Accuracy, F1-score and MCC were computed from the binary predictions obtained using the decision threshold selected on the validation set, whereas PR-AUC was calculated from the continuous predicted probabilities across the precision–recall curve. These metrics are widely known and used in various studies on violence detection and anomalous event recognition (Becerra-Suarez et al., 2025; Mahmoud et al., 2025). Bootstrap with 2,000 iterations and a seed of 2026 was used to estimate metric uncertainty, with 95% confidence intervals obtained from the 2.5th and 97.5th percentiles of the empirical bootstrap distribution. For paired model comparisons, the same test videos were resampled using identical stratified bootstrap indices for both models, and the 95% confidence interval of the MCC difference was estimated from the resulting paired bootstrap distribution.

To complement the evaluation of the proposed framework, a robustness assessment was performed considering nine frequent degradations in real-world video surveillance scenarios, namely:

• Low brightness: Pixel intensities multiplied by 0.55.
• High brightness: Pixel intensities multiplied by 1.35.
• Gaussian noise: Additive Gaussian noise with mean 0.0 and standard deviation 0.04
• Gaussian blur: Gaussian filter with 7 × 7 kernel and σ = 1.5
• JPEG compression: JPEG quality factor of 35
• Frame dropping: Frames at positions 1,4,7, … replaced by the immediately preceding frame.
• Early temporal selection: Frame selection shifted toward the beginning of each temporal segment (ρ = 0.25 )
• Late temporal selection: Frame selection shifted toward the end of each temporal segment (ρ = 0.75 )
• Clean reference condition: No visual perturbation; frames selected at the midpoint of each segment (ρ = 0.50).

This evaluation proved crucial because several previous studies, such as those based on lightweight CNNs, CNN-ConvLSTM, optical flow, and Vision Transformer (El-Fiky et al., 2026; Getsopon et al., 2025; Halder and Chatterjee, 2020; Mahmoud et al., 2025; Rendón-Segador et al., 2024; Shagufta et al., 2022; Tyagi et al., 2026), report high results in terms of accuracy, F1-score, or AUC, but their comparisons typically focus on metrics obtained under clean experimental conditions, without systematically examining performance degradation in response to input perturbations or temporal variations in sampling. Therefore, these perturbations were applied only during inference, without retraining the model or modifying the threshold selected during validation. Degradation was estimated by comparing each condition with the clean baseline evaluation.

2.4 Controlled temporal baselines

To perform a direct comparison under the same experimental conditions, two temporal reference models were implemented in addition to the proposed CNN-Transformer. The first model uses EfficientNet-B0 followed by mean pooling on the sampled frames, providing a simple temporal aggregation strategy without an explicit sequence model. The second replaces the Transformer with a GRU and uses its final temporal representation for classification.

The comparison was performed using the best temporal budget selected during the ablation analysis phase. All three models used the same video-level splitting, segmented temporal sampling, image resolution, preprocessing, EfficientNet-B0 initialization, data augmentation, optimization settings, checkpoint selection procedure and validation-based decision threshold. Each model was evaluated using seeds 42, 123, and 2026. This design allows for comparison of the temporal modeling strategy without introducing differences in the data or training protocol.

3 Results

3.1 Ablation study and configuration selection

The ablation study allowed us to examine how the number of frames used affected the performance of the complete CNN-Transformer architecture in identifying situations of violence. Three scenarios were considered, with 4, 8, and 16 frames per video, and each scenario was trained with three seed values of 42, 123, and 2026. The dataset, preprocessing, training protocol, and model architecture remained constant. The temporal budget was selected using only the validation set to avoid using test data during model selection.

The results shown in Table 3 indicate that the 4-frame configuration achieved the most favorable overall validation performance. The 8-frame configuration showed slightly lower performance, whereas the 16-frame configuration yielded the lowest mean validation MCC among the three temporal budgets. Based on this comparison, the 4F configuration was selected for the subsequent component ablation and controlled reference experiments.

Table 3. Multi-seed validation performance for the temporal-budget analysis.

^*Bold text: best result.

After selecting the 4-frame configuration, a second experiment was conducted to evaluate the contribution of the main components of the proposed architecture. All component ablations were evaluated using the selected 4F configuration and the same three seeds. The complete model CNN–Transformer achieved the best performance in all metrics reported in Table 4.

Table 4. Multi-seed validation performance of the component ablation under the selected 4-frame setting.

^*Bold text: best result and values correspond to validation performance.

The ablation results show that the model does not depend on a single component, but rather on the combined effect of temporal order, adaptive frame weighting, temporal interaction, and spatial adjustment. The small decrease observed without positional coding indicates that frame order contributes to discrimination, although the limited 4-frame sequence can still preserve much of the relevant information without it. A similar pattern appears when attention pooling is replaced by temporal averaging, demonstrating that learned weighting helps the model emphasize the most informative moments instead of treating all sampled frames equally. The larger decreases obtained after removing the Transformer or keeping EfficientNet-B0 frozen point to a greater contribution from temporal interaction and base architecture adaptation, as both mechanisms allow the model to refine how visual evidence is represented and combined over time.

3.2 Final performance in RLVS and controlled temporal comparison

To assess whether the proposed architecture offers advantages over the reference models, under the same 4F configuration, Figure 2 is presented. The three approaches achieved similar overall performance. Within this controlled comparison, the CNN-Transformer obtained the highest mean PR-AUC value, reaching 0.9915 ± 0.0008.

Although the GRU-based model obtained slightly higher values in Accuracy, F1-score and MCC metrics, the observed differences were small, so no clear overall advantage of a single model is evident. Furthermore, the performance achieved by the model with Mean pooling demonstrates that the 4F segmented sampling scheme provides important discriminatory information on its own; however, the CNN–Transformer maintained the highest mean PR-AUC within the controlled comparison.

To assess the overall classification balance using the MCC metric and to determine whether the observed differences between models were consistent across different seeds, paired bootstrap comparisons were performed using the same test videos for each model pair. As shown in Table 5, the 95% confidence intervals included zero in all three comparisons between the CNN-Transformer and the GRU-based model, indicating that the observed differences in MCC were inconclusive.

Table 5. Paired bootstrap comparison of MCC between the CNN–Transformer and controlled baselines.

A similar pattern was observed with the Mean Pooling model for seeds 42 and 123. Only in seed 2026 did the confidence interval remain entirely above zero, favoring the CNN-Transformer. These results demonstrate that the small differences observed in the mean MCC do not constitute sufficient evidence of consistent superiority of the GRU or Mean Pooling over the proposed architecture. On the contrary, the three approaches exhibit statistically comparable performance in most runs, while the CNN-Transformer maintains the highest PR-AUC values within the controlled comparison.

3.3 Robustness to disturbances

Robustness was assessed by applying perturbations only during inference, without retraining the model or modifying the threshold selected during validation. This new analysis was not intended to further compare temporal budgets, but rather to examine the ability of each architecture to preserve its performance in the face of different disturbance conditions. To ensure a homogeneous comparison, the 4F configuration was maintained in the three previously implemented models: EfficientNet-B0 + Mean pooling, EfficientNet-B0 + GRU and CNN–Transformer. Changes in performance were quantified with respect to the clean condition, paying special attention to MCC, G-Mean, Recall and the increase in false negatives, due to their relevance to characterize both the global balance of the classification and the ability to detect violent events under disturbed conditions.

The perturbation corresponding to Gaussian noise produced the greatest performance degradation and revealed clear differences in the response of the three models to the degraded visual information. The results are presented in Table 6, where the models exhibited very similar MCC values in noise-free conditions, which ranged between 0.9023 and 0.9084. However, under Gaussian noise, CNN-Transformer maintained an MCC of 0.7169 ± 0.0724, while Mean Pooling and GRU decreased to 0.6161 ± 0.2234 and 0.6143 ± 0.0368, respectively. With the rest of the results, the proposed architecture presented the smallest absolute reduction of the MCC (ΔMCC = −0.1853), compared to −0.2863 for Mean Pooling and −0.2941 for the GRU.

Table 6. Clean and Gaussian-noise performance, absolute and relative degradation, and false-negative increase for the evaluated temporal models.

Results are reported as mean ± standard deviation across three independent seeds (42, 123, and 2026). Absolute change was computed per seed as Δ = M Gaussian − M Clean, while relative degradation was calculated as [ ( M Gaussian − M Clean ) / M Clean ] × 100. ΔFN represents the increase in false negatives under Gaussian noise relative to the clean condition.

Gaussian noise affects visual information before it is encoded by EfficientNet-B0; therefore, its impact propagates to the temporal representation used by mean pooling, GRU, and CNN-Transformer. In the 4F environment, this effect becomes more relevant because each sample frame represents a substantial portion of the available temporal evidence, so degradation of a small number of frames can directly reduce the quality of the final prediction. Despite this limitation, CNN-Transformer showed lower performance loss than baseline methods, suggesting that self-attention and learned temporal weighting may contribute to preserving useful information when some frame-level representations become less reliable. This interpretation is consistent with the component ablation results, although the current robustness analysis does not isolate these mechanisms sufficiently to establish a direct causal relationship.

This trend was also observed in the other disturbances evaluated, although with different magnitudes of degradation between the models. As shown in Figure 3, CNN–Transformer showed the smallest MCC reduction against Gaussian noise (ΔMCC = −0.185) and JPEG compression (−0.033), while Mean pooling and GRU showed higher losses in both conditions. Under Gaussian blur, the differences were smaller and the GRU presented the least degradation (−0.113), followed by CNN–Transformer (−0.118) and Mean pooling (−0.129). On the other hand, frame loss and brightness changes produced relatively small variations. In particular, the CNN–Transformer remained practically stable against frame dropping (+0.001) and high brightness (≈0.000). Temporal shifts also had a limited effect, as both late and early frame selection produced slight increases in MCC on the CNN–Transformer (+0.008 in both cases).

Based on these results, perturbations that directly degrade visual information, especially Gaussian noise and blur, have a greater impact than moderate changes in temporal sampling. Under these conditions, the CNN–Transformer showed lower degradation under some of the most severe visual perturbations, particularly Gaussian noise, although this advantage was not consistent across all evaluated conditions.

3.4 Explainability analysis

To analyze the spatial and temporal evidence used by the CNN-Transformer, the Grad-CAM technique was applied to the final convolutional stage of EfficientNet-B0, and the attention pooling weights assigned to the 4F configuration were examined. The Grad-CAM maps represent the spatial regions that contribute to the predicted class, while w t indicates the relative contribution of each frame during temporal aggregation. As shown in Figure 4, in the true positive (TP) case, activation is concentrated primarily in the regions where physical interaction between people occurs, and simultaneously, the temporal weights increase from 0.220 in the first frame to 0.286 in the fourth. This behavior indicates a greater contribution from the segments where the interaction becomes more visually evident. When analyzing the true negative (TN), it presents more homogeneous weights (0.240–0.269) and a spatial distribution of activations associated with elements of a non-violent activity, which coincides with the low probability of violence obtained by the model ( P ( V ) = 0.011 ).

The error cases allow us to identify some limitations of the learned behavior. In the false positive (FP), the temporal attention weights increase considerably in the last two frames (0.318 and 0.350), while Grad-CAM concentrates activation on the athlete’s body and posture, resulting in a high probability of violence ( P ( V ) = 0.972 ). This suggests that certain body configurations or intense movements may share visual characteristics with patterns learned for violent events and generate an incorrect response. On the other hand, in the false negative (FN), the third frame receives the highest time weight (w t = 0.384); however, the model maintains a low probability of violence ( P ( V ) = 0.093 ). Because Grad-CAM explains the predicted non-violent class in this case, the activations show which regions contributed to that incorrect decision, even though the sequence contains physical interaction.

Based on these results, Figure 4 shows that the model does not assign the same relevance to all frames and that the classification arises from the combination of the spatial features extracted by EfficientNet-B0 and their temporal weighting. Furthermore, the FP and FN cases demonstrate that high temporal relevance does not guarantee correct classification, highlighting the model’s dependence on the quality and discriminability of the spatial representations.

3.5 Computational efficiency

The computational efficiency of the three controlled models was evaluated under the selected 4-frame configuration using the same hardware, precision mode, batch size, warm-up procedure, and number of repetitions. The reported latency corresponds exclusively to the model’s forward step and therefore excludes video decoding, temporal sampling, resizing, normalization, storage I/O, and host-to-device transfer.

The Mean Pooling model was the lightest configuration, with 4.34 million parameters and an estimated FP32 size of 16.55 MB, achieving an average latency of 9.16 ± 0.96 ms. The GRU contained 4.73 million parameters and required 18.05 MB, with a latency of 10.10 ± 1.10 ms. The CNN-Transformer contained 5.39 million parameters and required 20.57 MB, while achieving an average latency of 9.57 ± 1.25 ms and a throughput of 104.49 videos/s. Although the proposed architecture introduces a moderate increase in model size relative to the simpler baselines, its inference latency remains close to that of the Mean Pooling model and slightly lower than that of the GRU under the same measurement conditions. Therefore, the additional temporal modeling provided by Transformer does not result in a proportional increase in the inference time of the single model within the selected 4F configuration.

3.6 Independent benchmark evaluation and cross-dataset generalization

To extend the model evaluation beyond the RLVS dataset, additional experiments were conducted on Hockey Fight Videos and AIRTLab. Two complementary scenarios were considered. In the first, the architectures were trained and evaluated independently on each dataset. In the second, cross-domain generalization was analyzed by directly transferring models trained exclusively on RLVS to the external datasets, without weight adjustment, fine-tuning, or decision threshold recalibration.

The results of the independent evaluation are presented in Table 7. In Hockey Fight Videos, all three architectures achieved high and relatively stable performance. EfficientNet-B0 + Mean Pooling achieved the highest average values for Accuracy (0.9567 ± 0.0104), F1-score (0.9560 ± 0.0110), and MCC (0.9138 ± 0.0204), while EfficientNet-B0 + GRU achieved the highest PR-AUC (0.9891 ± 0.0015). The CNN-Transformer showed comparable performance, with an Accuracy of 0.9517 ± 0.0076, an F1-score of 0.9518 ± 0.0072, and an MCC of 0.9035 ± 0.0153. Taken together, these results show that all three configurations can be effectively trained within this domain.

Table 7. Results of the independent benchmark evaluation on Hockey Fight Videos and AIRTLab.

The values correspond to mean ± standard deviation for seeds 42, 123 and 2026.

As also shown in Table 7, AIRTLab produced a moderate reduction in performance compared to Hockey Fight Videos. EfficientNet-B0 with medium time grouping achieved the highest Accuracy (0.8952 ± 0.0082) and the highest MCC (0.7664 ± 0.0210), while CNN-Transformer achieved an Accuracy of 0.8857 ± 0.0143 and an MCC of 0.7493 ± 0.0299. However, the latter maintained a high PR-AUC of 0.9684 ± 0.0119. The decrease observed mainly in MCC could be due to AIRTLab being a more demanding classification environment, particularly for distinguishing non-violent interactions.

To evaluate a more stringent condition, models trained exclusively with RLVS were subsequently applied to Hockey Fight Videos and AIRTLab without adaptation to the target domain. The results of this experiment are summarized in Table 8. In the RLVS → Hockey Fight transfer, EfficientNet-B0 + GRU obtained the highest average MCC (0.3780 ± 0.1164), while the CNN-Transformer reached 0.3390 ± 0.0190. Although sensitivity remained high in all three architectures, specificity decreased considerably, indicating a greater tendency to classify non-violent sequences from the new domain as violent. Another aspect to highlight from Table 8 is the domain shift, which was more pronounced in the RLVS → AIRTLab transfer. The CNN-Transformer obtained the highest average Accuracy (0.6533 ± 0.0092) and the highest F1-score (0.7866 ± 0.0083).

Table 8. Generalization between datasets from RLVS to Hockey Fight Videos and AIRTLab, without adaptation to the target domain.

The comparison between Tables 7 and 8 reveals a substantial difference between reproducibility within a new dataset and direct generalization across domains. While independent training maintained high MCC and PR-AUC values in both Hockey Fight Videos and AIRTLab, transfer from RLVS resulted in a considerable reduction in MCC and G-Mean. This behavior indicates that a model capable of achieving high performance when using data representative of the target domain does not necessarily maintain the same level of discrimination in the face of changes in visual and behavioral distribution.

4 Discussion

This study analyzed the extent to which a segmented and reduced sampling strategy can maintain competitive performance in detecting violent situations. The results indicate that the number of frames does not produce a proportional improvement in performance. Under the same experimental conditions, the 4F configuration was selected as the most favorable, suggesting that distributed temporal coverage can preserve sufficient discriminative information without resorting to dense sequences. Furthermore, comparison with controlled baselines shows similar overall performance among the temporal strategies, while the CNN-Transformer strategy demonstrated specific advantages in PR-AUC and resilience to certain perturbations, particularly Gaussian noise. Independent benchmark evaluation showed that the models can achieve strong performance when trained within each target domain; however, the cross-dataset experiments revealed a substantial reduction in discrimination under domain shift.

In violent scenes, where certain postures, physical contact, or abrupt changes can be concentrated in brief intervals, processing more frames may introduce redundancy without necessarily providing new discriminative evidence. To contextualize this behavior in relation to representative works in the literature, Table 9 summarizes different approaches to video violence detection, considering the dataset used, the input method, the reported result, and its main comparative characteristic. As can be seen, several studies achieve higher accuracy values; however, they use heterogeneous experimental configurations, including 3D modeling, optical flow, recurrent processing, or longer time sequences. Therefore, the comparison presented in Table 9 should be interpreted as contextual and not as a direct performance comparison between architectures.

Table 9. Reference comparison with representative methods for detecting violence in video.

Furthermore, many comparisons focus primarily on metrics obtained under clean experimental conditions, without systematically analyzing performance degradation due to visual disturbances, partial frame loss, or variations in the temporal sampling position. This is particularly relevant in video surveillance systems, where capture conditions can be affected by changes in lighting, noise, blur, compression, or inconsistencies in the temporal availability of visual information. Therefore, an evaluation based solely on aggregated metrics from clean data may be insufficient to characterize the model’s stability in operational scenarios.

In this context, although the proposed framework does not surpass all the performance values reported in literature, its main contribution lies in the combination of computational efficiency, controlled evaluation, and robustness analysis. The proposed framework was evaluated not only on the main dataset but also under adverse conditions that approximate frequent situations in real-world video surveillance environments. Furthermore, multi-seed analysis, confidence interval estimation using bootstrap, comparison between temporal budgets, and disturbance evaluation allow for a broader characterization of the model’s behavior. Consequently, the proposed framework presents itself as a competitive alternative for detecting violence in video, especially in scenarios where stability, efficiency, and inference feasibility are relevant criteria alongside predictive performance.

Finally, the study has limitations that must be considered when interpreting the results. Although the experiments were repeated using multiple random seeds, the train, validation, and test partitions were kept fixed across runs; therefore, the reported variability reflects stochastic training effects but does not capture variability associated with alternative dataset splits. Although the evaluation includes RLVS, Hockey Fight Videos, and AIRTLab, all of these correspond to a binary formulation of violence/non-violence and do not fully represent the diversity of conditions present in real-world video surveillance scenarios. The drop observed in the cross-dataset experiments confirms that sensitivity to domain change remains a significant limitation, while the perturbations used, although useful for approximating adverse conditions, do not replace evaluation in real, continuous video streams.

Future research should explore domain adaptation and domain generalization strategies, multi-dataset training, multi-category classification of violent events, threshold calibration under domain change, and validation on edge platforms and in operational scenarios. These extensions would allow researchers to determine whether the efficiency observed with reduced temporal sampling can be maintained when diversity, uncertainty, and the constraints inherent in a real-world deployment increase.

Although the proposed 4F configuration showed favorable computational characteristics, the reported latency represents only model inference and should not be interpreted as end-to-end real-time performance. The framework was not evaluated on continuous live surveillance streams or resource-constrained edge devices. Therefore, real-world deployment validation remains an important next step, including the complete video-processing pipeline, camera acquisition, decoding, preprocessing, temporal sampling, inference, and communication overhead.

5 Conclusion

In this work, an efficient CNN–Transformer framework was proposed for video violence detection using segmented temporal sampling. The proposal integrated segmented frame selection, spatial extraction using EfficientNet-B0, positional coding, Transformer-based temporal modeling, and aggregation via attention pooling, allowing for the representation of video-level information without relying on optical flow, audio, recurrent modules, or extensive frame sequences. The experimental design allowed for the analysis of the effect of the temporal budget under controlled conditions, keeping the architecture, data partitions, and evaluation protocol constant. These results indicate that segmented temporal sampling can maintain competitive performance when spatial and temporal representations are modeled jointly. Furthermore, multi-seed evaluation, bootstrap confidence intervals, and robustness analysis allowed for a more comprehensive characterization of the model’s behavior than an evaluation based solely on aggregated metrics, the predominant approach in several previous studies analyzed in this work.

The proposed CNN-Transformer showed a favorable PR-AUC with the selected 4F configuration and exhibited less degradation than the controlled baselines under Gaussian noise, while maintaining a moderate computational footprint. These results support segmented temporal sampling as a viable strategy for efficient video violence detection. Instead of relying on dense temporal processing, the proposed framework demonstrates that temporally distributed observations can preserve relevant discriminative information, maintaining computational efficiency and robust predictive behavior under controlled evaluation conditions.

Generative AI statement

The author(s) declared that Generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Joey Tianyi Zhou, Agency for Science, Technology and Research (A*STAR), Singapore

Reviewed by: Marina Artiyasa, Nusa Putra University, Indonesia

Abu Bakar Siddique Mahi, Bangladesh University of Business and Technology, Bangladesh