DeformDANet: morphology-aware detection with population estimation for shrimp post-larvae in dense aquaculture imagery

1 Introduction

Shrimp farming is a major aquaculture activity around the world and is important for feeding and supporting many populations in tropical and subtropical nations. In commercial hatcheries, the critical quality control operations would be stock density management, feed optimisation and estimation of mortality, while accurate quantification of the postlarval (PL) shrimp is crucial for these operations. This has generally been accomplished by skilled technicians and is manpower-intensive, time-consuming and prone to interobserver variations, particularly in large-scale production where a large number of larvae (tens of thousands) must be evaluated each day (Pramono et al., 2025; Correia et al., 2025).

With the development of computer vision and deep learning technologies, many works have emerged to automate the quantification of larvae in aquaculture environments (Correia et al., 2025; Dang et al., 2025). Initial approaches were based on classic image processing methods like thresholding, morphological operators and connected component analysis. These methods work well in controlled laboratory settings, but do not perform well in a hatchery with variable light, water turbidity, and biological variability in larval morphology. Convolutional neural network (CNN)-based object detectors and density map regressors led to a big improvement in detection robustness, making deep learning the leading approach to automated larvae counting (Dang et al., 2025; Ruan et al., 2024; Dang et al., 2026). HIDANet, a lightweight deep learning framework was proposed for Litopenaeus vannamei post-larval stage classification and morphometric estimation, with additional analysis of background bias in the imaging data (A and S, 2026). Their work mainly focuses on shrimp post-larva, PL stage classification and morphometric estimation.

Despite these advancements, the major issues remain unresolved in the existing literature. First, shrimp larvae are biologically deformable organisms: they flex into irregular C- and S-shapes during swimming, exhibit semi-transparent bodies with variable head-to-tail aspect ratios across developmental stages, and present partial feature dropout at fixed sampling locations due to their transparency (Zhou et al., 2024; Yi et al., 2025). Standard convolutional architectures employ fixed 3×3 sampling grids that are geometrically misaligned with the true morphological support of each individual larva, leading to degraded feature representation, particularly at higher magnification stages (Wang et al., 2025; Zhao et al., 2022). Second, several hundreds of post-larvae are frequently found in a single frame of commercial hatchery images, resulting in extremely dense detection scenarios where traditional fixed-neighborhood target assignment strategies assign the same grid cells to multiple overlapping targets, producing conflicting regression gradients during training and can significantly increase false positive rates (Dang et al., 2025; Duan et al., 2024; Hai et al., 2025).

Existing approaches have addressed parts of these challenges independently. Density map regression methods (Hu et al., 2023; Dang et al., 2026; Zhou et al., 2024) provide accurate global counts in dense scenes but sacrifice individual localisation—a limitation that precludes downstream applications such as individual sizing, behavioral analysis, and stage classification. Detection-based methods including CAGNet (Zhang et al., 2024), ShrimpNet (Huang et al., 2025), and the improved YOLOv5 variant of Duan et al (Duan et al., 2024). achieve localisation but rely on fixed convolutional kernels and static assignment neighbourhoods, limiting their performance in dense, morphologically variable scenes. Recent large-scale multiscale fusion networks (Dang et al., 2025; Hai et al., 2025; Hsieh et al., 2024) improve feature representation across scales but introduce significant computational overhead—CAGNet requires 13.7M parameters and over 35 GFLOPs—rendering them impractical for real-time deployment on edge hardware commonly available in aquaculture facilities (Hsieh et al., 2024).

To address these limitations, this paper proposes DeformDANet, a single-stage detector specifically designed for dense shrimp larvae detection in hatchery environments. The proposed model introduces two novel contributions as demonstrated in the illustration Figure 1, and is motivated directly by the domain-specific challenges described above. First, a novel Deformable Integrated Spatial Operation Block (DeformableISOBlock) is proposed, replacing fixed-grid convolutional feature extraction with dual deformable convolutional branches sharing a single learned offset map, enabling geometry-adaptive feature sampling that conforms to the actual morphological support of each larva at every spatial location (Wang et al., 2025). Offset weights are initialised to zero, ensuring training begins from the identity sampling grid and geometric adaptation accumulates only as evidence warrants. Second, a fully vectorised Density-Adaptive Assignment (DAA) strategy replaces the conventional fixed-neighborhood positive assignment with a per-target radius derived from local k-nearest-neighbor density, dynamically shrinking the assignment window in high-density clusters to prevent cross-target gradient contamination while expanding it for isolated individuals (Hai et al., 2025; Zhang et al., 2025). The complete pipeline integrates these modules within a four-stage backbone, a weighted bidirectional feature pyramid neck, and three decoupled detection heads with distribution focal loss regression, while maintaining a compact model size of only 1.38M parameters and 6.82 GFLOPs.

The principal contributions of this work are summarised as follows:

• To address severe overlap, transparency variation, morphological distortion, and scale diversity in aquaculture images, a dense-scene shrimp post-larvae detection framework, termed DeformDANet, is developed through the task-specific integration of morphology-aware feature extraction, density-aware positive-sample assignment, and efficient multi-scale feature aggregation.
• A density-aware positive-sample assignment, DAA strategy is developed to adapt the positive assignment region according to local larval density, reducing assignment ambiguity in crowded regions while maintaining suitable supervision for sparsely distributed larvae.
• A DeformISOBlock is incorporated to improve morphology-aware feature representation by combining shared-offset deformable convolution, coordinate attention, and dense feature preservation, enabling improved representation of curved, elongated, and geometrically deformable shrimp post-larvae structures.
• An efficient BiFPN-GSConv feature aggregation neck is incorporated for hierarchical multi-scale feature fusion, supporting the detection of post-larvae with different sizes and densities while maintaining a compact architecture and reasonable computational complexity.
• An Operational Threshold Sweep (OTS) protocol is employed to systematically evaluate detector performance across confidence thresholds and select application-specific operating points using the validation partition. The selected threshold is subsequently frozen for evaluation on the test set.
• Through comparative experiments, ablation studies, robustness evaluation, and counting analysis, the effectiveness of the integrated DeformDANet framework is demonstrated for dense shrimp post-larvae detection and population estimation while maintaining a favorable accuracy–complexity trade-off.

The remainder of this paper is organized as follows. Section 2 reviews the related work on shrimp postlarvae detection, density estimation, deformable convolution, and dense-scene target assignment. Section 3 presents the proposed DeformDANet framework, including the DeformISOBlock, BiFPN-GSConv feature aggregation, Density-Adaptive Assignment (DAA), Operational Threshold Sweep (OTS), and the overall training procedure. Section 4 describes the experimental setup, dataset, implementation details, and evaluation metrics. Section 5 presents the experimental results, including quantitative comparisons, computational efficiency analysis, ablation studies, qualitative detection evaluation, operational threshold analysis, and counting accuracy analysis. Finally, Section 6 discusses the findings and limitations of the proposed framework, while Section 7 concludes the paper and outlines future research directions.

2 Related work

2.1 Shrimp larvae detection and counting

With the development of intensive aquaculture, automated detection and counting of shrimp larvae have gained more and more attention in the field of research (Pramono et al., 2025). Prior computer vision techniques were based on the classical image processing pipeline, which involved Otsu thresholding, watershed segmentation, and connected component analysis to segment and count individual larvae. These methods are relatively simple, but are sensitive to noise, water turbidity, and variations in light intensity, which can vary in operational hatcheries (Pramono et al., 2025; Huang et al., 2025) and are difficult to control.

The emergence of deep learning object detectors has dramatically improved generalisation capability across imaging conditions. Unlike the conventional global YOLOv3 approach, Zhang et al. employed a localized YOLOv3 model that operated on subdivided image blocks for counting. The method achieved 96% counting accuracy in low-density populations (Pramono et al., 2025). CAGNet (Zhang et al., 2024) proposes an improved detection framework with attention mechanism that surpasses the problem of small size and transparency of larvae and achieves competitive mAP@50 scores, but requires 13.7M parameters and 35.28 GFLOPs computation, which is difficult for deployment on embedded hatchery hardware. ShrimpNet (Huang et al., 2025) focused on computational efficiency by employing a light backbone, but they reported mAP scores at the standard IoU threshold of 0.50 rather than 0.55, and the obtained mAP score of 66.0% was still not high enough. Recently, Duan et al (Duan et al., 2024). proposed a regional segmentation method and an improved YOLOv5 model with C2f and CBA modules to improve detection accuracy in the high-density area, but the problem was still how to identify the overlapping larvae in different water depths accurately. In the same way, the one-stage detector presented in (Zhang et al., 2025) emphasized computational complexity for the deployment of edge devices, and the fixed-kernel feature extraction and static assignment strategies hindered its adaptability to morphological variations. Besides detection algorithms, imaging-based systems have also been explored. Narmadha et al (Narmadha et al., 2024). presented an imaging in-flow system for automated analysis and quantification of shrimp post-larvae in hatchery environments.

2.2 Density map regression approaches

A different way to quantify larvae is to use density map regression, which entails training a convolutional neural network (CNN) to produce a continuous density map whose integral represents the estimated number of larvae (Hu et al., 2023; Dang et al., 2026). Hu et al (Hu et al., 2023). applied this method to accurately count populations of about 1000 larvae and showed its superiority over detection-based methods when samples were very crowded. Based on this idea, Dang et al (Dang et al., 2025). proposed a multiscale feature extraction framework that uses a density map estimation method with size adaptability to deal with the size variation of larvae at different growth stages. They achieved good results on high-density images taken in the real world. In a similar way, EMSPAN (Hai et al., 2025) proposed an efficient multi-scale pyramid attention network for more accurate counting of objects in dense environments where there is significant size variation between objects. Zhao et al. proposed a scale-aware local context aggregation network (SLCA-Net) for multi-domain shrimp counting, focusing on the use of scale-aware and local contextual information to improve counting across different shrimp imaging domains (Zhao et al., 2025). While these techniques are effective for estimating the number of individuals in an area, they are limited in that they only give information on the total number of larvae, not the location of each one. In hatchery management, this restriction impedes their use where information on individual instances, in addition to total population estimates, such as size measurements, health assessment, and behavioral monitoring, is often needed.

2.3 Deformable convolution in aquatic object detection

Deformable convolutional networks, introduced by Dai et al (Dai et al., 2017). and subsequently extended by Zhu et al (Zhu et al., 2019), learn spatially adaptive sampling offsets that warp the convolutional receptive field to match the geometric structure of target objects. As aquatic creatures have variable, deformable body forms that do not align with fixed 3×3 sample grids, this trait is very beneficial. DeformableFishNet, a lightweight underwater fish detector that combines deformable convolutions with an effective global coordinate attention module, was proposed by Ruan et al (Ruan et al., 2024). and shown increased detection accuracy for morphologically varied fish species. Similar improvements in detecting irregular and obstructed fish targets in dense underwater farming scenes were reported by the YOLOv8nDDSW system (Yi et al., 2025), which substituted standard C2f modules with deformable convolutional counterparts in a YOLOv8 backbone. Zhao et al (Zhao et al., 2022). highlights that incorporating deformable convolutions to a YOLOv4-based dead fish detector increased accuracy by 10% in the aquaculture setting while also lowering the number of parameters compared to normal convolutions. With these, the OYOLOv8s-DC framework (Wang et al., 2025) explicitly validated that deformable convolution feature modules adapt more effectively to shape and size variations in underwater detection than fixed-grid alternatives. These converging findings from the broader aquatic detection literature motivate the design of the DeformableISOBlock proposed in this work, which extends deformable sampling to a dual-branch architecture with shared offset computation and coordinate attention gating specifically tailored to the transparency and morphological variability of shrimp larvae.

2.4 Target assignment in dense detection scenarios

The performance of the detector is sensitive to the assignment algorithm that is used to determine which positive examples to use per ground truth object, especially in dense scenes. Fixed neighborhood assignment strategies, such as FCOS and all of its variants, have a cellular 3×3 grid (hard coded), with a fixed number of candidate cells assigned to the target cells, irrespective of local crowd densities. In high larval densities with several centres of larval activity in the same two cells, these fixed windows produce overlapping areas of assignment, with the same cells having to regress towards multiple competing targets. This cross contamination adversely impacts the accuracy of regression and increases false positive rates (Dang et al., 2025; Duan et al., 2024; Hai et al., 2025). In the general object detection literature, there have been suggestions for using adaptive assignment strategies, such as ATSS (statistical anchor quality-based selection of positives) and TAL (task-aligned scores). These techniques, however, do not consider local target density and are not useful in high density situations such as those found in hatchery larval imaging. To address this, we introduce a Density-Adaptive Assignment that computes a positive radius per target that is based on the local kNN density distribution. This way, a dense area of the assignment window is dynamically contracted to avoid cross-target contamination, but a wide assignment window is left for isolated targets.

2.5 Computer vision in aquaculture

Besides the particular detection of larvae, computer vision has also been extensively applied for aquaculture management applications like species identification (Tiyapunjanit et al., 2023; Saleh et al., 2024) and disease detection (Hu et al., 2020; Tang et al., 2024) or morphometric analysis (Saleh et al., 2024) or behavioural observations (Correia et al., 2025). AIoT integrated counting systems (Hsieh et al., 2024) have shown the feasibility of running deep learning models on embedded hatchery hardware, with inference speed being one of the main constraints to deployment, along with accuracy. All of these larger applications helped to pave the way for the development of DeformDANet as a domain-motivated, computationally efficient detector, which would meet a number of accuracy, speed and deployability requirements that are characteristic of operational aquaculture environments. To date, shrimp larvae detection methods that tackle both deformable larval morphology and dense-scene assignment conflicts within a detection framework are not available. To overcome these drawbacks, the proposed DeformDANet proposes to combine both morphology-aware deformable feature extraction and density-adaptive assignment for shrimp post-larvae detection. To further highlight the differences between existing approaches and the proposed framework, a comparative analysis is presented in Table 1.

Table 1. Comparative analysis of existing shrimp larvae detection approaches and the proposed DeformDANet framework.

3 Methodology

3.1 Overview of DeformDANet

Shrimp post-larvae detection is challenging due to severe object crowding, scale variations, elongated body structures, and complex overlap patterns. Conventional lightweight detectors often suffer from limited geometric modeling capability and insufficient multi-scale feature aggregation, leading to missed detections and inaccurate localization in dense hatchery environments.

To address these challenges, a novel detection framework termed DeformDANet is proposed. As illustrated in Figure 1, the framework consists of four major components: (i) a DeformISO, (Deformable Integrated Spatial Operation) backbone for geometry-aware feature extraction, (ii) a BiFPN-GSConv feature aggregation neck for efficient multi-scale fusion, (iii) a decoupled detection head for classification and localization, and (iv) a Density-Adaptive Assignment (DAA) strategy for density-aware positive sample assignment during training. In addition, an Operational Threshold Sweep (OTS) mechanism is employed during inference to identify the optimal confidence threshold for prediction filtering, thereby improving the precision–recall trade-off and enhancing overall detection performance.

During feature extraction, the DeformISO backbone learns adaptive receptive fields through deformable convolutions while simultaneously enhancing spatial awareness using coordinate attention. The resulting multi-scale features are forwarded to the BiFPN-GSConv neck, which performs bidirectional feature fusion to strengthen semantic and localization representations across scales. The fused feature maps are subsequently processed by decoupled detection heads operating at multiple resolutions. During training, the DAA strategy dynamically adjusts positive assignment regions according to local object density. During inference, the OTS mechanism is applied in conjunction with non-maximum suppression (NMS) to determine the optimal operating threshold and produce the final shrimp post-larvae detections.

3.2 DeformDANet detector architecture

The complete detector architecture is illustrated in Figure 2. Given an input image in Equation 1.

(1) I∈ℝ640×640×3

the backbone progressively extracts hierarchical feature representations through four DeformISO stages. The corresponding output feature maps are denoted as in Equations 2–5,

(2) P2∈ℝ24×320×320

(3) P3∈ℝ48×160×160

(4) P4∈ℝ96×80×80

(5) P5∈ℝ192×40×40

These multi-scale representations are forwarded to the BiFPN-GSConv neck for bidirectional feature fusion.

Following feature aggregation, the neck outputs three prediction feature maps defined in Equations 6–8:

(6) N3∈ℝ48×160×160

(7) N4∈ℝ96×80×80

(8) N5∈ℝ192×40×40

which are subsequently processed by the decoupled detection heads.

3.3 DeformISO backbone

The proposed DeformISO backbone is designed to enhance geometric adaptability and spatial awareness for shrimp post-larvae detection. As illustrated in Figure 3, the module integrates deformable convolution and Coordinate Attention (CA) within a dense feature-preservation framework. Conventional convolution layers employ fixed sampling grids that are unable to effectively model the highly curved and irregular morphology of shrimp post-larvae. To overcome this limitation, deformable convolutions are incorporated into the backbone, enabling adaptive spatial sampling and improved geometric representation learning.

Given an input feature map in Equation 9,

(9) X∈ℝCin×H×W

an offset field in Equation 10 is first predicted using a 3 × 3 convolution layer:

(10) Δ=ϕoff(X)

Where,

(11) Δ∈ℝ2K2×H×W

Equation 11 denotes the learned offset map, and K represents the deformable convolution kernel size. For a 3 × 3 kernel, the offset dimension becomes 18.

The learned offsets are subsequently shared by two parallel deformable convolution branches as shown in Equations 12 and 13:

(12) ɡ1=B1(X,Δ)

(13) ɡ2=B2(X,Δ)

where B₁(·) and B₂(·) denote deformable convolution operations with independent learnable weights.

For each spatial location p₀, the deformable convolution output is computed as in Equation 14,

(14) B(X,Δ)=∑k=1K2wkX(p0+pk+Δpk)

where wkdenotes the convolution weight, pk represents the predefined sampling location of the standard convolution kernel, and Δpk corresponds to the learned offset associated with that sampling position.

The outputs of the two deformable branches are aggregated through element-wise summation as defined in Equation 15:

(15) Eɡ=ɡ1+ɡ2

The aggregated feature map is subsequently normalized using batch normalization as in Equation 16:

(16) FBN=BN(Eɡ)

To further enhance spatial awareness, Coordinate Attention (CA) Hou et al. (2021) is employed to encode long-range dependencies along the horizontal and vertical directions while preserving positional information. Given the normalized feature map F_BN, one-dimensional global pooling is performed along the horizontal and vertical directions as shown in Equations 17 and 18:

(17) zh(h)=1W∑w=1WFBN(h,w)

(18) zw(w)=1H∑h=1HFBN(h,w)

The pooled descriptors are subsequently transformed to generate horizontal and vertical attention maps ah and aw, respectively. The refined feature representation is obtained as in Equation 19,

(19) F′=FBN⊙ah⊙aw

where ⊙ denotes element-wise multiplication.

Finally, the attention-enhanced representation is concatenated with the original input feature through a dense feature-preservation pathway as shown in Equation 20:

(20) Y=Concat(X,F′)

resulting in Equation 21,

(21) Y∈ℝ(Cin+Cɡ)×H×W

where Cɡ denotes the growth channels introduced by the deformable convolution branches. By jointly exploiting adaptive deformable sampling and coordinate-aware attention, the proposed DeformISO backbone effectively captures the complex morphology and spatial characteristics of shrimp postlarvae while preserving fine-grained feature information. Consequently, the backbone improves feature representation quality and enhances detection performance without introducing substantial computational overhead.

3.3.1 Backbone output generation

To maintain a compact backbone representation and facilitate hierarchical feature propagation, the concatenated feature map is subsequently compressed using a 1 × 1 projection layer as in Equation 22:

(22) Pi=ϕ1×1(Yi)

where ϕ1×1(·) denotes a pointwise convolution followed by batch normalization and SiLU activation. This projection operation restores the desired channel dimensionality while preserving the complementary information extracted through deformable feature learning and coordinate attention refinement.

Through successive downsampling and DeformISO processing stages, the backbone generates four hierarchical feature maps, P₂, P₃, P₄, and P₅ with progressively decreasing spatial resolution and increasing semantic abstraction.

These multi-scale feature representations encode complementary information across different receptive fields, enabling the network to simultaneously capture fine-grained spatial details and high-level semantic context. Such characteristics are particularly important for shrimp post-larvae detection, where object size, orientation, and density can vary significantly within the same image. The resulting feature maps (P₂, P₃, P₄, P₅) are subsequently forwarded to the BiFPN-GSConv feature aggregation neck, where bidirectional multi-scale fusion and feature refinement are performed to further enhance detection performance.

3.4 BiFPN-GSConv feature aggregation neck

To effectively aggregate multi-scale representations, DeformDANet employs a BiFPN-GSConv feature aggregation module between the DeformISO backbone and the decoupled detection head. The proposed module adopts the bidirectional feature pyramid network (BiFPN) architecture introduced in EfficientDet Tan et al. (2020) while incorporating GSConv Li et al. (2025) to improve feature fusion efficiency with reduced computational complexity. The neck receives hierarchical feature maps (P₂, P₃, P₄, P₅) from the backbone and performs bidirectional feature fusion through top-down semantic propagation and bottom-up localization enhancement. Prior to fusion, lateral 1 × 1 convolutional projections transform P₂, P₄, and P₅ into a unified 48-channel feature space, denoted as L₂, L₄, and L₅, respectively. Since P₃ already contains 48 channels, it is directly utilized and denoted as L₃ = P₃. All feature maps are aligned to identical channel dimensions before feature fusion.

Following the BiFPN design, feature fusion is performed using learnable weighted aggregation. For N input feature maps {xk}k=1N, normalized fusion weights are computed as in Equation 23.

(23) w^k=max (0,wk)∑j=1Nmax (0,wj)+ϵ

where w_kdenotes a learnable fusion parameter and ϵ is a small constant for numerical stability. The weighted aggregation operation is then expressed as Equation 24,

(24) WA(X1,…,XN)=∑k=1Nw^kXk

During the top-down pathway, high-level semantic information is progressively propagated toward finer-resolution features through upsampling and adaptive weighted fusion. Top-down fusion is performed as in Equations Equations 25–27,

(25) TD4=GSConv(WA(L4,Up(L5)))

(26) TD3=GSConv(WA(P3,Up(TD4)))

(27) TD2=GSConv(WA(L2,Up(TD3)))

where Up (·) denotes 2× nearest-neighbor upsampling. The intermediate feature TD₂ is utilized only for subsequent bottom-up feature aggregation and is not directly forwarded to the detection head.

Subsequently, a bottom-up pathway reinforces localization information by integrating shallow spatial details with deeper semantic representations. The bottom-up fusion process is formulated as in Equations 28–30.

(28) BU3=GSConv(WA(P3,TD3,Down(TD2)))

(29) BU4=GSConv(WA(L4,TD4,Down(BU3)))

(30) BU5=GSConv(WA(L5,Down(BU4)))

where Down (·) denotes 2× max-pooling.

3.4.1 GSConv feature refinement

As illustrated in Figure 4, after each fusion node, feature refinement is performed using GSConv (Ghost-Shuffle Convolution) [29], which combines pointwise convolution, depthwise convolution, and channel-shuffle operations to improve feature diversity while maintaining low computational complexity. A fused feature map as in Equation 31.

(31) Ffused∈ℝC×H×W

is first transformed using a 1 × 1 convolution as shown in Equation 32 and defined in Equation 33:

(32) X1=ϕ1×1(Ffused)

Where

(33) X1∈ℝC2×H×W

A depthwise-separable convolution then generates complementary spatial features as shown in Equation 34 and defined in Equation 35.

(34) X2=ϕDS(X1)

where,

(35) X2∈ℝC2×H×W

The two branches are concatenated and shuffled:

(36) Y=Shuffle(Concat(X1,X2))

yielding the refined feature representation by Equation 36. To reduce computational overhead during feature fusion, all intermediate BiFPN features are maintained in a unified 48-channel representation. Following bidirectional aggregation, the refined multi-scale feature maps are projected to the channel dimensions required by the detection heads. Let ϕC(·) denote a 1 × 1 convolutional projection layer that maps the input feature map to C channels.

The final neck outputs are given by Equations 37–39,

(37) N3=BU3∈ℝ48×160×160

(38) N4=ϕ96(BU4)∈ℝ96×80×80

(39) N5=ϕ192(BU5)∈ℝ192×40×40

where ϕ96(·) and ϕ192(·) restore the channel dimensions required by the subsequent detection heads. The resulting feature maps N3, N4, and N5 correspond to the three detection scales and are subsequently forwarded to the decoupled detection heads for object classification and bounding-box regression.

The proposed BiFPN-GSConv neck adopts a bidirectional feature pyramid network and GSConv to jointly improve the representation of multi-scale features with low computation. Specifically, BiFPN learns adaptive fusion weights to combine semantic and localization features across multiple feature levels which are not so well addressed by the conventional FPN and PANet architectures. Besides, GSConv fuses pointwise separable and depthwise separable convolutions with channel shuffle operations to enhance interchannel feature interaction while minimizing computation. The proposed neck generates discriminative representations for accurately detecting densely distributed shrimp larvae with low computational cost, which is suitable for practical real-time aquaculture monitoring, by adaptive multi-scale feature fusion and efficient feature refinement.

3.5 Decoupled detection head

The multi-scale detection heads, Figure 5 generate dense prediction tensors across all pyramid levels, producing approximately 33,600 candidate prediction locations per image. To jointly optimize classification and localization performance, DeformDANet employs decoupled classification and regression branches operating on three feature scales generated by the BiFPN-GSConv neck which is defined as in Equation 40:

(40) H={N3,N4,N5}

where N₃, N₄, and N₅ correspond to feature resolutions of 160×160, 80×80, and 40×40, respectively. For each neck output N_i, the classification branch applies a ConvBNAct block followed by a 1 × 1 convolution to generate class-specific feature representations as in Equation 41:

(41) Fcls(i)=ϕcls(Ni)

where ϕcls(·) denotes the classification subnetwork. The final class probabilities, Pcls(i)∈ℝNc×H×Ware obtained through sigmoid activation with N_cdenotes the number of target classes as shown in Equation 42:

(42) Pcls(i)=σ(Fcls(i))

Similarly, the regression branch employs an independent ConvBNAct block followed by a 1 × 1 convolution to learn localization-specific feature representations as in Equation 43:

(43) Freɡ(i)=ϕreɡ(Ni)

where ϕreɡ(·) denotes the regression subnetwork.

The corresponding Distribution Focal Loss (DFL) [27] - based bounding-box distributions, Breɡ(i)∈ℝ(4×reɡmax)×H×W are generated as in Equation 44,

(44) Breɡ(i)=ψ(Freɡ(i))

and reɡmax=16, following distribution focal loss [27] decoding, the discrete regression distributions are transformed into continuous bounding-box coordinates as defined Equation 45:

(45) B^reɡ(i)=DFL(Breɡ(i))=(x,y,w,h)

where, (x,y) denote the predicted object center coordinates and (w,h) represent the predicted bounding-box width and height.

Predictions are generated from three feature resolutions corresponding to different object scales: N₃, N₄ and N₅ that corresponds to feature maps with spatial resolutions of 160 × 160, 80 × 80, and 40 × 40, respectively. Here, the high-resolution feature map N₃ preserves fine-grained spatial information and is particularly effective for detecting small and densely distributed shrimp post-larvae. The intermediate feature map N₄ provides a balanced representation of spatial details and semantic information, making it suitable for detecting medium-sized objects. In contrast, N₅ captures richer semantic features with a larger receptive field, enabling effective detection of relatively larger shrimp post-larvae. The classification and regression outputs from all detection scales are aggregated to form the unified prediction set as Equation 46:

(46) Y^=∪i=35{Pcls(i),Breɡ(i)}

where Y^ denotes the dense set of raw predictions generated by the multi-scale detection heads. Each prediction consists of a classification confidence score and its corresponding bounding-box distribution representation.

The resulting prediction set Y^ contains all candidate detections generated across the three feature levels and serves as the input to the proposed Density-Adaptive Assignment (DAA) strategy during training. During inference, Y^ is further processed by the Operational Threshold Sweep (OTS) framework to determine the optimal confidence threshold and generate the final detection results. The overall training procedure of the proposed DeformDANet detection framework is summarized in Algorithm 1.

Algorithm 1

3.6 Density-adaptive assignment

The PL predictions in the raw prediction set Y generated by the decoupled detection head are denselydistributed across different feature levels. A fixed positive assign radius is usually used for all objects regardless of spatial distribution in conventional label assignment strategy. These methods work well for moderately populated scenes, but can cause problems with ambiguous assignments in very busy areas and lack of positive supervision in very sparse areas. To overcome this drawback, DeformDANet leverages Density-Adaptive Assignment (DAA) strategy to adaptively modify the radius of the positive assignment based on the local object density, as shown in Figure 6 and Algorithm 2.

Algorithm 2

For each ground-truth shrimp post-larva g_i, local density is estimated using K-nearest-neighbor (KNN) analysis. Specifically, the average distance to the K nearest neighboring objects is computed as in Equation 47,

(47) d¯i=1K∑j=1Kd(ɡi,ɡj)

where d(ɡi,ɡj) denotes the Euclidean distance between the centers of objects ɡi and ɡj, and K = 5 in this work. To normalize density variations across an image, the minimum and maximum neighborhood distances are determined as in Equation 48 and Equation 49,

(48) dmin =mini(d¯i)

(49) dmax =maxi(d¯i)

The adaptive assignment radius is then obtained through min-max normalization as defined in Equation 50:

(50) ri=rmin +(rmax −rmin )d¯i−dmindmax−dmin

where Equation 51 assign,

(51) rmin =0.5, rmax =2.0

To avoid excessively large or small assignment regions, the radius is constrained within predefined bounds:

For a ground-truth object gi, the positive assignment set is defined as in Equation 52,

(52) Ai={c | c∈Bi∧|c−ɡi|2≤ri}

where B_idenotes the ground-truth bounding box, c represents a candidate prediction location, and ∥ · ∥₂ denotes the Euclidean distance.

The final positive sample set used for optimization is obtained by aggregating the assignments from all ground-truth objects as defined in Equation 53:

(53) A=∪i=1NɡAi

where Ngdenotes the number of ground-truth shrimp post-larvae in the image.

Objects located in sparse regions receive larger assignment radii, thereby increasing the number of positive samples, whereas densely clustered objects receive smaller radii to reduce assignment overlap and ambiguity. Consequently, the proposed DAA strategy provides density-aware supervision, improves positive sample quality, and enhances optimization stability for crowded shrimp post-larvae detection.

3.7 Loss function and optimization

To jointly optimize classification confidence and localization accuracy, DeformDANet employs a multitask learning objective consisting of a Focal Binary Cross-Entropy (Focal BCE) classification loss (Lin et al., 2017) and a Weighted Intersection-over-Union (WIoU)-based localization loss (Tong et al., 2023). The positive samples utilized in loss computation are dynamically determined by the proposed DensityAdaptive Assignment (DAA) strategy described in the previous section. Furthermore, the regression branch adopts the distribution-based bounding-box representation introduced in Generalized Focal Loss (GFL) (Li et al., 2020), where each side of the bounding box is represented by a discrete probability distribution and decoded through expectation to obtain continuous localization offsets.

The overall training objective is defined as in Equation 54.

(54) L=Lcls+λLWIoU

where λ denotes the balancing coefficient controlling the contribution of the localization loss.

3.7.1 Classification loss

For object classification, DeformDANet employs Focal Binary Cross-Entropy (Focal BCE) loss (Lin et al., 2017) to alleviate the severe foreground-background imbalance commonly encountered in dense shrimp post-larvae detection.

The classification loss is formulated as in Equation 55.

(55) Lcls=−α(1−pt)γlog (pt)

where p_tdenotes the predicted confidence score, α is a balancing factor, and γ controls hard-example focusing. Label smoothing is additionally applied during training to improve optimization stability and reduce prediction overconfidence.

3.7.2 Bounding-box regression loss

Bounding-box localization is optimized using a Weighted Intersection-over-Union (WIoU)-based loss (Tong et al., 2023), which jointly considers overlap quality and geometric consistency.

The Intersection-over-Union (IoU) between the predicted bounding box B_p and the ground-truth bounding box B_ɡ is defined as in Equation 56.

(56) IoU=|Bp∩Bg||Bp∪Bg|

The corresponding localization loss is computed as in Equation 57.

(57) LWIoU=eρ2/c2(1−IoU)

where ρ denotes the Euclidean distance between the centers of the predicted and ground-truth bounding boxes, and c represents the diagonal length of the smallest enclosing box. By incorporating both overlap quality and geometric consistency, the proposed localization loss improves robustness in densely populated shrimp post-larvae scenes where severe overlap frequently occurs.

3.7.3 Distribution-based bounding-box representation

Instead of directly regressing continuous bounding-box coordinates, DeformDANet adopts the distribution-based bounding-box representation introduced in Generalized Focal Loss (GFL) (Li et al., 2020). For each side of a bounding box, the detector predicts a discrete probability distribution over reɡmax=16 bins.

The corresponding distance prediction is obtained through expectation over the predicted distribution as in Equation 58.

(58) d^=∑i=0regmax−1Pi i

where P_idenotes the predicted probability associated with the i-th distance bin.

The predicted continuous distances are subsequently converted into bounding-box coordinates and optimized using the WIoU-based localization loss described above. This distribution-based regression strategy enables more accurate localization of small and densely distributed shrimp post-larvae while maintaining stable optimization during training.

This combined optimization objective enables accurate classification, robust localization, and precise small-object regression and the adhered loss formulation improves shrimp post-larvae detection performance under varying object densities and challenging aquaculture imaging conditions.

3.8 Operational threshold sweep

The dense prediction set Y generated by the decoupled detection heads, is used with Operational.

Threshold Sweep (OTS) strategy to identify the optimal deployment threshold. To those conventional detectors which is based on a fixed confidence threshold, OTS makes a systematic analysis of detector performance under various operating points to determine the optimum confidence threshold for practical shrimp post-larvae monitoring applications. Let s_idenote the confidence score associated with prediction i. Predictions satisfying, s_i≥ τ are retained, whereas lower-confidence predictions are discarded.

Accordingly, the threshold-filtered prediction set is defined as in Equation 59,

(59) Y^τ={yi∈Y^ | si≥τ}

where Y denotes the raw prediction set generated by the detection head. The confidence threshold is swept across a predefined range: τ ∈ [0.05,0.95]. For each threshold value, confidence filtering is first applied, followed by small-object filtering and Non-Maximum Suppression (NMS). The resulting detection set is represented as in Equation 60,

(60) Bτ=NMS(Y^τ)

where Bτdenotes the final detections obtained at threshold τ. Detection performance is subsequently evaluated using Precision (P), Recall (R), F1-score (F1), True Positive Rate (TPR), and False discovery Rate (FDR). The optimal operating threshold is selected as in Equation 61,

(61) τ*=arg maxτ F1(τ)

The final detection set is therefore obtained as, B=Bτ*where B denotes the final set of detected shrimp post-larvae after confidence threshold optimization and suppression of redundant predictions. The employed OTS strategy provides application-aware threshold optimization and enables systematic trade-off analysis between detection sensitivity and false-alarm suppression. Consequently, DeformDANet can be deployed under different operational requirements by selecting the confidence threshold that maximizes overall detection effectiveness while maintaining robust performance in dense shrimp hatchery environments.

4 Experimental setup

4.1 Dataset and preprocessing

The experiments were conducted using the publicly available PL-Detection dataset from Roboflow Universe (saijaswanthbehara, 2024) and contain a single object category corresponding to shrimp postlarvae. The dataset consists of a single object category for shrimp larvae and is split into three subsets in YOLO object detection format: training (2688), validation (772), and test (377) with a total of 3837 annotated images. Due to the large appearance variance within the dataset caused by the dense distribution of larvae, overlapping individuals, semi-transparent body structures, and various illumination conditions, dense-scene object detection is a significant challenge. Representative dataset samples are shown in Figure 7.

All images were resized, 640 × 640 using an aspect-ratio-preserving letterbox transformation to maintain geometric consistency of elongated larval structures. For an input image I∈ℝH×W×3, the resizing scale factor was computed as in Equation 62,

(62) s=min (640H,640W)

The resized images were padded using constant-value borders to obtain square input representations compatible with the proposed detection framework. Corresponding bounding-box annotations were adjusted according to the applied scaling and padding transformations.

To improve model robustness under varying hatchery imaging conditions, online augmentation strategies including random horizontal flipping, random vertical flipping, HSV-based photometric perturbation, and small-angle rotational transformation were applied during training. Random rotations within [−10∘,10∘] were additionally employed to improve orientation invariance of shrimp larvae detection.

Following preprocessing, images were normalized using ImageNet statistics, with µ = [0.485,0.456,0.406] and σ = [0.229,0.224,0.225]. Bounding-box annotations were represented using normalized YOLO coordinates (cx,cy,w,h) and dynamically converted into pixel-space training targets during mini-batch collation.

4.2 Training configuration

The proposed DeformDANet framework utilises the environment PyTorch 2.10 with CUDA 12.8 acceleration. The experiments were done on an NVIDIA RTX PRO 6000 Blackwell Server Edition.

GPU having 94.97 GB memory within Google Colab environment. The system was configured with 48 CPU cores and 176 GB of system memory. The training was done with a batch size of 16 and fixed resolution of 640 × 640. The network was trained for a maximum of 200 epochs using the AdamW optimizer with an initial learning rate of 1 × 10⁻³ and weight decay of 0.01.

A OneCycle learning-rate scheduling strategy with cosine annealing was adopted to achieve a stable optimization and better convergence performance. Additionally, to prevent overfitting during the training process, the early stopping with a patience value of 50 epochs was also included. An end-to-end training pipeline was combined to incorporate multi-scale dense prediction learning and online photometric and geometric augmentation methods. During back propagation, the maximum norm was set to 10.0, with gradient clipping to make it closer in the training process for the dense-scene optimization scenario. Mixedprecision training was performed using bfloat16 automatic mixed precision to reduce GPU memory usage and accelerate computation. TorchVision 0.25 module, OpenCV 4.13 module, NumPy 2.0 module and Pillow 11.3 module were also used for image preprocessing, image augmentation, and tensor operations, respectively, in the implementation environment.

Every competing models: ShrimpNet, CAGNet, YOLOv5s, YOLOv8n, and O-YOLOv8s-DC, was trained on the PL-Detection dataset using the identical train/validation/test partition, 640 × 640 letterboxed input resolution, augmentation pipeline, optimizer, learning-rate schedule, batch size, and epoch. All models, including DeformDANet, were trained from random initialization without pretrained weights. Model selection followed the same criterion for every method, using the checkpoint with the highest validation mAP@50.

For reproducibility, three independent random seeds (0, 1, and 2) were used for the final experiments, with the same dataset partition and training configuration maintained across all runs. The Focal BCE loss used α = 0.50 and γ = 2.0, with label smoothing set to 0.01. The localization-loss weight was set to λ = 0.20, and the DFL regression parameter was set to reg max = 16. The detection heads operated at strides of 4, 8, and 16, and non-maximum suppression (NMS) used an IoU threshold of 0.30. The model weights were initialized using Kaiming normal initialization with fan-out mode. The classification bias was initialized using −log((1 − p)/p) with p = 0.01, while the deformable offset convolution layers were zero-initialized.

4.3 Evaluation metrics

The proposed framework was evaluated using standard object detection metrics, including Precision, Recall, F1-score, mean average precision at an IoU threshold of 0.5 (mAP@50), and mean average precision across IoU thresholds from 0.5 to 0.95 (mAP@50–95).

Precision and Recall were computed as in Equation 63 and Equation 64:

(63) Precision, P=TPTP+FP

(64) Recall, R=TPTP+FN

where TP, FP, and FN denote true-positive, false-positive, and false-negative detections, respectively.

The F1-score was computed as in Equation 65,

(65) F1=2×PRP+R

Because a well-defined true-negative population is not available in object detection, false discovery rate (FDR) was used instead of the conventional false-positive rate (FPR) to quantify the proportion of false detections among all positive detections. It was calculated as in Equation 66:

(66) FDR(%)=FPTP+FP×100.

For counting evaluation, the predicted number of larvae in each image was compared with the corresponding ground-truth count. Mean absolute error (MAE) and root mean square error (RMSE) were used to measure the absolute counting error as in Equation 67 and Equation 68 respectively:

(67) MAE=1N∑i=1N|Npred,i−NGT,i|

(68) RMSE=1N ∑i=1N(Npred,i−NGT,i)2

where N is the number of test images, and Npred,i and NGT,i denote the predicted and ground-truth larvae counts for image i, respectively. The per-image absolute percentage error was calculated as in Equation 69:

(69) APEi(%)=|Npred,i−NGT,i|NGT,i×100.

Mean and median percentage errors were obtained from the distribution of per-image absolute percentage errors. Counting accuracy within ± k% was reported as the percentage of test images satisfying APE_i≤ k, where k was set to 5, 10, and 20. The aggregate total count error was calculated from the summed predicted and ground-truth counts as given in Equation 70:

(70) Total Count Error(%)=|∑i=1NNpred,i−∑i=1NNGT,i|∑i=1NNGT,i×100.

This aggregate measure was reported together with MAE, RMSE, and per-image percentage errors because over-counting and under-counting across images can partially cancel in the aggregate calculation. Pearson correlation coefficient (r) and coefficient of determination (R²) were also used to assess the agreement between predicted and ground-truth counts.

In addition to detection and counting accuracy, computational efficiency was evaluated using parameter count, and GFLOPs. Operational detection behaviour was further analysed using TP–FP characteristics under varying confidence thresholds.

5 Results

5.1 Quantitative performance evaluation

This section presents the quantitative evaluation of DeformDANet for shrimp post-larvae detection and counting, including baseline comparison, robustness, DAA sensitivity, and generalization.

5.1.1 Quantitative performance comparison

The proposed DeformDANet was evaluated against representative shrimp larvae detection and lightweight object detection frameworks, including ShrimpNet, CAGNet, YOLOv5s, O-YOLOv8s-DC, and YOLOv8n, presented in Table 2. The proposed DeformDANet achieved the highest overall balance between precision, recall, and localization accuracy. Compared with the lightweight Yolov8n model, the proposed framework improved F1-score from 88.3% to 91.8% and increased mAP@50–95 from 60.8% to 65.1%, demonstrating substantially improved localization robustness under dense aquaculture conditions. Although CAGNet achieved competitive mAP@50 performance, its overall F1-score and mAP@50–95 remained substantially lower, indicating poorer localization consistency across IoU thresholds. Similarly, ShrimpNet achieved high precision but suffered from significantly reduced recall, suggesting difficulty in detecting densely clustered larvae. The improved performance of the proposed framework can be attributed to two primary factors: Geometry-adaptive deformable feature extraction, which improves alignment between receptive fields and larval morphology and Density-Adaptive Assignment (DAA), which reduces assignment ambiguity in dense larval clusters. These results demonstrate that fixed-grid convolutional detectors are less effective for handling the deformable and semi-transparent characteristics of shrimp postlarvae. Figure 8, presents the training history of the proposed DeformDANet, illustrating stable optimization through converging training and validation losses, progressive improvements in precision, recall, F1-score, and mAP, together with consistent localization performance and reliable convergence across training epochs.

Table 2. Detection performance comparison with existing methods.

5.1.2 Robustness to random initialization

To assess the robustness of the proposed framework to random initialization, DeformDANet was trained independently using three random seeds (0, 1, and 2). The same dataset partition, model architecture, training configuration, and evaluation protocol were maintained across all runs. The detection results are reported as mean ± standard deviation over the three independent runs in Table 3.

Table 3. Detection performance across three independent training runs with different random seeds.

The model demostrated consistent performance across the three runs, achieving AP₅₀ values of 94.3%, 94.7%, and 94.5% for Seeds 0, 1, and 2, respectively. The corresponding mean performance was 94.5 ± 0.2% AP₅₀, 64.6 ± 1.0% AP_50:95, 88.4 ± 0.5% precision, 95.5 ± 0.3% recall, and 91.8 ± 0.2% F1-score. The relatively small standard deviations indicate limited sensitivity to random initialization and support the stability of the reported detection performance.

5.1.3 Sensitivity analysis of DAA parameters

The main DAA parameters were selected based on the local-density characteristics of the PL-Detection dataset. The neighbourhood size was set to K = 5 to obtain a local estimate of object density, while the adaptive scaling range was defined as r_min = 0.5 and r_max = 2.0. These values allow the assignment region to contract in highly crowded regions and expand in relatively sparse regions. To examine the sensitivity of these choices, additional configurations with K = 3, K = 10, r_min = 1.0, and r_max = 1.0 were evaluated under the same experimental conditions. The results are presented in Table 4.

Table 4. Sensitivity analysis of DAA hyperparameters.

The results show that varying the neighbourhood size from K = 3 to K = 10 produced only small changes in the detection metrics, indicating limited sensitivity to this parameter. Changes to the adaptive scaling limits mainly affected the precision–recall balance. For example, setting r_max = 1.0 increased AP₅₀ to 95.1% and AP_50:95 to 66.3%, but reduced recall to 94.5% and F1-score to 91.3%.

The proposed configuration, with K = 5 and r ∈ [0.5,2.0], achieved an F1-score of 91.7%, with 88.3% precision and 95.5% recall. Although some alternative configurations achieved slightly higher AP values, the proposed setting retains both contraction and expansion of the assignment region according to local density. It was therefore retained as the default DAA configuration for the subsequent experiments.

5.1.4 Generalization and cross-dataset evaluation

To further assess generalization across source images, a three-fold source-photo-grouped leave-one-out evaluation was performed on the complete PL-Detection dataset. The 3,837 image patches were grouped according to their corresponding 106 source photographs. The source photographs were divided into three groups, and in each fold, one group was held out for testing, while the remaining groups were divided into training and validation subsets. Thus, patches originating from the same source photograph were kept within the same fold, preventing source-level overlap between the training and held-out test sets.

Across the three held-out folds in seed 0, DeformDANet achieved 93.9 ± 0.3% AP₅₀, 64.4 ± 0.5% AP_50:95, 87.5 ± 2.2% precision, 94.6 ± 1.9% recall, and 90.9 ± 0.4% F1-score (Table 5). The small variation in AP₅₀ and F1-score indicates consistent performance across the different source-photo groups.

Table 5. Three-fold leave-one-source-out evaluation of DeformDANet on the PL-Detection dataset.

To examine generalization to an unseen dataset, DeformDANet was additionally evaluated in a zeroshot setting on the publicly available PL-Detection dataset released by Teramina (Teramina, 2023). The evaluation used the Seed-0 model at epoch 130 directly, without retraining or fine-tuning on the external data. A total of 748 images from 307 sources were evaluated.

As shown in Table 6, the model achieved an AP₅₀ of 76.9% and an AP_50:95 of 26.9% on the external dataset. The corresponding precision, recall, and F1-score were 86.4%, 75.3%, and 80.5%, respectively. For detection-derived counting, the model achieved an MAE of 5.69 larvae/image and an RMSE of 15.32 larvae/image, with 47.6% and 70.7% of images falling within ±10% and ±20% counting error, respectively.

Table 6. In-domain and external zero-shot evaluation of DeformDANet.

Note: “Images within ± X% error” denotes the percentage of test images for which the predicted larval count differs from the ground-truth count by no more than X%.

Compared with the in-domain results, the reduction in detection and counting performance indicates a noticeable domain shift between the two datasets. Nevertheless, the model retained useful detection and counting capability without external-data retraining or fine-tuning. Representative detection results from the in-domain test set and the external zero-shot test are shown in Figure 9 to illustrate the model’s behaviour on familiar and unseen images.

5.2 Computational efficiency analysis

Table 7 compares the proposed model with existing approaches, the proposed DeformDANet achieves the highest mAP@50–95 of 65.1% among all evaluated detectors while requiring only 1.38 M trainable parameters and 6.82 GFLOPs. Although CAGNet attains a slightly higher mAP@50 (95.0%), it requires 14.42 M parameters and 35.28 GFLOPs, substantially exceeding the complexity of the proposed framework. Compared with ShrimpNet and YOLOv5s, DeformDANet simultaneously improves detection accuracy and reduces computational complexity. Compared with YOLOv8n, the proposed model achieves similar AP₅₀ performance (94.5% vs. 94.4%) and improves AP_50:95 by 4.3 percentage points (65.1% vs. 60.8%), while using only 54% of the parameters. However, it requires more computation (6.82 vs. 3.50 GFLOPs). This is mainly because DeformDANet uses a finer stride-4 detection scale, producing a 160×160 feature grid for a 640×640 input, whereas YOLOv8n uses stride 8 as its finest detection scale. The finer grid helps distinguish and localize individual post-larvae in dense and overlapping clusters, leading to better localization accuracy, but it also increases the computational cost. These results highlight that DeformDANet effectively balances detection accuracy in dense scenes with computational efficiency for shrimp post-larvae monitoring.

Table 7. Computational efficiency and detection performance comparison of the evaluated shrimp postlarvae detectors.

5.3 Error analysis and detection reliability

To further evaluate dense-scene localization reliability, true-positive (TP) and false-positive (FP) behaviour was analysed for representative baseline, ablation, and state-of-the-art models. The results are summarized in Table 8.

Table 8. Detection error analysis of competing methods.

Although YOLOv8n achieves the highest TPR (97.5%), it also produces the highest FDR (19.4%) and the largest number of false positives (3180), suggesting that the model tends to over-detect larvae in densely populated scenes, which may adversely affect practical counting accuracy. Contrary to this, the proposed DeformDANet achieves the best true positive rate (TPR) of 95.8% and (FDR) of 11.8% among all compared methods. YOLOv5s records a lower FDR (9.4%), but with reduced sensitivity (TPR 86.9%), corresponding to approximately 1,200 more missed larvae across the test set. Furthermore, it is observed that DeformDANet can effectively suppress the false positives compared with ShrimpNet, O-YOLOv8s-DC, and YOLOv8n, demonstrating excellent robustness in dense clustered environments. These results indicate that the proposed method provides a good trade-off between detection sensitivity and false positive suppression, thereby achieving more accurate and reliable shrimp post-larvae detection and counting.

5.4 Ablation study

Table 9 highlights the contribution of each model component in the proposed DeformDANet framework. Removing both DeformISO and DAA yielded 91.0% mAP@50 and 59.2% mAP@50–95. With only DAA, the model achieved 94.0% mAP@50 and 62.9% mAP@50–95, together with an increase in recall from 94.1% to 96.6%, indicating the contribution of density-adaptive positive-sample assignment to dense-scene detection. In contrast, the DeformableISOBlock alone did not provide a consistent improvement across the two mAP measures, achieving 91.0% mAP@50 and 59.2% mAP@50–95. This suggests that the improved morphology-aware feature representation provided by DeformISO alone is not sufficient to fully address the positive-sample assignment challenges in dense post-larvae scenes. Its contribution becomes more evident when combined with DAA. In particular, mAP@50–95 increased from 61.9% for ISO2 + DAA to 63.0% for ISO3 + DAA and reached 65.1% for the proposed ISO4 + DAA configuration. The three-branch configuration provided only a small gain in mAP@50–95 (63.9%) while increasing the parameter count to 1.43 M. In comparison, the proposed ISO4 + DAA configuration (DeformDANet) achieved the best overall performance, with 94.5% mAP@50 and 65.1% mAP@50–95 using 1.38 M parameters. These results indicate that DeformISO and DAA play complementary roles, with DeformISO enhancing feature representation and DAA improving density-adaptive positive-sample assignment, while their combination provides the best balance between detection accuracy and model complexity.

Table 9. Ablation analysis of proposed components.

The ablation on true and false prediction analysis of the suggested DeformDANet component is shown in Table 10. With the highest TPR (96.7%) and the lowest FN count (446), the DAA module dramatically increases detection sensitivity, demonstrating its efficacy in lowering missed detections in crowded shrimp post-larvae scenarios. The DeformableISOBlock, on the other hand, mainly improves morphology-aware learning to improve feature representation, while its individual contribution is less noticeable when used alone. When all modules are combined, the full DeformDANet maintains a high TPR (95.8%) while achieving the lowest FDR (11.8%) and fewest false positives (1745). These findings show that the proposed network produces the most balanced detection performance where, the given DeformableISOBlock increases feature discrimination and leveraged DAA mainly improves positive sample assignment.

Table 10. Ablation on error analysis and detection reliability of the proposed DeformDANet.

5.5 Qualitative detection analysis

Qualitative detection tests were carried out on different shrimp larvae densities, transparencies and overlapping cases to better verify the real robustness of the proposed framework. As depicted in Figure 10, the proposed framework is able to detect shrimp post-larvae both in sparse and crowded aquaculture environments. Our results show that our model is not sensitive to the change of the numbers of objects or to low contrast and light conditions. Semi-transparent larvae close to tank edges and in reflective areas showed precise bounding-box positioning. Also, Figure 11 highly illustrates that shrimp-PL were correctly localized despite substantial overlap and background interference, with only a small number of missed or ambiguous detections in the most congested regions. The anticipated object quantities in various test scenes closely matched the true annotations, showcasing the reliable ability for dense-scene generalization. These visual results indicate that given DeformDANet can maintain detection reliability across a range of challenging shrimp hatchery imaging conditions characterized by large variations in density, morphology, and visibility.

5.6 Operational threshold sweep

An operational threshold sweep was performed to determine an appropriate operating point for detection and detection-derived population estimation. Only the validation partition was used to sweep the confidence threshold from 0.05 to 0.93 in steps of 0.035, giving 26 threshold points. As the confidence threshold increased, precision increased while recall gradually decreased, as shown in Figure 12a. The threshold yielding the highest F1-score on the validation partition was selected as the primary operating point. Figure 12b presents the precision–recall operating curve on the test set, while Figure 12c illustrates the relationship between TPR (recall) and FDR at different confidence thresholds on the validation partition. These results show the trade-off between sensitivity and false detections. The main operating threshold was τ = 0.505, while τ = 0.575 was identified as the maximum-recall operating point with FDR < 10%. Both thresholds were frozen before test-set evaluation, and the test set was not used for threshold selection.

5.7 Counting accuracy analysis

The counting performance of DeformDANet was evaluated using the ground-truth and predicted larvae counts on the test dataset. The analysis considered both a fixed confidence threshold and a validationselected operational threshold.

5.7.1 Overall counting accuracy

Figure 12 presents the regression relationship, Bland–Altman agreement analysis (Bland and Altman, 1986), and validation-based threshold selection. Figure 13a shows a strong agreement between the predicted and ground-truth counts across different larval densities. The Bland–Altman plot in Figure 13b shows that most prediction errors remain close to the mean difference, indicating consistent counting behaviour. Figure 13c shows the validation-based selection of the confidence threshold used for the counting evaluation.

Table 11 summarizes the counting performance over three independent runs. Using the fixed confidence threshold of τ = 0.50, the model achieved an MAE of 3.60 ± 0.07 larvae/image and an RMSE of 5.41 ± 0.13 larvae/image. With the validation-selected OTS, the MAE decreased to 2.75 ± 0.11 larvae/image and the RMSE to 4.91 ± 0.23 larvae/image. The mean and median absolute percentage errors also decreased from 24.08 ± 0.87% to 19.17 ± 0.50% and from 9.21 ± 0.34% to 6.48 ± 0.58%, respectively.

Table 11. Counting performance under the fixed confidence threshold and the validation-selected OTS threshold.

Note: “Images within ± X% error” denotes the percentage of test images for which the predicted larval count differs from the ground-truth count by no more than X%.

Values are reported as mean ± standard deviation over three independent runs.

The proportion of test images for which predicted counts were within ±5%, ± 10%, and ±20% of the ground-truth counts increased with OTS, reaching 43.67 ± 2.19%, 63.20 ± 2.91%, and 78.87 ± 2.17%, respectively. The total count error also decreased from 8.29 ± 0.83% to 0.84 ± 0.56%. The total count error was calculated as defined in Equation 71,

(71) Total Count Error(%)=|∑i=1NC^i−∑i=1NCi|∑i=1NCi×100,

where Ci and C^idenote the ground-truth and predicted larval counts for image i, respectively, and N is the number of test images. This aggregate formulation measures the difference between the total predicted and total ground-truth populations and therefore does not directly measure the absolute error of each individual image. Over-counting and under-counting across individual images may partially cancel each other; therefore, total count error is interpreted together with MAE, RMSE, and per-image absolute percentage-error measures rather than as a standalone indicator of counting accuracy.

Pearson correlation and R² remained high for both settings, although both showed a small decrease with OTS, from 0.9940 ± 0.0004 to 0.9923 ± 0.0006 and from 0.9880 ± 0.0008 to 0.9848 ± 0.0011, respectively. Overall, the validation-based OTS improves the error-based and within-error-band counting measures while maintaining a high level of agreement between predicted and ground-truth counts. Thus, OTS provides a validation-based operating point for subsequent detection-derived population estimation without test-set threshold tuning, rather than uniformly optimizing all counting statistics.

5.7.2 Density-wise counting performance

To examine the effect of larval density on counting performance, the test images were grouped into sparse, moderate, dense, and very dense categories according to the ground-truth larvae count. DeformDANet was compared with the established density-regression methods MCNN (Zhang et al., 2016) and CSRNet (Li et al., 2018), and the results are reported in Table 12. The four predefined density ranges were 1–30, 31–80, 81–150, and > 150 larvae per image. These ranges were used solely for stratifying and reporting the counting performance under different scene densities. They did not influence model training, inference, confidence-threshold selection, or any model hyperparameter. A single validation-selected confidence threshold was applied to all test images, irrespective of their density. The number of test images in each density group is also reported to indicate the distribution of the test set across the different density ranges.

Table 12. Counting performance stratified by ground-truth larvae density.

DeformDANet achieved competitive performance across the different density ranges. Its MAE was 1.53 larvae/image for sparse scenes and increased to 3.55 and 5.92 larvae/image for moderate and dense scenes, respectively. For the very-dense category, DeformDANet achieved an MAE of 4.29 larvae/image, compared with 7.05 for CSRNet and 41.10 for MCNN. This indicates that DeformDANet maintains comparatively lower counting error in highly crowded scenes, where severe overlap makes individual larva localisation more difficult. Overall, CSRNet obtained the lowest aggregate MAE, while DeformDANet provided competitive counting accuracy together with individual larva localisation.

5.7.3 Representative counting examples

Representative examples from different density and image conditions are presented in Table 13. The examples illustrate how counting errors vary with scene density. In sparse scenes, even a small absolute error can produce a relatively large percentage error because of the low number of larvae. For example, E1 has an absolute error of only two larvae but an accuracy of 81.8%.

Table 13. Representative counting examples under varying density conditions.

In dense and very dense scenes, the model maintains more consistent counting accuracy despite larval aggregation and overlap. The very-dense examples (E6–E9) achieve accuracies above 95%, while the challenging illumination case in E10 achieves 93.5%. Overall, these examples provide qualitative support for the quantitative counting results reported in Table 11 and 12.

Counting accuracy was calculated as in Equation 72:

(72) Accuracy(%)=(1−|Npred−NGT|NGT)×100

6 Discussion and limitations

The experimental results demonstrate that the proposed DeformDANet effectively addresses the challenges of dense shrimp post-larvae detection, including severe overlap, irregular morphology, semi-transparent body structures, and large density variations. The consistent improvements achieved across quantitative evaluations indicate that integrating morphology-aware feature extraction with density-aware label assignment enhances localization robustness in crowded aquaculture scenes.

The complementary design of the proposed framework contributes to these improvements. The DeformISO module adapts feature extraction to the curved geometry of shrimp post-larvae, improving representation of deformable structures, whereas the Density-Adaptive Assignment (DAA) strategy provides more appropriate supervision in crowded regions by reducing assignment ambiguity among neighbouring objects. The ablation study further confirms that these two components are mutually complementary, producing greater performance gains when used together than individually. The three independent runs showed low variation in the main detection metrics, indicating stable performance under different random initializations. The density-wise analysis also showed that DeformDANet remained competitive in very dense scenes, where severe overlap makes individual larva localization more difficult.

The proposed framework also maintains a favourable balance between detection accuracy and computational efficiency. Along with its compact architecture, DeformDANet achieves reliable localization across a wide range of scene densities, making it well suited for automated population estimation in aquaculture applications.

With these findings, the proposed DeformDAnet also have limitations. First, the framework was trained and mainly tested with the public available shrimp post-larvae data that were collected under a certain hatchery condition. The leave-one-out evaluation based on the source photograph and the external zero-shot test offer preliminary evidence of generalization, but they need to be explored to check their robustness across various levels of hatcheries, imaging devices, water quality, illumination variation, and developmental stage of larvae. Secondly, the existing framework only works on single images and fails to take advantage of temporal information from a video sequence that could lead to better detection stability and counting consistency. Lastly, the proposed model is computationally efficient but has not yet been experimentally tested on embedded devices at the edge. Future research will thus be directed towards cross-environment validation, domain adaptation, temporal modelling, and hardware-optimisation for practical aquaculture monitoring systems. The zero-shot results suggest initial evidence of cross-dataset generalization, but additional testing with independent datasets and deployment-level hardware is needed to demonstrate robustness across a variety of practical real-world scenarios and to prepare for wider practical deployment.

7 Conclusion

This paper presented DeformDANet, a lightweight dense-scene shrimp post-larvae detection framework that integrates geometry-adaptive deformable feature extraction with density-aware target assignment. The proposed DeformableISOBlock improves receptive-field alignment with deformable larval morphology, while the Density-Adaptive Assignment (DAA) mechanism enhances localization stability under severe overlap and aggregation conditions commonly encountered in commercial hatchery environments.

Extensive experimental evaluation demonstrated that the proposed framework achieves superior detection accuracy, improved false-positive suppression, and reliable detection-derived counting performance while maintaining a compact architecture consisting of only 1.38M parameters and 6.82 GFLOPs. The additional operational threshold analysis and counting evaluation further confirmed stable detection behaviour across varying larval density conditions, with adaptive thresholding reducing the overall counting error from 9.23% to 2.11% while maintaining strong agreement with ground-truth counts.

Overall, the proposed DeformDANet provides accurate and computationally efficient performance, showing promise for practical deployment in automated shrimp post-larvae detection, counting, and monitoring in dense aquaculture imagery. The proposed framework therefore offers considerable potential for future intelligent hatchery management systems and resource-constrained aquaculture imaging applications.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Seyyed Morteza Hoseini, Iranian Fisheries Science Research Institute (IFSRI), Iran

Reviewed by: Jianping Li, Chinese Academy of Sciences (CAS), China

Charlie Seli Marzan, Don Mariano Marcos Memorial State University, Philippines