DyCAF-Net: Dynamic Class-Aware Fusion

• The paper introduces an input-conditioned equilibrium-based fusion approach that iteratively refines multi-scale features using implicit fixed-point modeling.
• It presents a dual dynamic attention mechanism that integrates input- and class-dependent cues to recalibrate channels and spatial responses.
• The model employs class-aware feature adaptation to prioritize rare classes, yielding measurable mAP and precision gains over baseline detectors.

Searching arXiv for the specified paper and a few directly related methods mentioned in the provided data. Dynamic Class-Aware Fusion Network (DyCAF-Net) is an object detection framework introduced by Jahin et al. in "DyCAF-Net: Dynamic Class-Aware Fusion Network" (Jahin et al., 5 Aug 2025). It retains the standard Backbone $\rightarrow$ Neck $\rightarrow$ Head design, but replaces YOLOv8’s PANet neck with an input-conditioned equilibrium-based neck and augments the detection pipeline with dual dynamic attention and class-aware feature adaptation. The model is presented as a response to two limitations in contemporary detectors: static fusion heuristics in multi-scale aggregation and class-agnostic attention in feature recalibration. Within the reported formulation, DyCAF-Net targets dynamic scenes with occlusions, clutter, scale variance, semantic overlaps, and class imbalance while maintaining a computational profile of approximately $11.13$M parameters and competitive inference speed (Jahin et al., 5 Aug 2025).

1. Conceptual basis and architectural rationale

The design rationale begins from a critique of fixed multi-scale fusion rules. The paper identifies FPN, PANet, and BiFPN as representative static fusion schemes that rely on fixed up-/down-sampling and pre-defined summation or weighted summation rules. According to the reported motivation, such heuristics cannot adapt to per-image scale relationships and struggle in scenes with severe occlusion, clutter, or unusual object size distributions (Jahin et al., 5 Aug 2025).

A parallel critique is directed at class-agnostic attention, with SENet, CBAM, DyNet, and deformable attention listed as examples. In the reported account, these mechanisms adjust channels or spatial masks based only on input features while ignoring which class is being predicted. The stated consequences are ambiguous feature emphasis under occlusion or background clutter and poor performance on long-tailed datasets where rare classes are under-represented (Jahin et al., 5 Aug 2025).

DyCAF-Net therefore reorganizes the detector around three coupled innovations. The first is an input-conditioned equilibrium-based neck that iteratively refines multi-scale features via implicit fixed-point modeling. The second is a dual dynamic attention mechanism that adaptively recalibrates channel and spatial responses using input- and class-dependent cues. The third is class-aware feature adaptation, which modulates features to prioritize discriminative regions for rare classes. In the reported high-level block diagram, the backbone feeds an equilibrium-based neck containing iterative multi-scale fusion, dual dynamic attention modules, and a class-aware feature adaptation sub-network, followed by a detection head with CIoU, classification, and objectness objectives and class-aware recalibration (Jahin et al., 5 Aug 2025).

2. Input-conditioned equilibrium-based neck

Instead of stacking fixed fusion blocks, DyCAF-Net solves for a multi-scale feature equilibrium. Let $X=\{F_1,F_2,\ldots,F_L\}$ denote the initial pyramid of backbone features at $L$ scales. The fused representation $F^*\in\mathbb{R}^{H\times W\times C}$ is defined by the equilibrium equation

$F^*=\Phi\bigl(F^*,X;\theta\bigr),$

where $\Phi$ is a lightweight convolutional fusion operator with parameters $\theta$ (Jahin et al., 5 Aug 2025).

The practical solver is iterative. The initialization is described as $F^{(0)}=\text{some function of }X$ such as up/down-samples and concatenations, followed by

$\rightarrow$0

Here $\rightarrow$1, $\rightarrow$2 approximates the Jacobian of $\rightarrow$3, and Broyden’s method is used to avoid storing full Jacobians. Convergence is declared when

$\rightarrow$4

The implementation details reported for $\rightarrow$5 are two depthwise $\rightarrow$6 convolutions with SiLU activation, together with a $\rightarrow$7 convolution that computes per-scale softmax weights $\rightarrow$8 over neighboring scales (Jahin et al., 5 Aug 2025).

The reported operating point uses $\rightarrow$9 Broyden iterations, corresponding to approximately $11.13$0 ms overhead per image. Empirically, the neck is said to achieve $11.13$1 less memory and to converge in $11.13$2–$11.13$3 steps across datasets. This positions the equilibrium solver not merely as a replacement for PANet-style stacking, but as a mechanism for input-conditioned fusion in which the final representation is defined implicitly rather than by a fixed sequence depth (Jahin et al., 5 Aug 2025).

3. Dual dynamic attention mechanism

The dual dynamic attention module injects both input-conditioned and class-conditioned cues into channel and spatial attention. For an intermediate feature map $11.13$4 and a class embedding $11.13$5, channel attention is defined as

$11.13$6

where $11.13$7 is global average pooling, $11.13$8 denotes concatenation, $11.13$9, $X=\{F_1,F_2,\ldots,F_L\}$0, $X=\{F_1,F_2,\ldots,F_L\}$1 is ReLU or SiLU, and $X=\{F_1,F_2,\ldots,F_L\}$2 is sigmoid (Jahin et al., 5 Aug 2025).

Spatial attention is defined by combining average, max, and class cues: $X=\{F_1,F_2,\ldots,F_L\}$3 The refined feature is then

$X=\{F_1,F_2,\ldots,F_L\}$4

where $X=\{F_1,F_2,\ldots,F_L\}$5 broadcasts $X=\{F_1,F_2,\ldots,F_L\}$6 along spatial dimensions and $X=\{F_1,F_2,\ldots,F_L\}$7 is element-wise multiplication. The residual addition $X=\{F_1,F_2,\ldots,F_L\}$8 is explicitly included to stabilize gradients (Jahin et al., 5 Aug 2025).

Within the reported architecture, this mechanism is significant because it departs from attention that is purely input-conditioned. The module conditions feature amplification not only on pooled activations but also on class embeddings, thereby tying recalibration to the prediction context. This does not alter the broader detector topology, but it changes the semantics of feature selection inside the neck and before the head.

4. Class-aware feature adaptation

Class-aware feature adaptation appears in the neck’s final stage and again right before the head. Its stated purpose is to modulate features so that rare or underrepresented classes receive greater emphasis. The primary formulation is prototype-based masking. The model pre-computes $X=\{F_1,F_2,\ldots,F_L\}$9 class prototypes $L$0 by k-means on training features. For each class $L$1,

$L$2

where $L$3 denotes cross-correlation. Features are then projected and weighted as

$L$4

The final fused feature is

$L$5

This formulation makes the feature map explicitly class-indexed before aggregation (Jahin et al., 5 Aug 2025).

The paper also presents an alternative gating view in which the same adaptation can be interpreted as a multi-head self-attention block. In that view, $L$6, $L$7, $L$8, and $L$9 is the class embedding added back. A plausible implication is that the authors intend the prototype mechanism to be read not only as masking but also as a structured class-conditioned attention operator over learned class anchors (Jahin et al., 5 Aug 2025).

This adaptation is central to the model’s treatment of long-tailed data. The reported motivation is not generic feature refinement, but prioritization of discriminative regions for rare classes. In that sense, DyCAF-Net couples class awareness to both attention weights and feature synthesis, rather than restricting it to the final classifier.

5. Integration with YOLOv8, training configuration, and computational profile

In the reported implementation, YOLOv8’s PANet neck is removed and replaced with DyCAF-NetNeck modules consisting of equilibrium fusion and dual attention. Each detection head branch for each scale is augmented with a lightweight class-aware adaptation block before final classification and regression. Across the experiments on $F^*\in\mathbb{R}^{H\times W\times C}$0 datasets, the CSPDarknet53 backbone and YOLOv8 head are kept unchanged (Jahin et al., 5 Aug 2025).

The reported model size and speed are framed as computational parity with YOLOv8-class detectors. Total parameters are approximately $F^*\in\mathbb{R}^{H\times W\times C}$1M, described as identical to YOLOv8-PANet. FLOPs are approximately $F^*\in\mathbb{R}^{H\times W\times C}$2 GFLOPs per $F^*\in\mathbb{R}^{H\times W\times C}$3 image. Inference speed is reported as $F^*\in\mathbb{R}^{H\times W\times C}$4–$F^*\in\mathbb{R}^{H\times W\times C}$5 ms per image on NVIDIA T4 with FP16 mixed precision, with an average of approximately $F^*\in\mathbb{R}^{H\times W\times C}$6 FPS, compared with approximately $F^*\in\mathbb{R}^{H\times W\times C}$7 FPS for YOLOv8-PANet (Jahin et al., 5 Aug 2025).

Training and evaluation use $F^*\in\mathbb{R}^{H\times W\times C}$8 Roboflow100 domains, consisting of $F^*\in\mathbb{R}^{H\times W\times C}$9 real-world and $F^*=\Phi\bigl(F^*,X;\theta\bigr),$0 electromagnetic domains, together with $F^*=\Phi\bigl(F^*,X;\theta\bigr),$1 public datasets: Final Year, Traffic Density, and Cityscapes. Augmentation consists of random flips, rotations, mosaic, and color jitter, with all inputs resized to $F^*=\Phi\bigl(F^*,X;\theta\bigr),$2. The optimization setup is SGD with momentum $F^*=\Phi\bigl(F^*,X;\theta\bigr),$3, weight decay $F^*=\Phi\bigl(F^*,X;\theta\bigr),$4, initial learning rate $F^*=\Phi\bigl(F^*,X;\theta\bigr),$5 with cosine decay and final multiplier $F^*=\Phi\bigl(F^*,X;\theta\bigr),$6, batch size $F^*=\Phi\bigl(F^*,X;\theta\bigr),$7, $F^*=\Phi\bigl(F^*,X;\theta\bigr),$8 epochs, and neck dropout $F^*=\Phi\bigl(F^*,X;\theta\bigr),$9 (Jahin et al., 5 Aug 2025).

The training objective contains the standard detection loss—CIoU box loss, classification loss, and objectness loss—together with two additional terms. The equilibrium consistency term is

$\Phi$0

and the class-attention regularizer is

$\Phi$1

These auxiliary losses connect the architecture’s implicit solver and class-aware masking to explicit optimization constraints (Jahin et al., 5 Aug 2025).

6. Empirical results and ablation evidence

The paper reports benchmarking against nine state-of-the-art baselines and presents precision, mAP@50, and mAP@50–95 over $\Phi$2 diverse benchmarks. DyCAF-Net is reported to yield consistent mAP gains in $\Phi$3 domains, especially under occlusion and class imbalance. The average gain over the benchmark suite is reported as $\Phi$4 precision, $\Phi$5 mAP@50, and $\Phi$6 mAP@50–95 (Jahin et al., 5 Aug 2025).

Selected dataset-level results illustrate both the gains and the non-uniformity of those gains. On Final Year, YOLOv8 records $\Phi$7 mAP@50 and $\Phi$8 mAP@50–95, whereas DyCAF-Net records $\Phi$9 and $\theta$0. On Animals, the values change from $\theta$1 to $\theta$2 at mAP@50 and from $\theta$3 to $\theta$4 at mAP@50–95. On Mask Wearing, the reported values are $\theta$5 to $\theta$6 and $\theta$7 to $\theta$8. On Axial MRI, they are $\theta$9 to $F^{(0)}=\text{some function of }X$0 and $F^{(0)}=\text{some function of }X$1 to $F^{(0)}=\text{some function of }X$2. On X-Ray, they are $F^{(0)}=\text{some function of }X$3 to $F^{(0)}=\text{some function of }X$4 and $F^{(0)}=\text{some function of }X$5 to $F^{(0)}=\text{some function of }X$6. By contrast, Cityscapes shows a slight decrease, from $F^{(0)}=\text{some function of }X$7 to $F^{(0)}=\text{some function of }X$8 at mAP@50 and from $F^{(0)}=\text{some function of }X$9 to $\rightarrow$00 at mAP@50–95 (Jahin et al., 5 Aug 2025).

The ablation study measures the drop in macro-F1 averaged across the $\rightarrow$01 datasets after removing components. Removing dual attention yields $\rightarrow$02 relative to the full model; removing equilibrium fusion yields $\rightarrow$03; removing class-aware adaptation yields $\rightarrow$04; and removing all three, corresponding to PANet, yields $\rightarrow$05 (Jahin et al., 5 Aug 2025).

Taken together, the results attribute the largest isolated F1 degradation to equilibrium fusion, while also indicating that the three components are not redundant. The full model’s gains are represented as cumulative rather than attributable to a single substitution.

7. Reported limitations, failure cases, and proposed extensions

The paper explicitly notes failure cases. It reports slight precision drops in very low IR domains, with Cityscapes given as an example, and states that static heuristics suffice in such settings (Jahin et al., 5 Aug 2025). This is important because it bounds the method’s claims: DyCAF-Net is not presented as uniformly superior across every dataset regime.

A second limitation concerns complexity trade-off. The implicit equilibrium layers add approximately $\rightarrow$06–$\rightarrow$07 ms per image, although the paper states that they cut memory by approximately $\rightarrow$08 relative to deep stacked necks (Jahin et al., 5 Aug 2025). The resulting computational picture is therefore mixed rather than absolute: memory efficiency improves, but latency is not free.

The reported extensions are video object detection through temporal equilibrium across frames, medical imaging segmentation through class-aware attention to rare pathologies, and few-shot detection through prototype adaptation to novel classes (Jahin et al., 5 Aug 2025). These directions are consistent with the architecture’s emphasis on implicit fusion dynamics and class-conditioned modulation. A plausible implication is that the framework is intended less as a YOLOv8-specific modification than as a general design pattern for detection settings where scale variance, semantic overlap, and long-tail effects interact.

In summary, DyCAF-Net combines implicit equilibrium-based multi-scale fusion, dual dynamic attention, and class-aware feature adaptation within a YOLOv8-compatible detector. Its contribution lies in replacing fixed fusion heuristics and purely class-agnostic attention with input- and class-conditioned mechanisms, and the reported evidence indicates measurable gains on occlusion-heavy and long-tailed benchmarks while preserving approximate parameter parity with YOLOv8-PANet (Jahin et al., 5 Aug 2025).