Dynamic Fusion Gateway (DFG) in ACD-CLIP

• Dynamic Fusion Gateway (DFG) is a module that adaptively fuses multi-level text descriptors with visual features, addressing the fusion gap in zero-shot anomaly detection.
• It leverages visual context from multiple layers to tailor normal and abnormal text descriptors, thereby enhancing the segmentation accuracy in anomaly mapping.
• Empirical studies show that combining DFG with Conv-LoRA significantly improves pixel AUROC and image AUROC in the ACD-CLIP framework.

Dynamic Fusion Gateway (DFG) is a fusion module introduced in the Architectural Co-Design framework for zero-shot anomaly detection with CLIP-derived vision–LLMs. In that formulation, DFG addresses the “fusion gap” of static CLIP-based anomaly detection by using visual context to adaptively mix multi-level text semantics, producing level-specific normal and abnormal text descriptors that are then matched against multi-level visual features. Within the full ACD-CLIP design, DFG is paired with Conv-LoRA, which targets the separate “representation gap” by injecting local inductive bias into the visual encoder for dense prediction (Ma et al., 11 Aug 2025).

1. Problem setting and motivation

Zero-Shot Anomaly Detection (ZSAD) seeks to localize and classify anomalies without seeing anomaly examples during training. Pre-trained vision–LLMs are attractive in this regime because they already encode broad visual and textual semantics, but the CLIP backbone is optimized for global classification rather than pixel-level anomaly localization, and many CLIP-based ZSAD pipelines rely on rigid, static fusion between visual and textual features. The motivating critique is therefore twofold: a representation gap, caused by the lack of local inductive biases for dense prediction, and a fusion gap, caused by inflexible one-to-one alignment between visual feature levels and text feature levels (Ma et al., 11 Aug 2025).

In the static fusion paradigm described for prior CLIP-based ZSAD methods, visual features from a fixed layer are compared to pre-defined or learnable text prompts such as “normal object” and “defective object,” and the correspondence between visual and text features is fixed as layer $i \leftrightarrow$ text embedding $i$. The DFG proposal replaces that rigid pairing. Its central premise is that visual context at level $i$ should determine how text semantics from all text levels are mixed for that level, because anomalies vary in scale, texture, and semantics across images.

Within ACD-CLIP, DFG sits between the Conv-LoRA–enhanced vision encoder and the multi-level outputs of the text encoder. Its input is a bank of multi-level vision features $\{V_i\}_{i=1}^N$ together with multi-level normal and abnormal text features $\{(T_j^N, T_j^A)\}_{j=1}^N$. Its output is a pair of tailored descriptors $(T_i^N, T_i^A)$ for each visual level $i$, which are then used to generate the level-specific anomaly map $M_i$.

2. Placement within ACD-CLIP

The architectural context of DFG is defined by symmetric grouping on the vision and text sides. The CLIP vision encoder, implemented as a ViT, is partitioned into $N$ sequential Groups, and each group outputs a feature block $V_i$. A Conv-LoRA adapter is inserted into each group and operates on the sequence output of that group by low-rank projection, reshaping, multi-branch convolution, and projection back to a residual $i$0, which is added to the main stream. This produces richer vision features intended to be local-detail-aware (Ma et al., 11 Aug 2025).

The text encoder is likewise partitioned into $i$1 groups, yielding a hierarchy of text representations. For each level $i$2, the architecture defines $i$3 for the normal descriptor and $i$4 for the abnormal descriptor. These descriptors form the multi-level text feature bank: $i$5

The resulting division of labor is concise.

Component	Input	Output
Vision groups with Conv-LoRA	ViT group features	$i$6
Text groups	text encoder hierarchy	$i$7
DFG	$i$8 and $i$9	$i$0

This design makes DFG a level-wise mediator rather than a standalone encoder. It does not originate visual or textual features; instead, it adaptively composes the text hierarchy into descriptors specialized for each visual level. A plausible implication is that the module is best understood as a conditional selector over semantic depth rather than as a generic multimodal attention block.

3. Formal mechanism

For each visual level $i$1, DFG first extracts a global visual context from $i$2, then maps that context through a gating MLP to produce logits over the $i$3 text levels for the normal and abnormal branches separately. After Softmax normalization, it uses the resulting weights to form weighted sums over the multi-level text bank (Ma et al., 11 Aug 2025).

The weighting equations are: $i$4

$i$5

$i$6

The paper states that a global context vector is extracted from $i$7 and passed through $i$8 to produce logits over all $i$9 text feature levels. A typical instantiation given in the details is: $\{V_i\}_{i=1}^N$0

$\{V_i\}_{i=1}^N$1

with the output split into two vectors of length $\{V_i\}_{i=1}^N$2. In functional form,

$\{V_i\}_{i=1}^N$3

Two architectural constraints are explicit. First, DFG is not token-level cross-attention; it operates at the level of global context and text-level weights. Second, its granularity is group-wise. On the vision side it receives dense features $\{V_i\}_{i=1}^N$4 but compresses them to a single context vector $\{V_i\}_{i=1}^N$5; on the text side it operates over group-wise pooled embeddings rather than token sequences.

The abstract characterizes the surrounding system as enabling “bidirectional visual–text fusion.” In this architecture, text influences vision because anomaly maps are produced by comparing visual features to normal and abnormal text descriptors, while vision influences text because the visual context $\{V_i\}_{i=1}^N$6 determines how text features from different levels are fused. The “gateway” terminology refers to this conditioning step: vision reshapes the textual descriptor before the descriptor is matched back to vision.

4. Use in anomaly mapping and optimization

For each level $\{V_i\}_{i=1}^N$7, the model compares the dense visual features $\{V_i\}_{i=1}^N$8 against the fused descriptor pair $\{V_i\}_{i=1}^N$9 to obtain a level-specific anomaly map $\{(T_j^N, T_j^A)\}_{j=1}^N$0. The paper summarizes the aggregation rule as follows: the final anomaly map is the average of level-specific maps $\{(T_j^N, T_j^A)\}_{j=1}^N$1, each computed from the cosine similarity between $\{(T_j^N, T_j^A)\}_{j=1}^N$2 and its corresponding descriptor pair (Ma et al., 11 Aug 2025).

The segmentation branch therefore depends directly on DFG. The final segmentation map is

$\{(T_j^N, T_j^A)\}_{j=1}^N$3

The segmentation loss is defined as

$\{(T_j^N, T_j^A)\}_{j=1}^N$4

where $\{(T_j^N, T_j^A)\}_{j=1}^N$5 is Focal Loss and $\{(T_j^N, T_j^A)\}_{j=1}^N$6 is Dice Loss. In this setup, DFG is trained implicitly through the segmentation objective; there is no separate regularizer for the gate.

The image-level classification branch is intentionally decoupled from DFG. The framework computes cosine similarity between the final global visual feature $\{(T_j^N, T_j^A)\}_{j=1}^N$7 and the original, unfused text features from the last group $\{(T_j^N, T_j^A)\}_{j=1}^N$8, supervised with cross-entropy loss $\{(T_j^N, T_j^A)\}_{j=1}^N$9. The total objective is

$(T_i^N, T_i^A)$0

This separation is consequential for interpreting the module. DFG is not the universal fusion mechanism for all outputs in ACD-CLIP; it is specialized to the segmentation branch, while high-level classification semantics are stabilized by retaining the original final text features. A common misconception is therefore that DFG governs the entire anomaly-detection pipeline. In the reported design, it governs the fine-grained segmentation pathway rather than the image-level classification pathway.

5. Empirical profile, effects, and limits

The reported ablation study isolates the contributions of static fusion, Conv-LoRA, DFG, and their combination. The exact values given in Table 3 are as follows (Ma et al., 11 Aug 2025).

Variant	Pixel AUROC	Image AUROC
Baseline (static fusion, simple linear adapter)	82.3	81.2
+ Conv-LoRA only	89.1	84.1
+ DFG only	87.6	85.9
Full ACD-CLIP (Conv-LoRA + DFG)	91.7	90.9

These results support a division of function. Conv-LoRA alone yields the larger gain for pixel-level segmentation, whereas DFG alone yields the larger gain for image-level classification. The combined model achieves the best value on both metrics, which the paper interprets as evidence that fine-grained visual representation and dynamic fusion are complementary. The qualitative description associated with Figure 1 is that anomaly maps produced with DFG better align with anomaly regions, have less background noise and fewer false positives, and remain robust across industrial and medical domains.

Several limits are also explicit. DFG uses global visual context and therefore does not capture spatially varying text modulation. It fuses only at the level of hierarchy depth, not jointly across class and level. The details therefore point to a spatially-aware DFG, in which each patch or region would have its own gating weights, and to hierarchical schemes that would jointly gate across class and level, as natural extensions. This suggests that the current formulation should be regarded as a global feature modulation mechanism rather than a region-adaptive cross-attention mechanism.

6. Related usages of dynamic fusion and gateway concepts

The term “Dynamic Fusion Gateway” is specific to the CLIP-based ZSAD architecture above, but related mechanisms appear in several adjacent literatures. In “Dynamically Fused Graph Network for Multi-hop Reasoning,” the paper does not introduce a component literally called DFG; instead it defines a dynamic fusion block whose query-conditioned soft mask $(T_i^N, T_i^A)$1 gates message passing over an entity graph. Conceptually, that mask behaves as a gateway because it determines which entities are active sources of information at each hop, but the domain is multi-hop text-based question answering rather than vision–language anomaly detection (Xiao et al., 2019).

In “Dynamic Multimodal Fusion,” gating networks select modality-level experts or fusion-level operations on a per-sample basis, with a resource-aware loss that penalizes computational cost. The resemblance to DFG lies in data-dependent routing and conditional fusion, but the objective differs: DynMM optimizes the accuracy–efficiency trade-off by choosing a forward path, whereas DFG in ACD-CLIP reweights a hierarchy of text descriptors for dense anomaly perception (Xue et al., 2022).

In “Deep Fusion Clustering Network,” the SAIF module can be interpreted as a gateway between structure and attribute representations. Its learned scalar $(T_i^N, T_i^A)$2, local propagation through $(T_i^N, T_i^A)$3, self-correlation matrix $(T_i^N, T_i^A)$4, and learned $(T_i^N, T_i^A)$5 together regulate how multiple sources enter a consensus latent code. This is again conceptually related, but it concerns attributed-graph clustering rather than cross-modal anomaly mapping (Tu et al., 2020).

Outside machine learning, “Design and Implementation of a Multiprotocol IoT Gateway” uses “gateway” in the systems sense: a centralized intermediary for device management, protocol conversion, data transformation, and communication with the cloud. That usage is infrastructural and should not be conflated with the adaptive feature-fusion module in ACD-CLIP, even though both mediate between heterogeneous inputs and downstream processing (Macias et al., 2020).

Across these literatures, the recurring idea is not a single standardized module but a family of conditional mediation mechanisms. In the CLIP-based ZSAD setting, DFG denotes a specific vision-guided, level-wise, adaptive fusion of multi-level text features; related works illuminate parallel notions of gating, routing, and selective information flow, but they do not replace the precise architectural role that DFG plays in ACD-CLIP.