---
title: 'DA-Mamba: Hybrid CNN-SSM Architectures'
url: https://www.emergentmind.com/topics/da-mamba
type: topic
---

# DA-Mamba: Hybrid CNN-SSM Architectures

DA-Mamba denotes a family of Mamba-based selective state-space architectures whose “DA” qualifier is task-specific rather than universal. In current arXiv usage, the name is used most prominently for a **domain adaptive object detector** that augments a CNN detector with State Space Models to capture both global and local domain-invariant features, and also for a **dialogue-aware multimodal engagement estimator** that replaces attention-heavy dialogue encoders with Mamba-based selective state-space processing [2603.18757] [2509.17711]. The former is the more fully specified usage and defines DA-Mamba as a hybrid CNN-SSM architecture for global-local alignment in unsupervised domain adaptive object detection.

## 1. Name, scope, and disambiguation

In the current literature, the term **DA-Mamba** is not unique. It refers to at least two distinct methods.

| DA-Mamba usage | Task | Defining modules |
|---|---|---|
| DA-Mamba [2603.18757] | Domain Adaptive Object Detection | Image-Aware SSM, Object-Aware SSM |
| DA-Mamba [2509.17711] | Multimodal engagement estimation | Dialogue-Aware Encoder, Modality-Group Fusion, Partner-Group Fusion |

In the object-detection usage, DA-Mamba addresses **unsupervised domain adaptive object detection (DAOD)**, where a detector is transferred from a labeled source domain to an unlabeled target domain. Its central claim is that existing DAOD methods align source and target features mostly locally because CNN backbones and detection heads are locally connected, while transformer-based alternatives incur quadratic cost. DA-Mamba therefore combines CNNs with Mamba-style SSMs so that local domain-invariant features remain anchored by convolution while long-range dependencies are modeled with linear-time sequence processing [2603.18757].

The dialogue-aware usage retains the same acronym but applies it to conversational engagement estimation. That model is explicitly described as a **dialogue-aware multimodal architecture** with a **Dialogue-Aware Encoder**, **Modality-Group Fusion**, and **Partner-Group Fusion**, and is evaluated on **NoXi**, **NoXi-Add**, and **MPIIGI** using **CCC** as the main metric [2509.17711]. The reuse of the name is therefore a nomenclatural ambiguity rather than a shared architecture.

## 2. Domain adaptive object detection formulation

In the DAOD setting formalized by DA-Mamba, one has **source images** \(x_s\) with labels and **target images** \(x_t\) without labels. The paper’s diagnosis is that standard image-level adversarial alignment remains local because each feature location depends only on a local neighborhood. The appendix writes a pixel-wise adversarial objective as

\[
\mathcal{L}_{\text{adv}} = \frac{1}{HW} \sum_{i=1}^{H}\sum_{j=1}^{W} \left(\mathcal{D}(\mathbf{f}_{ij}) - y\right)^2,
\]

with feature \(\mathbf{f}_{ij}\) at spatial location \((i,j)\), domain classifier \(\mathcal{D}\), and domain label \(y\in\{0,1\}\). If the feature is only locally defined,

\[
\mathbf{f}^d_{ij}=\mathcal{F}(\mathbf{N}^d_{ij}),
\]

then alignment enforces only

\[
p\big(\mathcal{F}(\mathbf{N}^S_{i_s j_s})\big) = p\big(\mathcal{F}(\mathbf{N}^T_{i_t j_t})\big),
\]

which is local-only alignment. DA-Mamba instead augments each feature with global context,

\[
\mathbf{f}^d_{ij} = \big[\mathcal{F}(\mathbf{N}^d_{ij});\,\mathbf{c}^d_{ij}\big],
\]

where

\[
\mathbf{c}^d_{ij} = \mathcal{S}\Big(\big\{\mathbf{N}^d_{kl}\big\}_{k=1,l=1}^{H,W}\setminus\{\mathbf{N}^d_{ij}\}\Big),
\]

so adversarial alignment can encourage both local and global consistency [2603.18757].

This formulation motivates the paper’s notion of **global-local alignment**. The relevant global cues are not limited to scene texture. The paper explicitly includes **scene layout**, **inter-region relationships**, **object co-occurrence patterns**, and **semantic dependencies among categories**. The example given is that a **rider** often co-occurs with a **bicycle**, while **car** and **truck** are semantically related. These dependencies are beyond what a purely local convolutional alignment signal can represent [2603.18757].

The targeted domain shifts are correspondingly diverse: **weather** shifts such as **Cityscapes \(\rightarrow\) Foggy Cityscapes**, **field-of-view/camera bias** such as **Cityscapes \(\rightarrow\) BDD100K**, and **style** shifts such as **Pascal VOC \(\rightarrow\) Clipart** or **Comic**. This broad scope matters because it frames DA-Mamba as a detector for domain shift in general rather than a detector specialized to a single visual corruption [2603.18757].

## 3. Hybrid CNN-SSM detector architecture

DA-Mamba is built on a **one-stage YOLO-style architecture**, specifically **YOLO-World adapted to UDA**. The backbone begins with an initial **ResBlock** \(\mathcal{B}\) that extracts low-level feature \(\mathbf{C}_3\). The paper writes the subsequent backbone/FPN stream as

\[
\mathbf{C}_4 = \mathcal{B}_1(\mathcal{I}_1(\mathbf{C}_3)); \qquad
\mathbf{C}_5 = \mathcal{B}_2(\mathcal{I}_2(\mathbf{C}_4)); \qquad
\mathbf{P}_5 = \mathcal{I}_3(\mathbf{C}_5),
\]

where \(\mathcal{I}_i\) denotes **IA-SSM** and \(\mathcal{B}_j\) are residual downsampling blocks. Detection then uses **OA-SSM**-enhanced pyramid features,

\[
[Cls, Bbox] = \mathcal{H}(\mathcal{O}_1(\mathbf{P}_3);\mathcal{O}_2(\mathbf{P}_4);\mathcal{O}_3(\mathbf{P}_5)),
\]

with \(\mathcal{O}_i\) denoting **OA-SSM** and \(\mathcal{H}\) the detection head [2603.18757].

The insertion strategy is systematic. **IA-SSM** is integrated into the **backbone/FPN downsample stream**, while **OA-SSM** is inserted into the **detection head** at each scale \(\mathbf{P}_3,\mathbf{P}_4,\mathbf{P}_5\). This division gives the architecture two distinct alignment stages: image-level alignment in the backbone and instance-level alignment in the head [2603.18757].

A defining structural choice is the **dual pipeline** inside both modules. Each block contains a **convolution pipeline** for local domain-invariant feature extraction and an **SSM pipeline** for long-range/global modeling. The appendix compares **serial** and **parallel** compositions and adopts the **parallel** design, reporting the same mAP as serial but with **1.732M** parameters rather than **8.610M**, and **14.1 FPS** rather than **12.6 FPS** [2603.18757]. This supports the paper’s claim that DA-Mamba is not a simple “replace CNN with Mamba” architecture; it is a hybrid in which CNN locality and SSM globality are computed in parallel and fused.

The paper does not specify the exact 2D feature-map scan order used by the visual Mamba layer. It states that the Mamba branch operates on visual features and that the appendix mentions a **directed scan across the entire feature map**, but it does not give a raster, snake, or multi-directional scan definition. This makes DA-Mamba architecturally clear at the module level while leaving the internal visual serialization rule implicit [2603.18757].

## 4. IA-SSM and OA-SSM

### Image-Aware SSM

**IA-SSM** is designed to enhance **global domain awareness** at the image level. Given an input feature map

\[
\mathbf{f}^{in}\in \mathbb{R}^{B, C, H, W},
\]

it introduces a learnable **image-level visual prompt**

\[
\mathbf{v}^I\in \mathbb{R}^{C}.
\]

This prompt is broadcast to all spatial locations and concatenated with the input:

\[
\mathbf{f} = {\rm Concat}(\mathbf{f}^{in},Broadcast(\mathbf{v}^I)).
\]

To reduce redundancy and computational cost, IA-SSM first applies separate bottleneck embeddings for the SSM and convolution branches,

\[
\mathbf{f}^S = \mathcal{E}^S(\mathbf{f}); \qquad \mathbf{f}^C = \mathcal{E}^C(\mathbf{f}),
\]

with

\[
\mathbf{f}^S,\mathbf{f}^C \in \mathbb{R}^{B, C/r, H, W},
\]

and reduction ratio \(r=2.0\). The global branch then computes

\[
\mathbf{z}^S = \mathcal{M}(\mathbf{f}^S),
\]

while the local branch computes

\[
\mathbf{z}^C=\mathcal{C}^A(\mathbf{f}^c),
\]

where the paper’s notation almost certainly intends \(\mathbf{f}^C\). The two outputs are fused by

\[
\mathbf{f}^{out}=\mathcal{E}^O({\rm Concat(\mathbf{z}^S,\mathbf{z}^C)}),
\]

yielding a feature map that carries both local and long-range image information [2603.18757].

### Object-Aware SSM

**OA-SSM** addresses instance-level alignment by modeling **spatial dependencies among objects** and **semantic dependencies among categories**. Unlike IA-SSM, which uses a global image prompt, OA-SSM constructs a prompt that is both **pixel-wise** and **category-conditioned**. For input feature map

\[
\mathbf{f}^{in}\in \mathbb{R}^{B, C, H, W},
\]

it first projects the feature map into a category similarity tensor

\[
\mathbf{W}\in \mathbb{R}^{B,H,W,K},
\]

where \(K\) is the number of categories. The instance-level visual prompt is then

\[
\mathbf{v}^O = \mathbf{W}\otimes\mathbf{E}=\mathcal{C}^A(\mathbf{f}^{in})\mathbf{E},
\]

with \(\mathbf{E}\in\mathbb{R}^{K,C}\) denoting category prototypes. These prototypes are obtained from **CLIP’s text encoder** using prompts such as **“A photo of [Class]”** [2603.18757].

The purpose of this construction is to inject category semantics into the instance-level feature map before the dual conv-SSM pipeline. In the paper’s interpretation, a true object region should have coherent semantics, so all pixels in that region should share similar prompt vectors. OA-SSM therefore acts as a category-conditioned long-range relation model rather than a purely geometric contextualizer [2603.18757].

### Mamba layer inside DA-Mamba

The paper gives a simplified operational form for the Mamba layer used in these modules:

\[
\mathbf{a} = \sigma({\rm Linear}(\mathbf{f}^{in})),
\]

\[
\mathbf{f} = {\rm SSM}(\sigma({\rm Conv1D}({\rm Linear}(\mathbf{f}^{in})))),
\]

\[
\mathbf{f}^{out} = \mathcal{M}(\mathbf{f}^{in})={\rm Linear}(\mathbf{a}\cdot \mathbf{f}).
\]

This presentation is intentionally concise. The paper also recalls the continuous-time SSM background,

\[
\mathbf{h}'(t)=\mathbf{A}\mathbf{h}(t)+\mathbf{B}\mathbf{x}(t), \qquad
\mathbf{y}(t)=\mathbf{C}^\top\mathbf{h}(t),
\]

but does not restate the full selective scan recursion typically associated with Mamba. DA-Mamba is therefore mathematically explicit at the module and objective level, while relying on a standard Mamba layer as a black-box long-range operator [2603.18757].

## 5. Objective, training, and computational profile

DA-Mamba trains with four components: **source supervised detection**, **target pseudo-label classification**, **image-level adversarial alignment**, and **instance-level adversarial alignment**. Each **IA-SSM** output is connected to a domain discriminator \(\mathcal{D}_i^I\) through a **GRL**, giving the image-level adversarial loss

\[
\mathcal{L}^I_{adv}=-\sum_{i=1}^3[\mathbb{E}_{\mathbf{x}_s}||\mathcal{D}^I_i(\mathbf{f}^{out}_i)||_2^2+\mathbb{E}_{\mathbf{x}_t}||\mathcal{D}^I_i(\mathbf{f}^{out}_i)-\mathbf{1}||_2^2].
\]

For **OA-SSM**, instance-level adversarial alignment is masked by an instance mask \(\mathbf{M}\), generated from regions whose classification probability exceeds \(0.5\):

\[
\begin{split}
\mathcal{L}^O_{adv}=-\sum_{i=1}^3[&\mathbb{E}_{\mathbf{x}_s}||\mathbf{M}\cdot\mathcal{D}^O_i(\mathbf{f}^{out}_i)||_2^2 \\
+&\mathbb{E}_{\mathbf{x}_t}||\mathbf{M}\cdot(\mathcal{D}^O_i(\mathbf{f}^{out}_i)-\mathbf{1})||_2^2].
\end{split}
\]

The total objective is

\[
\mathcal{L}=\mathcal{L}^S_{cls} + \mathcal{L}^T_{lcs} + \lambda^I\mathcal{L}^I_{adv}+\lambda^O\mathcal{L}^O_{adv} + \mathcal{L}_{reg},
\]

with \(\lambda^I = 1.0\) and \(\lambda^O = 0.5\). The notation \(\mathcal{L}^T_{lcs}\) is clearly a paper typo for \(\mathcal{L}^T_{cls}\) [2603.18757].

Training uses **SGD**, a base learning rate of **\(1\mathrm{e}{-3}\)**, **linear warm-up** for **5 epochs**, decay starting at **epoch 20**, and a batch size of **2 source + 2 target** on **1 Tesla V100 GPU**. The target branch uses **high-confidence predictions as pseudo labels**, but the paper explicitly argues that DA-Mamba does not rely heavily on pseudo labels: in the pseudo-label ablation, the baseline gains **\(1.7\sim 2.4\%\)** from pseudo labels, whereas DA-Mamba gains only **\(0.7\sim1.5\%\)**, suggesting that most of the improvement comes from the alignment modules themselves [2603.18757].

The computational argument is central. On the **Cityscapes \(\rightarrow\) Foggy Cityscapes** comparison table, DA-Mamba reports **1307M** inference memory, **14.1 FPS**, **15500M** training memory, **0.782s** training time per iteration, **148G** FLOPs, and **58.1** mAP. In the same table, **Baseline+ViT Block** reports **2090M** inference memory, **10.6 FPS**, **19322M** training memory, **0.862s** per iteration, **208G** FLOPs, and **57.5** mAP; **DATR** reports **3206M**, **5.8 FPS**, **21606M**, **0.906s**, **279G**, and **53.4** mAP; **DA-Pro** reports **2549M**, **2.5 FPS**, **4034M**, **1.472s**, **243G**, and **55.9** mAP [2603.18757]. The practical implication is that DA-Mamba stays close to baseline detector throughput while achieving stronger cross-domain performance.

## 6. Experimental evidence, ablations, and other uses of the name

DA-Mamba is evaluated on four multi-category DAOD benchmarks: **Cityscapes \(\rightarrow\) Foggy Cityscapes**, **Cityscapes \(\rightarrow\) BDD100K**, **Pascal VOC \(\rightarrow\) Clipart**, and **Pascal VOC \(\rightarrow\) Comic**, using **mAP@0.5**. The reported headline results are **58.1** on Cityscapes \(\rightarrow\) Foggy Cityscapes, **48.7** on Cityscapes \(\rightarrow\) BDD100K, **52.5** on Pascal VOC \(\rightarrow\) Clipart, and **43.8** on Pascal VOC \(\rightarrow\) Comic. These improve over the corresponding **Baseline(UDA)** scores of **52.3**, **41.9**, **46.2**, and **37.9**, and also improve over the best compared DAOD baselines in the main tables, including **DA-Pro** on Cityscapes \(\rightarrow\) Foggy Cityscapes and **CAT** on Pascal VOC \(\rightarrow\) Clipart [2603.18757].

The ablations clarify what DA-Mamba is and what it is not. **IA-SSM alone** gives gains of **+4.5**, **+4.0**, **+3.8**, and **+2.6** on the four benchmarks, while **OA-SSM alone** gives **+3.1**, **+3.1**, **+3.7**, and **+2.7**. Using both together yields gains of **+5.8**, **+6.8**, **+6.3**, and **+5.9** over the baseline [2603.18757]. This is the clearest evidence that image-level and instance-level alignment are complementary rather than redundant.

A common misconception is that DA-Mamba is merely “Mamba inserted into a detector.” The paper directly tests this. Replacing the proposed domain-aware modules with plain Mamba blocks gives **55.0 / 44.8 / 48.7 / 38.9** on the four benchmarks, whereas DA-Mamba gives **58.1 / 48.7 / 52.5 / 43.8** [2603.18757]. The difference is substantial. The domain-aware prompt construction, dual conv-SSM fusion, and adversarial alignment design are therefore not peripheral engineering details; they define the method.

The paper also reports qualitative evidence. Feature maps suppress irrelevant background responses and highlight true foreground objects more clearly; similarity maps for a chosen **Person** region show stronger same-class long-range similarity and weaker unrelated similarity; t-SNE visualizations show tighter intra-class clusters and larger inter-class margins, while semantically related classes such as **car-truck** and **person-rider** remain close [2603.18757]. These observations are consistent with the architecture’s stated goal of aligning both global semantics and local evidence.

The second arXiv usage of the name broadens the term’s scope. In **“DA-Mamba: Dialogue-aware selective state-space model for multimodal engagement estimation”**, DA-Mamba is a **dialogue-aware** architecture that replaces **attention-heavy dialogue encoders** with **Mamba-based selective state-space processing** to achieve **linear time and memory complexity** while retaining **expressive cross-modal reasoning**. Its three core modules are a **Dialogue-Aware Encoder**, **Modality-Group Fusion**, and **Partner-Group Fusion**, and it reports improved **CCC** on **NoXi**, **NoXi-Add**, and **MPIIGI**, together with reduced training time and peak memory [2509.17711]. The coexistence of these two papers means that “DA-Mamba” is now best understood as an overloaded label rather than a single canonical architecture.

In that restricted but important sense, DA-Mamba names two different Mamba-era design programs. One is **domain-aware** and centered on **global-local alignment in DAOD**; the other is **dialogue-aware** and centered on **multimodal engagement estimation**. The former is the more fully specified formulation and establishes DA-Mamba as a hybrid CNN-SSM detector in which **IA-SSM** and **OA-SSM** make global domain awareness computationally practical within a one-stage detector [2603.18757].

Source: https://www.emergentmind.com/topics/da-mamba