LAMMA: Length-Adaptive Fusion for Multimodal Perception

• The paper introduces LAMMA, an attention-based fusion mechanism that preserves a unified semantic space across full, partial, and single-modality inputs.
• LAMMA fuses arbitrary modality subsets via additive fusion and length-adaptive queries, ensuring consistent feature representations even when sensors fail.
• Combined with the PAFR training strategy, LAMMA mitigates modality competition, yielding improved performance in multimodal, LiDAR-only, and camera-only scenarios.

Searching arXiv for the specified papers and directly related multimodal fusion work. Length-Adaptive Multi-Modal Fusion (LAMMA) is an attention-based fusion mechanism introduced within Single-Modality-Operable Multimodal Collaborative Perception (SiMO) to maintain multimodal collaborative perception under arbitrary sensor failures while preserving a unified semantic space across full, partial, and single-modality inputs (Wen et al., 9 Mar 2026). In SiMO, LAMMA addresses a specific failure mode of conventional multimodal fusion: once feature fusion maps modality-specific representations into a new joint space, downstream modules trained on that fused representation are no longer compatible with the original single-modality features when one sensor becomes unavailable. LAMMA is designed to fuse any subset of modalities using the same structural logic, keep the feature space before and after fusion identical through additive fusion in a unified space, process a variable number of modalities without changing parameters or control flow, and behave like a parallel circuit in which the system remains operational as long as one branch is healthy (Wen et al., 9 Mar 2026).

1. Problem formulation and motivation

In SiMO, the central requirement is that a multimodal collaborative perception system must continue functioning when any subset of sensors fails, while preserving each modality’s ability to function alone, avoiding semantic mismatch between single-modality features and the joint fusion and detection head, and enabling heterogeneous agents with different available modalities to collaborate in a unified feature space (Wen et al., 9 Mar 2026).

The motivating diagnosis is that common multimodal fusion induces a semantic space shift after fusion. The paper formalizes this as

$Z_{\text{mm}} = \phi\big(Z_L, Z_C\big),$

where $Z_L$ and $Z_C$ denote LiDAR and camera features, and $Z_{\text{mm}}$ lies in a different space than either input feature. Task heads $h(\cdot)$ are trained on $Z_{\text{mm}}$; if one modality fails, the remaining modality feature does not constitute a valid input in that fused semantic space, so the head fails (Wen et al., 9 Mar 2026). This is compounded by a hard dependency on all modalities, described as a series-circuit behavior, and by the collaborative setting in which different agents may retain different modalities, yet their BEV features must remain in a strictly unified semantic space for multi-agent fusion such as AttFusion or Pyramid Fusion (Wen et al., 9 Mar 2026).

This formulation distinguishes LAMMA from earlier adaptive multimodal fusion work that treated fusion as a learned function rather than a deterministic operator. "Adaptive Fusion Techniques for Multimodal Data" characterizes concatenation, sum, and tensor-product style constructions as static fusion, and proposes Auto-Fusion and GAN-Fusion as adaptive alternatives that learn how to combine modalities in latent space (Sahu et al., 2019). That work is not framed around collaborative perception or sensor failure, but it establishes the broader principle that multimodal fusion can itself be a learned transformation rather than a fixed operator (Sahu et al., 2019). A plausible implication is that LAMMA occupies a more constrained design point: it is not merely adaptive in weighting or compression, but explicitly engineered to preserve semantic compatibility across modality subsets.

2. Placement within the SiMO pipeline

In the SiMO architecture, each modality $m \in \{L, C, \dots\}$ has a modality-specific encoder $f_m$ and a modality-specific aligner $g_m$. For the LiDAR and camera instantiations reported in the paper, the encoders are PointPillars for LiDAR and LSS for cameras, while the aligners are ConvNeXt blocks (Wen et al., 9 Mar 2026). The aligned modality feature is

$Z_m = g_m\big(f_m(X_m)\big), \quad m \in \{L, C\}.$

LAMMA is applied after per-modality encoding and alignment: $Z_L$0 with $Z_L$1 instantiated as LAMMA (Wen et al., 9 Mar 2026). Multi-agent fusion $Z_L$2, such as AttFusion or Pyramid Fusion, is then performed on the fused BEV features from each agent: $Z_L$3 This makes LAMMA a middle-fusion module positioned between per-modality BEV formation and inter-agent fusion (Wen et al., 9 Mar 2026).

Its role in training is equally specific. Under the PAFR (Pretrain–Align–Fuse–RD) schedule, LAMMA is not learned jointly from scratch with all branches. It is trained only after single-modality pretraining of each branch and per-modality alignment training of the aligners, and is followed by fine-tuning with Random Modality Drop (RD) (Wen et al., 9 Mar 2026). This staging reflects the paper’s claim that naive joint training induces modality competition, especially causing LiDAR dominance and weak camera-only behavior. Accordingly, LAMMA supplies the structural mechanism for semantics-consistent fusion, while PAFR supplies the optimization regime needed to maintain branch independence (Wen et al., 9 Mar 2026).

3. Fusion architecture and length adaptivity

After encoder and aligner processing, each modality feature is represented as

$Z_L$4

where $Z_L$5 is batch size, $Z_L$6 is the number of BEV tokens, and $Z_L$7 is feature dimension (Wen et al., 9 Mar 2026). Before attention, SiMO adds a modality-agnostic positional embedding,

$Z_L$8

and may project features through $Z_L$9 to reduce Transformer compute (Wen et al., 9 Mar 2026).

LAMMA is defined as attention-based and set-size invariant. For two modalities $Z_C$0 and $Z_C$1, queries are constructed by concatenating modality tokens along the length dimension: $Z_C$2 For each modality $Z_C$3, keys and values are

$Z_C$4

The module then applies per-modality multi-head attention: $Z_C$5 Because $Z_C$6 contains queries from both modalities while $Z_C$7 and $Z_C$8 derive from only one modality, the same MHA call simultaneously contains self-attention and cross-attention effects (Wen et al., 9 Mar 2026).

The output $Z_C$9 is split into modality-origin segments and summed: $Z_{\text{mm}}$0 The final multimodal feature is then obtained by addition: $Z_{\text{mm}}$1 In the general case,

$Z_{\text{mm}}$2

The paper identifies this final addition as the critical operation that preserves the feature space of inputs and outputs, in contrast to concatenation-plus-convolution schemes that create a new semantic coordinate system (Wen et al., 9 Mar 2026).

The “length-adaptive” designation has two explicit meanings. First, the query length is proportional to the number of available modalities $Z_{\text{mm}}$3, since queries are formed by concatenating all available modality tokens. If some modalities are absent, their tokens are simply omitted; no branch structure or learnable parameter changes (Wen et al., 9 Mar 2026). Second, when one modality fails, the mechanism implicitly degenerates to self-attention on the remaining modality. For example, if modality $Z_{\text{mm}}$4 is absent, the paper describes

$Z_{\text{mm}}$5

and with $Z_{\text{mm}}$6, the second term vanishes, yielding

$Z_{\text{mm}}$7

Thus, the module preserves the same control flow and parameterization under modality removal, while changing only the effective token length (Wen et al., 9 Mar 2026).

4. Semantic alignment and modality preservation

LAMMA depends on a pre-fusion semantic alignment stage because direct fusion of raw LiDAR and camera features produces large semantic discrepancies arising from different distributions and geometric structure (Wen et al., 9 Mar 2026). SiMO therefore applies ConvNeXt-based aligners $Z_{\text{mm}}$8 and $Z_{\text{mm}}$9, each implemented as a 3-block ConvNeXt module, to align channel-wise and pixel-wise semantics and map modality features into a LAMMA-compatible space: $h(\cdot)$0 These aligners are trained per modality, with the remaining components frozen, so that each modality is brought into a compatible space without competition from the others (Wen et al., 9 Mar 2026).

The paper identifies two design decisions as responsible for semantic consistency before and after fusion. The first is shared processing architecture inside fusion: queries are treated equally regardless of modality, keys and values are processed via shared linear layers $h(\cdot)$1 and $h(\cdot)$2, and the Transformer block sees all modalities symmetrically (Wen et al., 9 Mar 2026). The second is additive fusion in a unified space: $h(\cdot)$3 Because the aligned per-modality features share a semantic basis, the channel meaning is preserved under addition; only activation content changes (Wen et al., 9 Mar 2026).

The paper provides both quantitative and qualitative evidence for this claim. In Procrustes analysis, disparity between LiDAR and camera features decreases from $h(\cdot)$4 before LAMMA to $h(\cdot)$5 after LAMMA alignment (Wen et al., 9 Mar 2026). The reported t-SNE visualizations show that LiDAR, camera, and fused clusters remain modality-separate, which the authors interpret as preservation of modality-specific information, while their internal topologies exhibit mirror symmetry after LAMMA, indicating aligned semantic structure across modalities and fused features (Wen et al., 9 Mar 2026).

This combination of alignment without identity collapse distinguishes LAMMA from fusion schemes that aggressively homogenize modality representations. A plausible implication is that LAMMA is intended not to erase modality provenance, but to make modality-specific representations interoperable under a shared downstream head.

5. Training regime and modality independence

LAMMA is coupled to the PAFR training strategy because the paper argues that fusion design alone does not solve modality competition (Wen et al., 9 Mar 2026). The four stages are:

Stage	Operation	Purpose stated in the paper
Pretrain	Train each encoder $h(\cdot)$6 separately with its own detection head	Learn strong single-modality features without interference
Align	Freeze encoders and train aligners $h(\cdot)$7 one by one	Map each modality into a LAMMA-compatible space without competition
Fuse	Freeze encoders and aligners; train LAMMA and shared downstream modules on multimodal inputs	Make downstream modules modality-independent
RD	Fine-tune with random modality drop	Expose fusion and downstream heads to available-modality combinations

In RD, the reported sampling probabilities are $h(\cdot)$8 for multimodal input, $h(\cdot)$9 for LiDAR-only, and $Z_{\text{mm}}$0 for camera-only in the basic setting, with additional patterns in heterogeneous settings (Wen et al., 9 Mar 2026). The paper states that this stage strengthens single-modality operability by randomly dropping one modal feature in LAMMA and thereby exposing the shared downstream modules to the full range of modality availability patterns (Wen et al., 9 Mar 2026).

The ablation findings are sharply stated. Naive training, with or without RD, cannot adapt to modal failure; SiMO without LAMMA, using BEVFusion-style fusion, cannot adapt even with RD; and only the combination of PAFR, LAMMA, and RD yields high performance in multimodal, LiDAR-only, and camera-only conditions simultaneously (Wen et al., 9 Mar 2026). In a normalized ablation table, the reported AP@70 values for the complete design are $Z_{\text{mm}}$1 for L+C, $Z_{\text{mm}}$2 for L-only, and $Z_{\text{mm}}$3 for C-only (Wen et al., 9 Mar 2026).

This training structure has a conceptual parallel with earlier adaptive fusion work that combined a fusion-specific loss with a task loss. In Auto-Fusion, the fused representation is constrained by a reconstruction objective $Z_{\text{mm}}$4, while GAN-Fusion uses adversarial alignment losses, both optimized jointly with the downstream task (Sahu et al., 2019). LAMMA does not adopt those losses; instead, it relies on staged training and modality drop. This suggests that LAMMA’s contribution is chiefly architectural and optimization-structural rather than based on explicit latent-space regularization beyond the alignment modules (Wen et al., 9 Mar 2026, Sahu et al., 2019).

6. Empirical behavior, comparisons, and limitations

The principal empirical evaluation reported for LAMMA is on OPV2V-H. Under normal operation, SiMO-AF with RD achieves AP@70 values of $Z_{\text{mm}}$5 for L+C, $Z_{\text{mm}}$6 for L-only, and $Z_{\text{mm}}$7 for C-only; SiMO-PF with RD achieves $Z_{\text{mm}}$8 for L+C, $Z_{\text{mm}}$9 for L-only, and $m \in \{L, C, \dots\}$0 for C-only (Wen et al., 9 Mar 2026). The camera-only result for SiMO-PF exceeds the corresponding Pyramid Fusion camera-only result of $m \in \{L, C, \dots\}$1, despite using the same camera encoder, which the paper interprets as evidence that LAMMA refines modality features through attention-based enhancement rather than merely combining them (Wen et al., 9 Mar 2026).

Under LiDAR degradation, the paper reports that Pyramid Fusion and BEVFusion degrade sharply and eventually fail as LiDAR points are reduced, whereas SiMO-PF degrades more slowly and converges to its camera-only performance as LiDAR becomes unusable (Wen et al., 9 Mar 2026). In heterogeneous modal-failure patterns across agents, SiMO-PF matches or exceeds HEAL in AP@30/50/70 for both LiDAR-ego and camera-ego cases, with only minor drop from homogeneous to heterogeneous failures, which the authors attribute to cross-agent semantic consistency provided by alignment and LAMMA (Wen et al., 9 Mar 2026).

The comparison to other fusion strategies is explicit. Relative to concatenation plus convolution, exemplified by BEVFusion, LAMMA avoids constructing a new fused space incompatible with single-modality features and inherently supports a variable number of modalities (Wen et al., 9 Mar 2026). Relative to Transformer-based fusion without length-adaptive design, LAMMA builds queries from only available modalities, uses the same structure regardless of modality presence, and degrades automatically to self-attention (Wen et al., 9 Mar 2026). Relative to UniBEV’s channel-normalized weighted sum, LAMMA is presented as compatible with heterogeneous encoders such as PointPillars and LSS, rather than depending on identical encoder architectures, and as directly addressing multi-agent heterogeneous failure (Wen et al., 9 Mar 2026).

Two limitations are expressly noted. First, because LAMMA performs fusion through attention plus addition without convolutional smoothing, noisy modalities can propagate noise more directly; the paper contrasts this with BEVFusion’s convolutional fusion, which can implicitly denoise (Wen et al., 9 Mar 2026). Second, the PAFR schedule increases training complexity and training time relative to naive end-to-end optimization (Wen et al., 9 Mar 2026).

Related multimodal fusion designs in other domains illustrate that “adaptive” need not mean the same thing across settings. In 4DRVO-Net, adaptive multimodal fusion is realized as an Adaptive Radar–Camera Fusion Module (A-RCFM) that uses radar-conditioned deformable attention to select image features around projected radar points at multiple scales (Zhuo et al., 2023). That mechanism is adaptive in a spatially varying, point-wise, and scale-wise sense, whereas LAMMA is adaptive in the number of available modalities and in preserving semantic consistency under missing-modality conditions (Zhuo et al., 2023, Wen et al., 9 Mar 2026). This suggests that “length adaptivity” in LAMMA refers specifically to token-length variation induced by modality subsets, not to temporal sequence length or geometric scale per se.

7. Broader interpretation and relation to adaptive fusion literature

Within SiMO, LAMMA can be defined succinctly as a length-adaptive, attention-based, additive fusion module operating on aligned BEV features from arbitrary modality subsets, with modality-agnostic parameters and invariant downstream semantics (Wen et al., 9 Mar 2026). Its distinguishing property is not simply that it learns fusion weights, but that it makes a multimodal collaborative perception system single-modality-operable by preserving semantic compatibility between single-modality and multimodal representations (Wen et al., 9 Mar 2026).

The wider literature supplies useful contrast. Auto-Fusion and GAN-Fusion propose lightweight adaptive networks that learn to compress and regularize multimodal latent spaces for tasks such as multimodal machine translation and emotion recognition (Sahu et al., 2019). Those methods treat adaptivity primarily as learned compression or adversarial latent alignment over fixed-dimensional modality embeddings. By contrast, LAMMA applies multi-head attention to BEV token sets and ties adaptivity to the variable cardinality of the modality set $m \in \{L, C, \dots\}$2, while enforcing additive closure in a shared semantic space (Wen et al., 9 Mar 2026). 4DRVO-Net, in turn, demonstrates another form of adaptive fusion in which deformable cross-attention is applied point-wise and multi-scale to fuse sparse radar with dense camera features for odometry (Zhuo et al., 2023).

Taken together, these works indicate that adaptive multimodal fusion can be organized along at least three axes present in the cited papers: learned latent compression and alignment (Sahu et al., 2019), spatially varying cross-modal sampling and confidence weighting (Zhuo et al., 2023), and semantic-space-preserving fusion under variable modality availability (Wen et al., 9 Mar 2026). LAMMA belongs to the third category. Its specific contribution is to make the fused representation structurally compatible with downstream collaborative perception modules across multimodal, partial-modality, and single-modality operation, thereby serving as the architectural basis for single-modality-operable multimodal collaborative perception (Wen et al., 9 Mar 2026).