---
title: Trans-Chunk BiMamba in ASR & Point Clouds
url: https://www.emergentmind.com/topics/trans-chunk-bimamba-tc-bimamba
type: topic
---

# Trans-Chunk BiMamba in ASR & Point Clouds

Searching arXiv for TC-BiMamba and closely related ASR Mamba papers to ground the article.
Trans-Chunk BiMamba (TC-BiMamba) denotes a chunk-structured Mamba-based design that has been used in at least two distinct research contexts. In automatic speech recognition, it refers to an architecture for unified streaming and non-streaming ASR built around bidirectional Mamba (BiMamba) and a Trans-Chunk mechanism for dynamic chunk size training [2602.11546]. In a separate point-cloud literature context, the same label appears in the in-depth description of a design instantiated within the PoinTramba framework, where chunked Transformer processing is coupled with Mamba-based inter-group modeling [2405.15463]. The ASR usage is the explicit title-bearing formulation, while the point-cloud usage shows the term applied to a related chunk-to-sequence modeling pattern.

## 1. Unified streaming and non-streaming ASR setting

In the ASR formulation, TC-BiMamba is introduced for “unified streaming/non-streaming ASR,” where a single model is expected to support both offline decoding and streaming decoding with various latency settings [2602.11546]. The immediate technical background is the bidirectional Mamba encoder, which processes each input frame twice—once forward and once backward—to capture both past and future context in a linear-time sequence model. Existing BiMamba-based streaming approaches, notably LC-BiMamba, freeze the chunk size at training time. The stated consequences are that they cannot flexibly adapt to different latency settings at inference, and that naïvely applying dynamic chunk-size training causes substantially increased training overhead.

The problem arises from chunk-wise splitting, especially in the backward pass. With an input batch of shape $(B,T,D)$, chunk-wise processing reshapes the sequence into $(B\times N,\,T'/C,\,D)$ where $N=T'/C$, $C$ is chunk size, and $T'=\lceil T/C\rceil C$. The cited formulation identifies this reshaping as a major source of GPU memory growth and reduced throughput. TC-BiMamba addresses this by training both bidirectional sequences in an “offline style” regardless of chunk size, so that dynamic chunk-size training requires only one reshape per batch and one bidirectional pass rather than separate backward passes for each chunk [2602.11546].

A recurring source of confusion is to equate Trans-Chunk with conventional chunk-wise training. In the ASR formulation, the mechanism is explicitly presented as a replacement for per-chunk backward passes rather than a restatement of them.

## 2. Encoder, decoder, and state-space formulation

The ASR architecture consists of a convolutional subsampling front-end that reduces frame rate by $4\times$, followed by a stack of $L$ BiMamba encoder blocks, and a “hybrid” BiTransformer decoder in which each masked self-attention is replaced by a Mamba module while standard cross-attention remains unchanged [2602.11546]. Each encoder block contains a forward Mamba pass, a backward Mamba pass with independent parameters, a learnable fusion weight $\beta$, and a convolution module with kernel size $k$ to capture local patterns.

The Mamba component is expressed through the discrete SSM formulation
$$
h_t = \bar A_t\,h_{t-1} + \bar B_t\,x_t,\quad
y_t = C\,h_t + D\,x_t,
$$
with
$$
\bar A_t = \exp(\Delta_t A),\quad
\bar B_t = (\Delta_t A)^{-1}\bigl(\exp(\Delta_t A)-I\bigr)\,\Delta_t B,
$$
and $\Delta_t,B_t,C_t = f(x_t)$ [2602.11546]. Within each encoder block, the bidirectional outputs are fused as
$$
\mathrm{Fused}(x)
= \beta\,\mathrm{FwdMamba}(x) + (1-\beta)\,\mathrm{BwdMamba}(x).
$$

This design places linear-time sequence modeling at the encoder core while retaining a two-directional contextual pathway. The convolution module supplements the state-space pathway with local pattern modeling, and the hybrid decoder preserves cross-attention while replacing masked self-attention with Mamba. The result is not a pure SSM stack but a mixed encoder-decoder system specialized for ASR deployment modes that vary by latency.

## 3. The Trans-Chunk mechanism

The Trans-Chunk mechanism is the defining procedural component of TC-BiMamba in ASR [2602.11546]. Its purpose is to replace per-chunk backward passes with a single offline-style pass while still training for dynamic chunk sizes.

Given $X\in\mathbb R^{B\times T\times D}$ and chunk size $C$ with minimum 2, the forward pass ignores $C$ and computes
$$
H_f = \mathrm{FwdMamba}(X)\in\mathbb R^{B\times T\times D}.
$$
The backward path branches on whether $C=T$.

When $C=T$, the method reverses the full sequence, applies $\mathrm{BwdMamba}$ once, and reverses the output back. When $C<T$, it pads the sequence to $T'=\lceil T/C\rceil C$, reshapes the padded tensor to $\mathbb R^{(B\cdot N)\times C\times D}$ with $N=T'/C$, reverses within each chunk-sequence, applies $\mathrm{BwdMamba}$ once, reshapes back to $\mathbb R^{B\times T'\times D}$, crops to length $T$, and reverses again along the chunk dimension [2602.11546]. Fusion and convolution then produce
$$
H = \beta\cdot H_f + (1-\beta)\cdot H_b,\qquad
H_{\mathrm{out}} = \mathrm{ConvModule}(H).
$$

The central claim is that, because the backward computation is not split into $N$ separate forward calls, peak memory and compute remain similar to offline BiMamba while the model still learns to handle any chunk size dynamically. The formulation therefore distinguishes chunk-aware training behavior from chunk-wise execution overhead.

## 4. Training objective, decoding, and latency behavior

TC-BiMamba uses joint CTC + E2E training and two-pass decoding [2602.11546]. The training objective uses $\lambda$ to balance $L_{\mathrm{CTC}}$ and $L_{\mathrm{E2E}}$, while the E2E term uses $\alpha$ to combine left-to-right and right-to-left losses. The decoding score is likewise defined by a weighted combination of CTC, left-to-right, and right-to-left scores.

This setup is directly tied to the latency claims of the model. Dynamic chunk-size training is intended to produce a single checkpoint that can be deployed for any streaming latency by choosing different chunk sizes at inference, or for full-context offline decoding by setting chunk size equal to the full sequence length [2602.11546]. The minimum chunk size is fixed at 2 rather than 1, so the formulation explicitly notes that extremely low-latency settings below 2 frames may be unsupported.

The combination of dynamic chunk-size training and bidirectional processing is presented as the basis for “unified streaming and offline ASR in one model.” A plausible implication is that the method aims to collapse what would otherwise be separate offline and streaming training pipelines into one encoder-decoder configuration, though the concrete claim made in the source is deployment flexibility rather than pipeline simplification.

## 5. Efficiency and empirical performance in ASR

The computational argument compares TC-BiMamba with traditional chunk-wise training as in LC-BiMamba [2602.11546]. In the chunk-wise baseline, forward cost per batch is $\mathcal O(B\cdot T\cdot D)$, backward cost consists of $N=T'/C$ separate passes of $\mathcal O(B\cdot C\cdot D)$ each, and total backward compute is still $\mathcal O(B\cdot T\cdot D)$ but with $N\times$ more launch overhead and memory for reshaped tensors. In TC-BiMamba, there is only one backward pass of cost $\mathcal O(B\cdot T\cdot D)$, and the reshape from $(B,T,D)$ to $((B\cdot N),C,D)$ and back is described as negligible compared to $N$ passes.

Empirically on LibriSpeech, training throughput is reported as 1.3× faster than the chunk-splitting baseline, specifically “TC-BiMamba(L) vs. TC-BiMamba(L)-CS,” and peak GPU memory is reduced by approximately 50% [2602.11546]. Analytically, when $T/D/C$ is large, the overhead of $N$ kernel launches and extra temporary buffers in chunk splitting is described as scaling roughly as $O(N)$, whereas Trans-Chunk eliminates this factor.

Two model variants are reported: TC-BiMamba(S), a small model with approximately 47 M parameters, 9 encoder blocks, and model dimension 256; and TC-BiMamba(L), a large model with approximately 70 M parameters, 17 encoder blocks, and model dimension 320 [2602.11546]. On AISHELL-1, TC-BiMamba(S) achieves 5.25 offline (Res), 5.92 offline (CTC), 5.70 chunk16 (Res), and 6.60 chunk16 (CTC), compared with U2++ at 5.31, 6.12, 5.75, and 6.78. On AISHELL-2, TC-BiMamba(S) records 6.00, 6.88, 6.43, and 7.53, compared with U2++ at 6.41, 7.08, 6.74, and 7.72.

On LibriSpeech clean, U2++ reports 3.32 offline (Res), 3.76 offline (CTC), 3.80 chunk16 (Res), and 4.54 chunk16 (CTC); TC-BiMamba(S) reports 3.12, 3.66, 3.66, and 4.40; TC-BiMamba(L) reports 2.97, 3.39, 3.51, and 4.14; and LC-BiMamba reports 3.00 offline and 3.70 chunk16 with 118.70 M parameters [2602.11546]. On LibriSpeech other, U2++ gives 8.67, 9.50, 10.38, and 11.52; TC-BiMamba(S) gives 8.19, 9.05, 9.78, and 11.13; TC-BiMamba(L) gives 7.72, 8.62, 9.44, and 10.58; and LC-BiMamba gives 7.10 offline and 8.90 chunk16. The stated summary is that TC-BiMamba(S) consistently beats U2++ in both offline and streaming modes, and TC-BiMamba(L) matches or exceeds LC-BiMamba on clean data with approximately 40% fewer parameters. The source also notes that streaming WER remains close to offline WER, which it interprets as effective cross-chunk context modeling.

## 6. Separate point-cloud instantiation under the same label

A distinct use of the label appears in the PoinTramba literature, where the “Trans-Chunk BiMamba (TC-BiMamba) design” is described as instantiated in a hybrid Transformer-Mamba framework for point cloud analysis [2405.15463]. In that setting, the input is a raw point cloud
$$
P=\{p_i\in\mathbb R^3\}_{i=1}^N.
$$
The method first samples $G$ anchor points via Farthest Point Sampling and, for each anchor, collects its $K$ nearest neighbors, forming $G$ disjoint groups. Each point is projected into a point embedding and a positional embedding,
$$
z_{g,k}^0 = E_{\rm pt}(p_{g,k}) + PE_{\rm pt}(p_{g,k}) \in \mathbb R^D,
$$
or, in the mathematical formulation,
$$
z_{g,k}^0 = W_{\rm pt}\,p_{g,k}+{\rm PE}(p_{g,k})\in\mathbb R^D.
$$
A standard Transformer is then applied independently within each group for $T$ layers, and the group representation is obtained by mean pooling,
$$
e_g = \frac{1}{K}\sum_{k=1}^K Z_g^T[k].
$$

The point-cloud formulation introduces a Bi-Directional Importance-aware Ordering (BIO) strategy. It predicts a scalar importance score $I_g$ for each $e_g$, sorts the group embeddings once in descending order and once in ascending order, and concatenates the two ordered lists into a sequence of length $2G$. Inter-group relationships are then modeled by standard Mamba layers, and importance-aware pooling discards final embeddings whose predicted score satisfies $I_g<0$ and sums the rest to form the global feature $f$ [2405.15463]. Supervision for the importance scores uses a cosine-similarity score
$$
S_g=\frac{\hat e_g^\top \hat f}{\|\hat e_g\|\,\|\hat f\|},
$$
a SmoothL1 importance loss, and an alignment loss denoted AM-CL.

The reported hyperparameters are $G=256$ for classification or $128$ for segmentation, $K=16$ for classification or $32$ for segmentation, Transformer depth $T=4$, Mamba depth $M=12$, embedding dimension $D=384$, and projection dimension $d=256$ [2405.15463]. The parameter count is approximately 19.5 M, compared with approximately 22 M for a vanilla Transformer on $N=2048$ and 12.3 M for PointMamba. The rough per-sample FLOP estimate is approximately 50 M for the intra-Transformer, approximately 2.36 M for the inter-Mamba, and approximately 52 M total, versus approximately 1.6 G for a vanilla full-cloud Transformer.

The experimental ablations attribute a progression on ScanObjectNN PB-T50-RS from 82.5% for Base Mamba (no chunking), to 86.5% with +Intra-Transformer, to 88.6% with +BIO ordering, and to 89.1% total with +IAP pooling [2405.15463]. On ModelNet40, the method reports 92.7%, beating 92.4% for PointMamba and 92.3% for PointMAE. On ShapeNetPart, it reports 85.7% versus 85.8% for PointMamba with 25 M parameters. The ordering ablation gives 87.2% for random ordering, 87.0% for coordinate $z$, 87.3% for Hilbert, 88.1% for single-direction importance-aware ordering, and 89.1% for bi-directional importance-aware ordering. The source summarizes this design as combining the long-range modeling power of Transformer on small chunks with Mamba’s linear-time inter-chunk modeling, using BIO to provide a “meaningful” order rather than random noise, and using IAP to focus global pooling on positive, high-importance group embeddings.

This separate usage suggests a broader naming pattern in which “Trans-Chunk BiMamba” denotes chunk-level local processing followed by Mamba-based modeling across chunk representations, but the concrete mechanisms, objectives, and benchmarks differ substantially between the ASR and point-cloud settings.

Source: https://www.emergentmind.com/topics/trans-chunk-bimamba-tc-bimamba