Trans-Chunk BiMamba in ASR & Point Clouds

• Trans-Chunk BiMamba is a chunk-structured design that employs dynamic chunk-size training and bidirectional Mamba processing to support both unified streaming/offline ASR and point cloud analysis.
• It replaces traditional per-chunk backward passes with a single Trans-Chunk mechanism, reducing GPU memory overhead and accelerating training throughput.
• Its point cloud variant integrates Transformer modules with bi-directional importance-aware ordering to merge local and global feature modeling, achieving competitive performance on classification and segmentation benchmarks.

Searching arXiv for TC-BiMamba and closely related ASR Mamba papers to ground the article. Trans-Chunk BiMamba (TC-BiMamba) denotes a chunk-structured Mamba-based design that has been used in at least two distinct research contexts. In automatic speech recognition, it refers to an architecture for unified streaming and non-streaming ASR built around bidirectional Mamba (BiMamba) and a Trans-Chunk mechanism for dynamic chunk size training (She et al., 12 Feb 2026). In a separate point-cloud literature context, the same label appears in the in-depth description of a design instantiated within the PoinTramba framework, where chunked Transformer processing is coupled with Mamba-based inter-group modeling (Wang et al., 2024). The ASR usage is the explicit title-bearing formulation, while the point-cloud usage shows the term applied to a related chunk-to-sequence modeling pattern.

1. Unified streaming and non-streaming ASR setting

In the ASR formulation, TC-BiMamba is introduced for “unified streaming/non-streaming ASR,” where a single model is expected to support both offline decoding and streaming decoding with various latency settings (She et al., 12 Feb 2026). The immediate technical background is the bidirectional Mamba encoder, which processes each input frame twice—once forward and once backward—to capture both past and future context in a linear-time sequence model. Existing BiMamba-based streaming approaches, notably LC-BiMamba, freeze the chunk size at training time. The stated consequences are that they cannot flexibly adapt to different latency settings at inference, and that naïvely applying dynamic chunk-size training causes substantially increased training overhead.

The problem arises from chunk-wise splitting, especially in the backward pass. With an input batch of shape $(B,T,D)$, chunk-wise processing reshapes the sequence into $(B\times N,\,T'/C,\,D)$ where $N=T'/C$, $C$ is chunk size, and $T'=\lceil T/C\rceil C$. The cited formulation identifies this reshaping as a major source of GPU memory growth and reduced throughput. TC-BiMamba addresses this by training both bidirectional sequences in an “offline style” regardless of chunk size, so that dynamic chunk-size training requires only one reshape per batch and one bidirectional pass rather than separate backward passes for each chunk (She et al., 12 Feb 2026).

A recurring source of confusion is to equate Trans-Chunk with conventional chunk-wise training. In the ASR formulation, the mechanism is explicitly presented as a replacement for per-chunk backward passes rather than a restatement of them.

2. Encoder, decoder, and state-space formulation

The ASR architecture consists of a convolutional subsampling front-end that reduces frame rate by $4\times$, followed by a stack of $L$ BiMamba encoder blocks, and a “hybrid” BiTransformer decoder in which each masked self-attention is replaced by a Mamba module while standard cross-attention remains unchanged (She et al., 12 Feb 2026). Each encoder block contains a forward Mamba pass, a backward Mamba pass with independent parameters, a learnable fusion weight $\beta$, and a convolution module with kernel size $k$ to capture local patterns.

The Mamba component is expressed through the discrete SSM formulation

$$h_t = \bar A_t\,h_{t-1} + \bar B_t\,x_t,\quad y_t = C\,h_t + D\,x_t,$$

with

$(B\times N,\,T'/C,\,D)$0

and $(B\times N,\,T'/C,\,D)$1 (She et al., 12 Feb 2026). Within each encoder block, the bidirectional outputs are fused as

$(B\times N,\,T'/C,\,D)$2

This design places linear-time sequence modeling at the encoder core while retaining a two-directional contextual pathway. The convolution module supplements the state-space pathway with local pattern modeling, and the hybrid decoder preserves cross-attention while replacing masked self-attention with Mamba. The result is not a pure SSM stack but a mixed encoder-decoder system specialized for ASR deployment modes that vary by latency.

3. The Trans-Chunk mechanism

The Trans-Chunk mechanism is the defining procedural component of TC-BiMamba in ASR (She et al., 12 Feb 2026). Its purpose is to replace per-chunk backward passes with a single offline-style pass while still training for dynamic chunk sizes.

Given $(B\times N,\,T'/C,\,D)$3 and chunk size $(B\times N,\,T'/C,\,D)$4 with minimum 2, the forward pass ignores $(B\times N,\,T'/C,\,D)$5 and computes

$(B\times N,\,T'/C,\,D)$6

The backward path branches on whether $(B\times N,\,T'/C,\,D)$7.

When $(B\times N,\,T'/C,\,D)$8, the method reverses the full sequence, applies $(B\times N,\,T'/C,\,D)$9 once, and reverses the output back. When $N=T'/C$0, it pads the sequence to $N=T'/C$1, reshapes the padded tensor to $N=T'/C$2 with $N=T'/C$3, reverses within each chunk-sequence, applies $N=T'/C$4 once, reshapes back to $N=T'/C$5, crops to length $N=T'/C$6, and reverses again along the chunk dimension (She et al., 12 Feb 2026). Fusion and convolution then produce

$N=T'/C$7

The central claim is that, because the backward computation is not split into $N=T'/C$8 separate forward calls, peak memory and compute remain similar to offline BiMamba while the model still learns to handle any chunk size dynamically. The formulation therefore distinguishes chunk-aware training behavior from chunk-wise execution overhead.

4. Training objective, decoding, and latency behavior

TC-BiMamba uses joint CTC + E2E training and two-pass decoding (She et al., 12 Feb 2026). The training objective uses $N=T'/C$9 to balance $C$0 and $C$1, while the E2E term uses $C$2 to combine left-to-right and right-to-left losses. The decoding score is likewise defined by a weighted combination of CTC, left-to-right, and right-to-left scores.

This setup is directly tied to the latency claims of the model. Dynamic chunk-size training is intended to produce a single checkpoint that can be deployed for any streaming latency by choosing different chunk sizes at inference, or for full-context offline decoding by setting chunk size equal to the full sequence length (She et al., 12 Feb 2026). The minimum chunk size is fixed at 2 rather than 1, so the formulation explicitly notes that extremely low-latency settings below 2 frames may be unsupported.

The combination of dynamic chunk-size training and bidirectional processing is presented as the basis for “unified streaming and offline ASR in one model.” A plausible implication is that the method aims to collapse what would otherwise be separate offline and streaming training pipelines into one encoder-decoder configuration, though the concrete claim made in the source is deployment flexibility rather than pipeline simplification.

5. Efficiency and empirical performance in ASR

The computational argument compares TC-BiMamba with traditional chunk-wise training as in LC-BiMamba (She et al., 12 Feb 2026). In the chunk-wise baseline, forward cost per batch is $C$3, backward cost consists of $C$4 separate passes of $C$5 each, and total backward compute is still $C$6 but with $C$7 more launch overhead and memory for reshaped tensors. In TC-BiMamba, there is only one backward pass of cost $C$8, and the reshape from $C$9 to $T'=\lceil T/C\rceil C$0 and back is described as negligible compared to $T'=\lceil T/C\rceil C$1 passes.

Empirically on LibriSpeech, training throughput is reported as 1.3× faster than the chunk-splitting baseline, specifically “TC-BiMamba(L) vs. TC-BiMamba(L)-CS,” and peak GPU memory is reduced by approximately 50% (She et al., 12 Feb 2026). Analytically, when $T'=\lceil T/C\rceil C$2 is large, the overhead of $T'=\lceil T/C\rceil C$3 kernel launches and extra temporary buffers in chunk splitting is described as scaling roughly as $T'=\lceil T/C\rceil C$4, whereas Trans-Chunk eliminates this factor.

Two model variants are reported: TC-BiMamba(S), a small model with approximately 47 M parameters, 9 encoder blocks, and model dimension 256; and TC-BiMamba(L), a large model with approximately 70 M parameters, 17 encoder blocks, and model dimension 320 (She et al., 12 Feb 2026). On AISHELL-1, TC-BiMamba(S) achieves 5.25 offline (Res), 5.92 offline (CTC), 5.70 chunk16 (Res), and 6.60 chunk16 (CTC), compared with U2++ at 5.31, 6.12, 5.75, and 6.78. On AISHELL-2, TC-BiMamba(S) records 6.00, 6.88, 6.43, and 7.53, compared with U2++ at 6.41, 7.08, 6.74, and 7.72.

On LibriSpeech clean, U2++ reports 3.32 offline (Res), 3.76 offline (CTC), 3.80 chunk16 (Res), and 4.54 chunk16 (CTC); TC-BiMamba(S) reports 3.12, 3.66, 3.66, and 4.40; TC-BiMamba(L) reports 2.97, 3.39, 3.51, and 4.14; and LC-BiMamba reports 3.00 offline and 3.70 chunk16 with 118.70 M parameters (She et al., 12 Feb 2026). On LibriSpeech other, U2++ gives 8.67, 9.50, 10.38, and 11.52; TC-BiMamba(S) gives 8.19, 9.05, 9.78, and 11.13; TC-BiMamba(L) gives 7.72, 8.62, 9.44, and 10.58; and LC-BiMamba gives 7.10 offline and 8.90 chunk16. The stated summary is that TC-BiMamba(S) consistently beats U2++ in both offline and streaming modes, and TC-BiMamba(L) matches or exceeds LC-BiMamba on clean data with approximately 40% fewer parameters. The source also notes that streaming WER remains close to offline WER, which it interprets as effective cross-chunk context modeling.

6. Separate point-cloud instantiation under the same label

A distinct use of the label appears in the PoinTramba literature, where the “Trans-Chunk BiMamba (TC-BiMamba) design” is described as instantiated in a hybrid Transformer-Mamba framework for point cloud analysis (Wang et al., 2024). In that setting, the input is a raw point cloud

$T'=\lceil T/C\rceil C$5

The method first samples $T'=\lceil T/C\rceil C$6 anchor points via Farthest Point Sampling and, for each anchor, collects its $T'=\lceil T/C\rceil C$7 nearest neighbors, forming $T'=\lceil T/C\rceil C$8 disjoint groups. Each point is projected into a point embedding and a positional embedding,

$T'=\lceil T/C\rceil C$9

or, in the mathematical formulation,

$4\times$0

A standard Transformer is then applied independently within each group for $4\times$1 layers, and the group representation is obtained by mean pooling,

$4\times$2

The point-cloud formulation introduces a Bi-Directional Importance-aware Ordering (BIO) strategy. It predicts a scalar importance score $4\times$3 for each $4\times$4, sorts the group embeddings once in descending order and once in ascending order, and concatenates the two ordered lists into a sequence of length $4\times$5. Inter-group relationships are then modeled by standard Mamba layers, and importance-aware pooling discards final embeddings whose predicted score satisfies $4\times$6 and sums the rest to form the global feature $4\times$7 (Wang et al., 2024). Supervision for the importance scores uses a cosine-similarity score

$4\times$8

a SmoothL1 importance loss, and an alignment loss denoted AM-CL.

The reported hyperparameters are $4\times$9 for classification or $L$0 for segmentation, $L$1 for classification or $L$2 for segmentation, Transformer depth $L$3, Mamba depth $L$4, embedding dimension $L$5, and projection dimension $L$6 (Wang et al., 2024). The parameter count is approximately 19.5 M, compared with approximately 22 M for a vanilla Transformer on $L$7 and 12.3 M for PointMamba. The rough per-sample FLOP estimate is approximately 50 M for the intra-Transformer, approximately 2.36 M for the inter-Mamba, and approximately 52 M total, versus approximately 1.6 G for a vanilla full-cloud Transformer.

The experimental ablations attribute a progression on ScanObjectNN PB-T50-RS from 82.5% for Base Mamba (no chunking), to 86.5% with +Intra-Transformer, to 88.6% with +BIO ordering, and to 89.1% total with +IAP pooling (Wang et al., 2024). On ModelNet40, the method reports 92.7%, beating 92.4% for PointMamba and 92.3% for PointMAE. On ShapeNetPart, it reports 85.7% versus 85.8% for PointMamba with 25 M parameters. The ordering ablation gives 87.2% for random ordering, 87.0% for coordinate $L$8, 87.3% for Hilbert, 88.1% for single-direction importance-aware ordering, and 89.1% for bi-directional importance-aware ordering. The source summarizes this design as combining the long-range modeling power of Transformer on small chunks with Mamba’s linear-time inter-chunk modeling, using BIO to provide a “meaningful” order rather than random noise, and using IAP to focus global pooling on positive, high-importance group embeddings.

This separate usage suggests a broader naming pattern in which “Trans-Chunk BiMamba” denotes chunk-level local processing followed by Mamba-based modeling across chunk representations, but the concrete mechanisms, objectives, and benchmarks differ substantially between the ASR and point-cloud settings.