Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generation

Abstract: Discrete diffusion LLMs (dLLMs) recover masked tokens in parallel, offering significant speedups over autoregressive (AR) generation. However, such promising frameworks face a fundamental architectural design dilemma: \ding{182} Adopting bidirectional attention achieves strong generation quality by allowing each position to access the full context, but is inherently incompatible with KV caching, limiting inference throughput in batch-serving scenarios; \ding{183} Conversely, causal attention enables efficient cached inference but loses all right-side context, substantially degrading generation quality. This paper introduces Bifocal dLLMs, a new paradigm that resolves this dilemma through \emph{asymmetric bidirectional context}. Analogous to bifocal lenses, we instantiate the paradigm as \textbf{R2LM} (Right-to-Left Mamba), which combines two complementary mechanisms: $a$) standard causal attention providing precise left-context with full KV cache compatibility, while $b$) a lightweight reverse Mamba SSM sidecar supplying compressed right-side context without breaking cacheability. Comprehensive experiments on continued pretraining of Qwen3-1.7B with 60B tokens demonstrate that R2LM achieves $2.4\times$ to $12.9\times$ higher throughput than bidirectional dLLMs and $1.9\times$ to $2.9\times$ speedup over AR baselines in batch serving through parallel decoding with KV caching, while exceeding the causal baseline on most benchmarks and surpassing the bidirectional dLLM on average.

• The paper presents a dual-path architecture using a causal Transformer and a reverse SSM to maintain KV caching while injecting right-context for improved generation.
• It achieves up to 12.9x throughput improvement and up to 32% memory savings on long sequences compared to bidirectional models.
• Experimental results demonstrate enhanced performance on key benchmarks, surpassing both causal and bidirectional approaches in quality and efficiency.

Bifocal Diffusion LLMs: Asymmetric Bidirectional Context for Parallel Generation

Introduction and Motivation

Diffusion-based LLMs (dLLMs) have emerged as an alternative to conventional autoregressive (AR) LLMs by enabling parallel recovery of masked tokens for text generation through iterative denoising. However, achieving fast parallel decoding raises an architectural compromise: bidirectional attention provides full context but prohibits efficient key-value (KV) caching for inference, resulting in high compute and memory cost, while causal attention enables KV caching but discards right-side context, leading to reduced generation quality. Existing attempts (block-wise, hybrid, or reordered-attention models) only partially mitigate this dilemma by trading off context completeness, cacheability, and model complexity.

This work presents a new architectural paradigm termed Bifocal Diffusion LLMs (Bifocal dLLMs), which achieves asymmetric bidirectional context—distinct mechanisms process left and right context—offering a resolution to the context-cacheability dichotomy. The core instantiation, named R2LM (Right-to-Left Mamba), utilizes a standard causal Transformer to fully exploit left-side context and maintain KV-cache compatibility, while a lightweight reverse Mamba state space model (SSM) supplies compressed right-side context via a parallel sidecar pathway. This design allows efficient batch-serving inference with parallel generation and validates the notion that right context can be injected without breaking prefix-caching.

Architectural Contributions

Bifocal dLLMs decouple left and right context processing as follows:

1. Causal Transformer Backbone: The backbone provides left-context information strictly through causal attention, fully compatible with KV caching, ensuring fast and efficient inference.
2. Reverse Mamba SSM Sidecar (R2L pathway): The right context is supplied through a residual stream using a reverse-direction Mamba (an SSM), which scans the sequence right-to-left, flipped for correct aggregation, and injects compressed right-side context as a residual correction to the backbone hidden states. Gated and normalized connections ensure stable additive integration.
3. Zero-Initialized Residual Gate: R2LM is initialized to be functionally identical to the causal model, enabling seamless integration into pretrained checkpoints and progressive opening of the right-context path during continued pretraining.
4. Plug-in Adaptation: The R2LM-PI variant allows freezing a pretrained causal dLLM and training only the R2L pathway, demonstrating transferability and flexibility.

The primary theoretical insight is additive log-posterior decomposition, where total conditional information at each masked position can be separated into left- and right-context contributions; the model operationalizes this by independently computing each and summing them, bounded by design limitations such as non-linearities in Transformer architectures.

Experimental Results

Throughput and Memory Efficiency

R2LM achieves a substantial throughput advantage over bidirectional dLLMs due to its retention of KV cacheability:

• In batch-serving regimes (e.g., batch size 8, sequence length 4096), R2LM provides a 2.4x to 12.9x throughput improvement over bidirectional dLLMs, while maintaining a consistent computational and memory profile close to pure causal dLLMs.
• Peak GPU memory usage grows sublinearly with sequence length for R2LM, contrasting with linear memory growth in bidirectional models, yielding up to 32% memory savings at large sequence lengths (16384 tokens).

Generation Quality

On seven multiple-choice benchmarks spanning both multi-token and single-token target tasks (ARC-Challenge, HellaSwag, PIQA, WinoGrande, OpenBookQA, BoolQ, MMLU), R2LM is evaluated on a continued-pretrained Qwen3-1.7B backbone:

• On long-target tasks, R2LM attains an average accuracy of 47.44%, outperforming both the causal baseline (43.90%) and the bidirectional dLLM (44.78%).
• On short-target tasks, R2LM closes most of the gap to bidirectional dLLM and surpasses causal.
• The R2LM-PI plug-in variant, despite training only 185M parameters on 5B tokens (one-tenth the compute budget), matches or surpasses all baselines on ALL (47.76%) and Short (52.15%) averages.
• Variants without correct directionality (MLP Adapter, L2R Mamba) confirm directionality is essential and mere parameter increase is insufficient; only the reverse pathway recovers the bidirectional information upper bound.

Notably, R2LM and R2LM-PI exceed both the causal and bidirectional dLLM endpoints on key benchmarks on average, indicating the efficacy of asymmetric context supply.

Theoretical and Practical Implications

The Bifocal framework redefines the context-efficiency trade-off in parallel-convergent text generation. By showing that non-attention right-context modules can deliver position-aware, compressed information without impacting KV-caching, the work revises aspirations for batch-inference text generation—demonstrating that full-sequence context does not necessitate full recomputation per generation step.

• The clean integration via forward hooks and zero-gate initialization facilitates non-destructive, efficient adaptation of any cached Transformer checkpoint into a dLLM with enhanced context aggregation.
• The approach exposes an open design space: the nature of the right-context module (here, SSM-based) can be flexibly defined without modifying the backbone, suggesting applicability for other sequence models, potentially impacting architectures reliant on masked token prediction or iterative denoising.
• The setup allows for data- and compute-efficient right-context adaptation, which can be critical for applying large pretrained LMs to new domains or tasks where right-context dependency is high.

Limitations and Future Work

The presented results are obtained in the moderately-sized Qwen3-1.7B regime with 60B tokens; the behavior and scaling properties at larger model/data scales remain to be explored. The overhead of the R2L pathway, while minor, can impose a small penalty in single-request (B=1) settings. Exploring lower-rank, shared, or factored variants for the right-context module could offer further efficiency. Investigating the interplay between backbone adaptation and right-context signal, as well as alternative right-context mechanisms, constitutes a rich avenue for architectural research.

Conclusion

Bifocal dLLMs, instantiated as R2LM, present a principled architectural framework for discrete diffusion-based language generation that reconciles the context-quality advantage of bidirectional models with the inference efficiency of causal models. By using parallel pathways for asymmetric context—causal backbone with KV cache for left, SSM-based sidecar for right—this approach produces state-of-the-art throughput and quality among parallel dLLMs. The effectiveness of plug-in adaptation and empirical dominance over both classical endpoints underscore the broader implication: parallel language generation can closely approach (and sometimes exceed) AR model quality without sacrificing deployment efficiency, challenging previous boundaries between context optimization and inference throughput (2606.27732).