---
title: Adaptive Bi-directional Cyclic Diffusion (ABCD)
url: https://www.emergentmind.com/topics/adaptive-bi-directional-cyclic-diffusion-abcd
type: topic
---

# Adaptive Bi-directional Cyclic Diffusion (ABCD)

Adaptive Bi-directional Cyclic Diffusion (ABCD) is an adaptive, search-based inference framework for diffusion models that dynamically scales computational effort during sampling, allowing for instance-specific allocation of compute and principled early stopping. In contrast to conventional uni-directional, fixed-schedule denoising, ABCD iteratively refines a population of candidate samples through bi-directional diffusion cycles, automatically tuning the depth of exploration and termination criterion to maximize a task-specific reward or verifier. ABCD consists of three main components: Cyclic Diffusion Search, Automatic Exploration-Exploitation Balancing, and Adaptive Thinking Time, and has demonstrated empirical effectiveness across generative and reasoning tasks by concentrating compute where needed without sacrificing efficiency [2505.14036].

## 1. Formal Definition and Architecture

Let $p_\theta(x_{0:T})$ denote a pretrained diffusion model characterized by a forward noising kernel $q(x_{t+1}|x_t) = \mathcal{N}(x_{t+1}; \sqrt{\alpha_{t+1}} x_t, (1-\alpha_{t+1})I)$ and a reverse denoiser $p_\theta(x_{t-1}|x_t)$. Given a task-specific reward function $r(x_0)$ and a population of $N$ candidate samples ("particles"), standard sampling runs a uni-directional reverse chain $x_T \sim \mathcal{N}(0, I) \rightarrow \dots \rightarrow x_0$ to produce one output. ABCD reframes this as a search process that:

- Iteratively seeks to maximize $r(x_0)$ by refining particle populations.
- Allocates increased computation adaptively to more difficult instances.
- Dynamically terminates inference when further improvement is unlikely.

The core algorithm introduces three elements:  
(i) Bi-directional cycling through the diffusion timeline (combining both forward "go-back" noising and reverse denoising steps).  
(ii) Multi-level allocation of particles at different stages of the diffusion process ("temperature pool").  
(iii) Adaptive termination dictated by the dynamic progress of top candidates.

## 2. Cyclic Diffusion Search: Mechanics and Mathematical Formulation

Cyclic Diffusion Search (CDS) is the operational foundation of ABCD. Evaluation steps are as follows:

- Define the "temperature pool" $T = \{t_1, \dots, t_M\}$ ($0 = t_1 < \dots < t_M = T$), which specifies the set of possible "go-back" levels for each cycle.
- Maintain a population of $N$ particles $\{x_0^{(i)}\}_{i=1}^N$ at each cycle $c$. Each particle is associated with a verifier score $r_i = r(x_0^{(i)})$.
- Each cycle consists of:
  - **Fast Denoising**: Initialize by applying $x_0^{(i)} \leftarrow \text{DDIM}_\text{jump}(x_T^{(i)}; \text{steps}~\Delta T)$ for all $i$.
  - **Selection-and-Copy**: Select the indices $S(c)$ of the top-$K$ particles by $r_i$, then replicate each selected particle $J$ times ($K\cdot J$ total).
  - **Noising (Forward)**: For each replica and each $t' \in T$, sample $x_{t'}^{k,j} \sim q(x_{t'}|x_0^{k,j})$.
  - **Denoising (Reverse)**: Run $\text{DDIM}_{\text{jump}}(x_{t'}^{k,j};~t' \rightarrow 0)$ for all noised particles to produce new $x_0^{k,j}$.
  - **Population Update**: Merge $K\cdot J$ outputs, evaluate $r(\cdot)$, and proceed to the next cycle.

Forward (noising) and reverse updates use DDIM-like transitions:
\[
x_{t'} = \sqrt{\bar{\alpha}_{t'}} x_0 + \sqrt{1-\bar{\alpha}_{t'}}\epsilon, \quad \epsilon \sim \mathcal{N}(0,I)
\]
\[
x_{t-1} = \frac{1}{\sqrt{\alpha_t}}\left(x_t - \frac{1-\alpha_t}{\sqrt{1-\bar{\alpha}_t}}\epsilon_\theta(x_t,t) \right) + \sigma_t z, \quad z \sim \mathcal{N}(0,I)
\]

The process continues until the adaptive stopping criterion is triggered.

## 3. Automatic Exploration–Exploitation Balancing

ABCD provides a mechanism for adjusting exploration depth via the temperature pool, denoted as Automatic Exploration–Exploitation Balancing (AEEB). Instead of a fixed go-back parameter $t'$, the process distributes each selected survivor across all $M$ temperature levels in $T$ during each cycle. For survivors $S(c) = \{k_1, ..., k_K\}$:

\[
\{ x_{t_i}^{k_j,m} \sim q(\cdot|x_0^{k_j}) \mid i=1..M, j=1..K, m=1..\frac{J}{M} \}
\]

No explicit regularizer is applied; underperforming go-back levels are naturally filtered out by selection in subsequent cycles. The hyperparameters controlling this trade-off are $M$ (pool size), $J$ (replicas per anchor), and $K$ (survivors per cycle). This structure inherently supports both global exploration (large $t_i$) and fine local refinement (small $t_i$), adaptively tailored per instance.

## 4. Adaptive Thinking Time: Cycle Termination and Instance Difficulty

Adaptive Thinking Time (ATT) governs termination by monitoring which go-back levels produce the current top-$K$ solutions across cycles. For multiset $D(c) \subset T$ of the go-back indices for the present top-$K$:

- Define $t_{\min}(c) = \min D(c)$.
- Inference terminates at cycle $C$ if $t_{\min}(\ell) = 0$ for all $\ell \in \{C-w+1, ..., C\}$, for given persistence $w$.

This allows cycles to persist longer on more challenging instances, as reflected by the observed history of $t_{\min}(c)>0$, and terminate rapidly for easier ones.

The hyperparameters are $w$ (controls duration before termination upon repeated exploitation) and $\text{max\_iter}$ (enforces an upper limit on the number of cycles).

## 5. Computational Complexity and Convergence Guarantees

Pseudocode formalizes the ABCD algorithm with per-cycle computational cost:

\[
O(N \cdot L + N \log N + K \cdot J \cdot L / M)
\]
where $L$ is the number of DDIM steps and choosing $J/K \approx M$ attains overall per-cycle cost $O(N \cdot L)$. The algorithm is guaranteed to terminate in finite steps with probability 1, provided the reward is bounded above by $r^*$, the denoising step has full support, and selection is monotonic:

- **Theorem 1 (Finite-time termination)**: Using ATT, termination occurs in finite $c$ almost surely, with the best reward in the final top-$K$ converging to $r^*$.
- Proof outline: Each cycle has constant nonzero probability $p$ of hitting the global maximum reward $x^*$. Once $x^*$ appears in the top-$K$, subsequent cycles quickly enter persistent exploitation, triggering ATT's stopping criterion.

## 6. Empirical Evaluation and Ablation Studies

ABCD has been benchmarked across six diverse tasks using multiple baselines: Base Diffusion, Best-of-N (BoN), Diffusion Beam Search (BS), Sequential Monte Carlo (SMC), and Search-over-Paths (SoP).

| Task                | Key Metric(s)    | Core Results                                                                  |
|---------------------|------------------|-------------------------------------------------------------------------------|
| Mixture of Gaussians| success rate     | Only ABCD attains 100% success with adaptive go-backs within 25 cycles        |
| Sudoku              | accuracy, time   | ABCD achieves 100% accuracy; SoP caps at ~95.5%; superior time-accuracy trade |
| Pixel Maze          | success rate     | Near-100% success for sizes up to 15 with far less time than baselines        |
| Molecule Generation | stability, time  | Peaks at ~0.99 stability in 20s; SoP only reaches ~0.94 in much longer        |
| OGBench PointMaze   | success rate     | Only ABCD to achieve 100% on giant; greater speed for all environments        |
| Text-to-Image       | compressibility, | Target levels achieved $\geq$ baselines in $<$1/4 time; fewer seconds needed  |
| (Stable Diffusion)  | aesthetic, pref. | for matched human preference score (e.g. 81s vs $>$300s for SoP)              |

Ablation analysis reveals:

- Temperature pool yields higher performance than any fixed go-back.
- Adaptive cycle endpoint (ATT) improves trade-off over fixed cycle counts.
- Per-instance cycle count/time correlates with instance difficulty.
- Sample diversity is preserved even as convergence to high-reward regions occurs (assessed via CLIP cosine similarity).

## 7. Relation to Prior Methods and Applicability

Uni-directional approaches—Base, BoN, SMC, BS—lack adaptability, leading to wasted computation on easy cases or under-exploration on difficult ones. SoP introduces backward steps but is restricted to a fixed schedule. ABCD generalizes these approaches by:

- Allowing bi-directional cycling and flexible exploration at every cycle.
- Distributing computation both globally and locally via the temperature pool.
- Providing a principled, instance-wise stopping criterion (ATT) with convergence guarantees.

ABCD is especially well-suited for tasks with heterogeneous instance difficulty—including reasoning/planning (e.g., maze navigation, Sudoku), generative tasks that require focused computation (e.g., stable molecule synthesis, high-fidelity image generation), or scenarios with nondifferentiable rewards. As it relies only on a black-box verifier, ABCD supports a broad range of problems beyond differentiable or likelihood-based domains.

In summary, ABCD advances inference-time scaling in diffusion-based models by treating sampling as an adaptive search process, offering flexible resource allocation, and furnishing practical guarantees across a variety of domains [2505.14036].

Source: https://www.emergentmind.com/topics/adaptive-bi-directional-cyclic-diffusion-abcd