LucidFlux: Caption-Free Diffusion Restoration

• LucidFlux is a caption-free universal image restoration framework that leverages the Flux.1 diffusion transformer to adaptively recover high-quality images from unknown degradation mixtures.
• It employs a lightweight dual-branch conditioner that fuses information from both degraded inputs and lightly restored proxies to balance detail preservation and artifact suppression.
• The framework utilizes timestep- and layer-adaptive modulation along with a large-scale, structure-rich data curation pipeline to ensure semantic fidelity without relying on text-based guidance.

Searching arXiv for the LucidFlux paper and closely related context papers. LucidFlux is a caption-free universal image restoration (UIR) framework that adapts the large Flux.1 Rectified-Flow Diffusion Transformer for restoration under unknown mixtures of degradations while preserving semantics (Fei et al., 26 Sep 2025). It is designed for settings in which discriminative restorers and UNet-based diffusion priors often oversmooth, hallucinate, or drift, and it replaces caption-based semantic guidance with image-derived conditioning. Its central design couples a lightweight dual-branch conditioner, a timestep- and layer-adaptive modulation schedule, caption-free semantic alignment via SigLIP features, and a large-scale structure-rich data curation pipeline (Fei et al., 26 Sep 2025).

1. Problem setting and design objective

Universal image restoration aims to recover high-quality, semantically faithful images from inputs degraded by unknown mixtures of real-world noise, blur, compression, and related artifacts (Fei et al., 26 Sep 2025). The paper frames the core difficulty as a compound problem: degradations are diverse and usually unlabeled; semantic preservation is required; prompt- or caption-based guidance can introduce latency, variability, and bias; UNet-based backbones are often inadequate on complex degradations; and large diffusion transformers require substantially more and better data than standard restoration datasets provide (Fei et al., 26 Sep 2025).

LucidFlux addresses this setting by eliminating runtime and training dependence on textual captions. The framework uses the degraded image and a lightly restored proxy as complementary conditioning signals, then routes them across the Flux.1 backbone according to diffusion timestep and network depth. In the terminology of the paper, the degraded input is rich in detail but noisy, whereas the proxy is cleaner but smoother. The architecture is therefore organized around a division of labor: one branch anchors geometry and the other suppresses artifacts (Fei et al., 26 Sep 2025).

A common misunderstanding is to treat LucidFlux primarily as a larger restoration model. The paper argues the opposite emphasis: for large DiTs, “when, where, and what to condition on” is the governing lever, rather than adding parameters or relying on text prompts (Fei et al., 26 Sep 2025). In that sense, LucidFlux is not simply a scale-up of prior diffusion restorers; it is an adaptation strategy for a pretrained large DiT under caption-free constraints.

2. Backbone adaptation and dual-branch conditioning

LucidFlux uses Flux.1 as a frozen backbone during training, with all backbone parameters frozen for efficiency and to preserve the pretrained generative prior; only small, task-specific adapters and conditioners are trained (Fei et al., 26 Sep 2025). The paper describes Flux.1 as a large, open-weight, rectified-flow diffusion transformer with strong generative priors and flexible multimodal conditioning.

The principal conditioning module is a lightweight dual-branch conditioner. The two inputs are the original degraded image $I_{\text{LQ}}$ and a “Lightly Restored Proxy” image $I_{\text{LRP}}$. Both are encoded via the frozen Flux.1 VAE:

$$z_{\text{LQ}} = E(I_{\text{LQ}}), \qquad z_{\text{LRP}} = E(I_{\text{LRP}}).$$

Each latent is then processed by a dedicated 2-block MM-DiT transformer with no weight sharing:

$$\phi_{\text{LQ}} = \mathrm{DBC}(z_{\text{LQ}}), \qquad \phi_{\text{LRP}} = \mathrm{DBC}(z_{\text{LRP}}).$$

The paper’s interpretation of this split is explicit: conditioning only on degraded images propagates noise and artifacts, while conditioning only on a strong restorer oversmooths and loses detail; using both signals is meant to retain texture while stabilizing structure (Fei et al., 26 Sep 2025).

The conditioner is described as minimal, with constant small overhead relative to approaches that require ControlNet-scale duplication. This architectural choice situates LucidFlux within the adapter-based line of DiT adaptation rather than full-backbone finetuning. A plausible implication is that the model attempts to preserve the pretrained Flux prior as much as possible, intervening mainly in the conditioning pathway rather than in the backbone weights themselves.

3. Timestep- and layer-adaptive control and caption-free semantics

LucidFlux’s second defining mechanism is a timestep- and layer-adaptive modulation schedule. The paper states that, in diffusion transformers, early timesteps and shallow layers restore global structure, while later timesteps and deeper layers recover fine details (Fei et al., 26 Sep 2025). To match this behavior, the model computes sinusoidal positional encodings of $(t/T, l/L)$ for timestep $t$ and layer $l$, then predicts per-channel scale and bias for each conditioning branch:

$$\boldsymbol\alpha^{t,l}_\bullet, \boldsymbol\beta^{t,l}_\bullet = \mathrm{Modulation}_\bullet(\mathrm{PE}(t/T, l/L)), \qquad \bullet \in \{\mathrm{LQ}, \mathrm{LRP}\}.$$

The branch features are modulated as

$$\tilde{\phi}_\bullet^{t,l} = \boldsymbol\alpha^{t,l}_\bullet \odot \phi_\bullet + \boldsymbol\beta^{t,l}_\bullet,$$

and fused by a learned convex gating:

$$\gamma^{t,l} = \sigma(\mathrm{Gate}(\mathrm{PE}(t/T, l/L))),$$

$I_{\text{LRP}}$0

This schedule is intended to produce coarse-to-fine and context-aware updates that protect global structure while recovering texture (Fei et al., 26 Sep 2025).

The semantic pathway is caption-free. Instead of generating captions with a VLM or MLLM, LucidFlux extracts SigLIP features from the lightly restored proxy:

$I_{\text{LRP}}$1

These features are used as a drop-in replacement for text embeddings in the Flux DiT context (Fei et al., 26 Sep 2025). The rationale is twofold. First, it removes external captions at both training and runtime. Second, it avoids “degradation leakage”: the paper reports that 17\%-24\% of MLLM-generated captions for real degradations explicitly reference blur, noise, and related artifacts, which can misguide restoration (Fei et al., 26 Sep 2025).

This directly addresses another common misconception: LucidFlux is not “caption-lite” or “caption-assisted.” Its semantic alignment is explicitly caption-free, and the paper presents this as both a robustness measure and a latency reduction measure.

4. Data curation and supervision regime

A central component of LucidFlux is its large-scale, structure-rich data curation pipeline. The paper argues that standard restoration datasets such as DIV2K and LSDIR are insufficient in scale and quality for training large DiTs for UIR (Fei et al., 26 Sep 2025). To address this, it constructs a three-stage automatic pipeline over 2.3M web images plus 557K images from Photo-Concept-Bucket.

The first stage is Blur Detection, using Laplacian variance:

$I_{\text{LRP}}$2

The second stage is Flat Region Filtering, which removes images dominated by textureless patches using Sobel edge variance per $I_{\text{LRP}}$3 patch and discards an image if more than 50\% of patches are flat ($I_{\text{LRP}}$4) (Fei et al., 26 Sep 2025). The third stage is IQA Perceptual Ranking, which ranks the remaining images by CLIP-IQA perceptual score and keeps the top 20\%.

After these stages, and after including the LSDIR high-quality subset, the final training set is 342K images, producing 1.36M degraded/clean pairs via Real-ESRGAN-based synthetic degradation (Fei et al., 26 Sep 2025). The paper describes this curated corpus as more diverse, structured, and of higher perceptual quality than prior restoration datasets.

This curation pipeline is not a peripheral engineering detail. The ablations identify the large-scale high-quality data as the source of the largest jump in improvement when incorporated (Fei et al., 26 Sep 2025). A plausible implication is that LucidFlux treats restoration quality as jointly dependent on conditioning design and supervision quality, rather than assuming that a strong generative prior alone is sufficient.

5. Evaluation, baselines, and ablation evidence

LucidFlux is evaluated on both synthetic and real-world benchmarks: DIV2K, LSDIR, RealSR, DRealSR, and RealLQ250, at $I_{\text{LRP}}$5 resolution (Fei et al., 26 Sep 2025). The reported metrics include perceptual and semantic no-reference scores such as CLIP-IQA+, Q-Align, MUSIQ, MANIQA, and NIMA, together with reference metrics PSNR, SSIM, and LPIPS.

Across these settings, the paper states that LucidFlux consistently outperforms strong open-source and commercial baselines, including ResShift, StableSR, SinSR, SeeSR, DreamClear, SUPIR, HYPIR, Seedream, Topaz, and MeiTuSR, especially on real-world mixed degradations (Fei et al., 26 Sep 2025). The qualitative characterization is correspondingly specific: sharper edges, more faithful textures, and superior semantic fidelity.

The model uses a 12B backbone with 1.6B trainable adapter overhead, while the backbone remains frozen (Fei et al., 26 Sep 2025). The paper also emphasizes that, despite this large backbone, inference avoids caption preprocessing entirely.

Model	Total Inference (s)	Caption Preproc (s)
SeeSR	22.5	0.1
SUPIR	22.5	5.9
DreamClear	37.6	8.7
LucidFlux	23.6	0

This comparison is important for interpreting the paper’s efficiency claims. LucidFlux does not have the smallest backbone, and its total inference time is reported as 23.6 seconds in the excerpted table. Its efficiency advantage is instead tied to the complete removal of caption preprocessing, which avoids the caption-generation overhead present in several competing methods (Fei et al., 26 Sep 2025).

The ablation results support the necessity of each module. The paper reports that the dual-branch conditioner, SigLIP semantic alignment, the timestep- and layer-adaptive modulation, and the large-scale high-quality data each independently and additively improve quality metrics; removing either temporal or hierarchical modulation, or reverting to caption-based semantic guidance, leads to inferior results or instability (Fei et al., 26 Sep 2025).

6. Position within Flux-based diffusion research

LucidFlux belongs to a broader line of work that adapts Flux-based diffusion transformers to tasks that require precise control over pretrained generative priors. In a separate Flux-based study on content–style disentanglement, “SplitFlux: Learning to Decouple Content and Style from a Single Image” reports that Single Dream Blocks are essential for image generation, that early single stream blocks mainly control content, and that later blocks govern style (Yang et al., 19 Nov 2025). That work is not a restoration method, but it provides a useful architectural comparison: Flux is not treated as a homogeneous stack, and blockwise specialization matters.

This suggests a conceptual affinity with LucidFlux’s own design. Its timestep- and layer-adaptive modulation schedule assumes that different stages of the backbone contribute differently to global structure and fine detail, and therefore should receive different conditioning signals at different times (Fei et al., 26 Sep 2025). The two papers target different problems—UIR in LucidFlux, content–style disentanglement in SplitFlux—but both foreground the internal organization of Flux rather than treating conditioning as a uniform injection process.

LucidFlux should therefore not be conflated with personalized image generation methods or with prompt-centric restoration pipelines. Its target is universal image restoration under unknown mixed degradations; its semantics are image-derived rather than caption-derived; and its principal claim is that robust restoration in the wild depends on conditioning design, semantic grounding without captions, and structure-rich supervision at scale (Fei et al., 26 Sep 2025).