LoongX: Neural Signal Image Editing

• LoongX is a hands-free image editing framework that translates EEG, fNIRS, PPG, and motion data into controllable semantic editing operations.
• It utilizes a cross-scale state space encoder and dynamic gated fusion to align and integrate multimodal signals with language cues.
• LoongX establishes a benchmark for neural signal-driven generative modeling, enabling accessible, high-fidelity image edits regardless of motor or language abilities.

LoongX is a hands-free image editing framework that translates multimodal neurophysiological and physiological signals—including electroencephalography (EEG), functional near-infrared spectroscopy (fNIRS), photoplethysmography (PPG), and inertial measurement unit (IMU) motion data—into controllable, semantic editing operations via a diffusion transformer backbone. By fusing multiscale brain and body signals with language or speech cues, LoongX enables high-fidelity, intuitive image editing for users irrespective of motor or language abilities and serves as a benchmark for neural signal-driven generative modeling (Zhou et al., 7 Jul 2025).

1. Problem Definition and Framework Overview

LoongX aims to supplant traditional, text- or sketch-driven image editing with a paradigm that operates directly on non-invasive, wireless brain–computer interface (BCI) data. The target scenario involves sequential acquisition of multiple synchronized signals as a user views an image, receives or generates a free-form editing instruction, and responds using natural cognitive and physiological states.

The system leverages a conditional diffusion transformer (DiT) model, fine-tuned to map fused latent representations derived from EEG, fNIRS, PPG, and motion channels (optionally speech/text) to edited image outputs. The principal pipeline integrates two novel modules: the Cross-Scale State Space (CS³) encoder for each modality and a Dynamic Gated Fusion (DGF) block that aggregates paired modality features. A comprehensive L-Mind dataset with 23,928 before/after editing pairs and paired multimodal signals provides the training backbone (Zhou et al., 7 Jul 2025).

2. Multimodal Signal Composition and Preprocessing

LoongX integrates four primary neural and physiological recording modalities:

• EEG: Four-channel scalp data (Pz, Fp2, Fpz, Oz; 250 Hz) band-pass filtered (1–80 Hz), notch filtered (48–52 Hz), and baseline-corrected in a 2 s window around the editing prompt. Trials with excessive artifacts are automatically rejected (Bai et al., 21 Dec 2025).
• fNIRS: Six channels (735 nm, 850 nm) processed to ΔHbO, ΔHbR, ΔHbT using the Modified Beer–Lambert Law, band-pass filtered (0.01–0.5 Hz), and hemispherically averaged.
• PPG: Four channels (0.5–4 Hz), pooled by hemisphere.
• IMU Motion: Six-axis accelerometer and gyroscope (12.5 Hz).

Each trial in the dataset also links to its respective “before/after” image pair and a natural language editing instruction. Ground-truth edited images are precisely pixel-aligned with originals, supporting direct evaluation and mask-based supervision.

3. Cross-Scale State Space (CS³) Encoding

The CS³ module addresses the heterogeneity and temporal dependencies of each signal modality. For a modality input $S \in \mathbb{R}^{C \times L_0}$:

1. Pyramid Downsampling: Adaptive Average Pooling (AAP) produces a pyramidal set $\{P_i\}$, with each scale $s_i$ proportional to increasingly coarse resolutions.
2. Cross-shaped State-Space Modeling: Two Structured State-Space Models (S3Ms) encode both temporal and channel-wise properties. Each S3M solves a diagonal linear system $\dot{e}(t) = \hat{A}e(t) + \hat{B}s(t)$, yielding latent codes $\tilde{Z}^{(1)}$ (temporal) and $\tilde{Z}^{(2)}$ (spatial).
3. Aggregation: Features from both streams and pyramid pooling are concatenated and projected through an Adaptive Nonlinear Projection (ANP) head, resulting in a compact embedding $F \in \mathbb{R}^{C'\times d'}$.

This architecture enables LoongX to capture both global sequence context and localized, channel-specific activations for each physiological signal (Zhou et al., 7 Jul 2025).

4. Dynamic Gated Fusion and Latent Alignment

DGF fuses two modality embeddings, for example EEG (content) and PPG (condition), as follows:

• Gated Statistics Mixing: Combines instance-normalized and layer-normalized statistics per channel via a learned gating network.
• Adaptive Affine Modulation: Uses an MLP on globally pooled “condition” features to produce scale and shift parameters $(\gamma, \beta)$, modulating normalized content features.
• Dynamic Channel Masking: Computes channel importances, retaining the top $k = \lfloor \rho C \rfloor$ channels ($\rho=0.7$), applied via a binary mask $\{P_i\}$0.

A residual connection allows for flexible combination with language or speech features where present. This enables precise alignment of fused neural representations with editing semantics, facilitating downstream conditional generation (Zhou et al., 7 Jul 2025).

5. Conditional Diffusion Transformer and Training Objectives

LoongX employs a DiT backbone for image editing. The DiT takes as input the original image $\{P_i\}$1, applies a forward noising process, and iteratively denoises conditioned on fused neural features:

1. Condition Injection: The fused feature $\{P_i\}$2 from DGF is injected at each step.
2. Diffusion Update Rule:

$\{P_i\}$3

where $\{P_i\}$4 is the predicted velocity field.

1. Training Loss:

$\{P_i\}$5

Encoders are initially pretrained using a symmetric NT-Xent contrastive loss to align neural representations with CLIP-embedded language instructions, encouraging rich semantic associations regardless of modality (Zhou et al., 7 Jul 2025).

6. LoongX Benchmark Dataset and Evaluation Protocol

The L-Mind/LoongX dataset comprises 23,928 total trials (22,728 train / 1,200 test) from 12 participants. Each sample contains synchronized multimodal signals, original and edited images, and a free-form textual edit instruction. Edits span four categories: background, object swaps, style/tone changes, and text overlays.

Preprocessing for each modality is strictly defined. Image regions subject to edits are annotated via pixel-aligned masks. Text instructions are tokenized and embedded using a frozen CLIP-T5 encoder. Experiments examine both neural-only, text-only, and combined control conditions (Zhou et al., 7 Jul 2025, Bai et al., 21 Dec 2025).

LoongX Editing Benchmarks

Method	L1↓	L2↓	CLIP-I↑	DINO↑	CLIP-T↑
OminiControl (Text)	0.2632	0.1161	0.6558	0.4636	0.2549
LoongX (Neural signals only)	0.2509	0.1029	0.6605	0.4812	0.2436
LoongX (Signals + Speech)	0.2594	0.1080	0.6374	0.4205	0.2588

Multimodal neural signals alone match or surpass text-only methods on semantic (CLIP-I) and structure (DINO) metrics; fusing neural signals with speech drives further gains on text-consistency (CLIP-T) (Zhou et al., 7 Jul 2025).

7. Impact, Insights, and Future Directions

Experimental ablation demonstrates that the CS³ module is critical for extracting actionable feature representations from long, multichannel physiological time series. DGF is essential for robust pairwise modality integration and supports seamless language–neural fusion. The DiT backbone, when conditioned on neural and/or hybrid embeddings, preserves editing intent and achieves high-fidelity outputs comparable to strong text-driven generative models.

Key insights:

• Parameter efficiency and extensibility: The LoRA-based variant of Uni-Neur2Img achieves better L1/L2 and DINO metrics by leveraging parameter-efficient adapters and causal attention tailored to the temporal nature of EEG, confirming the value of latent neural feature injection and temporal structure (Bai et al., 21 Dec 2025).
• Generalization and adaptation: LoongX can potentially be extended to non-EEG modalities, subject-specific adaptation, and online BCI feedback loops. Synchronized multimodal datasets and contrastive pretraining appear critical for stable cross-modal alignment.
• Assistive and creative potential: By decoupling generative control from manual or linguistic input, LoongX opens new directions for universal creative interfaces and accessibility tools.

A plausible implication is the emergence of unified frameworks for brain–computer-mediated content generation, supporting closed-loop, multimodal, and context-aware editing tools with broad application prospects in human–AI interaction, assistive systems, and creative domains (Zhou et al., 7 Jul 2025, Bai et al., 21 Dec 2025).