Clinically Controllable Generative Framework

• The framework is designed to incorporate explicit clinical factors—such as diagnostic classes, anatomical masks, and pathology stages—into generative models, ensuring semantic control and meaningful synthesis.
• It employs advanced architectures like conditional VQ-VAE, multimodal diffusion, and disentangled representations to maintain fidelity, balance, and clinical interpretability across diverse medical modalities.
• Empirical studies demonstrate improved downstream performance in data augmentation, restoration, and privacy-preserving synthesis, while incorporating safeguards against semantic drift and noise.

Clinically controllable generative framework denotes a class of generative systems in which the controllable variables are explicitly tied to clinically meaningful factors—diagnostic class, anatomy, morphology, pathology stage, imaging style, privacy attributes, or target molecular properties—rather than left implicit in an unconstrained latent space. Recent work instantiates this idea in respiratory sound augmentation through class-conditioned discrete tokens and prototypes, pain-face synthesis through action units and PSPI targeting, corneal confocal microscopy through joint mask-and-prompt conditioning, breast ultrasound through BI-RADS-aligned text plus structural masks, visually guided structure/style disentanglement for medical images, training-free restoration through dual-latent steering, privacy-preserving clinical text generation through entity-aware control codes, and controllable molecular generation through motif-aware or span-aware editing (Ma et al., 1 Jun 2026, Lin et al., 20 Sep 2025, Zhang et al., 14 Feb 2026, Pan et al., 10 Jul 2025, Huang et al., 11 Mar 2026, Liu, 26 Mar 2026, Zhao et al., 30 Sep 2025, Zhu et al., 14 May 2026, Liu et al., 15 Feb 2025).

1. Clinical motivation and problem setting

Across modalities, the rationale for clinical controllability is consistent: clinically relevant datasets are often small, imbalanced, noisy, privacy-sensitive, or ethically difficult to expand, while naive augmentation can damage the very cues that diagnosis depends on. In respiratory sound analysis, real-world auscultation datasets are described as small, contaminated by environmental and device noise, and markedly imbalanced across classes, so masking or time/pitch scaling can occlude or distort crackle spikes and other delicate spectral transients (Ma et al., 1 Jun 2026). In automated pain assessment, the motivating benchmark includes only 25 participants and is heavily imbalanced, with 88.7% of frames having PSPI $\le 1$, while severe pain cannot be induced and privacy constraints limit collection and sharing (Lin et al., 20 Sep 2025). In diabetic neuropathy screening with corneal confocal microscopy, the cited cohort contains 318 individuals, and the target anatomy consists of ultra-thin nerve fibers with tortuosity, branching, beadings, and modality-specific noise, making anatomical fidelity a first-order requirement rather than a cosmetic one (Zhang et al., 14 Feb 2026).

The same pressure appears in breast ultrasound, pathology, and medical sequence analysis. The BUS framework is motivated by six public datasets totaling 1600 images, with labels requiring experienced sonographers or radiologists and with clinically important features distributed across morphology, margin characteristics, and echogenicity (Pan et al., 10 Jul 2025). Pathology synthesis is framed around scarce, imbalanced, and privacy-sensitive datasets, especially for rare phenotypes, and explicitly targets semantic instability and morphological hallucinations in histology generation (Guan et al., 15 Dec 2025). For sequence classification, ultrasound videos and cine-MRI volumes are clinically challenging because both semantic control and temporal or stereoscopic coherence are required; underrepresented high-risk populations and out-domain conditions further amplify the problem (Zhou et al., 2024).

A clinically controllable framework therefore differs from generic conditional generation in two ways. First, the condition space is meant to correspond to variables that clinicians already use—class labels, BI-RADS descriptors, AUs, PSPI, cancer types, segmentation masks, disease-stage prompts, or privacy entity types. Second, the framework typically includes safeguards against semantic drift, hallucinated structure, mislabeled synthesis, or privacy leakage, rather than treating conditional consistency as a by-product.

2. Control variables and disentangled representations

The core design choice is the form of control. In C2GA, “clinical controllability” is operationalized by separating local acoustic events from global class cues. A conditional VQ-VAE constructs a discrete latent sequence $K=\{k_t\}_{t=1}^{T'}$ and separately maintains class prototypes $\{\mu_y^{(\ell)}\}$, later fused through

$\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$

with $$p_y=\mathrm{Concat}(\mu_y^{(1)},\dots,\mu_y^{(L_s)})$$. This makes label consistency depend simultaneously on token-level local semantics and prototype-level global class guidance (Ma et al., 1 Jun 2026).

In facial pain synthesis, the control variables are explicitly clinical. The supervisory and generative target is the Prkachin and Solomon Pain Intensity index,

$$\text{PSPI} = \text{AU}_4 + \max(\text{AU}_6, \text{AU}_7) + \max(\text{AU}_9, \text{AU}_{10}) + \text{AU}_{43},$$

and the framework exposes direct AU intensity control through Neural Face Rigging, structural control through FLAME identity, age, ethnicity, and gender parameters, and pain-region heatmaps derived from 3D vertex displacements (Lin et al., 20 Sep 2025).

Other frameworks use clinically grounded control pairs rather than a single condition. The CCM model conditions jointly on nerve segmentation masks and disease-specific prompts for Control, T1NoDPN, and T1DPN, interpreting masks as topology constraints and prompts as semantic constraints over disease-stage morphology and photometric appearance (Zhang et al., 14 Feb 2026). The BUS framework splits control between a tumor mask, which governs morphology and spatial configuration, and BI-RADS-aligned text, which governs echogenicity and boundary clarity; shape is excluded from text specifically to avoid conflicting guidance (Pan et al., 10 Jul 2025). The visually guided DiT framework likewise separates “Structure” from “Style,” using independent text projections aligned to visual priors, while DRAI separates clinically meaningful content $c$ from non-clinical style $z$ through dual adversarial inference and explicit disentanglement regularization (Huang et al., 11 Mar 2026, Havaei et al., 2020).

Beyond imaging, private synthetic clinical text uses entity-aware control codes over PERSON, CODE, LOC, ORG, DEM, DATETIME, QUANTITY, and MISC, with optional privacy levels that specify replacement, removal, or masking behavior (Zhao et al., 30 Sep 2025). In molecular generation, CoMole uses task embeddings and target values in a motif-aware diffusion policy, whereas ControllableGPT exposes edit locations and edit length directly through tokens such as $\texttt{<mask\_i:n>}$ and $\texttt{<s2s\_i\_L\^t:\dots>}$, allowing expansion, reduction, or mutation while preserving designated anchors (Zhu et al., 14 May 2026, Liu et al., 15 Feb 2025).

Framework	Domain	Explicit control variables
C2GA (Ma et al., 1 Jun 2026)	Respiratory sounds	Class label, discrete tokens, class prototypes
3DPain (Lin et al., 20 Sep 2025)	Pain faces	AU intensities, PSPI, FLAME identity and demographics
WDLoRA CCM (Zhang et al., 14 Feb 2026)	Corneal confocal microscopy	Nerve masks, disease-specific prompts
BUS text+mask (Pan et al., 10 Jul 2025)	Breast ultrasound	Tumor masks, BI-RADS-style text
VG-MedGen / DRAI (Huang et al., 11 Mar 2026, Havaei et al., 2020)	Medical images	Structure/style or content/style factors
CARE (Liu, 26 Mar 2026)	Restoration	$K=\{k_t\}_{t=1}^{T'}$0 operating mode, uncertainty, structural reliability
Private synthetic text (Zhao et al., 30 Sep 2025)	Clinical text	Entity-aware control codes, privacy levels
CoMole / ControllableGPT (Zhu et al., 14 May 2026, Liu et al., 15 Feb 2025)	Molecular design	Task embeddings, motif actions, span-aware edits

3. Architectural patterns

Although the application domains differ, several architectural patterns recur. One pattern is staged discrete-latent generation. C2GA first learns a semantically rich discrete latent space with a conditional VQ-VAE, then trains a decoder-only Transformer prior over label-conditioned token sequences: $K=\{k_t\}_{t=1}^{T'}$1 At synthesis time, sampled token sequences are fused with class prototypes and decoded into high-fidelity log-Mel spectrograms (Ma et al., 1 Jun 2026).

A second pattern is multimodal diffusion with separated condition streams. The CCM framework adapts Qwen-Image-Edit with a Multimodal Diffusion Transformer and WDLoRA, inserting magnitude-direction decoupled adapters into query, key, value, output, and MLP projections so that directional updates emphasize nerve geometry and magnitude updates emphasize photometric contrast (Zhang et al., 14 Feb 2026). The BUS model uses a stable-diffusion-style latent diffusion backbone with ControlNet, but replaces purely textual conditioning with residual structural features from the mask and semantic features from a CLIP text encoder (Pan et al., 10 Jul 2025). The visually guided DiT framework goes further by explicitly disentangling text into structural and style subspaces and injecting them through a Hybrid Feature Fusion Module: $K=\{k_t\}_{t=1}^{T'}$2 so that structure and style occupy separated channels in cross-attention (Huang et al., 11 Mar 2026).

A third pattern is geometry-first synthesis. 3DPain proceeds through diverse 3D mesh generation with FLAME, depth-conditioned face generation with Kandinsky 2.2 + ControlNet, PBR texturing with Hunyuan3D 2.1, AU-driven face rigging through Neural Face Rigging, and final inpainting for hair and background completion (Lin et al., 20 Sep 2025). This architecture enforces identity preservation and viewpoint consistency at the 3D level rather than attempting to recover them from 2D prompt semantics alone.

A fourth pattern is explicit separation of fidelity and prior. CARE is training-free and forms a fidelity latent $K=\{k_t\}_{t=1}^{T'}$3 and a prior latent $K=\{k_t\}_{t=1}^{T'}$4, then fuses them by

$K=\{k_t\}_{t=1}^{T'}$5

with the restored image obtained through a frozen decoder. The same $K=\{k_t\}_{t=1}^{T'}$6 also gates prior guidance in the image-domain update, allowing the system to move continuously between conservative and enhancement-focused restoration without retraining (Liu, 26 Mar 2026).

Finally, some frameworks implement controllability through inference rather than through a diffusion backbone. DRAI uses a generator $K=\{k_t\}_{t=1}^{T'}$7, separate style and content encoders, joint discriminators for $K=\{k_t\}_{t=1}^{T'}$8 and $K=\{k_t\}_{t=1}^{T'}$9, an image-cycle discriminator, code-cycle consistency, gradient-reversal mutual-information regularization, and self-supervised invariance/variance constraints to make content clinically interpretable and style non-clinical (Havaei et al., 2020). ControllableGPT uses a GPT-style sequence model with a Causally Masked Seq2seq objective so that control over location and span length is encoded directly in the prompt tokens rather than appended by a separate controller (Liu et al., 15 Feb 2025).

4. Guidance, regularization, and reliability mechanisms

Clinical controllability is rarely implemented as conditioning alone; it is usually reinforced by additional training objectives, inference-time guidance, or post-generation filtering. In pathology synthesis, CRAFTS introduces a semantic consistency loss

$\{\mu_y^{(\ell)}\}$0

to align the relational geometry of image latents with that of text features, and a weighted category guidance loss

$\{\mu_y^{(\ell)}\}$1

to inject cancer-type priors only when prompts are sufficiently category-typical (Guan et al., 15 Dec 2025). The visually guided DiT framework uses cosine alignment between text and visual priors plus a reconstruction term for the base ClinicalBERT embedding, and adds an online color distribution regularizer during diffusion fine-tuning (Huang et al., 11 Mar 2026).

Several frameworks use explicit safeguards against noisy or incoherent synthesis. Ctrl-GenAug filters synthetic sequences in three stages: semantic filtering with a real-data-trained classifier, inner-sequence coherence filtering through

$\{\mu_y^{(\ell)}\}$2

and inter-sequence diversity filtering by groupwise cosine similarity in VAE latent space (Zhou et al., 2024). CARE computes uncertainty as ensemble variance across stochastic trajectories and structural reliability through patchwise SSIM in measurement space, then sets

$\{\mu_y^{(\ell)}\}$3

so that prior influence decreases when uncertainty rises or measurement consistency falls (Liu, 26 Mar 2026).

Other frameworks regularize the control channel itself. DRAI introduces a gradient reverse layer to make style predictable from content and content predictable from style only for the auxiliary predictors, thereby pushing the main encoders toward lower shared information, and combines that with self-supervised content invariance under rotations and flips (Havaei et al., 2020). Private synthetic text uses a three-term objective consisting of a language-modeling loss, a contrastive divergence term on PHI tokens, and a KL term on non-PHI tokens, while the ICL variant blocks known PHI tokens through $\{\mu_y^{(\ell)}\}$4 and can apply rejection sampling if residual leakage is detected (Zhao et al., 30 Sep 2025). C2GA reinforces class consistency by injecting label embeddings into the encoder and decoder, using a learnable class token for the autoregressive prior, and broadcasting class prototypes into decoder skip levels; its ablations show that removing the Transformer prior, prototype fusion, or Stage-1 class conditioning degrades F1 (Ma et al., 1 Jun 2026).

These mechanisms are important because “clinically controllable” does not simply mean “conditioned.” In the cited literature, controllability is typically paired with a reliability layer: semantic consistency losses, prototype stabilization, geometry constraints, coherence filters, privacy audits, or uncertainty-aware steering.

5. Empirical performance and downstream utility

The reported empirical gains span augmentation, restoration, privacy-preserving synthesis, and inverse design. In respiratory sound classification, C2GA achieves the best mean performance across both reported datasets: on the binary dataset, Acc 78.20%, Recall 76.40%, and F1 77.15%; on the noisy 3-class dataset, Acc 49.85%, Recall 49.20%, and F1 49.50%. It surpasses AudioLDM2 by +1.35 pp F1 on Dataset 1 and +2.20 pp on Dataset 2, while ablations show F1 drops from 49.50% to 44.20% without the Transformer prior, to 46.40% without prototype fusion, and to 47.15% without Stage-1 class conditioning (Ma et al., 1 Jun 2026).

In clinically guided face synthesis, 3DPain produces 82,500 frames, 25,000 pain expression heatmaps, and 2,500 synthetic identities balanced across age, gender, and ethnicity categories as specified in the paper. ViTPain then improves real-world pain recognition on UNBC-McMaster: the baseline AUC is 0.83, augmentation with 3DPain yields 0.90, and adding heatmap supervision yields AUC 0.91 with F1 = 0.54 at PSPI $\{\mu_y^{(\ell)}\}$5 (Lin et al., 20 Sep 2025).

For CCM synthesis, the WDLoRA framework reports FID $\{\mu_y^{(\ell)}\}$6, PSNR $\{\mu_y^{(\ell)}\}$7, and SSIM $\{\mu_y^{(\ell)}\}$8, outperforming SPADE, MAISI, and Qwen+LoRA on the reported averages across classes. In downstream evaluation on real test data, hybrid training with real plus WDLoRA synthetic images improves diagnostic accuracy from $\{\mu_y^{(\ell)}\}$9 to $\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$0 and segmentation mIoU from $\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$1 to $\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$2 (Zhang et al., 14 Feb 2026). In BUS generation, quantitative quality on BUSI is FID/KID $\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$3 for the full text+mask model, and augmentation improves downstream performance; for example, BUSI DenseNet121 AUC rises from $\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$4 to $\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$5 with 25% synthetic data, while external STU segmentation performance improves markedly in several settings (Pan et al., 10 Jul 2025).

The broader medical image literature reports analogous gains. The visually guided DiT framework reaches FID 51.56, HFD 3.22, and KID 0.036 on HAM10000, and synthetic augmentation yields F1 0.619, BACC 0.348, and AUC 0.830 in downstream classification (Huang et al., 11 Mar 2026). CRAFTS reports PLIP-FID 11.32, PLIP-I 85.74%, PLIP-T 29.24%, and cluster separability 34.37%, and pathologists give it the highest semantic alignment score, 3.27. Augmented training improves classification across BACH, BRACS, BreakHis, and Lunghist, and also improves cross-modal retrieval, self-supervised transfer, and VQA metrics (Guan et al., 15 Dec 2025). CARE reports PSNR 33.6248 dB and SSIM 0.9004 on low-dose CT, 4× MRI SSIM 0.964, and 8× MRI SSIM 0.955; its balanced mode achieves PSNR 33.62, Structure score 0.938, and Hallucination risk 0.11, while conservative mode lowers hallucination risk to 0.08 at the cost of some fidelity (Liu, 26 Mar 2026).

Sequence augmentation shows similar patterns. Ctrl-GenAug improves carotid, thyroid ultrasound, and ACDC classification across 11 backbones and 3 training paradigms; for carotid ID, I3D improves from 79.59/0.737 Acc/AUROC to 85.03/0.813 under Real-finetune, and underrepresented carotid cohorts show F1 improvements from 25.92% to 52.73% for moderate stenosis and from 43.44% to 60.40% for severe stenosis (Zhou et al., 2024).

Controllability also extends beyond image augmentation. In private synthetic text, the ICL variant with privacy enhancement achieves PIPP 1.2%, ELP 0.5%, and ROUGE-L 0.4263 on MIMIC-III, while prefix tuning with masking reaches PIPP 4.8%, ELP 1.2%, and ROUGE-L 0.0874, reflecting a different privacy–utility trade-off (Zhao et al., 30 Sep 2025). In molecular design, CoMole ranks first in controllability on all nine targets across three benchmarks, reduces MAE by up to 48.2% relative to the strongest baselines, and maintains validity above 0.94 without rule-based correction or post-hoc filtering (Zhu et al., 14 May 2026). ControllableGPT reports Avg Norm Reward 0.671 on 3CLPro and 0.678 on RTCB for mask+seq2seq control, with high similarity and improved docking/solubility relative to competing baselines (Liu et al., 15 Feb 2025).

6. Limitations, misconceptions, and future directions

A recurrent misconception is that controllability guarantees clinical validity. The cited papers do not support that interpretation. C2GA generates spectrograms only and does not specify waveform reconstruction; it also depends on label quality, prototype quality, and careful calibration of synthetic ratio $\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$6 and loss weight $\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$7 (Ma et al., 1 Jun 2026). 3DPain reports strong downstream transfer, but the paper does not report formal realism metrics such as FID or clinician validation studies specifically for realism; it also notes that texture and hair inpainting may introduce artifacts and that PSPI captures only a subset of pain-related facial cues (Lin et al., 20 Sep 2025). CARE explicitly frames restoration as a trade-off between faithful reconstruction and prior-driven enhancement, and its own ablations show that enhancement mode increases hallucination risk relative to conservative mode (Liu, 26 Mar 2026).

Another misconception is that clinical control is equivalent to formal privacy or fairness. The synthetic-text framework explicitly contrasts its approach with differential privacy and states that it does not provide an $\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$8-$\tilde{z}_t = W_f\,( e_{k_t} \oplus p_y ),$9 guarantee (Zhao et al., 30 Sep 2025). The pain-face work reduces reliance on patient data and addresses demographic imbalance, yet the released identity distribution still includes only 82 Black synthetic identities relative to larger counts in other groups (Lin et al., 20 Sep 2025). Pathology and BUS augmentation are motivated partly by privacy, but their primary validation is utility and realism rather than formal privacy analysis (Guan et al., 15 Dec 2025, Pan et al., 10 Jul 2025).

The main technical limitations are also domain-specific. WDLoRA is trained on data from a single device and identifies multi-center domain-adaptive fine-tuning as future work; rare morphologies and noisy masks remain failure modes (Zhang et al., 14 Feb 2026). BUS synthesis does not model ultrasound physics explicitly and can saturate or slightly degrade when the synthetic ratio becomes too large (Pan et al., 10 Jul 2025). VG-MedGen depends on strong visual priors and notes that residual entanglement may persist because cosine alignment alone may not fully decorrelate structure and style (Huang et al., 11 Mar 2026). Ctrl-GenAug still incurs substantial diffusion sampling cost and benefits from access to target-domain unlabeled data during generator training for out-domain robustness (Zhou et al., 2024). In molecular control, CoMole depends on learned oracles and explicitly notes distribution-shift and oracle-bias risks, while ControllableGPT acknowledges imperfect docking surrogates and trade-offs involving synthesizability (Zhu et al., 14 May 2026, Liu et al., 15 Feb 2025).

The future directions in the cited literature are convergent. They include richer conditioning variables such as waveform decoders and clinical metadata for respiratory audio, finer-grained clinical metadata for faces, multi-center adaptation for CCM, physics-informed priors for ultrasound, CT/MRI/X-ray extensions for disentangled text-guided generation, uncertainty-aware and audit-ready restoration, richer multimodal pathology control, and additional ADMET or toxicity endpoints for molecular design (Ma et al., 1 Jun 2026, Lin et al., 20 Sep 2025, Zhang et al., 14 Feb 2026, Pan et al., 10 Jul 2025, Huang et al., 11 Mar 2026, Liu, 26 Mar 2026, Guan et al., 15 Dec 2025, Zhu et al., 14 May 2026). This suggests that the mature form of a clinically controllable generative framework is not a single architecture, but a design doctrine: expose clinically interpretable control variables, disentangle them from nuisance variation, constrain generation with modality-specific priors, and validate not only fidelity but also semantic integrity, risk, and downstream clinical utility.