Prompt-Conditioned FiLM

• The paper introduces prompt-conditioned FiLM, a method that leverages learned MLPs to map external prompts into per-channel scaling and bias parameters.
• It employs dual prompt injection pathways—soft prompt concatenation and FiLM modulation—to improve signal compression in IRS systems and quality assessment in low-dose CT imaging.
• Empirical results show significant NMSE improvements and efficient decoding with reduced parameters, demonstrating robust generalization across diverse operating conditions.

Prompt-conditioned Feature-wise Linear Modulation (FiLM) is a neural conditioning technique in which external prompts dynamically gate and shift network activations at various depths using learned, channel-wise affine transformations. This scheme enables deep models to adaptively modulate intermediate representations based on input prompts that encode side-information or task directives. Recent work in prompt-conditioned FiLM integrates both soft prompt embeddings (vector or token concatenation) and prompt-conditioned scaling/shifting, with demonstrated advantages in flexible signal compression (as in IRS phase control) and text-guided medical image assessment (Yu et al., 5 Nov 2025, Demiroglu et al., 15 Nov 2025). The paradigm combines prompt-driven semantic biasing via concatenation with channel-local statistical adaptation via FiLM layers, often implemented with lightweight, learnable MLPs that map prompts to per-channel scaling and bias signals.

1. Formal Definition and Core Mechanisms

Prompt-conditioned FiLM establishes a conditioning interface in which network activations are modulated as a function of external prompt information. For a feature tensor $X \in \mathbb{R}^{C \times H \times W}$ and a prompt embedding $z_p$, the modulation is defined by

$$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$

where $\gamma_c$ and $\beta_c$ are per-channel scale and shift parameters generated by mapping $z_p$ through a neural MLP, optionally gated for adaptive strength.

Two main prompt-injection pathways are typically deployed:

• Soft Prompt Concatenation: Learnable prompt embeddings $S \in \mathbb{R}^{P \times d}$ are concatenated with the tokenized feature map $X_{in} \in \mathbb{R}^{N \times d}$ to yield $[\;S;\;X_{in}\;] \in \mathbb{R}^{(N+P)\times d}$. This enforces global semantic priors before self-attention or transformation stages.
• FiLM Modulation: The prompt (or side-information, e.g., SNR, channel type, compression rate) is mapped to FiLM parameters $(\gamma,\beta)$ via a small MLP. In IRS phase shift compression, a gating scalar $z_p$0 further controls signal injection:

$z_p$1

In MedSigLIP-based IQA, the modulation adopts

$z_p$2

with $z_p$3 a learned "FiLM strength" and $z_p$4 the patch embedding matrix (Demiroglu et al., 15 Nov 2025).

2. Architectural Integration in IRS Compression and MedSigLIP

Adaptive PSI Compression for IRS Systems:

The prompt-conditioned FiLM mechanism appears in a hybrid encoder for compressing phase shift information (PSI) in intelligent reflecting surfaces (Yu et al., 5 Nov 2025). The encoder receives as input a rasterized PSI map and side-information vector $z_p$5 (holding SNR, channel type, and compression ratio details). The architectural workflow is:

• Features $z_p$6 are linearly embedded from the input PSI map.
• The side-information $z_p$7 is processed along two parallel tracks:
  1. FiLM branch: Local, per-layer activation scaling/shifting via prompt-conditioned FiLM layers using $z_p$8.
  2. Soft prompt branch: $z_p$9 is mapped to prompt tokens concatenated before transformer blocks.

• The conditioned features traverse multiple Vision Transformer (ViT) blocks to produce an ordered latent vector $$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$0 suitable for variable-rate compression (by binary masking).

• The subsequent DWCG decoder reconstructs phase information without prompt input, leveraging depthwise convolutions and gated activations for efficiency.

Prompt-Conditioned MedSigLIP for CT Quality Assessment:

(Demiroglu et al., 15 Nov 2025) applies prompt-conditioned FiLM in the MedSigLIP framework for low-dose CT image quality assessment. The workflow is:

• The MedSigLIP vision backbone encodes image patches into $$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$1.
• Clinical text prompts are encoded via the frozen MedSigLIP text tower, yielding $$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$2.
• A two-layer MLP $$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$3 produces FiLM parameters $$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$4.
• FiLM modulates image tokens channel-wise as in the formalism above.
• Multi-scale pooling (global, local, texture) produces feature summaries for regression heads.
• Outputs are fused in a lightweight MLP and trained with a pairwise ranking loss for quality.

3. Mathematical Formulations and Training

Across applications, the FiLM parameters are derived using prompt-to-parameter MLPs, conditioned either on structured side-info (e.g., SNR, rate) or text embeddings. For IRS compression (Yu et al., 5 Nov 2025):

• Let $$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$5, mapped to $$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$6.
• $$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$7 are produced by learned projections.
• A variable compression rate $$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$8 is injected via prompt and latent masking, with normalization ensuring variance consistency.

The normalized MSE loss guides joint encoder-decoder training:

$$\hat{X}_{c,i,j} = \gamma_c(z_p) X_{c,i,j} + \beta_c(z_p)$$9

For MedSigLIP IQA (Demiroglu et al., 15 Nov 2025), pairwise ranking loss (RankNet-style) focuses model capacity on preserving clinical quality orderings.

4. Empirical Outcomes and Performance Characteristics

IRS Compression Performance:

Prompt-conditioned models maintain NMSE below –25 dB across 0–25 dB SNR in varying fading environments, outperforming non-prompted baselines (which degrade by up to 5 dB NMSE at high SNR). A single prompt-conditioned model closely tracks or exceeds the best per-rate baseline over a wide range of compression ratios, with NMSE within 0.5 dB. Generalization to larger IRS sizes is robust, as shown by nearly constant NMSE for $\gamma_c$0 without retraining.

Ablation results demonstrate that encoder-side FiLM is critical (2–3 dB NMSE gain over soft prompts alone), decoder prompt conditioning offers negligible benefit, and a DWCG decoder achieves equivalent or better NMSE with 85% fewer parameters and 90% lower FLOPs compared to attention-based decoders (Yu et al., 5 Nov 2025).

MedSigLIP for LDCT IQA:

Prompt-conditioned FiLM achieves PLCC = 0.9575, SROCC = 0.9561, KROCC = 0.8301 on the public LDCTIQA2023 dataset (1,000 training, 300 test), surpassing previous challenge benchmarks. The method exhibits strong, data-efficient adaptation with only 1,000 training images.

5. Practical Applications and Adaptation Strategies

Prompt-conditioned FiLM is deployed to address:

• Dynamic adaptation to environmental or task parameters (e.g., SNR, channel characteristics, compression ratio in wireless systems).
• Injection of clinical or semantic context for vision tasks (e.g., image quality assessment conditioned on textual directives).
• Joint support for multiple operating regimes in a single unified model, realized via prompt-hierarchical latent ordering and masking for variable-rate scenarios (Yu et al., 5 Nov 2025).
• Data-efficient learning and generalization, exploiting prompt-driven modulation to align representations with prompt-specified context in settings with limited ground-truth data (Demiroglu et al., 15 Nov 2025).

6. Comparative Analysis and Ablation Insights

Comparative ablations confirm that:

• Dual-path prompt conditioning (soft concatenation plus FiLM gating) provides superior robustness and rate-distortion performance compared to either method alone.
• Encoder-local FiLM modulation is more beneficial than prompt-conditioning at decoder or only soft prompts.
• Lightweight gated convolutional decoders retain performance with substantial parameter/FLOP reduction.
• Prompt variation elicits interpretable channel responses: as SNR increases, $\gamma_c$1 transitions from attenuating ($\gamma_c$2) to amplifying ($\gamma_c$3) critical channels, with $\gamma_c$4 recentering activations, demonstrating fine-grained adaptation (Yu et al., 5 Nov 2025).

7. Broader Implications and Future Outlook

Prompt-conditioned FiLM provides a generic and computationally efficient scheme for context-adaptive neural processing in both control and perception settings. The hybrid prompt interface enables seamless integration of structured numeric or semantic inputs, scalable parameter sharing across tasks/rates, and data-efficient learning. A plausible implication is that similar designs can generalize to other domains where rapid task/context adaptation with minimal overhead is critical. Continued research may focus on theoretical convergence properties, limits of prompt expressivity, and the extension to multi-modal, continual, or federated adaptation scenarios, building directly on the core findings of (Yu et al., 5 Nov 2025) and (Demiroglu et al., 15 Nov 2025).