Unified Channel-wise Conditioning

• Unified channel-wise conditioning is a structural principle that applies fixed conditioning along a channel axis to preserve an anchor variable across different domains.
• It unifies diverse mechanisms—from per-symbol OT in communication to full channel mixing in vision—allowing consistent application of conditioning in varied model architectures.
• This approach balances computational efficiency with robust performance, enabling faster sampling and improved metrics in tasks such as crowd counting, wireless simulation, and image compression.

Unified channel-wise conditioning denotes a family of mechanisms in which conditioning information is applied consistently along a channel axis or per fixed condition, so that a model preserves an anchor variable while modulating channel-dependent statistics, features, or outputs. In recent literature, the phrase has been used for per-symbol transport of conditional channel laws in learned communication simulators, joint spatial/channel feature conditioning in crowd counting, side-information-conditioned mean and covariance modeling of wireless channels, reference-conditioned processing of microphone streams, channel-autoregressive entropy modeling of compression latents, and channel-wise fusion in multi-reference image generation, biological diffusion, and multimodal video understanding (Fritschek et al., 16 Jun 2026, Gao et al., 2019, Böck et al., 2024, Aldarmaki et al., 2024, Minnen et al., 2020, Xu et al., 12 May 2026, Zhang et al., 24 Jun 2025, Zhang et al., 17 May 2025).

1. Core concept and terminological scope

Across these works, the term channel is not uniform. In wireless communications it may denote a stochastic physical channel or the time/frequency/space axes of a channel tensor; in vision and multimodal learning it usually denotes feature channels or aligned modality channels; in speech enhancement it denotes microphone streams; in learned compression it denotes latent slices. What is unified is the insistence that conditioning be imposed by a single mechanism across all channel instances, rather than by ad hoc per-case rules.

Domain	Channel object	Unified conditioning form
Learned channel simulation	Conditional output law $p(y\mid x)$ at fixed transmitted symbol $x$	Per-$x$ entropic OT and one-shot generator regression (Fritschek et al., 16 Jun 2026)
Crowd counting and image generation	Feature or token channels	Parallel attention or early channel concatenation before a shared head (Gao et al., 2019, Xu et al., 12 May 2026)
Wireless channel statistics	Time, frequency, and space axes	Side-information-conditioned $\mu_h(s)$ and $\Sigma_h(s)$ with Toeplitz/BTTB structure (Böck et al., 2024)
Speech enhancement and compression	Microphone streams or latent slices	Reference-conditioned stacking or channel-autoregressive factorization (Aldarmaki et al., 2024, Minnen et al., 2020)
Biological and multimodal data	Measured marker channels or modality features	Hierarchical injection, channel attention, and channel/modality fusion (Zhang et al., 24 Jun 2025, Zhang et al., 17 May 2025)

This suggests that unified channel-wise conditioning is best understood as a structural principle rather than a single algorithm. The principle is to keep the conditioning semantics fixed while allowing the modeled variable to vary along a designated channel dimension.

2. Condition-preserving transport in learned communication systems

In "Condition-Wise Sinkhorn Drifting for One-Shot Learned Channel Simulation" (Fritschek et al., 16 Jun 2026), unified channel-wise conditioning is defined at the level of conditional channel laws. The simulator preserves the transmitted symbol $x$ and transports only the family $p(y\mid x)$, with

$$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$

The generator is one-shot,

$$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$

and in the reported implementation $G_\theta$ is a two-hidden-layer MLP with latent dimension $x$0, hidden width $x$1, SiLU activations, and concatenated conditioning $x$2 (Fritschek et al., 16 Jun 2026).

The central claim is that a global coupling over $x$3 can match the joint cloud while misaligning the conditional fibers at fixed $x$4. Condition-wise Sinkhorn drifting instead builds independent entropic OT couplings for each anchor $x$5. For each condition, costs are

$x$6

with Sinkhorn scaling

$x$7

The aggregated objective sums per-condition entropic OT terms, and the paper states that the debiased conditional Sinkhorn divergence is nonnegative, with equality iff $x$8 for $x$9-a.e. $x$0 (Fritschek et al., 16 Jun 2026).

Training uses barycentric velocities computed from the coupling and then detached particle regression. Instead of differentiating through the OT solver, the method forms stop-gradient targets

$x$1

then minimizes

$x$2

The same work adds self-transport subtraction,

$x$3

as a collapse-reduction term (Fritschek et al., 16 Jun 2026).

The empirical role of this conditioning is sharply operational. On AWGN, Rayleigh fading, SSPA nonlinearity, and compact TDL channels, condition-wise Sinkhorn is reported as strongest among the evaluated one-shot drifting-family variants under conditional diagnostics and symbolic-coding checks, while diffusion remains strongest on the hardest downstream SER curves. Representative values include global SWD $x$4 versus $x$5 on AWGN for condition-wise versus direct drifting, Rayleigh SER $x$6 versus $x$7 for condition-wise versus joint Sinkhorn, and TDL SER $x$8 versus $x$9 for condition-wise versus joint Sinkhorn (Fritschek et al., 16 Jun 2026).

The computational argument is equally central. One-shot drifting requires a single forward pass, reported at approximately $\mu_h(s)$0 per sample on an RTX 5060 Ti in fp32, whereas DDPM-100 is reported at approximately $\mu_h(s)$1 and DDIM-100 at approximately $\mu_h(s)$2. Training-time complexity is $\mu_h(s)$3 for condition-wise Sinkhorn, versus $\mu_h(s)$4 for joint Sinkhorn over the expanded batch (Fritschek et al., 16 Jun 2026). The result is a condition-preserving one-shot simulator designed for settings where channel calls occur millions of times inside differentiable training loops.

3. Feature-space conditioning in vision and multi-reference generation

In "SCAR: Spatial-/Channel-wise Attention Regression Networks for Crowd Counting" (Gao et al., 2019), unified channel-wise conditioning is implemented as a pair of non-local streams built on the same feature tensor $\mu_h(s)$5. The backbone is the first 10 convolution layers of VGG-16 followed by a dilation module with six $\mu_h(s)$6 dilated convolutions of dilation $\mu_h(s)$7 and channel schedule $\mu_h(s)$8-$\mu_h(s)$9-$\Sigma_h(s)$0-$\Sigma_h(s)$1-$\Sigma_h(s)$2-$\Sigma_h(s)$3, yielding $\Sigma_h(s)$4 at $\Sigma_h(s)$5 resolution. Spatial-wise Attention Model (SAM) forms an $\Sigma_h(s)$6 attention matrix and outputs

$\Sigma_h(s)$7

whereas Channel-wise Attention Model (CAM) forms a full $\Sigma_h(s)$8 affinity matrix and outputs

$\Sigma_h(s)$9

The fused representation is

$x$0

with $x$1 a $x$2 regressor (Gao et al., 2019).

SCAR explicitly contrasts CAM with pooling-based channel attention such as SENet or CBAM. Instead of diagonal multiplicative gating from pooled scalars, CAM performs full $x$3 channel mixing conditioned on spatially detailed channel vectors. The paper reports that this suppresses background false positives and strengthens head-region cues. On ShanghaiTech Part B, the backbone-only FCN achieves MAE $x$4, MSE $x$5, PSNR $x$6, and SSIM $x$7; FCN+CAM improves to MAE $x$8, MSE $x$9, PSNR $p(y\mid x)$0, and SSIM $p(y\mid x)$1; SCAR with both SAM and CAM reaches MAE $p(y\mid x)$2, MSE $p(y\mid x)$3, PSNR $p(y\mid x)$4, and SSIM $p(y\mid x)$5 (Gao et al., 2019).

In "UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation" (Xu et al., 12 May 2026), the same phrase is used for token-level fusion prior to VLM encoding. For each reference image, aligned ViT and VAE token sequences

$p(y\mid x)$6

are concatenated along channels and linearly projected back to $p(y\mid x)$7: $p(y\mid x)$8 Initialization is identity-preserving,

$p(y\mid x)$9

so that training gradually injects appearance information while remaining compatible with the frozen Qwen2.5-VL encoder (Xu et al., 12 May 2026).

UniCustom extends unified conditioning from feature fusion to slot binding. Reference images are serialized as “Picture $$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$0” followed by unified visual tokens, and slot-wise binding regularization

$$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$1

forces the hidden states for slot $$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$2 to retain recoverable VAE-level details for the corresponding reference. Stage 1 uses $$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$3K steps at learning rate $$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$4; stage 2 uses $$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$5K steps at learning rate $$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$6. The paper reports that single-image reconstruction PSNR approaches $$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$7 dB in stage 1, and that UniCustom achieves the best open-source average on OmniContext ($$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$8) and MICo-Bench ($$p(dx,dy)=\mu(dx)p_x(dy), \qquad q_\theta(dx,dy)=\mu(dx)q_{\theta,x}(dy).$$9) (Xu et al., 12 May 2026).

Taken together, these works show two distinct forms of feature-space unification: SCAR unifies spatial and channel conditioning through parallel non-local branches feeding a shared regressor, whereas UniCustom unifies semantic and appearance conditioning through early channel concatenation feeding a frozen VLM. In both cases, the design prevents late-stage decoupling from discarding the association between channel content and the target prediction.

4. Statistical conditioning of wireless channels by side information

"A Statistical Characterization of Wireless Channels Conditioned on Side Information" (Böck et al., 2024) gives a probabilistic interpretation of unified channel-wise conditioning that differs from feature attention or OT transport. Here conditioning is a single operator

$$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$0

that updates first- and second-order channel statistics consistently across time, frequency, and space. The central structural requirement is that $$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$1 remain axiswise Toeplitz, or jointly block-Toeplitz-with-Toeplitz-blocks, whenever the side information preserves the WSSUS assumptions (Böck et al., 2024).

The paper starts from a wideband time-varying MIMO baseband tensor

$$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$2

with i.i.d. $$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$3, independent of the remaining path parameters $$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$4. Under this condition, $$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$5, time/frequency/space covariances are Toeplitz, and joint covariances are BTTB. The key theorem states that if $$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$6 for all $$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$7, then $$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$8 and $$\hat y = G_\theta(x,z), \qquad z\sim\mathcal N(0,I),$$9 remains in the Kronecker product of Toeplitz covariance cones (Böck et al., 2024).

This yields a binary taxonomy of side information. If $G_\theta$0 informs only $G_\theta$1, such as cluster labels, scene metadata, array configuration, or non-phase-resolving geometry, conditioning preserves zero mean and Toeplitz/BTTB structure. If $G_\theta$2 is a descendant of $G_\theta$3, as in pilots, feedback, or precise path-length information at carrier-wavelength scale, then $G_\theta$4 in general, so conditioning may induce nonzero mean and destroy Toeplitzness or stationarity (Böck et al., 2024).

The paper makes this distinction explicit through Bayesian-network factorization. In the modeling/sensing case, $G_\theta$5, so $G_\theta$6 is d-separated from $G_\theta$7 and phase uniformity is preserved. In the direct inference case, $G_\theta$8 and $G_\theta$9, so observing $x$00 opens dependence between $x$01 and $x$02. A practical consequence is the conditioned LMMSE estimator

$x$03

Reported qualitative findings are that sensing-only side information without pilots matches the zero estimator, pilot-only improves NMSE, and joint conditioning with pilot plus $x$04-informed side information improves further (Böck et al., 2024).

The same framework is extended to generative modeling and clustering. In a VAE, if the latent $x$05 does not encode path phases, decoder means converge toward zero and decoder covariances toward Toeplitz/BTTB structure; in channel clustering, GMMs with zero-mean Toeplitz covariances outperform k-means when the discriminative information lies in covariances rather than means (Böck et al., 2024). This is a statistical, rather than architectural, form of unified conditioning.

5. Reference-based and autoregressive conditioning in speech enhancement and compression

"RelUNet: Relative Channel Fusion U-Net for Multichannel Speech Enhancement" (Aldarmaki et al., 2024) implements unified channel-wise conditioning by anchoring every microphone stream to the same reference channel. For a fixed reference $x$06, each channel $x$07 is represented by

$x$08

Shared encoder-decoder weights process all $x$09 in parallel, so conditioning is identical for every channel. Decoded per-channel features are concatenated and a $x$10 convolution predicts a complex mask $x$11, which is applied to the reference spectrogram: $x$12 The paper uses six downsampling and six upsampling stages, SeLU, batch normalization, and optional GCN or GAT bottlenecks (Aldarmaki et al., 2024).

The argument is that early relative conditioning lets the network learn cross-spectral relations at every scale rather than only at late fusion. On CHiME-3 simulated test data, reported averages are PESQ $x$13 and STOI $x$14 for a standard multi-channel U-Net, versus PESQ $x$15 and STOI $x$16 for RelUNet; with GAT bottlenecks, U-Net+GAT gives PESQ $x$17, STOI $x$18, while RelUNet+GAT gives PESQ $x$19, STOI $x$20. The model is reported to add only about $x$21 parameters over the standard U-Net (Aldarmaki et al., 2024).

"Channel-wise Autoregressive Entropy Models for Learned Image Compression" (Minnen et al., 2020) relocates the same principle into latent coding. Instead of spatial autoregression, the latent tensor $x$22 is split into channel slices $x$23, and the entropy model factorizes as

$x$24

with no spatial autoregressive dependence inside a slice. Hyperprior outputs $x$25 and $x$26 provide forward adaptation, while previously decoded slices $x$27 provide backward adaptation. Per-symbol likelihoods are discretized Gaussians, and latent residual prediction refines reconstruction through

$x$28

This keeps spatial processing parallel while reducing serial decoding to $x$29 slice steps (Minnen et al., 2020).

Empirically, the paper reports average rate savings of $x$30 on Kodak and $x$31 on Tecnick relative to a context-adaptive baseline, with up to $x$32 savings over the baseline and up to $x$33 over BPG at low bit rates. The largest model, using 10 channel-conditioned slices with LRP and round-based training, achieves an average BD-rate saving of $x$34 over BPG on Kodak (Minnen et al., 2020). In this setting, unified channel-wise conditioning is an efficiency device: it preserves most of the adaptivity of causal entropy models while avoiding pixel-by-pixel serialism.

6. Flexible channel conditioning in biological diffusion and multimodal fusion

"Controllable diffusion-based generation for multi-channel biological data" (Zhang et al., 24 Jun 2025) treats channels as aligned biological measurements and learns a conditional model over arbitrary observed subsets. A mask $x$35 selects observed channels, the conditional input is $x$36, and the training objective is

$x$37

Conditioning is hierarchical: a contextual encoder produces multi-resolution features $x$38, which are injected into the diffusion U-Net by SE-gated addition,

$x$39

The model further combines latent-space channel attention with output-space channel attention,

$x$40

so that inter-channel dependencies are modeled throughout denoising and again at the prediction head (Zhang et al., 24 Jun 2025).

The reported ablations on Breast IMC show average Pearson $x$41 for the full single-channel model and $x$42 for the full multi-channel model. Removing output-space channel attention reduces performance to $x$43; removing latent-space channel attention reduces it to $x$44; replacing hierarchical injection by element-wise addition yields $x$45; and the unconditional baseline gives $x$46. On IMC protein imputation, reported scores are $x$47 on Breast and $x$48 on Lung for the single-channel setting, both exceeding ControlNet, STEM, and MULTIPLAI in the listed comparisons. On CITE-seq datasets, reported $x$49 values are $x$50 for PBMC, $x$51 for CBMC, $x$52 for BMNC, and $x$53 for HSPC (Zhang et al., 24 Jun 2025).

"Enhanced Multimodal Hate Video Detection via Channel-wise and Modality-wise Fusion" (Zhang et al., 17 May 2025) uses the term in a different but related sense. Video, audio, and text are each projected to a common width $x$54; video and audio first undergo temporal cross-gating by 1D convolutions,

$x$55

and each modality then passes through a channel-wise multi-head linear transform with $x$56 heads. Modality-wise gates are computed as

$x$57

followed by summation $x$58 (Zhang et al., 17 May 2025).

On the HateMM dataset, CMFusion reports Accuracy $x$59, F1 $x$60, Precision $x$61, and Recall $x$62, exceeding the listed HateMM baseline at $x$63. An ablation with channel-wise and modality-wise fusion but without the full configuration reports $x$64, while the full CMFusion configuration achieves $x$65 (Zhang et al., 17 May 2025). Here unified channel-wise conditioning is not about spatial channels or physical channels, but about forcing all modalities through the same channel partitioning and gating grammar before fusion.

7. Cross-cutting principles, misconceptions, and limitations

The surveyed literature suggests several recurring principles. First, unified channel-wise conditioning usually preserves an anchor and conditions only the complementary degrees of freedom: transmitted symbols are fixed while $x$66 is transported in condition-wise Sinkhorn drifting; reference microphone channels anchor relative speech features in RelUNet; biological channels are zero-masked rather than geometrically rearranged; image-reference slots in UniCustom are kept distinct by explicit identifiers and slot-local binding (Fritschek et al., 16 Jun 2026, Aldarmaki et al., 2024, Zhang et al., 24 Jun 2025, Xu et al., 12 May 2026).

Second, the phrase does not imply a specific mechanism. It can mean per-condition OT, full $x$67 non-local channel mixing, early token concatenation with linear projection, side-information-conditioned covariance operators, channel-autoregressive priors, or multi-head channel transforms. A common misconception is to equate channel-wise conditioning with simple diagonal channel gating. SCAR explicitly distinguishes its CAM from SENet/CBAM because it performs full channel mixing rather than pooled scalar reweighting, and UniCustom explicitly states that no additional gating or attention is used in the fusion layer, only channel-wise concatenation and a single linear projection (Gao et al., 2019, Xu et al., 12 May 2026).

Third, computational trade-offs are central. Condition-wise Sinkhorn reduces sampling latency relative to diffusion but can still trail diffusion on the hardest SER curves. SCAR’s SAM scales quadratically in $x$68. Biological diffusion notes that transformer-style channel attention scales as $x$69, so SE-style attention is preferred for stability and efficiency. Channel-autoregressive compression reduces serial complexity from spatial to slice-wise ordering but remains dependent on slice order and grouping (Fritschek et al., 16 Jun 2026, Gao et al., 2019, Zhang et al., 24 Jun 2025, Minnen et al., 2020).

Fourth, conditioning quality depends on what the conditioning signal can legitimately preserve. In the wireless-statistical setting, if side information preserves phase uniformity, then zero mean and Toeplitz/BTTB structure should be preserved; if it resolves carrier-scale phase or directly observes the channel, deterministic means and nonstationarities may be required instead (Böck et al., 2024). A plausible implication is that unified conditioning is most robust when the invariants it preserves are matched to the data-generating process rather than imposed purely for architectural convenience.

Finally, limitations recur across domains: sparse repeated outputs per $x$70 can destabilize conditional barycenters in learned channel simulation; extreme multi-reference similarity can still entangle slots in UniCustom; fixed-reference conditioning in RelUNet is not permutation-invariant; very large channel counts make quadratic channel attention costly in biological diffusion; and CMFusion’s temporal cross-attention is convolutional cross-gating rather than QKV attention, which may limit long-range inter-modality interactions (Fritschek et al., 16 Jun 2026, Xu et al., 12 May 2026, Aldarmaki et al., 2024, Zhang et al., 24 Jun 2025, Zhang et al., 17 May 2025). These limitations indicate that unified channel-wise conditioning is a powerful organizing principle, but not a universal substitute for domain-specific structure.