Probabilistic Prompting Method (PPM)

• Probabilistic Prompting Method (PPM) is a mechanism that dynamically routes control cues via a probabilistic posterior over latent tokens and spatial priors.
• It integrates depth and mask conditions into a latent diffusion U-Net to maintain 360° coherence while achieving fine-grained semantic control in panoramic street-view generation.
• Empirical evaluations show that PPM improves drivability IoU and image quality metrics, enhancing downstream BEV segmentation for autonomous driving.

Searching arXiv for the cited papers to ground the article and confirm bibliographic details. Probabilistic Prompting Method (PPM) denotes a prompt-conditioning mechanism introduced in Percep360 for controllable panoramic street-view generation in autonomous driving. In that setting, PPM dynamically selects the most relevant control cues by treating the choice and strength of each prompt cue as a latent random variable, computing a posterior over “which cue matters where,” and softly routing only the most relevant signals into each spatial location of the denoising network. The method is designed for a setting in which panoramic generation must jointly preserve coherence across the 360° field of view and support fine-grained controllability over semantic or style attributes, and it is implemented as a depth- and mask-conditioned attention module inside a latent diffusion U-Net (Teng et al., 9 Jul 2025).

1. Problem setting and design objective

Percep360 is presented as a panoramic generation method for autonomous driving in which controllable generation is conditioned on stitched panoramic data. Within that framework, PPM addresses the controllability side of the problem, while Local Scenes Diffusion Method (LSDM) addresses coherence. The motivating difficulty is that panoramic street-view generation must balance two competing demands: coherence across the 360° field of view, meaning no seams or discontinuities at the stitch boundaries, and fine-grained controllability over semantic or style attributes such as “daytime,” “rainy,” or road topology (Teng et al., 9 Jul 2025).

The stated motivation is that prior diffusion-based pipelines learn from one fixed panorama distribution and inherit stitching misalignments when asked to impose new control cues. In that regime, controllability drops as the model “overwrites” local structure in pursuit of global consistency. PPM is introduced to avoid feeding all control signals in equally. Instead, it treats the choice and strength of each prompt cue as a latent random variable and computes a posterior over which cue matters where, so that only the most relevant signals are routed into each spatial location of the denoising network.

This formulation makes PPM more specific than a generic text-prompting heuristic. In Percep360, it is an internal probabilistic routing mechanism operating at diffusion time and at the level of spatial tokens. A plausible implication is that the “prompt” in PPM refers not merely to an external text instruction, but to a structured, learned allocation of textual control cues across a panoramic latent field.

2. Probabilistic formulation

The method is defined over latent U-Net features and a set of control-prompt embeddings. The variables are introduced as follows:

• $x \in \mathbb{R}^n$: the current latent tokens, described as a flattened U-Net feature map of size $H \times W \rightarrow N = H \cdot W$
• $C = \{c_1,\ldots,c_L\}$: a set of $L$ control-prompt embeddings, described as text tokens from CLIP
• $M_d, M_m$: depth-based and binary-mask spatial priors, each upsampled or interpolated to size $H \times W$ and then flattened or replicated to shape $N \times L$

A uniform prior is placed on prompts:

$p(c_i) = \frac{1}{L}, \qquad i=1,\ldots,L.$

At each diffusion denoising step $t$, PPM computes a soft selection over prompts at each spatial position $j \in \{1,\ldots,N\}$. Query, key, and value projections are

$H \times W \rightarrow N = H \cdot W$0

The base scaled-dot-product attention logits are

$H \times W \rightarrow N = H \cdot W$1

The two spatial priors are then injected with learnable scalars $H \times W \rightarrow N = H \cdot W$2:

$H \times W \rightarrow N = H \cdot W$3

$H \times W \rightarrow N = H \cdot W$4

These cue-specific logits are merged via element-wise maximum,

$H \times W \rightarrow N = H \cdot W$5

and normalized over prompts to obtain a posterior distribution,

$H \times W \rightarrow N = H \cdot W$6

The attended embedding at each spatial location is then

$H \times W \rightarrow N = H \cdot W$7

and the output is fused back into the U-Net by a residual projection,

$H \times W \rightarrow N = H \cdot W$8

This is the core probabilistic statement of the method: prompt injection is recast as posterior inference over prompt relevance at each spatial token. The formulation is explicitly spatial, and its posterior is conditioned not only on latent features but also on depth and mask priors (Teng et al., 9 Jul 2025).

3. Diffusion-step algorithm

For one diffusion step, PPM is applied inside each U-Net block where control is injected. The procedure can be summarized in the same order used in the paper:

1. Feature extraction and projections: from the feature map at layer $H \times W \rightarrow N = H \cdot W$9, obtain $C = \{c_1,\ldots,c_L\}$0 of shape $C = \{c_1,\ldots,c_L\}$1, then compute

$C = \{c_1,\ldots,c_L\}$2

2. Base attention logits: compute

$C = \{c_1,\ldots,c_L\}$3

with shape $C = \{c_1,\ldots,c_L\}$4.

3. Spatial-prior injection: upsample and flatten the depth and mask priors to obtain $C = \{c_1,\ldots,c_L\}$5 and $C = \{c_1,\ldots,c_L\}$6 of shape $C = \{c_1,\ldots,c_L\}$7, then form

$C = \{c_1,\ldots,c_L\}$8

4. Merge and normalize: apply the element-wise maximum

$C = \{c_1,\ldots,c_L\}$9

then compute

$L$0

so that each row sums to $L$1.

5. Residual feature update: compute the weighted sum

$L$2

then inject it back through

$L$3

after which the tokens are folded back to the $L$4 feature map.

After these injections, the standard U-Net forward pass continues to predict $L$5, and the loss remains the standard latent diffusion mean-squared error,

$L$6

During inference, the same per-layer PPM injections are performed on each denoising iteration while sampling $L$7 from $L$8. This places PPM directly in the denoising loop rather than as a preprocessing or postprocessing device (Teng et al., 9 Jul 2025).

4. Architectural integration and implementation

PPM is integrated into a Latent Diffusion U-Net, described as following Rombach et al. The control branch is a parallel pathway that encodes 2-D cues such as RGB, depth, and binary mask via a frozen VAE encoder and BEV encoder. PPM sits inside the control branch at each U-Net resolution level. Rather than simple concatenation, it replaces the standard cross-attention with a depth- and mask-conditioned attention over text prompts (Teng et al., 9 Jul 2025).

The data flow is given in four stages. First, there is multi-modal encoding of image $L$9, depth $M_d, M_m$0, mask $M_d, M_m$1, BEV layout $M_d, M_m$2, and text $M_d, M_m$3. Second, LSDM applies a random circular shift to $M_d, M_m$4, $M_d, M_m$5, $M_d, M_m$6, and the BEV features to enforce continuity. Third, at each U-Net scale $M_d, M_m$7, PPM fuses $M_d, M_m$8. Fourth, the standard noise-prediction head outputs $M_d, M_m$9.

The reported implementation details are concise but specific:

Item	Reported setting
Number of prompts $H \times W$0	equals the length of the CLIP token sequence for the descriptive text, typically 16–32
Embedding dimension $H \times W$1	768
Softmax temperature $H \times W$2	1.0 by default
Spatial-prior scalars $H \times W$3	initialized to 0.1 and learned jointly
Diffusion steps	20 during both training and inference
Training objective	standard MSE on noise prediction $H \times W$4; no additional KL or prompt-specific regularizers were needed

The implementation implies a relatively light modification of the standard latent-diffusion stack. This suggests that the probabilistic part of PPM lies in how prompts are routed, not in replacing the diffusion objective or introducing an auxiliary probabilistic loss.

5. Empirical evaluation and observed behavior

The evaluation is reported from three perspectives: image quality assessment, controllability, and utility in real-world Bird’s Eye View segmentation. Controllability metrics are Drivable-area IoU (“Driv.”) on a pre-trained panoramic BEV segmentation network and mean IoU (“Mean”) across all BEV classes. Quality metrics include no-reference BRISQUE and PIQE, reference SSIM, and FID (Teng et al., 9 Jul 2025).

An ablation isolates prompt usage through four conditions: Baseline (no PPM), LSDM only, LSDM + depth prior (“LSDM + D”), and LSDM + full PPM (depth + mask), which corresponds to Percep360. The summarized result is that adding the PPM module improves drivable-area IoU from $H \times W$5 and mean IoU from $H \times W$6, while also obtaining top rankings on BRISQUE and PIQE and competitive FID. Qualitatively, the reported comparison shows that PPM preserves road boundaries and scene elements more faithfully under prompt variations such as “foggy morning” and “nighttime city.”

The paper’s abstract further states that the generated data consistently outperforms the original stitched images in no-reference quality metrics and enhances downstream perception models. In that experimental framing, PPM is not evaluated only as a prompt interface; it is evaluated as a component whose contribution is visible in both controllability metrics and downstream utility.

A plausible implication is that the posterior over prompts functions as a locality-preserving control mechanism. Because the control signal is spatially modulated by depth and mask priors, prompt variation need not be expressed as a uniform global rewrite of the panorama.

6. Terminological scope, limitations, and future directions

The acronym PPM is not unique across recent arXiv literature. In “String Seed of Thought: Prompting LLMs for Distribution-Faithful and Diverse Generation,” PPM is defined as any prompting strategy that steers an LLM’s non-deterministic output to follow a specified distribution $H \times W$7, with the objective of minimizing $H \times W$8 or total variation distance (Misaki et al., 24 Oct 2025). In “Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory,” the authors propose a probabilistic method to efficiently predict scaling performance and identify the best prompting strategy under large sampling times (Liu et al., 16 May 2025). This suggests a terminological overlap: in one line of work, PPM refers to distribution-faithful sampling in LLMs, whereas in Percep360 it refers to posterior-guided prompt routing in diffusion-based panoramic generation.

Within Percep360 itself, the reported strengths are that PPM turns control-cue selection into a per-pixel posterior inference problem, dynamically focuses on the most relevant prompts and spatial priors, requires no hand-tuned fusion weights or per-prompt heuristics because everything is learned end-to-end, and combines naturally with LSMD’s wrap-around diffusion to guarantee full 360° coherence. The reported limitations are equally specific: PPM relies on the accuracy of the depth and mask priors, so large errors there can mislead the attention; and it adds only a light supervisory signal, so global attributes such as weather may still bleed across regions (Teng et al., 9 Jul 2025).

The future directions named in the paper are to learn the priors jointly rather than using off-the-shelf estimators, to introduce a variational prompt-selection loss that could encourage sparse or low-entropy $H \times W$9 distributions, and to extend PPM to video for panorama-consistent long sequences and to 3D scene reconstruction. Taken together, these directions preserve the central interpretation of PPM: prompt injection as probabilistic routing conditioned on spatial structure, rather than prompt concatenation or uniform cross-attention.