PrecisionDiff: Optimizing Diffusion & LLM Testing

• PrecisionDiff is a framework that treats numerical precision as an explicit design variable for optimizing diffusion model inference by jointly allocating bit-widths and selecting denoising timesteps.
• It employs manifold-aware layer ranking and dynamic activation quantization to balance efficiency and accuracy under strict hardware constraints.
• In LLM evaluation, PrecisionDiff serves as a differential-testing method that detects precision-induced behavioral disagreements to enhance safety alignment.

Searching arXiv for "PrecisionDiff" and closely related papers to ground the article. PrecisionDiff is used in current arXiv literature for two distinct precision-sensitive procedures. In diffusion-model deployment, it denotes the mechanism inside DiffPro that jointly allocates per-layer bit-widths and selects denoising timesteps under hardware-faithful integer-kernel constraints, producing a single budgeted deployable plan for Diffusion Transformers (DiTs) (Amin et al., 14 Nov 2025). In LLM evaluation, it denotes an automated differential-testing framework that searches for precision-induced behavioral disagreements across numerical precisions such as BF16, FP16, INT16, and INT8 (Wang et al., 2 Apr 2026). In both cases, numerical precision is treated as an explicit design variable rather than a passive implementation detail, but the operational objectives differ: DiffPro’s PrecisionDiff targets efficient inference, whereas the LLM framework targets reliability and alignment verification.

1. Terminological scope and neighboring concepts

Within the supplied literature, the name “PrecisionDiff” does not designate a single method family. One use appears as a component-level mechanism in “DiffPro: Joint Timestep and Layer-Wise Precision Optimization for Efficient Diffusion Inference” (Amin et al., 14 Nov 2025). The other use appears as the title of “Hidden Reliability Risks in LLMs: Systematic Identification of Precision-Induced Output Disagreements,” where PrecisionDiff is an automated testing framework for LLMs (Wang et al., 2 Apr 2026).

The term should be distinguished from “computational divided differencing,” introduced as a program-transform technique closely analogous to forward-mode automatic differentiation that rewrites a routine computing $f:\mathbb{R}^n\to\mathbb{R}$ so that it also produces the finite difference $\Delta f=f(x+s)-f(x)$ with high relative accuracy even when direct subtraction suffers catastrophic cancellation (Vavasis, 2013). It is also distinct from the “precision increase by averaging” approach to numerical differentiation, in which estimates $D_h[f](x)$ are averaged over multiple step sizes $h_i$, reducing the random-error component by the familiar $1/\sqrt{n}$ factor while leaving the bias at order $O(h^p)$ (Liptaj, 2017).

These earlier methods concern accurate numerical differencing itself. By contrast, both contemporary PrecisionDiff usages concern the effects of precision choices inside modern ML systems: one for post-training efficiency optimization in diffusion inference, and one for cross-precision behavioral divergence in aligned LLMs.

2. PrecisionDiff in DiffPro: unified precision and schedule planning

In DiffPro, PrecisionDiff is presented as a post-training, hardware-faithful framework that works with the exact integer kernels used in deployment and jointly tunes timesteps and per-layer precision in DiTs to reduce latency and memory without any training (Amin et al., 14 Nov 2025). The mechanism combines three parts: a manifold-aware sensitivity metric to allocate weight bits, dynamic activation quantization to stabilize activations across timesteps, and a budgeted timestep selector guided by teacher-student drift.

The organizing object is a single “deployable plan” consisting of per-layer bits, group sizes, activation bits by time-bin, and a pruned step schedule. This plan is not a floating-point proxy. The detailed description explicitly states that all linear and convolution layers use INT8$\times$INT8$\to$INT32 GEMMs with per-output-channel weight scales, symmetric quantization, and zero-point $0$. This emphasis on exact deployment kernels is central to the method’s characterization as hardware-faithful.

A plausible implication is that PrecisionDiff should be understood less as an isolated quantizer and more as a joint budget-allocation procedure over multiple interacting sources of inference cost: weight precision, activation precision, denoising-step count, memory traffic, and kernel overhead.

3. Manifold-aware layer ranking and dynamic activation quantization

For per-layer weight-bit allocation, DiffPro computes a combined sensitivity score that fuses a curvature proxy and a manifold proxy (Amin et al., 14 Nov 2025). For a layer $\ell$, the curvature term is formed from recorded activations $\Delta f=f(x+s)-f(x)$0 over a calibration set: $\Delta f=f(x+s)-f(x)$1 This is normalized across layers to

$\Delta f=f(x+s)-f(x)$2

The manifold proxy is derived by collecting up to $\Delta f=f(x+s)-f(x)$3 activation rows per layer into $\Delta f=f(x+s)-f(x)$4 and running incremental PCA up to $\Delta f=f(x+s)-f(x)$5 components. With

$\Delta f=f(x+s)-f(x)$6

DiffPro defines

$\Delta f=f(x+s)-f(x)$7

so that low-rank, low-spill layers get a small score. The final manifold-aware sensitivity is

$\Delta f=f(x+s)-f(x)$8

Layers are then tiered into low, mid, and high sensitivity. A seed plan sets low-risk layers to $\Delta f=f(x+s)-f(x)$9 bits, medium layers to $D_h[f](x)$0–$D_h[f](x)$1 bits, and the most sensitive layers to $D_h[f](x)$2 bits or even FP16.

Dynamic Activation Quantization (DAQ) addresses the fact that DiT activations shift dramatically over diffusion time. At step $D_h[f](x)$3, layer $D_h[f](x)$4 has activations $D_h[f](x)$5. Channels are divided into groups of size $D_h[f](x)$6, each sample-group pair is flattened to $D_h[f](x)$7, and the quantizer uses a percentile clip threshold

$D_h[f](x)$8

a scale

$D_h[f](x)$9

and the quantized activation

$h_i$0

Operationally, weights are packed once with group-wise scales via GPTQ. At runtime, a tiny pre-hook computes $h_i$1, requantizes inputs to INT8, and then runs INT8$h_i$2INT8$h_i$3INT32 GEMM. DAQ is applied to $h_i$4, $h_i$5, $h_i$6, the output projection, and both MLP layers; LayerNorms and residual adds stay in high precision. The method also allows different bins of timesteps, such as early, mid, and late, to use different activation bit-widths.

4. Drift-guided timestep pruning, hardware constraints, and measured trade-offs

DiffPro couples precision allocation with timestep reduction through a budgeted timestep selector based on teacher-student drift (Amin et al., 14 Nov 2025). For each candidate step $h_i$7, the method measures

$h_i$8

A protected tail of late timesteps is fixed as

$h_i$9

From the remaining steps, the schedule of size $1/\sqrt{n}$0 is formed as

$1/\sqrt{n}$1

The stated purpose is to preserve late-time fidelity while keeping early and mid steps that the quantized student handles least well.

The hardware-faithful evaluation protocol measures end-to-end latency on an NVIDIA A30 GPU via synchronized CUDA events, including DAQ scale computation, requantize/dequantize overhead, and fused residual adds and LayerNorm. Results are reported for batch-size $1/\sqrt{n}$2 and batch-size $1/\sqrt{n}$3, channels-last. DiffPro also tracks

$1/\sqrt{n}$4

as a secondary metric. Reported complexity includes $1/\sqrt{n}$5 for sensitivity and PCA, $1/\sqrt{n}$6 for step pruning, and $1/\sqrt{n}$7 per candidate generation in joint search.

On ImageNet $1/\sqrt{n}$8 with $1/\sqrt{n}$9 DDIM samples and guided scale $O(h^p)$0, the reported full-precision baseline has $O(h^p)$1, latency $O(h^p)$2 s/image, and model size $O(h^p)$3 MB. DiffPro with bits, DAQ, and pruned steps reports $O(h^p)$4, latency $O(h^p)$5 s/image ($O(h^p)$6 faster), and model size $O(h^p)$7 MB ($O(h^p)$8 smaller). The ablations are: No-DAQ with optimized bits and schedule, $O(h^p)$9 and latency $\times$0 s/image; No-Prune with optimized bits and DAQ, $\times$1 and latency $\times$2 s/image; and Uniform W8/G32 with no DAQ and no pruning, $\times$3 and latency $\times$4 s/image.

A typical DiffPro plan for DiT-XL/2 with $\times$5 layers assigns approximately $\times$6 of layers to W4, approximately $\times$7 to W8, and approximately $\times$8 to FP16. Higher-sensitivity late attention heads and final MLPs are boosted to W8 or FP16. The reported Pareto behavior is likewise explicit: at $\times$9 ms versus $\to$0 ms, the drift-aware search nudges MSE lower with no latency increase, and pruning alone saves approximately $\to$1 of steps for only approximately $\to$2 FID points loss.

5. PrecisionDiff for LLMs: formalization as cross-precision differential testing

In the LLM literature, PrecisionDiff is defined as an automated differential-testing framework for systematically detecting precision-induced behavioral disagreements (Wang et al., 2 Apr 2026). Let $\to$3 denote a fixed LLM with parameters $\to$4 evaluated under numerical precision $\to$5. A precision-induced output disagreement occurs when two precisions $\to$6 and $\to$7 produce semantically different outputs on the same input, even though the model weights and prompt are otherwise identical.

The framework introduces both a token-level disagreement score and a binary oracle disagreement. For prompt $\to$8,

$\to$9

and a generic disagreement score is

$0$0

When the focus is safety alignment, disagreement is determined by a safety oracle $0$1, where $0$2 denotes safe or refuse and $0$3 denotes harmful or jailbreak.

Its central search procedure is a dual-precision adversarial suffix search. The goal is to find $0$4 that maximizes the chance that $0$5 while $0$6. The optimization target is

$0$7

where $0$8 is cross-entropy, $0$9 is a canonical affirmative jailbreak prefix, $\ell$0 is a canonical refusal prefix, and $\ell$1 in the experiments. Because $\ell$2 lies in a discrete token space, the method adapts Greedy Coordinate Gradient with a momentum buffer: $\ell$3 Top-$\ell$4 candidate tokens are selected per position by the largest negative entries in $\ell$5, a batch of suffixes is evaluated by the exact dual objective, and the best suffix is retained.

After each candidate suffix is selected, the framework performs cross-precision comparative analysis by generating $\ell$6 and $\ell$7 with greedy decoding, feeding each output into the safety oracle, and flagging a precision-induced disagreement whenever the oracle labels differ. The reported implementation evaluates five open-source models—Llama-2-7B-chat-hf, Meta-Llama-3-8B, Vicuna-7B-v1.5, Mistral-7B-Instruct-v0.2, and Guanaco-7B-HF (QLoRA)—under BF16 versus FP16, BF16 versus INT16, and INT16 versus INT8. The task is safety alignment verification on $\ell$8 harmful user queries sampled from AdvBench across eight categories, with greedy decoding ($\ell$9). Hyperparameters are $\Delta f=f(x+s)-f(x)$00, batch size $\Delta f=f(x+s)-f(x)$01, Top-$\Delta f=f(x+s)-f(x)$02, and momentum $\Delta f=f(x+s)-f(x)$03.

6. Empirical behavior, root-cause analysis, and implications

The reported success rates show that precision-induced behavioral disagreements are widespread across multiple aligned LLMs and precision settings (Wang et al., 2 Apr 2026). Averaged over $\Delta f=f(x+s)-f(x)$04 test queries, BF16 versus FP16 yields $\Delta f=f(x+s)-f(x)$05 success with $\Delta f=f(x+s)-f(x)$06 iterations on average; BF16 versus INT16 yields $\Delta f=f(x+s)-f(x)$07 success with $\Delta f=f(x+s)-f(x)$08 average iterations; and INT16 versus INT8 reaches approximately $\Delta f=f(x+s)-f(x)$09 success, with average $\Delta f=f(x+s)-f(x)$10 and $\Delta f=f(x+s)-f(x)$11 iterations. Model-specific results include, for BF16 versus FP16, Guanaco-7B at $\Delta f=f(x+s)-f(x)$12 with $\Delta f=f(x+s)-f(x)$13 iterations, Llama-2-7B at $\Delta f=f(x+s)-f(x)$14 with $\Delta f=f(x+s)-f(x)$15 iterations, Mistral-7B at $\Delta f=f(x+s)-f(x)$16 with $\Delta f=f(x+s)-f(x)$17 iterations, and Vicuna-7B at $\Delta f=f(x+s)-f(x)$18 with $\Delta f=f(x+s)-f(x)$19 iterations.

Against baseline search methods at $\Delta f=f(x+s)-f(x)$20, the paper reports Random Search, Fuzzing, and Genetic methods at $\Delta f=f(x+s)-f(x)$21–$\Delta f=f(x+s)-f(x)$22 success; standard single-precision GCG at $\Delta f=f(x+s)-f(x)$23 on Vicuna-7B and $\Delta f=f(x+s)-f(x)$24 on Llama-2-7B; and PrecisionDiff at $\Delta f=f(x+s)-f(x)$25 on Vicuna-7B with $\Delta f=f(x+s)-f(x)$26 iterations and $\Delta f=f(x+s)-f(x)$27 on Llama-2-7B with $\Delta f=f(x+s)-f(x)$28 iterations. Additional ablations report stochastic decoding success of $\Delta f=f(x+s)-f(x)$29–$\Delta f=f(x+s)-f(x)$30 on Llama-3 8B and $\Delta f=f(x+s)-f(x)$31–$\Delta f=f(x+s)-f(x)$32 on Llama-2 7B, with convergence in $\Delta f=f(x+s)-f(x)$33–$\Delta f=f(x+s)-f(x)$34 iterations, and reduced-budget success of $\Delta f=f(x+s)-f(x)$35–$\Delta f=f(x+s)-f(x)$36 across four models.

The layer-wise root-cause analysis attaches forward hooks at each transformer layer. For layer $\Delta f=f(x+s)-f(x)$37, with activations $\Delta f=f(x+s)-f(x)$38 and $\Delta f=f(x+s)-f(x)$39 under $\Delta f=f(x+s)-f(x)$40 and $\Delta f=f(x+s)-f(x)$41,

$\Delta f=f(x+s)-f(x)$42

Critical layers $\Delta f=f(x+s)-f(x)$43 are those with $\Delta f=f(x+s)-f(x)$44 above the $\Delta f=f(x+s)-f(x)$45th percentile. Across all models, the primary amplification sites are the input embedding layer, the initial attention projections $\Delta f=f(x+s)-f(x)$46 and $\Delta f=f(x+s)-f(x)$47, and the output LayerNorm and final head.

The paper frames several implications for deployment. As a drop-in differential tester, PrecisionDiff can be run on a held-out suite of safety queries under the exact precision mix planned for deployment. Suggested remedies include quantization-aware alignment, selective precision elevation for critical layers $\Delta f=f(x+s)-f(x)$48, and dynamic mixed precision governed by a divergence monitor. The authors summarize the framework as the first systematic, gradient-guided differential-testing framework that automatically crafts precision-sensitive prompts, detects hidden alignment failures across precision pairs with up to $\Delta f=f(x+s)-f(x)$49 success, and pinpoints the internal layers where tiny numerical differences snowball into safety violations.

Taken together, the two PrecisionDiff usages delineate two complementary research directions. In diffusion inference, precision is an optimization resource to be allocated jointly with timesteps under exact deployment kernels. In LLM safety evaluation, precision is a perturbation source whose behavioral consequences must be stress-tested. This suggests that precision-sensitive reasoning is becoming a distinct systems concern spanning both efficient deployment and robustness verification.