Single-Pass Scoring Primitive

• Single-pass scoring primitive is a computational strategy that integrates scoring directly into the main data processing pass, eliminating the need for additional evaluation rounds.
• It encompasses techniques such as multimodal reward modeling, dynamic programming over weighted automata, hidden-state probe classification, and rank-based token verification.
• By merging score extraction with inference, it significantly reduces latency and resource usage while maintaining effective decision-making in real-time applications.

Searching arXiv for the cited papers to ground the article and verify metadata. Single-pass scoring primitive denotes, across several recent lines of work, a scoring mechanism that is executed during the primary inference, generation, or stream-processing pass rather than through repeated response-by-response evaluation, a second model invocation, or a traceback phase. In current usage, the pattern includes multimodal reward modeling that scores all candidate responses in one forward pass, FPGA-resident weighted automata that emit alignment scores without backtracking, hidden-state probes that classify in the same forward pass used for generation, and CPU-only verification that scores each generated token from precomputed rank statistics (Yang et al., 13 Apr 2026, Karakchi, 2024, Meyoyan et al., 19 Jan 2026, Bogdan et al., 28 Apr 2026).

1. Scope and common computational pattern

Across these systems, the primitive is defined operationally: score computation is fused into the main pass over the relevant structure. In one case the structure is a concatenated prompt-response sequence; in another it is a weighted non-deterministic finite automaton (WNFA) driven one symbol per cycle; in another it is the full token-layer hidden-state tensor of a serving LLM; and in another it is a streaming token sequence scored by global rank and optional model log-probability.

Setting	Single-pass mechanism	Eliminated extra pass
Multimodal reward modeling	Concatenate multiple responses with separator tokens; apply cross-entropy over scalar scores	One forward pass per response
WNFA sequence alignment	Fully pipelined Viterbi-style dynamic programming over the WNFA	Backtracking or traceback pass
LLM orchestration classification	Train lightweight probes on hidden states from the same forward pass used for generation	Separate safety or classification model
Output verification	O(1) token lookup and arithmetic using token-to-rank tables and optional log-probabilities	Sampling-based detector passes

This suggests a common structure: a shared representation is produced once, candidate-specific or state-specific scores are extracted locally, and an aggregation operator selects or combines them before the computation advances. The exact operator varies by domain. Multi-response reward modeling uses softmax and cross-entropy over response scores. NAPOLY+ uses local $\max$-reduction over competing predecessor paths. Probe-based classification uses token-level and layer-level aggregation over $H \in \mathbb{R}^{L \times T \times d}$. Real-time verification uses a rank-based prior and, when available, a product-of-experts combination with $\log P_{LLM}(w \mid c)$.

2. Joint response scoring in multimodal reward models

"You Only Judge Once" formulates a discriminative multimodal reward model that scores all candidate responses in a single forward pass by concatenating a prompt $x$ with $N$ candidate responses $\{y_1,\dots,y_N\}$ into

$$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$

Hidden states $H \in \mathbb{R}^{L \times d}$ are produced once, the last-token state $h_i = H_{e_i}$ is extracted for each response, and a lightweight 2-layer MLP value head maps $h_i \in \mathbb{R}^d$ to a scalar $H \in \mathbb{R}^{L \times T \times d}$0. Scores are normalized by

$H \in \mathbb{R}^{L \times T \times d}$1

and training minimizes multi-response cross-entropy with respect to the ground-truth best-response index $H \in \mathbb{R}^{L \times T \times d}$2. When $H \in \mathbb{R}^{L \times T \times d}$3, this recovers the Bradley-Terry pairwise loss. The reported instantiation uses a pretrained vision-language backbone, Molmo2-4B with hidden dimension $H \in \mathbb{R}^{L \times T \times d}$4, a frozen vision tower, LoRA on the language transformer with rank $H \in \mathbb{R}^{L \times T \times d}$5, $H \in \mathbb{R}^{L \times T \times d}$6, dropout $H \in \mathbb{R}^{L \times T \times d}$7, and a value head with hidden width $H \in \mathbb{R}^{L \times T \times d}$8 and SiLU activation; training uses AdamW with learning rate $H \in \mathbb{R}^{L \times T \times d}$9, no weight decay, linear decay schedule, 3 epochs, effective batch size $\log P_{LLM}(w \mid c)$0, max sequence length $\log P_{LLM}(w \mid c)$1, and random shuffling of response order per sample (Yang et al., 13 Apr 2026).

The paper’s central claim is that joint scoring changes both efficiency and learning dynamics. A conventional single-response discriminative reward model requires $\log P_{LLM}(w \mid c)$2 full forward passes, whereas the multi-response design uses one pass over the concatenated sequence. Under the condition that the vision prefix dominates, ideal speedup is approximately $\log P_{LLM}(w \mid c)$3. On Molmo2-4B running on H100, the empirical figures are $\log P_{LLM}(w \mid c)$4 latency and $\log P_{LLM}(w \mid c)$5 FLOPs reduction at $\log P_{LLM}(w \mid c)$6, and up to $\log P_{LLM}(w \mid c)$7 latency with $\log P_{LLM}(w \mid c)$8 FLOPs reduction at $\log P_{LLM}(w \mid c)$9. The reported scaling behavior is that multi-response latency is approximately constant as $x$0 grows, while single-response latency is proportional to $x$1.

The benchmark program extends beyond pairwise preference data. MR$x$2Bench-Image contains 240 real user prompts plus images, divided into 80 VQA, 80 safety, and 80 reasoning examples. For each prompt-image pair, 8 responses are generated from 8 diverse models—GPT-5, Claude-4.5, Gemini-2.5, Qwen3-VL (2B/32B), Qwen-7B, and LLaVA-7B—and human annotators provide a complete ranking of all 8. For best-of-4 evaluation, 4 of the 8 responses are randomly sampled while preserving relative order. MR$x$3Bench-Video contains 497 video questions over 489 CC-licensed videos, responses from 19 models, and approximately 94K crowdsourced pairwise human judgments; rankings are denoised via Preference Graph Ensemble & Denoising, which builds per-annotator preference graphs, aggregates them into an ensemble graph with approximately 57,998 edges, removes cycles by greedy edge deletion to obtain a DAG with approximately 45,036 edges, and produces a total ranking by topological sort. After filtering, 495 questions remain for 4-response evaluation.

On six multimodal reward benchmarks, the main 4B open-source model table reports 71.2% average accuracy for Molmo2-4B Multi-RM, 65.1% for Qwen3-VL-4B Multi-RM, and approximately 63.0% for the next best model, Skywork-VL-7B. In a direct comparison on Molmo2-4B, multi-response cross-entropy reaches 64.8% average accuracy, while single-response Bradley-Terry reaches 54.0%. The same reward model is then used with Group Relative Policy Optimization to fine-tune a Molmo2-4B policy over 50 K open-ended prompts, with $x$4 rollouts per prompt, full fine-tuning of the policy with frozen vision, 500 steps, learning rate $x$5, KL coefficient $x$6, batch size $x$7, and 2 PPO epochs per step. On GPT-4-judged open-ended benchmarks, WildVision win% rises from 54.6 to 60.2 under +Multi-RM, LLaVA-Bench score from 92.4 to 97.0, and MMHal average score from 3.98 to 4.25; the single-response RM baseline improves less and, for LLaVA-Bench, declines from 92.4 to 91.6. Figure 1 is summarized as showing steadily increasing validation reward for the multi-response RM and flat or unstable reward signal for the single-response RM, with some configurations even “reward hack.”

3. Single-pass scoring in weighted automata processors

"A Scored Non-Deterministic Finite Automata Processor for Sequence Alignment" defines the primitive in formal automata terms. Sequence alignment is modeled as a weighted NFA

$x$8

where $x$9 assigns real-valued weights to transitions. For an accepting path $N$0, the accumulated path weight is

$N$1

and the score assigned to an input string $N$2 is the maximum over all accepting paths that spell $N$3:

$N$4

NAPOLY+ implements this as a fully pipelined, cycle-by-cycle Viterbi-style dynamic programming procedure over the WNFA without any backtracking pass. At time $N$5, only the start STE+ is active with score 0. For each incoming symbol, new-symbol fan-in drives the start STE+ with score 0, every active incoming edge computes a candidate score $N$6, local $N$7-reduction keeps only the best predecessor for each STE+, and accepting states write out their current score and offset immediately. The summary states explicitly that no second pass or traceback is ever needed (Karakchi, 2024).

The microarchitecture is a 2-D array of STE+ tiles connected by global horizontal buses and local vertical fan-in and fan-out wires. Each tile contains symbol-matching memory implemented with distributed RAM, a small register file for outgoing edge weights, an adder-plus-comparator datapath for up to $N$8 incoming scores in a cycle, a local score register, and fan-out drivers to up to $N$9 successor STE+s. The pipeline is organized into three stages: symbol broadcast and compare, predecessor score plus weight and max-reduction, and write-back plus outgoing fan-out. At steady state the throughput is one symbol per cycle regardless of how many states are active.

The principal state-space control mechanism is local max-reduction per state, which collapses an exponential number of nondeterministic paths into at most $\{y_1,\dots,y_N\}$0 live scores each cycle. A second mechanism is static hardware limits on fan-in and fan-out. If theoretical out-degree exceeds the configured $\{y_1,\dots,y_N\}$1, the summary notes two remedies: replicate STE+s in multiple banks or apply a coarse-grained beam cut that suppresses any activation whose score falls below $\{y_1,\dots,y_N\}$2. The baseline prototype relies solely on local max-reduction.

The implementation results are reported on a ZCU104 board with XCZU7EV, 225 K LUTs, 445 K registers, and 64 K distributed RAM. For a 64 K STE+ array, measured utilization is 214 K/225 K LUTs (95%), 423 K/445 K registers (95%), 64 K/64 K distributed RAM (100%), and 0 DSPs. Maximum frequency is 380 MHz at 1 K STE+, 350 MHz at 16 K STE+, and 300 MHz at 64 K STE+. Throughput is exactly one symbol per cycle, so at 300 MHz the design scores 300 M symbols/sec. Latency per symbol is exactly three pipeline stages; end-to-end latency is stated as $\{y_1,\dots,y_N\}$3 cycles but hidden in streaming mode. The results are described as preliminary, with ongoing testing on real datasets and applications such as BLAST.

4. Hidden-state probe classification in the serving pass

"A BERTology View of LLM Orchestrations" uses the phrase in a model-serving setting: the serving LLM’s hidden states are reused and lightweight probes predict labels in the same forward pass used for generation. The backbone is frozen and the probe operates over the full hidden-state tensor

$\{y_1,\dots,y_N\}$4

where $\{y_1,\dots,y_N\}$5. The architecture is a two-stage aggregator. Stage 1 summarizes tokens within each layer, producing $\{y_1,\dots,y_N\}$6 from $\{y_1,\dots,y_N\}$7; Stage 2 aggregates across the layer summaries to produce a single vector $\{y_1,\dots,y_N\}$8 for classification. Three variants are reported for both stages: direct pooling, a scoring-attention gate, and a downcast multi-head self-attention probe. The scoring-attention gate uses approximately 100 K parameters overall, with Stage 1 parameters $\{y_1,\dots,y_N\}$9 and an additional $$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$0 in Stage 2. The MHA probe uses a downcast factor 32 and approximately 35 M parameters, with parameter count $$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$1 when $$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$2. Classification then applies a linear head with cross-entropy loss (Meyoyan et al., 19 Jan 2026).

The reported efficiency comparison uses Llama-3.2-3B at batch size 1, max_length 512, and max_new_tokens 1. Base throughput is 37.8 samples per second, latency is 26.4 ms, and GPU VRAM is 6,497 MB. Adding a pooling probe yields 33.7 sps, 29.7 ms, and 6,497 MB; adding a scoring probe yields 32.4 sps, 30.9 ms, and 6,749 MB; adding an MHA probe yields 24.8 sps, 40.3 ms, and 6,969 MB. The separate Guard→Serve pipeline is much costlier at 8.1 sps, 123.2 ms, and 22,769 MB. The paper therefore places single-pass probes between two extremes: logit-only reuse and a separate guard-model pipeline.

Empirically, the trade-off is between probe complexity and downstream accuracy. On ToxicChat in-distribution with a frozen Llama-3B backbone, MULI reaches F1 77.8 and AUPRC 0.829 with 0.13 M added parameters; direct pooling reaches 73.53±0.68 and 0.812±0.005; the scoring-attention gate reaches 80.49±1.17 and 0.854±0.008 with 0.10 M parameters; and the MHA probe reaches 84.51±0.43 and 0.898±0.005 with 35 M parameters. On WildGuardMix in-distribution, the same ordering holds, with MHA at 88.55 and scoring-attention at 85.98. On sentiment and emotion tasks, MHA reaches 95.15±0.06 on IMDB, 95.39±0.32 on SST-2, and 87.68±1.09 on Emotion, while the scoring-attention gate reaches 95.05±0.11, 94.42±0.64, and 84.58±0.67. The paper’s stated interpretation is that a two-stage aggregator over the full hidden-state tensor can discover where the classification signal resides rather than committing a priori to final-layer pooling or first-token logits.

5. Rank-based verification and streaming anomaly scoring

"The Surprising Universality of LLM Outputs" introduces a different single-pass scoring primitive based on an observed regularity in token rank-frequency distributions. Tokens are ranked globally by empirical frequency in a 4 billion-token Wikipedia snapshot, and across six independent LLMs and five held-out domains the empirical rank-frequency curve is reported to be well approximated by a two-parameter Mandelbrot distribution,

$$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$3

Fit quality is summarized by 34 of 36 model-by-domain fits exceeding $$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$4, all 36 exceeding 0.90, and 35 of 36 favoring Mandelbrot over Zipf by AIC. The reported global-aggregate parameter examples are $$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$5 for GPT-5.1; $$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$6 for Gemini 2.5 Pro; $$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$7 for Qwen 2.5 7B; $$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$8 for Llama 3.1 8B; $$s = [x; y_1; \langle|\mathrm{resp\_sep}|\rangle; y_2; \langle|\mathrm{resp\_sep}|\rangle; \dots; \langle|\mathrm{resp\_sep}|\rangle; y_N].$$9 for Claude 4.6; and $H \in \mathbb{R}^{L \times d}$0 for Mistral Large (Bogdan et al., 28 Apr 2026).

From this reference distribution, the paper derives several scores. In rank-only mode, the local rank deviation in bits is

$H \in \mathbb{R}^{L \times d}$1

where $H \in \mathbb{R}^{L \times d}$2 is the token’s local rank within the passage. In hybrid mode, the model’s context-conditioned probability is combined with the reference prior through

$H \in \mathbb{R}^{L \times d}$3

which yields the per-token log-score

$H \in \mathbb{R}^{L \times d}$4

and corresponding energy

$H \in \mathbb{R}^{L \times d}$5

Because $H \in \mathbb{R}^{L \times d}$6 depends only on a precomputed token-to-rank table, streaming evaluation reduces to O(1) lookups and arithmetic per token.

The efficiency figures are central. On an 8-core AMD Ryzen CPU, pure-rank plus entity filtering mode excluding NER runs at 0.139 ms per passage or approximately 2.6 $H \in \mathbb{R}^{L \times d}$7s per token. Adding spaCy-based NER adds about 10 ms per passage. By contrast, sampling-based detectors such as Semantic Entropy or SelfCheckGPT require $H \in \mathbb{R}^{L \times d}$8 additional forward passes per example, with $H \in \mathbb{R}^{L \times d}$9, each costing approximately 4 s on GPU or about 20 s on CPU-only inference; at $h_i = H_{e_i}$0 this is summarized as at least 20,000 ms per passage, giving an approximately 100,000$h_i = H_{e_i}$1 cost difference.

The reported behavior is selective rather than universal. On FRANK, overall span-level ROC-AUC is 0.585, with AUC 0.646 for out-of-article entity errors and approximately 0.50 for coreference and discourse errors. On TruthfulQA multi-choice, baseline accuracy rises from 61.4% to 63.8% under rank-only reranking; timer or locational indexical errors have AUC greater than 0.82, while science or politics errors in normal vocabulary have AUC approximately 0.42–0.45. On HaluEval, the QA subset is dominated by a length artifact and the primitive adds no net signal; the dialogue subset improves from length-only AUC 0.703 to 0.720; the summarization subset is approximately at chance. The paper therefore positions the primitive as a first-pass triage layer rather than a replacement for sampling-based or source-conditioned verification.

6. Cross-cutting properties, limitations, and recurrent misconceptions

The surveyed usage suggests that “single-pass” is not a single algorithmic family but a systems property: scoring is fused into the primary pass, while the decisive operation is an in-pass aggregation or reduction. In multimodal reward modeling, that operation is $h_i = H_{e_i}$2-way comparative reasoning over concatenated responses with cross-entropy. In NAPOLY+, it is local max-reduction over predecessor paths. In probe-based classification, it is token- then layer-selective aggregation over hidden states. In rank-based verification, it is per-token scoring from a precomputed prior with optional composition with model log-probabilities. A plausible implication is that the term is best understood by what secondary computation it removes: repeated forward passes, traceback, a separate classifier, or sampling-based re-evaluation.

A common misconception is that single-pass scoring implies negligible overhead or universally stronger semantics. The four cases do not support that interpretation. The probe paper reports a throughput drop from 37.8 sps for the base model to 24.8 sps for the MHA probe, even though the system remains far cheaper than a Guard→Serve pipeline. The reward-model paper reports gains that depend on response count and dominance of the vision prefix; the architecture supports larger $h_i = H_{e_i}$3 within context length, but experiments are only reported up to $h_i = H_{e_i}$4. NAPOLY+ still faces hardware limits on fan-in and fan-out and exhibits declining maximum frequency as state count grows. The verification primitive attains very low latency, but its strongest signal is for Tier 1 distributional anomalies such as unsupported entities and rare lexical insertions, while Tier 2 pure reasoning or world-knowledge errors remain approximately at chance.

The limitations are domain-specific. For the multi-response reward model, MR$h_i = H_{e_i}$5Bench-Video remains difficult, with best-of-4 accuracy approximately 50.7%; the authors also state that a discriminative reward model cannot produce natural-language rationales, unlike generative judges. For NAPOLY+, the experimental results are preliminary and real-dataset evaluation is still ongoing. For probe-based classification, direct pooling is inexpensive but weaker, while MHA is stronger but materially heavier in added parameters and latency. For the verification primitive, the paper explicitly recommends deployment as a first-pass triage layer in a cascade that forwards only flagged cases to heavier verifiers.

Taken together, these works characterize the single-pass scoring primitive as a design choice for embedding supervision, selection, or verification into the first computational sweep over data. The benefit is reduced repeated computation and lower operational complexity. The cost is that each primitive inherits the representational and structural limits of the pass into which it is embedded.