Personalized Semantic Judge (PSJ)

• Personalized Semantic Judge (PSJ) is an evaluator that scores candidate outputs based on user-specific semantic criteria rather than a fixed global standard.
• It employs a two-stage structure where discrete aspect evidence and a learned user weight vector are combined to produce a holistic, interpretable reward.
• Empirical results demonstrate that PSJ enhances recommendation metrics (e.g., NDCG, CTR, GMV) by aligning semantic evaluation with human preferences in an efficient offline setup.

Personalized Semantic Judge (PSJ) denotes an evaluator that scores candidate outputs under user-conditional semantics rather than a fixed global notion of quality. In its most explicit formulation, within S-GRec, PSJ is an offline LLM-based semantic reward module that produces interpretable aspect evidence and a user-conditional holistic score for generative recommendation, and that score is used to supervise a lightweight online generator without real-time LLM inference (Jiang et al., 11 Feb 2026). In a broader research sense, this suggests a family of judges conditioned on user or task preferences rather than a single architecture, spanning explicit human preference descriptions, representation-based evaluators, and preference-conditioned decoding schemes (Li et al., 2 Jun 2026, Li et al., 30 Jan 2026, Zhang et al., 17 Apr 2025).

1. Canonical formulation in generative recommendation

Within S-GRec, PSJ sits between behavioral data and reinforcement learning. The online generator receives unified user context $x$ and produces a Semantic ID sequence $y=(y_1,\dots,y_T)$ according to

$$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$

after which the SID is mapped to an item $a=\mathrm{Map}(y)$. Offline reward computation combines a business reward $r_{\mathrm{biz}}(x,y)$ with a semantic reward $r_{\mathrm{sem}}(x,a)$, and PSJ provides the latter (Jiang et al., 11 Feb 2026).

The architectural split is strict. Only the lightweight generator runs in serving, while PSJ is offline and train-time only. In the reported system, the online generator is Qwen2.5-1.5B, whereas PSJ is built on Qwen3-4B with DeepSeek-R1 used for initial labeling; PSJ scores only a sampled subset of training instances, often $p=0.05$, so latency and cost remain comparable to a standard generative recommender (Jiang et al., 11 Feb 2026). This design makes PSJ an offline semantic reward provider rather than an online reranker.

The motivation is equally specific. Behavioral logs supply weak supervision, are noisy and biased proxies for user intent, and do not explain why an item matches a user. Direct LLM judging is also unstable, being sensitive to prompt wording, candidate order, and context window, while raw semantic scores may conflict with business metrics such as CTR, GMV, or eCPM. PSJ is introduced to convert LLM reasoning into stable, personalized, and interpretable scalar rewards that can be safely fused with business optimization (Jiang et al., 11 Feb 2026).

2. Two-stage structure: aspect evidence and user-conditional aggregation

The canonical PSJ in S-GRec has two tightly coupled parts. Part A performs aspect-level semantic scoring. Given user profile, behavior history, context, and candidate item text, it outputs

$$\mathbf{s}(x,a) = [s_1(x,a), s_2(x,a), s_3(x,a), s_4(x,a)]^\top \in \mathbb{R}^4,$$

where the four dimensions are User Profile Relevance, Future Interest Relevance, Novelty, and Contextual Relevance. These scores are discrete and rubric-based, with scales such as $\{-1,-0.5,0,0.5,1\}$ or $\{0,0.5,1\}$ depending on the aspect (Jiang et al., 11 Feb 2026).

Part B learns how a particular user trades off those aspects. It maps the same user context to a weight vector

$y=(y_1,\dots,y_T)$0

and produces the holistic semantic score

$y=(y_1,\dots,y_T)$1

That scalar becomes the semantic reward $y=(y_1,\dots,y_T)$2 used for training the generator (Jiang et al., 11 Feb 2026).

Personalization is therefore explicit rather than implicit. The aspect vector describes what the item looks like semantically, while the weight vector describes how the user values those dimensions in context. The aggregation model is trained from pairwise preference tuples $y=(y_1,\dots,y_T)$3, constructed from same-request candidates and clicked-versus-un-clicked exposures, so the judge learns user-conditional trade-offs directly from preference contrasts rather than from a single opaque score target (Jiang et al., 11 Feb 2026).

Training data mirror this decomposition. S-GRec reports about 20k human-verified point-wise samples for aspect scoring and about 40k human-verified pairwise samples for aggregation, both sourced from the production ad system. A case study further shows that learned weights can emphasize Future Interest and Novelty, which explains why semantically complementary but previously unseen items are promoted for some users (Jiang et al., 11 Feb 2026).

3. Broader personalization mechanisms

Outside recommendation, several systems instantiate closely related PSJ mechanisms by changing how preference is represented. SenseJudge conditions judgment on explicit textual preference descriptions $y=(y_1,\dots,y_T)$4, turning evaluation into a function of query, two candidate responses, and a preference description. It constructs a pool of persona-like preferences from a small labeled set using DeepSeek-R1, selects a subset $y=(y_1,\dots,y_T)$5 that maximizes agreement with a particular annotator’s past choices, and applies majority vote across that subset on new examples. Its human study reports that only 49% of instances had full agreement across all 3 annotators, which directly motivates personalized rather than universal judging (Li et al., 2 Jun 2026).

A different route is Representation-as-a-Judge. INSPECTOR feeds aspect-specific evaluation prompts $y=(y_1,\dots,y_T)$6 to a small LM, extracts hidden states and attention statistics, and trains separate probes per aspect. The paper formalizes the Semantic Capacity Asymmetry Hypothesis: evaluation requires significantly less semantic capacity than generation and can be grounded in compressed internal representations of small LLMs (Li et al., 30 Jan 2026). A plausible implication is that some PSJs can be implemented as compact, decoding-free evaluators even when generation remains expensive.

Persona-judge pushes personalization into decoding itself. The same base LLM acts as a draft model under one persona and as a judge under another persona. If the draft proposes token $y=(y_1,\dots,y_T)$7 with distribution $y=(y_1,\dots,y_T)$8 and the judge assigns $y=(y_1,\dots,y_T)$9, the token is accepted with probability $$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$0; otherwise it is resampled from $$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$1. Because personas are expressed as natural-language prefixes, the method is training-free and applies to unseen preferences at inference time (Zhang et al., 17 Apr 2025).

An earlier antecedent appears in Semantic Sort, which learned user-specific semantic relatedness $$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$2 by reweighting corpus contexts from pairwise preference labels over term pairs. There personalization resides in the learned context weights $$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$3, not in a universal similarity function, and the resulting model ranks term pairs according to the semantics of a specific user or group (El-Yaniv et al., 2013).

4. Learning, reward construction, and efficient scoring

PSJ training is typically staged rather than monolithic. In S-GRec, aspect scoring is learned by supervised fine-tuning on human-verified discrete labels and then aligned with GRPO using an aspect reward that combines exact label matches and relative ordering across the four dimensions. Aggregation is trained with a pairwise reward

$$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$4

so that the final semantic reward becomes $$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$5 (Jiang et al., 11 Feb 2026).

That semantic reward is not fused naively with business reward. A2PO first standardizes both signals into group-relative advantages and then defines

$$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$6

where $$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$7 is nonzero only when business and semantic advantages have the same sign and is magnitude-bounded so that

$$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$8

Business reward therefore remains the primary direction and scale, while semantic reward refines updates only when aligned (Jiang et al., 11 Feb 2026).

A more general judge-training formulation appears in FairJudge, which models judging as a policy

$$\pi_{\theta}(y \mid x) = \prod_{t=1}^{T} \pi_{\theta}(y_t \mid x, y_{<t}),$$9

over judgment outputs conditioned on evaluation input $a=\mathrm{Map}(y)$0, criteria context $a=\mathrm{Map}(y)$1, and evaluation mode $a=\mathrm{Map}(y)$2. FairJudge couples a high-information-density judging dataset with a curriculum-style SFT-DPO-GRPO pipeline so that rubric adherence is learned first, debiasing is introduced through preference optimization, and cross-mode consistency is later regularized by reward design, all while avoiding catastrophic forgetting (Yang et al., 6 Feb 2026).

Low-cost scoring can also be made intrinsic to the judge. *-PLUIE replaces free-form evaluation with a perplexity-based log-odds score,

$a=\mathrm{Map}(y)$3

using task-specific prompt variants for French paraphrase detection, network-policy equivalence, and revision instruction following. The resulting score is continuous, deterministic, and does not require textual generation or post-processing (Lemesle et al., 17 Feb 2026).

Prompt-space optimization constitutes another training route. The Multi-Agent LLM Judge framework iteratively selects representative examples, evaluates the current prompt, and rewrites it until prompt quality crosses a threshold, improving AUC on Instruct-QA from 0.78 for the initial prompt to 0.91 and Pearson correlation on STSB from 0.67 to 0.81 (Cao et al., 1 Apr 2025). A complementary engineering view organizes judge development around general and contextualized constitutions and decomposed principle-based scoring, treating rubric specialization as a first-class systems artifact (Lin et al., 2024).

5. Reliability, bias, and consistency

A persistent misconception is that conditioning a strong LLM on a persona is sufficient for reliable personalized judging. Direct LLM-as-a-Personalized-Judge evaluation is reported as less reliable than previously assumed, with low and inconsistent agreement with human ground truth; however, verbal uncertainty estimation raises agreement to above 80% on high-certainty samples for binary tasks (Dong et al., 2024). This establishes selectivity as a core PSJ property: the judge must know when it does not know.

SenseJudge attacks the same problem through explicit preference ensembling and order-invariant evaluation. Its framework substantially reduces position bias, improves consistency for weaker judges, and shows that preference induction quality matters sharply: for Qwen3-14B, self-generated preferences yield 69.09% accuracy, whereas DeepSeek-R1-generated preferences yield 82.65% (Li et al., 2 Jun 2026).

SEMJ treats disagreement itself as signal. For $a=\mathrm{Map}(y)$4 language-specific judgments $a=\mathrm{Map}(y)$5, it computes

$a=\mathrm{Map}(y)$6

and uses low agreement to trigger self-reflection with multilingual rationales as memory. Across multiple benchmarks, this iterative re-evaluation improves both accuracy and cross-lingual consistency over monolingual reflection and voting baselines (Fu et al., 6 Jun 2026). The same logic generalizes naturally to multi-persona or multi-rubric PSJs.

BabelJudge makes reliability auditable even without human preference labels by constructing gold pairs through controlled degradation and measuring position bias, verbosity susceptibility, order inconsistency, and cross-lingual degradation. Its composite reliability score

$a=\mathrm{Map}(y)$7

formalizes the principle that raw accuracy is insufficient when the judge is unstable or shortcut-driven (KC, 21 Jun 2026). FairJudge encodes the same diagnosis at training time, treating debiasing and pointwise–pairwise consistency as explicit policy objectives rather than incidental outcomes (Yang et al., 6 Feb 2026).

6. Empirical impact, applications, and open problems

The most detailed PSJ evaluation remains S-GRec. Its aspect scorer improves from PairAUC 0.3987 and PointAcc 0.4043 for pre-trained Qwen3-4B to PairAUC 0.8116 and PointAcc 0.8687 after SFT+GRPO on 1k held-out samples. In downstream ablations, full PSJ gives the best NDCG@10 on both Office and Industrial datasets, moving Office from 0.1242 to 0.1308 and Industrial from 0.1167 to 0.1202, while online A/B testing in WeChat Channels reports +1.19% GMV, +1.55% GMV-Normal, +1.16% CTR, and −2.02% dislike rate without online LLM inference (Jiang et al., 11 Feb 2026).

Beyond recommendation, SenseJudge shows that explicit preference-set induction can raise Qwen3-14B-Instruct from 66.90% to 82.65% overall accuracy on LLMs-as-personalized-judges, with the framework averaging 16.08% higher accuracy than baselines and producing model rankings aligned with human sense (Li et al., 2 Jun 2026). Representation-based judging reaches a related conclusion on efficiency: probing hidden states of 0.6B–1.7B models yields +20% or more absolute F1 over prompting-based small LMs on binary evaluation tasks and can approximate full LLM judges closely enough for data filtering in distillation pipelines (Li et al., 30 Jan 2026).

Prompt-optimized and domain-specific judges broaden the application space. The Multi-Agent LLM Judge framework improves both QA correctness discrimination and semantic similarity correlation through iterative prompt rewriting (Cao et al., 1 Apr 2025). JudgeMeNot, in a legal setting, shows that parameter-efficient personalization from judicial decisions can make model-generated outputs statistically indistinguishable from human judges’ reasoning under an authorship-classification test, while also highlighting risks of gaming the system, bias amplification, and misuse in high-stakes environments (Razumenko et al., 20 Apr 2026).

Open problems remain consistent across the literature. S-GRec identifies domain shift, bias and misalignment in LLM semantics, limited aspect coverage, annotation cost, judge robustness, and the restriction to two-reward fusion as central limitations; notably, its PSJ relies on 20k human-verified point-wise samples and 40k human-verified pairwise samples (Jiang et al., 11 Feb 2026). SenseJudge adds limited numbers of explicit human preferences, domain-transfer gaps, and the scalability problem of exhaustive preference-subset selection (Li et al., 2 Jun 2026). A plausible implication is that future PSJs will combine richer user-conditioned criteria, stronger auditing for bias and inconsistency, and offline-to-online distillation so that personalized semantic evaluation remains interpretable, scalable, and operationally safe.