JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew

Abstract: Despite significant advances in LLMs, personalizing them for individual decision-makers remains an open problem. Here, we introduce a synthetic-organic supervision pipeline that transforms raw judicial decisions into instruction-tuning data, enabling parameter-efficient fine-tuning of personalized models for individual judges in low-resource settings. We compare our approach to state-of-the-art personalization techniques across three different tasks and settings. The results show that Causal Language Modeling followed by synthetically generated instruction-tuning significantly outperforms all other baselines, providing significant improvements across lexical, stylistic, and semantic similarity. Notably, our model-generated outputs are indistinguishable from the reasoning of human judges, highlighting the viability of efficient personalization, even in low-resource settings.

• The paper introduces a synthetic-organic alignment pipeline that transforms Hebrew court judgments into high-fidelity instructions for emulating judge-specific reasoning.
• It employs parameter-efficient fine-tuning with QLoRA adapters on Gemma-3 models, achieving significant gains in BLEU, ROUGE-L, and BERTScore for style and content fidelity.
• The study demonstrates that modest data and low-rank tuning yield judge-specific models nearly indistinguishable from authentic judicial writing in low-resource settings.

Emulating Judicial Reasoning: Personalized LLMs in Low-Resource Hebrew Legal Domain

Motivation and Conceptual Foundations

The paper "JudgeMeNot: Personalizing LLMs to Emulate Judicial Reasoning in Hebrew" (2604.18041) addresses the longstanding challenge of modeling individual decision-maker reasoning, specifically within the legal domain. The prevailing approach in legal NLP has typically treated judges as interchangeable actors, reducing interpretive diversity to statistical noise. This study asserts that judge-specific rhetorical and philosophical patterns are central to legal outcomes, thereby necessitating high-fidelity personalization for tasks such as counterfactual analysis and granular trust by legal scholars. The research situates its contribution within deliberative reasoning personalization, where most current methods lack the granularity to capture slow, cognitive decision processes distinctive to individual judges.

Synthetic-Organic Supervision and Data Pipeline

A fundamental technical contribution is the synthetic-organic alignment pipeline for instruction-tuning. The corpus consists of publicly available single-judge summary judgments from Israeli magistrate and district courts, targeting Hebrew—a morphologically rich, low-resource language, thereby stress-testing standard benchmarks and transferability. The pipeline utilizes agentic workflows leveraging multiple LLMs to:

• Segment verdicts into explicit reasoning sentences,
• Generate synthetic questions mapped to reasoning statements,
• Validate both extraction and question fidelity.

This yields 62,051 question-reason pairs across 29 judges, empirically validated for quality (Gwet's AC1 of 0.75 for statement extraction, 0.90 for question generation; 83% reasoning, 91% question alignment). The structured dataset supports instruction-tuning without manual annotation, significantly reducing the supervision bottleneck.

Personalization Strategies and Fine-Tuning Protocols

The study operationalizes personalization via Parameter-Efficient Fine-Tuning (PEFT), specifically QLoRA adapters on multilingual Gemma-3 models at 1B and 4B parameter scales:

• Causal Language Modeling (CLM): Fine-tunes LoRA adapters per judge on raw judgements.
• Instruction Tuning: Further adapts each judge's model on synthetic question-reason pairs.
• Chain-of-LoRA (CoLA): Residual learning by merging the CLM adapter into base weights before instruction-tuning, as in chain-of-residuals [xia2024chainloraefficientfinetuning].
• Retrieval-Augmented Generation (RAG): In-context retrieval from judge-specific instruction pairs, evaluated with Gemini-3-Pro and Gemma-3, at inference, without parameter updates.

Judicial fidelity is assessed via lexical, semantic, and stylistic metrics (BLEU, ROUGE-L, BERTScore, POS-JSD), as well as authorship discernment classifiers using DictaBERT.

Quantitative Evaluation and Specificity

The personalized CoLA-4B model delivers consistent, statistically significant improvements over baselines on both question-answering and next-token prediction tasks. Lexical, semantic, and stylistic metrics all indicate judge-level adaptation efficacy:

• BLEU and ROUGE-L gains are pronounced relative to vanilla and multi-judge baselines.
• BERTScore indicates robust semantic fidelity.
• POS-JSD reduction (for most baselines), although less specific than content measures.
• In cross-judge evaluation, performance gaps ($\Delta_j$) strongly favor judge-matched models, evidencing specificity beyond generic improvement.

  Figure 1: Cross-judge specificity visualized; personalized CoLA-4B models outperform non-matched models on their respective judge’s test sets, with lexical and semantic measures predominantly positive.

Authorship classifiers reveal a critical dichotomy: baseline models (vanilla, multi-judge, RAG) remain readily distinguishable from real judge writing (accuracy in 70–76% range). CoLA and instruction-tuning reduce accuracy to chance (∼50%), rendering generated outputs indistinguishable from authentic judge sentences.

Ablation Analysis: Data and Adapter Efficacy

Ablations demonstrate:

• Performance plateaus after modest data increases (50% of available examples); cross-judge semantic and stylistic alignment improve sharply with increased supervision signal.
• LoRA rank effects are subject to diminishing returns; substantial improvements occur at the lowest rank ($r=2$), plateauing rapidly.
• Data quantity uptrumps adapter size in effect magnitude, suggesting personalization hinge predominantly on supervision structure, not model capacity.

Qualitative and Robustness Assessment

Blind human evaluations show 71–80% positive ratings for generated answers in legal responsiveness, coherence, and authenticity. Failure modes include overly terse, generic, or irrelevant responses. Catastrophic forgetting is ruled out: CLM and CoLA maintain perplexity on Hebrew out-of-domain text and general accuracy on English commonsense benchmarks.

Practical and Theoretical Implications

The findings challenge assumptions that reasoning personalization requires massive corpora and compute; synthetic-organic instruction signals allow models as small as 4B parameters with LoRA adapters to capture "cognitive fingerprints" in low-resource settings. Theoretically, the paper elucidates the surface-style/deep-reasoning dichotomy: RAG excels at stylistic mimicry but underperforms on substantive legal reasoning, whereas parametric adaptation (CoLA) achieves the inverse. This bifurcation points to layer-wise persona modeling, optimal for simultaneous stylistic and epistemic fidelity.

Figure 2: Illustrative demonstration of two personalized LLMs emulating distinct judge philosophies—textualist and purposivist—on the same legal question.

Limitations and Ethical Concerns

The study does not address sparse case-level reasoning or shifts in judge philosophy over time. Strict legal correctness is not enforced at the personalization objective; outputs are tailored for fidelity rather than factuality. Ethical safeguards include removal of identities, non-distribution of judge-specific adapters, and explicit disclaimers regarding misuse and bias amplification.

Conclusion

Personalization of LLMs to emulate judicial reasoning signatures is achievable efficiently in low-resource languages when leveraging synthetic-organic supervision. CoLA-style sequential adaptation on well-structured instructions outperforms prior baselines in Hebrew legal text, yielding judge-level models indistinguishable from authentic court writings. The research unlocks avenues for personalized expert modeling in domains with dense reasoning traces and challenges foundational assumptions on the scale and structure required for cognitive personalization. Future directions include expansion to other expert domains, hybrid optimization of style and reasoning, and integration with planning-based tasks.