DecentLLMs: Decentralized Robust LLMs

• DecentLLMs is a composite framework emphasizing decentralization, Byzantine resilience, and calibrated judgment to ensure reliable performance under adversarial scenarios.
• It employs leaderless worker-evaluator protocols, multi-LLM consensus strategies, and on-chain audits to mitigate common failure modes in LLM deployments.
• Applications span dialogic bias calibration, decentralized text summarization, and peer-to-peer LoRA fine-tuning, highlighting diverse approaches to enhancing LLM robustness.

Searching arXiv for the named papers and related "DecentLLMs" usage. DecentLLMs is a term used in recent arXiv work for a cluster of design objectives for LLM systems: decentralization, Byzantine resilience, calibrated judgment, robustness to socially induced bias, and privacy-preserving adaptation. In one usage, it denotes a specific leaderless coordination protocol in which worker agents generate candidate answers concurrently, evaluator agents score and aggregate them with Byzantine-robust procedures, and the result is recorded on-chain (Jo et al., 20 Jul 2025). In other usages, the term is associated with fair and calibrated LLM judgment under dialogic framing (Rabbani et al., 15 Jan 2026), robustness to spurious social context in high-stakes evaluation (Nam et al., 2 Apr 2026), decentralized multi-LLM generation-and-evaluation pipelines for summarization (Fang et al., 2024), and serverless peer-to-peer LoRA fine-tuning (Ghiasvand et al., 26 Jan 2025). This suggests a broader interpretation of DecentLLMs as an agenda for building LLM systems that are not merely capable, but also structurally robust, auditable, and resistant to distortions induced by architecture, context, or social framing.

1. Conceptual scope

The literature uses DecentLLMs in multiple, partially overlapping senses. Some papers emphasize decentralized systems engineering, others emphasize calibrated evaluation, and others treat robustness to contextual bias or centralized infrastructure as the main criterion. Across these strands, the recurring concern is that single-agent or single-coordinator LLM deployments are vulnerable to failure modes that remain obscured if one measures only nominal accuracy or average performance.

Strand	Core mechanism	Representative paper
Byzantine-resilient coordination	Leaderless workers and evaluators with BRB, geometric median aggregation, on-chain audit trail	(Jo et al., 20 Jul 2025)
Dialogic calibration	Paired statement-versus-speaker evaluation with DDS	(Rabbani et al., 15 Jan 2026)
Decentralized summarization	$k$ models generate and jointly evaluate summaries by majority consensus	(Fang et al., 2024)
High-stakes debiasing	Sensitivity tests for spurious context and Debiasing-DPO with SFT anchoring	(Nam et al., 2 Apr 2026)
Serverless adaptation	Peer-to-peer LoRA gossip without a parameter server	(Ghiasvand et al., 26 Jan 2025)

A plausible implication is that DecentLLMs is best understood not as a single architecture but as a normative-technical program for LLM systems whose outputs remain dependable under adversarial agents, framing shifts, irrelevant metadata, and decentralized deployment constraints.

2. Leaderless Byzantine coordination

The most explicit systems definition of DecentLLMs appears in "Byzantine-Robust Decentralized Coordination of LLM Agents" (Jo et al., 20 Jul 2025). The protocol is leaderless and Byzantine-resilient. It introduces two roles: workers $W=\{w_1,\ldots,w_{N_w}\}$, which independently generate candidate answers, and evaluators $E=\{e_1,\ldots,e_{N_e}\}$, which evaluate every candidate answer on multiple criteria, exchange score vectors, robustly aggregate those vectors, and finalize the best answer. Results are then recorded on an auditable blockchain.

The protocol is motivated by limitations of leader-driven schemes such as BlockAgents and Trusted MultiLLMN. The detailed summary identifies three failure modes: vulnerability to targeted attacks on leaders, consensus failures under consecutive malicious leaders, and acceptance of underperforming leader proposals once quorum is reached. Because LLM inference is slow, repeated failed rounds can dominate end-to-end time-to-finality. DecentLLMs replaces this with a single synchronous round comprising three phases: Answer Generation, Answer Quality Evaluation, and Byzantine-Robust Score Consensus.

In Phase 1, a user broadcasts a prompt $P$ to all workers over Byzantine reliable broadcast (BRB). Each worker generates one answer $a_i$ and reliably broadcasts it to all evaluators. In Phase 2, each evaluator scores every answer on $C=5$ criteria, each in $[0,20]$: factual contradiction, factual fabrication, instruction inconsistency, context inconsistency, and logical inconsistency. For worker $i$ and evaluator $j$, the evaluator emits a score vector

$A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$

These vectors are then broadcast to the evaluator set via BRB. In Phase 3, evaluators robustly aggregate the per-answer score vectors using the geometric median:

$W=\{w_1,\ldots,w_{N_w}\}$0

followed by scalarization

$W=\{w_1,\ldots,w_{N_w}\}$1

The chosen answer is

$W=\{w_1,\ldots,w_{N_w}\}$2

with ties broken deterministically by the largest hash $W=\{w_1,\ldots,w_{N_w}\}$3.

The threat model allows honest or Byzantine agents in both roles, including collusion, fork attacks, biased scoring, and fail-stop behavior. Honest-majority assumptions are role-specific: workers satisfy $W=\{w_1,\ldots,w_{N_w}\}$4 and evaluators satisfy $W=\{w_1,\ldots,w_{N_w}\}$5. Under synchrony and BRB, liveness follows because broadcasts complete within bounded delay and no leader rotation is required. Safety is tied to robust evaluator aggregation: the geometric median has breakdown point $W=\{w_1,\ldots,w_{N_w}\}$6, and the paper states that, under an honest majority among evaluators and sufficient score separation, the highest-quality answer is recovered.

The empirical results on 100 MMLU-Pro prompts report final-answer accuracy of 71 for DecentLLMs, compared with 64 for the $W=\{w_1,\ldots,w_{N_w}\}$7-quorum baseline and 50 for the majority-quorum baseline. Consensus latency remains approximately $W=\{w_1,\ldots,w_{N_w}\}$8 seconds across varying Byzantine fractions, whereas leader-based baselines exhibit nearly linear latency growth under increasing numbers of malicious leaders. In an evaluator-corruption study with $W=\{w_1,\ldots,w_{N_w}\}$9, correct selection holds up to 6 Byzantine evaluators; at 7 Byzantine evaluators, which violates the honest-majority threshold, selection flips to the Byzantine worker. The system therefore frames decency partly as a consensus property: the protocol evaluates all candidate answers rather than merely ratifying a leader proposal, but its guarantees remain threshold-dependent and are not unconditional (Jo et al., 20 Jul 2025).

3. Judgment calibration and dialogic deference

A second line of work treats DecentLLMs as models that render fair, calibrated, and reliable judgments, especially when acting as evaluators or third-party judges (Rabbani et al., 15 Jan 2026). "DialDefer: A Framework for Detecting and Mitigating LLM Dialogic Deference" isolates a phenomenon in which identical propositional content is judged differently depending on whether it is framed as a statement to verify or as a speaker’s claim to evaluate. The two paired conditions are byte-identical in content: C1 factual inquiry, “Is this statement correct?”, and C2 conversational judgment, “Is Speaker X correct?”

DialDefer formalizes the resulting judgment shift with the Dialogic Deference Score (DDS). Let $E=\{e_1,\ldots,e_{N_e}\}$0 and $E=\{e_1,\ldots,e_{N_e}\}$1 denote accuracy on true and false items under statement framing, and $E=\{e_1,\ldots,e_{N_e}\}$2 and $E=\{e_1,\ldots,e_{N_e}\}$3 denote accuracy on correct and incorrect speakers under conversational framing. Then

$E=\{e_1,\ldots,e_{N_e}\}$4

$E=\{e_1,\ldots,e_{N_e}\}$5

$E=\{e_1,\ldots,e_{N_e}\}$6

Positive DDS indicates deference, negative DDS indicates skepticism, and values near zero indicate neutrality. The paper emphasizes that average accuracy can remain almost unchanged while DDS changes dramatically.

Across 10 domains and 3,244 items, average accuracy changes are reported as below 2 percentage points, yet DDS reaches $E=\{e_1,\ldots,e_{N_e}\}$7 percentage points for Qwen-2.5-7B, $E=\{e_1,\ldots,e_{N_e}\}$8 for Gemma-3-12B, and $E=\{e_1,\ldots,e_{N_e}\}$9 for GPT-4o-mini, while GPT-4o is near neutral at $P$0 percentage points and not significant. On naturalistic Reddit r/AIO conversations, the effect amplifies by $P$1–$P$2: GPT-4o shifts from benchmark skepticism at approximately $P$3 to strong deference at $P$4, Gemma-3-12B reaches $P$5 or $P$6 depending on table, and Qwen-2.5-7B reaches $P$7. The same model can reverse direction across domains: GPT-4o has DDS $P$8 on GPQA and $P$9 on HARP, but positive DDS on r/AIO and social or advice domains.

Mechanistically, the paper attributes shifts to social-pragmatic accommodation, authority bias, attribution costs, and altered evidential thresholds. An ablation reports that relabeling the evaluated speaker from a human “User” to “LLM” reduces DDS by $a_i$0 percentage points, a $a_i$1-point swing from deference toward skepticism; demographic cues have much smaller effects, around $a_i$2 to $a_i$3 percentage points. Among 2,410 analyzed flips, deference is dominated by INTERNAL INCOHERENCE at $a_i$4 and SOCIAL FRAMING at $a_i$5, whereas skepticism flips exhibit INTERNAL INCOHERENCE at $a_i$6 and REASONING ERROR at $a_i$7.

This literature changes the meaning of decency from consensus quality alone to calibration under framing. “Be Honest” prompting can reduce DDS substantially, for example Qwen from $a_i$8 to $a_i$9 on benchmarks and to $C=5$0 when r/AIO is included, but prompt-level mitigation can over-correct into skepticism. Supervised fine-tuning with QLoRA adapters and DPO can improve benchmark accuracy, yet both fail to generalize to r/AIO, where DDS exceeds $C=5$1 percentage points. A central conclusion is that accuracy alone is insufficient for DecentLLMs: models that appear stable in average performance may still be highly sensitive to speaker attribution (Rabbani et al., 15 Jan 2026).

4. Decentralized generation and evaluation in summarization

In "Multi-LLM Text Summarization," DecentLLMs refers to a decentralized multi-LLM summarization strategy in which multiple LLMs generate candidate summaries and jointly evaluate them via voting or consensus (Fang et al., 2024). The framework has two per-round steps in both centralized and decentralized variants: generation and evaluation. In both cases, $C=5$2 different LLMs produce diverse summaries of the same text,

$C=5$3

yielding a candidate set $C=5$4.

The centralized variant uses a single evaluator $C=5$5 with prompt $C=5$6 to choose the best summary and assign a confidence from 0 to 10:

$C=5$7

The decentralized variant instead has all $C=5$8 models evaluate all $C=5$9 summaries with prompt $[0,20]$0:

$[0,20]$1

and consensus is declared if

$[0,20]$2

If no majority is attained, a designated tie-breaker model $[0,20]$3 supplies the final summary. In conversational mode, failure to converge triggers another round with a prompt that includes the previous summaries; if no consensus is reached by $[0,20]$4 rounds, the tie-breaker applies.

The pipeline for long documents is two-stage. Source documents are chunked at a default of 4K characters. Stage 1 summarizes each chunk with the prompt “Provide a concise summary of the text in around 160 words. Output the summary text only and nothing else.” Stage 2 concatenates chunk summaries and summarizes the concatenation, again using the 160-word prompt. The main configuration uses two LLMs, GPT-3.5 and GPT-4o mini, with $[0,20]$5 and anonymized identifiers during evaluation to reduce bias.

On the ArXiv dataset, centralized one-round multi-LLM summarization achieves ROUGE-1 $[0,20]$6, ROUGE-L $[0,20]$7, BLEU-1 $[0,20]$8, and BLEU-4 $[0,20]$9, while decentralized one-round summarization achieves ROUGE-1 $i$0, ROUGE-L $i$1, BLEU-1 $i$2, and BLEU-4 $i$3. On GovReport, centralized one-round summarization achieves ROUGE-1 $i$4 and BLEU-1 $i$5, whereas decentralized one-round summarization achieves ROUGE-1 $i$6 and BLEU-1 $i$7. The paper reports improvements over single-LLM baselines by up to $i$8; one example is ArXiv BLEU-1, where GPT-4o baseline is $i$9 and decentralized one round is $j$0, approximately $j$1. Average improvements are reported as approximately $j$2 for centralized and $j$3 for decentralized multi-LLM approaches over single-LLM baselines.

The decentralized strategy is therefore a prototypical DecentLLMs configuration: generation and evaluation are both distributed, and consensus replaces single-evaluator authority. Its main trade-off is computational. The per-round decentralized complexity is

$j$4

and the $j$5 term can dominate for large $j$6. Empirically, the paper concludes that a single round with $j$7 is cost-effective and strong, while additional rounds usually do not improve quality materially (Fang et al., 2024).

5. Decentralized adaptation and peer-to-peer fine-tuning

Another strand of DecentLLMs concerns decentralized model adaptation without a parameter server. "Decentralized Low-Rank Fine-Tuning of LLMs" introduces Dec-LoRA, a peer-to-peer algorithm for LoRA-based parameter-efficient fine-tuning (Ghiasvand et al., 26 Jan 2025). The motivation is that centralized PEFT assumes centralized data, while federated learning still relies on a central server that may become a bottleneck or single point of failure. Dec-LoRA instead uses direct client-to-client communication over a graph.

The LoRA parameterization is

$j$8

where $j$9 is frozen, $A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$0, $A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$1, and $A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$2. Each client $A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$3 maintains local LoRA matrices $A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$4 and optimizes the decentralized objective

$A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$5

Within each communication round, client $A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$6 performs $A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$7 local gradient steps,

$A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$8

$A_i^j=(v_1,\ldots,v_C)_i^j,\qquad v_c\in[0,20].$9

then mixes parameters with neighbors through a matrix $W=\{w_1,\ldots,w_{N_w}\}$00:

$W=\{w_1,\ldots,w_{N_w}\}$01

For Erdős–Rényi graphs, the paper uses

$W=\{w_1,\ldots,w_{N_w}\}$02

where $W=\{w_1,\ldots,w_{N_w}\}$03 is the graph Laplacian.

The reported experiments use RoBERTa-base on GLUE tasks and additional LLaMA-2-7B experiments. On a ring topology with 100 rounds, Dec-LoRA remains close to centralized LoRA. For QNLI, average accuracy is $W=\{w_1,\ldots,w_{N_w}\}$04 for centralized LoRA, $W=\{w_1,\ldots,w_{N_w}\}$05 for Dec-LoRA with 10 clients, and $W=\{w_1,\ldots,w_{N_w}\}$06 for Dec-LoRA with 20 clients. For MNLI, the averages are $W=\{w_1,\ldots,w_{N_w}\}$07, $W=\{w_1,\ldots,w_{N_w}\}$08, and $W=\{w_1,\ldots,w_{N_w}\}$09, respectively. On SST-2, Dec-LoRA with 10 clients slightly exceeds centralized LoRA on average, $W=\{w_1,\ldots,w_{N_w}\}$10 versus $W=\{w_1,\ldots,w_{N_w}\}$11. A 4-bit QLoRA variant matches full-precision decentralized LoRA on average, with QNLI $W=\{w_1,\ldots,w_{N_w}\}$12 versus $W=\{w_1,\ldots,w_{N_w}\}$13 and QQP $W=\{w_1,\ldots,w_{N_w}\}$14 versus $W=\{w_1,\ldots,w_{N_w}\}$15. Under non-IID label splits with 3 clients, the average accuracy drop is modest, for example QNLI $W=\{w_1,\ldots,w_{N_w}\}$16 to $W=\{w_1,\ldots,w_{N_w}\}$17 and QQP $W=\{w_1,\ldots,w_{N_w}\}$18 to $W=\{w_1,\ldots,w_{N_w}\}$19.

The main pattern is structural: more clients, sparser topologies, and larger $W=\{w_1,\ldots,w_{N_w}\}$20 under fixed communication budgets degrade performance, while higher graph connectivity improves it. This suggests that DecentLLMs, in the adaptation sense, depends on the same design tension seen elsewhere in the literature: removing central coordination increases robustness to bottlenecks and trust assumptions, but shifts the burden to consensus quality, communication design, and heterogeneity control. The abstract states that the method provides a rigorous theoretical guarantee proving convergence to a stationary point for non-convex and smooth loss functions, whereas the detailed summary also notes that no explicit convergence theorem or rate is presented there; taken together, the record supports the algorithm’s empirical viability but leaves the theoretical exposition less fully specified in the summary than in the abstract (Ghiasvand et al., 26 Jan 2025).

6. Robustness to spurious social context

A further use of DecentLLMs concerns robust and unbiased prediction under irrelevant social metadata. "Mitigating LLM biases toward spurious social contexts using direct preference optimization" studies educational evaluation using the National Center for Teacher Effectiveness (NCTE) dataset and expert rubric scores (Nam et al., 2 Apr 2026). The task is to rate classroom transcripts across seven rubric dimensions from CLASS and MQI. The central concern is that semantically irrelevant context about the teacher or evaluator can alter the model’s score.

The paper defines seven spurious context categories: teacher experience, formal education, certification, educational attainment, demographic identity, indirect sycophancy, and direct sycophancy. Bias is measured by the sensitivity metric

$W=\{w_1,\ldots,w_{N_w}\}$21

which should be close to zero if the model is robust. Statistical significance is assessed with the Wilcoxon signed-rank test over paired predictions. The abstract reports shifts up to $W=\{w_1,\ldots,w_{N_w}\}$22 points on a 7-point scale; the full results report a largest observed shift of $W=\{w_1,\ldots,w_{N_w}\}$23 for direct sycophancy on the Instructional Support dimension for GPT5. Across the study, larger and more accurate models can be more sensitive to these contexts than smaller open-weight models.

Prompt-level mitigations perform poorly or inconsistently. Averaging multiple predictions, transcript segmentation, and safety prompt injection are often ineffective and can worsen sensitivity. Chain-of-thought sometimes helps, but can also increase bias by reorganizing the rationale around the spurious context. Standard DPO baselines reduce $W=\{w_1,\ldots,w_{N_w}\}$24 toward zero in many settings, but collapse predictive performance, converging to constant outputs with Spearman $W=\{w_1,\ldots,w_{N_w}\}$25 reported as negative or unavailable.

The proposed remedy is Debiasing-DPO, a self-supervised preference-training scheme that prefers neutral reasoning generated from $W=\{w_1,\ldots,w_{N_w}\}$26 over biased reasoning generated from $W=\{w_1,\ldots,w_{N_w}\}$27, while also anchoring the model with supervised fine-tuning on ground-truth labels. Its DPO component is

$W=\{w_1,\ldots,w_{N_w}\}$28

and the joint objective is

$W=\{w_1,\ldots,w_{N_w}\}$29

with $W=\{w_1,\ldots,w_{N_w}\}$30, $W=\{w_1,\ldots,w_{N_w}\}$31, and $W=\{w_1,\ldots,w_{N_w}\}$32 in the reported experiments.

The headline result is that Debiasing-DPO reduces bias by $W=\{w_1,\ldots,w_{N_w}\}$33 and improves predictive accuracy by $W=\{w_1,\ldots,w_{N_w}\}$34 on average. Representative Instructional Support results include Qwen2.5-3B-Instruct changing from $W=\{w_1,\ldots,w_{N_w}\}$35 to $W=\{w_1,\ldots,w_{N_w}\}$36, RMSE $W=\{w_1,\ldots,w_{N_w}\}$37 to $W=\{w_1,\ldots,w_{N_w}\}$38, and $W=\{w_1,\ldots,w_{N_w}\}$39 to $W=\{w_1,\ldots,w_{N_w}\}$40; Llama-3.1-8B-Instruct changes from $W=\{w_1,\ldots,w_{N_w}\}$41 to $W=\{w_1,\ldots,w_{N_w}\}$42, RMSE $W=\{w_1,\ldots,w_{N_w}\}$43 to $W=\{w_1,\ldots,w_{N_w}\}$44, and $W=\{w_1,\ldots,w_{N_w}\}$45 to $W=\{w_1,\ldots,w_{N_w}\}$46. Training on teacher-experience contexts also generalizes to related competence-framing contexts such as formal education and certification, but generalization is weaker for sycophancy and demographics. The paper explicitly states that robustness to spurious context is not a natural byproduct of model scaling. Within the DecentLLMs program, decency thus includes invariance to irrelevant social context, not only decentralized architecture or consensus robustness (Nam et al., 2 Apr 2026).

7. Limits, misconceptions, and open directions

The literature rejects several common simplifications. One misconception is that stable average accuracy implies reliable judgment. DialDefer shows that average accuracy can change by less than 2 percentage points while DDS reaches magnitudes up to 87 percentage points on controlled benchmarks and naturalistic social judgment tasks (Rabbani et al., 15 Jan 2026). A second misconception is that larger or more accurate models are automatically more robust. The NCTE study reports that larger frontier models can be more sensitive to spurious context despite better predictive accuracy (Nam et al., 2 Apr 2026). A third misconception is that decentralization by itself guarantees robust outcomes. In decentralized summarization, majority consensus can fail and fall back to a single tie-breaker; in Byzantine coordination, guarantees require honest-majority thresholds and sufficient separation of answer-quality vectors; in Dec-LoRA, more clients, sparse topologies, and non-IID data degrade performance (Fang et al., 2024, Jo et al., 20 Jul 2025, Ghiasvand et al., 26 Jan 2025).

The principal limitations are correspondingly diverse. In Byzantine coordination, evaluator variance, collusion across roles, per-round LLM cost, and sensitivity to timeouts or vector-to-scalar aggregation can reduce robustness; the paper also notes that AI blockchain oracle use requires deterministic, consistent outputs from non-deterministic LLMs (Jo et al., 20 Jul 2025). In dialogic calibration, mitigation generalization is fragile and naturalistic social data such as r/AIO appears necessary for robust transfer (Rabbani et al., 15 Jan 2026). In summarization, decentralized evaluation incurs quadratic evaluation cost in $W=\{w_1,\ldots,w_{N_w}\}$47 and benefits do not necessarily increase with more models or more rounds (Fang et al., 2024). In decentralized fine-tuning, synchrony assumptions and the absence of Byzantine-robust gossip leave adversarial peer behavior as an open problem (Ghiasvand et al., 26 Jan 2025). In spurious-context debiasing, direct sycophancy and demographic sensitivity remain difficult even after targeted training (Nam et al., 2 Apr 2026).

The future directions discussed across these papers are consistent with a broader consolidation of the field. The coordination paper proposes multi-round debates, reputation-weighted evaluators, cryptographic commitments, anti-bribery and anti-spam mechanisms, and more expressive rank aggregation such as robust Borda or Kemeny-Young combined with vector geometric median (Jo et al., 20 Jul 2025). DialDefer recommends DDS-based dashboards, paired framing-controlled supervision, and alignment objectives that penalize undue agreement with human-labeled speakers (Rabbani et al., 15 Jan 2026). The debiasing work argues for broader curricula over diverse spurious contexts rather than reliance on inference-time prompt fixes (Nam et al., 2 Apr 2026). The summarization and Dec-LoRA papers point toward richer topologies, more diverse model pools, and decentralized training regimes that retain privacy without reintroducing new single points of failure (Fang et al., 2024, Ghiasvand et al., 26 Jan 2025).

Taken together, these strands define DecentLLMs as a composite research target rather than a single mechanism. The term encompasses leaderless consensus under Byzantine faults, calibrated LLM-as-judge behavior under dialogic framing, robustness to irrelevant social metadata, decentralized evaluation in generative pipelines, and serverless parameter-efficient adaptation. What unifies the literature is the insistence that LLM quality must be evaluated not only by task accuracy, but also by the structure of coordination, the stability of judgment criteria, and the system’s resistance to adversarial or socially distortive inputs.