Knowledge-Free Correlated Agreement for Incentivizing Federated Learning

Abstract: We introduce Knowledge-Free Correlated Agreement (KFCA) to reward client contributions in federated learning (FL) without relying on ground truth, a public test set, or distribution knowledge. Under categorical reports and an honest majority, KFCA is strictly truthful, addressing the label-flipping vulnerability of Correlated Agreement (CA). We evaluate KFCA on federated LLM adapter tuning and a real-world PCB inspection task, showing efficient real-time reward computation suitable for decentralized and blockchain-based incentive designs.

• The paper introduces KFCA, a novel mechanism that incentivizes federated learning without the need for external ground truth by leveraging categorical report agreement.
• The paper demonstrates that KFCA achieves strict truthfulness and robustness against label-flipping attacks while operating with linear computational complexity compared to quadratic alternatives.
• The paper validates KFCA through empirical benchmarks in real-world tasks such as PCB inspection and federated LLM adapter fine-tuning, and outlines its integration with blockchain for decentralized deployment.

Knowledge-Free Correlated Agreement: Incentivizing Federated Learning without Ground Truth

Introduction and Motivation

Federated learning (FL) fundamentally changes collaborative machine learning by enabling decentralized clients to jointly optimize a global model while preserving data privacy. A longstanding barrier to robust, scalable FL deployment, however, is the design of incentive mechanisms that both reliably measure individual client contributions and offer provable guarantees against low-effort or adversarial participation. Traditional methodologies—most notably Shapley value computations—require external ground truth, are computationally prohibitive for large client pools, and are often infeasible in scenarios lacking a public test set.

Statistical peer-prediction mechanisms such as Correlated Agreement (CA) have been explored for bypassing ground truth, scoring clients based on the statistical dependencies in their reports rather than direct performance evaluation. However, CA is hampered by two critical practical limitations: quadratic overhead due to the need for joint report correlation matrix estimation, and severe vulnerability to coordinated label-flipping attacks, which can subvert the incentive structure.

This work introduces Knowledge-Free Correlated Agreement (KFCA), a peer-prediction mechanism for FL that is strictly truthful, label-flip robust, and requires no knowledge of the latent report-generation distributions. KFCA leverages a categorical-world condition—a mild constraint on report correlation structure typically satisfied in standard FL regimes—to achieve incentive alignment and computational efficiency that is well-suited for decentralized and real-time reward computation.

Multi-Task Peer Prediction Framework for FL

KFCA is grounded in the multi-task peer prediction (MTPP) paradigm, which designs payment rules eliciting truthful and effortful behavior by comparing correlated reports across multiple tasks. An FL client participates in several classification tasks (e.g., image labeling or LLM adapter fine-tuning), yielding signals based on the task's unobserved latent truth (such as the correct class or optimal parameter direction). The core goal of the MTPP incentive mechanism is to ensure that clients maximize their expected reward only by exerting effort (e.g., model training) and reporting their private signal truthfully, even in the absence of observable ground truth.

Figure 1: Illustration of the multi-task peer prediction mechanism (MTPP) as instantiated by KFCA for both public test set and sign-quantized update regimes.

A critical definition is the delta matrix $\Delta$, which, for each pair of possible categorical reports, encodes excess correlation above what would be expected had the reports been independent. The expected client reward is then expressed as a function of this delta matrix and the scoring rule applied to reports from each pair of clients. In CA, the scoring rule requires estimating the full $\Delta$; in KFCA, this is circumvented by exploiting structural properties of $\Delta$ induced by how FL client signals are generated.

Limitations of Correlated Agreement

CA, while optimal when the full report distribution is known, encounters two issues that undermine its practical use in federated settings. First, it mandates global access to all client reports for per-round correlation estimation, fundamentally conflicting with the privacy and decentralization ethos of FL. Second—and more seriously—it is vulnerable to informed coordinated attacks such as global label permutation (label flipping): CA rewards all strategies preserving the sign pattern of $\Delta$ equally, meaning even fully adversarial, globally coordinated label-flipping clients receive maximal reward, as shown both analytically and empirically.

Figure 2: Under a label-flip attack, CA fails to penalize malicious reports, while KFCA robustly differentiates honest and adversarial participants by reward.

Knowledge-Free Correlated Agreement (KFCA) Mechanism

KFCA introduces a significant simplification: it rewards clients only for categorical report agreement, bypassing the need for $\Delta$ estimation entirely. This is theoretically justified under the categorical-world condition—a sign constraint on the report correlation matrix: $\Delta(a, a) > 0$ and $\Delta(a, b) < 0$ for $a \ne b$. This holds whenever matching reports from different clients are positively correlated (as is typical in correctly functioning FL) and mismatches are negatively correlated.

Figure 3: KFCA logistics for both public test set (KFCA-D) and quantized parameter update (KFCA-QP) instantiations, illustrating how non-categorical raw signals are transformed into categorical alignments suitable for knowledge-free agreement.

Strong Theoretical Guarantees:

• Strict Truthfulness: Under $\Delta$ satisfying the categorical-world condition, any truthful (or uniformly permuted) strategy uniquely maximizes expected reward. Any deviation (including label flipping, partial manipulation, or lagged reporting) is strictly suboptimal unless a coalition exceeding 50% of clients colludes on a global relabeling, matching the standard Byzantine-fault-tolerance threshold.
• Robustness and Decentralization: KFCA requires only local, per-peer comparisons for real-time reward computation, avoiding quadratic client scaling or delayed payout, and thus is deployable in both centralized and fully decentralized (blockchain-based) infrastructures.

Practical Deployment and Empirical Results

Computational Efficiency Relative to Shapley and CA

Strong claims are made regarding computational resource requirements: KFCA's cost is linear in the number of clients and peers, contrasted with the quadratic scaling of CA and the exponential bottleneck of Shapley value estimators. Empirical benchmarking on standard FL benchmarks (e.g., MNIST) confirms orders-of-magnitude speedup while maintaining high statistical agreement with exact Shapley value orders, especially in identifying both high- and low-quality contributors.

Real-World and LLM-Focused Applications

Industrial PCB Inspection

KFCA-QP is deployed in a real factory pipeline for PCB defect detection, where pooling client data is infeasible and public ground truth is unavailable. By quantizing client parameter updates, KFCA-QP assigns substantially higher reward to high-quality clients and penalizes low-quality/poorly performing sites, in alignment with intuitive ground-truth contribution, but without the need for a test set.

Federated LLM Adapter Fine-Tuning

In cross-domain federated LLM tasks evaluated on the FlowerTune leaderboard, KFCA-QP is instantiated with 1-bit quantized LoRA/DoRA adapter updates and per-round peer comparison, covering challenging domains such as general NLP, financial sentiment, medical Q&A, and code synthesis.

Figure 4: Per-round FL pipeline for federated LLM fine-tuning with quantized LoRA/DoRA adapters and KFCA-QP scoring.

Across all domains and attack scenarios—including adversarial sign flips, random/zero attacks, and sparse/lagged reporting—KFCA-QP displays clear, monotonic incentive compatibility:

Figure 5: KFCA-QP reward trajectories: honest reporting consistently achieves the highest rewards, while adversarial and low-effort strategies are strictly penalized, validating incentive compatibility across federated LLM fine-tuning tasks.

Blockchain-enabled Decentralized FL

The lightweight, knowledge-free design of KFCA enables direct integration into blockchain smart contracts, supporting decentralized, auditable, and real-time reward distribution without reliance on centralized servers or heavy off-chain computation. Figure 6 demonstrates the on-chain workflow for committing, verifying, and rewarding client contributions using KFCA in a decentralized setting.

Figure 6: Decentralized and incentivized FL using KFCA on blockchain infrastructure, detailing client registration, report commitment, unbiased peer pairing, and automated on-chain reward distribution.

Implications and Future Directions

Pragmatically, KFCA unlocks scalable, robust, and privacy-preserving incentive mechanisms for federated learning in scenarios where ground truth is either unavailable or fundamentally ill-defined. This includes privacy-sensitive domains (healthcare, finance), industrial anomaly detection, and large-scale LLM adaptation to proprietary data. Its strong theoretical guarantees ensure strict incentive compatibility up to standard FL security thresholds, and its computational tractability positions it as a practical building block for FL on public or consortium blockchains.

Theoretically, the work highlights the full exploitability of peer-prediction under the categorical-world condition and sets a new lower bound on the operational complexity of FL incentive mechanisms in the absence of ground truth.

Future developments will likely explore:

• Further relaxing the categorical-world structure through adaptive or partially supervised transformation pipelines for more heterogeneous signal spaces;
• On-chain aggregation and further decentralization of the FL process using verifiable computation and MPC/TEE primitives;
• Robustness against advanced Sybil and collusion attacks in permissionless and open FL ecosystems.

Conclusion

This paper formalizes and instantiates Knowledge-Free Correlated Agreement as a practical and theoretically sound mechanism for incentivizing FL without external ground truth. KFCA's ability to combine strict truthfulness, robustness to informed attacks, and real-time computation positions it as an enabling technology for the next generation of decentralized, privacy-preserving, and incentive-aligned collaborative machine learning systems (2605.04747).