zkLoRA: Zero-Knowledge Proofs for LoRA

• zkLoRA is a protocol family that uses zero-knowledge proofs to verify LoRA-based model adaptations while keeping proprietary weights hidden.
• It encompasses two main schemes: one for ensuring module compatibility during inference and another for end-to-end verifiable fine-tuning in untrusted environments.
• The approach leverages succinct cryptographic proofs and specialized circuits to handle both arithmetic and non-arithmetic operations in Transformer architectures.

zkLoRA denotes the use of zero-knowledge proofs for Low-Rank Adaptation (LoRA) in LLM workflows conducted in distributed, decentralized, contract-based, or otherwise untrusted environments. In the current literature, the name has been used for two closely related but distinct objectives: verification that a delivered LoRA module is compatible with a specified base model without exposing proprietary LoRA weights, and end-to-end verifiable LoRA fine-tuning in which forward propagation, backward propagation, and parameter updates are all proven correct while model parameters and training data remain private (Roy et al., 21 Jan 2025, Liao et al., 29 Aug 2025).

1. Terminological scope and problem setting

LoRA is described as a “widely adopted method for customizing large-scale LLMs,” and its parameter-efficient structure makes it a natural target for cryptographic verification because only small trainable low-rank matrices are added to a frozen base model (Roy et al., 21 Jan 2025). The zero-knowledge setting arises when a model owner, base-model user, or client delegates adaptation work to an external party but still requires correctness, privacy, and protection of intellectual property.

Work	Scope	Core claim
"ZKLoRA: Efficient Zero-Knowledge Proofs for LoRA Verification" (Roy et al., 21 Jan 2025)	LoRA verification	Verifies LoRA-base model compatibility without exposing LoRA weights
"zkLoRA: Fine-Tuning LLMs with Verifiable Security via Zero-Knowledge Proofs" (Liao et al., 29 Aug 2025)	LoRA fine-tuning	Provides end-to-end verifiability for forward propagation, backward propagation, and parameter updates

The verification-oriented formulation begins from two explicit requirements: “the base model user must confirm that the LoRA weights are effective when paired with the intended base model,” and “the LoRA contributor must keep their proprietary weights private until compensation is assured” (Roy et al., 21 Jan 2025). The fine-tuning-oriented formulation begins from a broader gap: although LoRA reduces compute and memory relative to full fine-tuning, “ensuring the security and verifiability of fine-tuning under zero-knowledge constraints remains an unresolved challenge” (Liao et al., 29 Aug 2025).

This usage suggests that zkLoRA is best understood not as a single algorithmic primitive, but as a family of protocols that attach cryptographic correctness guarantees to LoRA-centric adaptation pipelines.

2. Verification of proprietary LoRA modules

In the verification protocol, the central task is to prove that a secret LoRA adapter works correctly with a public base model, while revealing nothing about the LoRA parameters beyond correctness of the claimed computation. The paper formalizes a three-party structure around a base model user, a LoRA contributor, and the proof system, with the base model user characterized as “honest-but-curious” and the contributor seeking remuneration and control over intellectual property (Roy et al., 21 Jan 2025).

The protocol unfolds in three steps. First, Multi-Party Inference (MPI) is executed. The base model user runs the public base model locally; when inference reaches a layer containing a LoRA module, partial activations (“Base Acts”) are sent securely to the LoRA contributor. The contributor applies the secret low-rank adapter and returns “LoRA Acts.” The LoRA contribution is written as

$\mathbf{\Delta} = \mathbf{BAx},$

and the resulting module output is

$h = \mathbf{Wx} + \mathbf{BAx}.$

Second, the contributor generates a proof for each LoRA module. The module computation is compiled into a cryptographic circuit, a key setup phase produces the proving key and verification key, and a witness is constructed from the input–output relation induced by the secret adapter. Proof generation is expressed as

$$\mathcal{P}_m = \operatorname{Prove}_{pk}(C_m, w_m).$$

Third, the base model user verifies the proofs for all LoRA modules. Acceptance requires successful verification of every proof:

$$\operatorname{Verify}_{vk}(\mathcal{P}_m) \overset{?}{=} 1.$$

The paper characterizes these guarantees as succinct, deterministic, and zero-knowledge. Succinctness refers to compact proofs and low verification latency; determinism refers to the fact that “the circuit encodes only the LoRA transformation for the observed input, so the proof is tied to the actual computation performed”; and zero-knowledge refers to mathematical assurance that the proofs leak no information about the LoRA parameters (Roy et al., 21 Jan 2025). The protocol also claims compatibility and lineage verification, so that the recipient can confirm that the delivered LoRA module “works as claimed.”

3. End-to-end verifiable LoRA fine-tuning

The fine-tuning framework extends the zero-knowledge objective from adapter verification to the entire learning process. Its stated contribution is “the first framework to integrate LoRA fine-tuning with zero-knowledge proofs (ZKPs), achieving provable security and correctness” (Liao et al., 29 Aug 2025). The setting is explicitly motivated by outsourced fine-tuning and other untrusted environments in which both correctness and privacy are at risk.

The prover–verifier division is direct. The prover, such as an untrusted cloud provider, commits to “all key intermediate values and updates,” while the verifier is assured that “all training steps followed the LoRA protocol exactly, without gaining access to the proprietary data/model” (Liao et al., 29 Aug 2025). Unlike verification-only protocols, this framework covers the complete training loop:

• Forward propagation: query, key, and value projections; residuals; output layers; softmax; SwiGLU; element-wise products; and transpositions.
• Backward propagation: chain-rule-based gradient computation through both arithmetic and non-arithmetic components.
• Parameter update: correctness of the LoRA-specific gradient descent step.

The update rule is written in the paper as

$$A_\ell^{t+1} = A_\ell^t - \eta \frac{\partial \mathcal{L}^t}{\partial A_\ell^t}, \qquad B_\ell^{t+1} = B_\ell^t - \eta \frac{\partial \mathcal{L}^t}{\partial B_\ell^t}.$$

This framework is specific to Transformer-based architectures and emphasizes that verifiable fine-tuning is technically harder than verifiable inference because it must handle “both arithmetic and non-arithmetic operations,” as well as backpropagation and updates (Liao et al., 29 Aug 2025). A plausible implication is that LoRA is especially attractive in this setting because the cryptographic workload can focus on low-rank trainable components rather than full-model adaptation.

4. Cryptographic mechanisms and proof decomposition

The two zkLoRA formulations use different proof architectures, reflecting their different goals. The verification protocol relies on “succinct proofs,” “recursive argument systems such as Nova and HyperNova,” and “incrementally verifiable computation” (Roy et al., 21 Jan 2025). Its unit of proving is the individual LoRA module, and the proof statement is narrowly scoped to the correctness of the LoRA transformation for a given observed activation.

The fine-tuning framework uses a more heterogeneous proof stack. It explicitly combines lookup arguments, sumcheck protocols, and polynomial commitments to verify “both arithmetic and non-arithmetic operations in Transformer-based architectures” (Liao et al., 29 Aug 2025). Arithmetic operations are handled by a matrix-verification protocol based on multilinear extensions and sumcheck. For matrix multiplication, if $C = AB$, the checked relation is

$C[i, j] = \sum_{k=1}^{D_1} A[i, k] B[k, j].$

The multilinear-extension form is then verified through the sumcheck protocol over committed polynomials.

Non-arithmetic operations are handled through lookup-based protocols. The paper names specialized components including zkMat, zkSwiGLU, zkSoftmax, zkElementProd, and zkTranspose (Liao et al., 29 Aug 2025). The lookup design proves that secret outputs belong to valid tables of quantized or decomposed values without revealing the actual values. One of the set-membership identities used in the framework is

$$\sum_{i\in [D]} \frac{1}{\beta+s_i} = \sum_{j\in [N]} m_j \frac{1}{\beta+t_j},$$

where the secret set is checked against a public lookup table.

Privacy is enforced through hiding commitments and the zero-knowledge property itself. The fine-tuning paper states that “frozen weights, LoRA matrices, intermediate activations, gradients, and training data are never revealed,” and that the Fiat–Shamir heuristic is used to obtain fully non-interactive proofs (Liao et al., 29 Aug 2025). By contrast, the verification protocol exposes neither LoRA weights nor full model weights, but does exchange partial activations during MPI (Roy et al., 21 Jan 2025).

5. Empirical characteristics and systems performance

The verification-oriented protocol emphasizes low-latency checking at module granularity. It reports that each LoRA module can be validated in “only 1–2 seconds on state-of-the-art LLMs,” and describes this as enabling “nearly real-time verification” (Roy et al., 21 Jan 2025). The paper further states that with “dozens of LoRA modules (e.g., 80 modules for 70B Llama-3), total verification for all adapters is completed in a few minutes.” Proof generation and circuit setup are more expensive, requiring “tens of seconds per module,” but these costs are incurred once per delivered proof.

The benchmark table in that work reports the following averages:

Model	# LoRAs	Avg Settings / Avg Proof
distilgpt2	24	38.0 s / 31.6 s
Llama-3.3-70B-Instruct	80	54.9 s / 46.9 s
Mixtral-8x7B-Instruct-v0.1	32	86.1 s / 73.7 s

The fine-tuning framework targets larger end-to-end proving workloads and therefore reports substantially different timings. Its implementation uses CUDA acceleration for “elliptic curve arithmetic, polynomial ops, zero-knowledge proofs” and Python orchestration for “high-level logic, dataset management, and model integration” (Liao et al., 29 Aug 2025). Evaluation was run on a server with a “192-core AMD EPYC CPU, 503 GB RAM” and an “NVIDIA A100 GPU (80 GB VRAM).”

The supported model set includes LLaMA-3.2 with “3B, 11B params,” LLaMA-2 with “7B, 13B params,” and OPT with “6.7B, 13B params,” using the C4 dataset (Liao et al., 29 Aug 2025). For one mini-batch step, the paper reports:

• Proof generation time: “LLaMA-3B: 122 sec” and “LLaMA-13B: 249 sec”
• Verification time: “sub-4 seconds”
• Polynomial Commitment time: “up to 550 sec for 13B param models”
• Commitment sizes: “up to ~233 MB”
• GPU Memory: “Peak usage per batch <80 GB”

These measurements distinguish two performance regimes within the zkLoRA literature: rapid verification of already-constructed module proofs, and heavier but still practical proving for full fine-tuning steps.

6. Security guarantees, limitations, and relation to adjacent LoRA research

The two zero-knowledge lines share a common goal—cryptographic assurance for LoRA workflows—but they differ in what is being certified. The verification protocol certifies that a delivered adapter is correct for a designated base model and observed inference path, with “deterministic correctness guarantees” and protection of contributor intellectual property (Roy et al., 21 Jan 2025). The fine-tuning framework certifies the correctness of the learning procedure itself and gives formal guarantees of soundness, completeness, and zero-knowledge (Liao et al., 29 Aug 2025). Its soundness analysis includes the bound

$$\varepsilon_{\mathrm{sound}} \leq \frac{m\cdot d_{\mathrm{max}}}{|\mathbb{F}|} + \frac{C}{|\mathbb{F}|} + \varepsilon_{\mathrm{binding}}.$$

The literature also makes clear that zkLoRA is not synonymous with LoRA fusion, LoRA merging, or zeroth-order LoRA optimization. "K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs" is a “training-free LoRA fusion approach” for diffusion models that compares the Top-K elements in each LoRA to be fused at each attention layer, with the aim of preserving subject and style rather than providing cryptographic guarantees (Ouyang et al., 25 Feb 2025). "Tensorized Clustered LoRA Merging for Multi-Task Interference" addresses heterogeneous multi-task adapter merging through embedding-space clustering and joint Canonical Polyadic decomposition; its discussion of “zkLoRA settings” is explicitly framed as relevance to privacy-preserving or black-box scenarios rather than as a zero-knowledge proof system in itself (Su et al., 6 Aug 2025). "AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning" addresses a different bottleneck—“the rank paradox” in black-box zeroth-order optimization—and restores rank-invariant active finite-difference signal via topology-aware scaling $\gamma=\alpha r$, again without introducing a zero-knowledge proof layer (Chen et al., 19 May 2026).

This broader context suggests that zkLoRA occupies a specialized subfield within LoRA research: it is concerned less with improving adaptation quality, merging fidelity, or query efficiency than with making LoRA-based customization verifiable, privacy-preserving, and suitable for untrusted execution environments.