Parametric Memory Decoding for Zero-Shot Routing in LoRA-Based External Parametric Memory

Abstract: With the rise of parametric memory, LoRA-based External Parametric Memory (EPM) has emerged as a modular solution, but existing routing methods often introduce additional training, deployment, and maintenance overhead. This raises a natural question: can a LoRA-based EPM bank be routed without maintaining an additional routing component? However, existing zero-shot LoRA routing methods still face two problems under the EPM setting: (1) their evaluations are scattered across different task settings rather than organized around EPM access, and (2) their routing signals lack a unified perspective to guide systematic improvement. To address these problems, we organize PMD-Bench, covering document-level, domain-level knowledge, and task-skill, and propose Parametric Memory Decoding (PMD), the first framework designed to systematically improve zero-shot LoRA routing by reframing it as decoding activations over external parametric memory. Based on PMD, we further instantiate PMDRouter, which scores each LoRA by its response magnitude from a single base-model prefill. Experiments on PMD-Bench show that PMDRouter achieves the strongest internal-signal performance across multiple zero-shot routing settings. These results demonstrate the feasibility of zero-shot LoRA routing and suggest that PMD can serve as a general framework for improving zero-shot routing methods. Sources: Github (https://anonymous.4open.science/r/Parametric-Memory-Decoding-872A/)

• The paper proposes PMD, reframing routing as decoding query-conditioned responses from LoRA weights to achieve zero-shot routing without auxiliary training.
• The PMDRouter implementation leverages normalized response energy to select adapters, consistently achieving high routing accuracy over document, domain, and task benchmarks.
• A training-side signal-writing objective enhances adapter addressability, reducing routing design complexity by approximately 40x compared to traditional matching paradigms.

Parametric Memory Decoding for Zero-Shot Routing in LoRA-Based External Parametric Memory

Motivation and Framework

The emergence of parameter-efficient fine-tuning (PEFT) has highlighted LoRA-based External Parametric Memory (EPM) as a modular, scalable solution for storing knowledge, domains, and task skills in LLMs without retraining a monolithic backbone. EPM architectures expose adapter banks, raising the challenge of query-dependent routing—that is, dynamically selecting the right LoRA module for a given input. Conventional routing solutions employ learned routers or retrieval-augmented gating, which escalate training, deployment, and maintenance costs. The main question this paper tackles is whether routing can be achieved in a zero-shot fashion: using only internal signals without any additional router training.

To address the lack of systematic benchmarking and unified signal construction, the authors introduce PMD-Bench—a multi-granularity testbed spanning document-level (PaperQA), domain-level (NQ-DomainLoRA), and task-level (Task-LoRA) LoRA memories. The paper proposes Parametric Memory Decoding (PMD), reframing the routing problem as decoding query-conditioned responses from LoRA activations rather than static adapter matching. PMD supports both zero-shot and training-free scenarios by constraining routing signals to observable LoRA weights and base-model activations.

Figure 1: Three LoRA routing paradigms: learned routers, retrieval-augmented routing, and zero-shot PMD routing, which utilizes query-side signals and LoRA weights.

Methodology

Formalization

For a frozen backbone $f_{base}$ and a LoRA bank $\mathcal{M} = \{L_k\}_{k=1}^K$, each adapter is parameterized by $(A_k, B_k)$. The routing objective is to compute—without an auxiliary router—a score $\mathcal{R}(h(q), A_k, B_k)$ for each LoRA given a query $q$, where $h(q)$ is the query representation aggregated from backbone activations.

PMD builds routing signals by constructing a query-conditioned response object:

$\rho_k(q) = \mathrm{Resp}(h(q), \theta_k)$

decoded into a scalar score via a signal decoder $D$:

$s_k(q) = D(\rho_k(q))$

Routing selects the adapter with maximal score, aiming for high response-margin separability.

PMDRouter Instantiation

PMDRouter instantiates PMD with a linear-response operator, applying each candidate LoRA update to the query-conditioned backbone activation and measuring the induced response energy, normalized for input and update scale:

$$E_{\ell,k}(x) = \frac{\|B_{\ell,k} A_{\ell,k} u(x)\|^2}{\|u(x)\|^2 \|B_{\ell,k} A_{\ell,k}\|_F^2 + \epsilon}$$

Scores are aggregated and calibrated for hard routing (adapter selection).

Importantly, PMDRouter requires only one backbone prefill per query and omits any learned routing module, thus satisfying zero-shot constraints.

Figure 2: PMD and PMDRouter workflow: response construction, signal separability, routing pathway, and signal shaping.

Training-Side Signal Writing

PMD further proposes a signal-writing objective during LoRA training, directly augmenting the response-separability used at inference. Auxiliary loss terms maximize the response margin or energy, shaping adapters for stronger addressability without extra inference-time computation.

Experimental Results

Benchmarking and Baselines

PMDRouter is evaluated against Backbone-only, LoRA-only, joint Backbone-LoRA signal routes, and retrieval-based non-parametric baselines, using PMD-Bench (PaperQA, NQ-DomainLoRA, Task-LoRA) and multiple LLM backbones.

Routing Accuracy and Comparative Performance

PMDRouter consistently achieves top routing accuracy across all three benchmarks:

• PaperQA: 0.600–0.658, outperforming all signal baselines.
• Task-LoRA: 0.663–0.923, showing high decodability for task/skill LoRAs.
• NQ-DomainLoRA: Up to 0.790, competitive with but occasionally surpassed by LAG for domain-level routing.

Text retrieval (BM25) remains effective in certain document/knowledge settings, but PMDRouter demonstrates substantial gains, especially in modular skill scenarios.

Figure 3: Routing accuracy increases when moving from single-side to joint and PMD-based response paths, across backbones and benchmarks.

Design Complexity and Ablations

PMD reduces routing design complexity by ~40x compared to direct matching paradigms, collapsing query- and adapter-choice axes into a response-centered representation. Ablations indicate that projection-based response decoding captures most of the signal, with full write-back sometimes introducing non-separability due to adapter-specific scale bias.

Theoretical and Practical Implications

The PMD framework generalizes the construction of zero-shot routing policies, demonstrating that query-conditioned linear response energy establishes a robust criterion for adapter selection. This signal is strongly predictive of routing success—higher response margins are correlated with higher accuracy. The training-side signal-writing strategy further improves adapter bank addressability, aligning response energy with task-induced improvements.

Practically, PMD and PMDRouter eliminate the need for additional routing parameters, supporting scalable deployment of LoRA modular memory with minimal operational overhead. Theoretically, PMD reframes the routing problem as decoder design, opening the path for future research on response-object construction, layer-specific signals, and broader adapter-bank scenarios.

Conclusion

The paper advances the study of zero-shot routing for LoRA-based EPM by introducing a response-centric decoding framework and a benchmark suite for systematic evaluation. PMDRouter achieves strongest zero-shot routing performance in skill/task and mixed-domain settings, validating the PMD principle of query-conditioned response energy as the prime decodability signal. PMD substantially reduces routing design complexity and enables effective training-side improvements without external routers. Future directions include extending PMD to multi-hop memory access, continual learning, and integrating richer response objects or multi-modal adapters.