Retrieval-based Prior Selection (RPS)

• Retrieval-based Prior Selection is a design pattern that selects or constructs prior distributions from external evidence to improve prediction in contexts with sparse or evolving data.
• It is applied across diverse domains—such as responsive survey design, linear model shrinkage, robotics, and dialogue elicitation—to adapt prior information based on task-relevant signals like precision, stability, or compatibility.
• RPS emphasizes the benefit of adaptive prior incorporation to address challenges like partial observability, nonstationarity, and model misspecification rather than relying on a fixed prior assumption.

Retrieval-based Prior Selection (RPS) denotes a family of procedures in which a model does not fix its prior arbitrarily, but instead derives, selects, or reweights prior information from an external evidence source or from a library of prior-like candidates. In the literature considered here, the retrieved object varies by domain: a multivariate normal prior over response-propensity coefficients in responsive survey design, a shrinkage penalty interpreted as a prior in linear models, a repertoire-specific Gaussian-process prior in robotics, or a prompt strategy selected from a predefined pool in information elicitation dialogue. The unifying structure is the use of externally grounded or precompiled prior information to improve inference, prediction, or control when current observations are limited or evolving (West et al., 2019, Dustin et al., 2022, Kaushik et al., 2019, Wang et al., 15 Apr 2026).

1. Scope and conceptual definition

Across these works, RPS is not a single standardized algorithm. This suggests that it is better understood as a cross-domain design pattern: identify candidate priors or prior-like objects, retrieve or score them using task-relevant evidence, and then use the selected object to regularize prediction or action selection.

Setting	Prior-like object	Selection or construction signal
Responsive survey design	$\beta \sim N(\mu,\Sigma)$	historical quarters or screened literature
Linear models	shrinkage penalty / implied prior	perturbation-based predictive stability or GA fitness
Robotics	repertoire-specific GP prior	posterior over policy and repertoire, with UCB over repertoires
Dialogue elicitation	prompt from a prompt pool	normalized information gain reward

A terminological complication arises because the acronym RPS is itself overloaded. In the dialogue paper, RPS stands for Reinforcement Prompt Selection, not Retrieval-based Prior Selection. However, that work explicitly frames the prompt pool as a library of prior elicitation behaviors and treats selection as adaptive choice among predefined strategies, making it conceptually adjacent to retrieval- or prior-based selection rather than a purely generative prompting method (Wang et al., 15 Apr 2026).

2. Informative-prior retrieval in responsive survey design

A direct and explicit instantiation of retrieval-style prior construction appears in Bayesian prediction of daily response propensity for Responsive Survey Design (RSD). The target quantity is the probability that sampled unit $i$ responds at contact attempt $t$, modeled through a discrete-time hazard / logistic regression,

$$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$

with predictors drawn from fielding day of the quarter, paradata, sampling frame information, linked commercial data, and manually retained variables; after variable selection, the final model contained 72 coefficients. The Bayesian formulation places a multivariate normal prior on the coefficient vector,

$\beta \sim N(\mu,\Sigma).$

The paper evaluates three informative or near-informative strategies plus a baseline. The principal historical-data strategy is PWP (precision-weighted prior), which fits the same discrete-time logit model to each of the eight prior quarters and combines quarter-specific coefficient estimates and covariance matrices so that more precise quarters receive more weight. The prior mean for coefficient $c$ is

$$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$

and the covariance is based on the inverse of the average precision, with a ridge-type stabilization to ensure invertibility. The simpler LAST strategy uses only the immediately preceding quarter,

$$\mu=\hat\beta_{q-1}, \qquad \Sigma=\widehat{\mathrm{Var}(\hat\beta_{q-1})},$$

while LASTZ removes off-diagonal covariances and uses independent normal priors. The literature-based strategy, LIT, searches methodological literature and official documentation, retains eight documents after screening, maps analogous variables across studies, and sets

$$\mu_c=\frac{1}{S_c}\sum_{s=1}^{S_c}\hat\beta_{sc}, \qquad \Sigma_{cc}=\frac{1}{S_c}\sum_{s=1}^{S_c}\widehat{\mathrm{Var}(\hat\beta_{sc})},$$

with a diagonal $\Sigma$ because cross-study covariance information was generally unavailable. If no usable literature evidence exists for coefficient $i$0, the prior is set to $i$1 and $i$2. One probit-based study is converted to the log-odds scale by multiplying coefficients and standard errors by 1.61 (West et al., 2019).

Posterior updating begins after day 7 of the quarter, using current data through day $i$3 and PROC MCMC. Daily predictive probabilities are estimated by Monte Carlo averaging over 5000 posterior draws, and performance is summarized by daily mean difference from a final benchmark prediction, its standard error, and

$i$4

Evaluation uses five recent NSFG quarters as target collections and the eight preceding quarters as historical evidence. The field period is divided into Early (days 7–30), Middle (days 31–60), and Late (days 61–84) windows, and the final hazard models are additionally described with Nagelkerke pseudo-$i$5, Hosmer–Lemeshow goodness-of-fit, and AUC.

The empirical pattern is specific and operationally important. PWP generally performed best or among the best, with lower mean differences from the final benchmark earlier in the quarter and faster convergence toward zero. Improvements were especially notable in the middle period, when RSD interventions are often considered. RMSE also tended to be lower under Bayesian methods than under the standard approach. LAST was unstable and often poor, plausibly because a single quarter yielded noisy covariance estimates; LASTZ improved stability by removing covariances. LIT was competitive in some settings and often better than the non-informative baseline, though less consistently strong than PWP. The paper further notes that the NSFG is unusually well suited to the PWP strategy because quarterly design and response behavior are stable over time. In this setting, RPS is most literal: prior evidence is retrieved either from internal longitudinal history or from published external studies and mapped into a prior that improves early and middle-field prediction when current data are still sparse (West et al., 2019).

3. Prior selection through shrinkage equivalence in linear models

A broader theoretical framing comes from linear-model shrinkage, where prior selection is treated as mathematically equivalent to penalty selection. The starting point is a penalized optimization problem,

$i$6

together with an adaptive version using weights $i$7. When the first term is a negative log-likelihood, exponentiation yields a posterior-mode problem with density

$i$8

Thus $i$9 acts as a prior factor for $t$0, so the penalty is the negative log-prior up to constants. Ridge corresponds to a Gaussian prior, LASSO to a Laplace prior, and nonconvex penalties such as SCAD or MCP to more complicated priors (Dustin et al., 2022).

The paper argues that asymptotic oracle property (OP) considerations are useful but insufficient for practical prior or penalty selection. Its OP formulation requires, for null coefficients $t$1, both $t$2 and asymptotic normality for active coefficients. Regularity conditions imply familiar scaling requirements: for active coefficients, $t$3 must be small enough that $t$4; for inactive coefficients, $t$5 must be large enough that $t$6. Yet the paper’s core claim is that OP is not decisive because it is asymptotic, many penalties satisfy it under mild conditions, and methods with OP can still predict poorly in finite samples, while some methods without OP can predict well.

The proposed finite-sample criterion is predictive instability under data perturbation. Responses are perturbed as

$t$7

followed by train-test splitting, refitting, and evaluation of

$t$8

This is averaged over many random splits and perturbations, with a grid $t$9 for $$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$0. A good method has low instability values, smooth curves, and curves that increase slowly with $$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$1; a decreasing curve under increasing perturbation is treated as a warning sign of fundamental instability or misspecification.

The simulation recommendations are regime dependent. For $$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$2 and $$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$3, where $$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$4, variable selection is poor for all methods and RR and EN are generally best; under Toeplitz correlation and high sparsity, SCAD2 and MCP can become preferable. For $$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$5, EN is typically best for low or medium sparsity. For $$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$6, LM can do well when sparsity is low and SCAD1 often becomes very competitive or best. At $$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$7, many methods are nearly indistinguishable, OP methods are often near-perfect in variable selection, but several non-OP methods still predict well. The methods most often recommended against are RR, LAD-LASSO, and ASCAD1, and sometimes LM when sparsity is high. The paper’s real-data superconductivity example shows that these recommendations can shift under model misspecification, where a less sparse linear approximation can outperform a sparser one.

The paper’s second contribution is a genetic-algorithm (GA) procedure for learning a custom penalty, hence a custom prior. It uses a shifted penalty,

$$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$8

with $$\log\left(\frac{P(Y_{it}=1\mid X_{it})}{1-P(Y_{it}=1\mid X_{it})}\right)=\beta^\top X_{it},$$9 a $\beta \sim N(\mu,\Sigma).$0-consistent pilot estimate, and searches over a polynomial penalty class

$\beta \sim N(\mu,\Sigma).$1

In the high-dimensional case $\beta \sim N(\mu,\Sigma).$2, the construction reduces to a penalty on $\beta \sim N(\mu,\Sigma).$3. In the $\beta \sim N(\mu,\Sigma).$4 example, the GA returns $\beta \sim N(\mu,\Sigma).$5, exactly recovering ridge regression; in the $\beta \sim N(\mu,\Sigma).$6 example, it returns $\beta \sim N(\mu,\Sigma).$7, a custom penalty that produced the best predictive error among compared methods. Within an RPS interpretation, this work shifts attention from retrieving external empirical summaries to selecting among, or constructing, prior families using predictive stability as the operative criterion rather than asymptotic optimality alone (Dustin et al., 2022).

4. Adaptive prior retrieval in repertoire-based robotics

In robotics, prior selection appears as adaptive choice among multiple repertoires learned offline in simulation. The setting is online adaptation under an unknown situation $\beta \sim N(\mu,\Sigma).$8, such as robot damage, changes in floor friction, or unknown object properties, with task-space dynamics

$\beta \sim N(\mu,\Sigma).$9

where $c$0 is Gaussian noise and the robot does not know either $c$1 or the current situation $c$2. The robot does, however, have access to a low-fidelity simulator $c$3 and to a subset of likely situations. The paper’s contribution is APROL (Adaptive Prior selection for Repertoire-based Online Learning), which relaxes the single-repertoire assumption and instead lets the system select the most useful prior online (Kaushik et al., 2019).

APROL has three phases. In offline repertoire generation, MAP-Elites is run for each probable situation $c$4, producing repertoires $c$5 containing tuples

$c$6

and forming a repertoire set

$c$7

In online learning, each repertoire supplies the prior mean-function for a Gaussian process that maps simulated transitions to real transitions. For dimension $c$8,

$c$9

with prior mean

$$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$0

and squared-exponential kernel

$$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$1

The online selection rule is a MAP decision over both policy and repertoire:

$$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$2

Here $$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$3 is either uniform over policies or restricted to policies whose simulated transition lies near the desired transition; $$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$4 is a Gaussian likelihood favoring policies whose repertoire-specific GP predicts motion toward the current sub-goal; and $$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$5 measures repertoire compatibility with reality. Compatibility begins with a closeness score

$$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$6

then uses UCB to balance exploitation and exploration,

$$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$7

followed by normalization across repertoires. The result is a prior-selection mechanism in which repertoire identity is itself a latent variable updated online.

The empirical evaluation covers object pushing with a robotic arm, goal reaching with a damaged hexapod in simulation, and a real damaged hexapod. Simulation comparisons use 40 replicates and include CP-L, SP-L, SP-NL, and APROL-NL. APROL solves both simulated tasks in less interaction time than baselines, matches or approaches the close-prior baseline, and significantly outperforms wrong-single-prior and no-learning variants. In the real damaged-hexapod experiment, the robot reaches the goal with 100% success within a maximum of 30 steps and recovers up to 88% of the intact robot’s capability in time to goal. The robotics case therefore instantiates RPS as retrieval over a discrete prior library, where the retrieved object is not a coefficient prior but a repertoire-specific transition prior for planning and adaptation (Kaushik et al., 2019).

5. Prompt-pool selection as a prior-selection analogue in dialogue elicitation

In open-ended dialogue, the paper titled "RPS: Information Elicitation with Reinforcement Prompt Selection" defines RPS differently: it is a reinforcement learning framework for selecting prompts from a predefined prompt pool to elicit concealed or incompletely expressed user information. The paper explicitly states that it is not a classic retrieval system, but it also states that the prompt pool acts like a library of prior elicitation behaviors, making the method conceptually related to prior-based prompt selection and retrieval over strategies (Wang et al., 15 Apr 2026).

The formalization is sequential. User information is divided into

$$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$8

representing positive, neutral, and negative information. At dialogue round $$\mu_c=\frac{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}\hat\beta_{qc}}{\sum_{q=1}^{8}\widehat{\mathrm{Var}(\hat\beta_{qc})}^{-1}},$$9, the state is

$$\mu=\hat\beta_{q-1}, \qquad \Sigma=\widehat{\mathrm{Var}(\hat\beta_{q-1})},$$0

the action is a prompt $$\mu=\hat\beta_{q-1}, \qquad \Sigma=\widehat{\mathrm{Var}(\hat\beta_{q-1})},$$1, and the selected prompt conditions the query-generation process,

$$\mu=\hat\beta_{q-1}, \qquad \Sigma=\widehat{\mathrm{Var}(\hat\beta_{q-1})},$$2

After the user responds, the model extracts new information $$\mu=\hat\beta_{q-1}, \qquad \Sigma=\widehat{\mathrm{Var}(\hat\beta_{q-1})},$$3 and updates $$\mu=\hat\beta_{q-1}, \qquad \Sigma=\widehat{\mathrm{Var}(\hat\beta_{q-1})},$$4. The reward is a normalized information gain,

$$\mu=\hat\beta_{q-1}, \qquad \Sigma=\widehat{\mathrm{Var}(\hat\beta_{q-1})},$$5

designed to stabilize learning and avoid reward vanishing as the elicitation gap shrinks. The paper also gives the policy-gradient style objective

$$\mu=\hat\beta_{q-1}, \qquad \Sigma=\widehat{\mathrm{Var}(\hat\beta_{q-1})},$$6

The synthetic validation uses a Gaussian Mixture Model user-disclosure environment with two settings: an unbiased user, where $$\mu=\hat\beta_{q-1}, \qquad \Sigma=\widehat{\mathrm{Var}(\hat\beta_{q-1})},$$7, and a biased user, where disclosure weights favor positive information. Performance is averaged over 10 independent runs. The RL querying strategy consistently achieves lower KL divergence than a random querying baseline, remains stable around 0.01 KL divergence in both settings, and the random baseline worsens under bias, with divergence increasing from 3.36 to 3.51.

The main benchmark is IELegal, derived from the CAIL2018 Chinese criminal case corpus. It contains 2,000 cases in two subsets, IELegal-base and IELegal-augment, each with 500 train / 500 test. Case entries include case content, case type, plaintiff/defendant, key information, classified information, and negative/positive/neutral subsets. Evaluation uses semantic similarity between cumulative extracted information and a reference fact set via cosine similarity over Sentence-BERT embeddings. The baselines include five fixed prompt strategies—Normal, Exploratory Information Gathering, Precise Evidence Confirmation, Corroborative Detail Verification, and Confrontational Disclosure Questioning—plus Adaptive Generative Questioning, GRIPS, and RLPrompt. On IELegal-base and IELegal-augment, RPS consistently outperforms all baselines by the end of dialogue; Normal performs worst; and GRIPS and RLPrompt do not surpass RPS and often do no better than the original fixed strategies. In this setting, the “prior” is the prompt pool itself: a precompiled set of elicitation strategies from which the policy retrieves the most contextually useful option at each turn (Wang et al., 15 Apr 2026).

6. Shared structure, boundary conditions, and recurring misconceptions

Despite substantial heterogeneity, the four lines of work share a common architecture. First, there is a candidate prior space: earlier survey quarters and screened documents, a menu of penalties or a GA-search class, a library of repertoires, or a prompt pool. Second, there is a selection signal: coefficient precision, perturbation-based predictive stability, observed reality-versus-repertoire closeness with UCB, or normalized information gain. Third, the selected prior-like object directly affects downstream prediction or control: posterior response-propensity estimates, regularized linear prediction, policy choice for robotic adaptation, or dialogue strategy selection. This suggests that RPS is less a domain-specific method than a recurrent solution to a common problem: current data alone are not yet sufficient, so prior information must be chosen rather than assumed.

Several misconceptions are explicitly contradicted by the source materials. One is that a single prior is enough. APROL is motivated precisely by the failure mode of single-repertoire adaptation when the deployment situation differs too much from the assumed prior, and the survey paper shows that priors based on only the immediately preceding quarter can be unstable. Another is that asymptotic optimality settles prior choice. The shrinkage paper argues that the oracle property is too broad and too asymptotic to decide finite-sample predictive performance. A third is that retrieval must mean document retrieval. The survey paper uses both internal longitudinal history and published literature; robotics retrieves among repertoires; dialogue retrieves among strategy prompts. Retrieval, in these materials, is therefore object-dependent rather than document-specific (West et al., 2019, Dustin et al., 2022, Kaushik et al., 2019, Wang et al., 15 Apr 2026).

The same papers also define the principal boundary conditions. Historical-data priors are strongest when there is substantial historical data from the same survey with stable design and response behavior; when such history is absent, literature-based priors are a reasonable alternative. If behavior changes over time or there is seasonality, older history may need to be downweighted or replaced by more recent data. In linear models, the best penalty or implied prior depends strongly on the sample-size-to-sparsity regime, dependence structure, and model misspecification. APROL assumes access to a low-fidelity simulator and a subset of likely situations. Reinforcement Prompt Selection assumes that a prompt pool is available, that user disclosure is state-dependent and biased, and that reward can be approximated through divergence or semantic similarity. These are not incidental details; they define where each RPS instantiation is expected to work.

A plausible synthesis is that retrieval-based prior selection is most valuable in problems with partial observability, nonstationary evidence accumulation, and meaningful prior heterogeneity. The survey study shows the benefit when fieldwork is ongoing and managers need reliable early guidance; the shrinkage study shows why finite-sample predictive stability matters when asymptotics are uninformative; the robotics study shows that uncertainty over the correct prior can itself be modeled and updated online; and the dialogue study shows that a library of prior strategies can be adaptively selected to uncover concealed information. In that sense, RPS names a methodological orientation: prior information should be elicited, retrieved, or selected in a task-aware manner, rather than treated as fixed background structure.