--- title: Correlation-based Prior Generation (CPG) url: https://www.emergentmind.com/topics/correlation-based-prior-generation-cpg type: topic --- # Correlation-based Prior Generation (CPG) Searching arXiv for papers directly relevant to correlation-based prior generation and acronym disambiguation. Search query: "correlation prior generation random function priors correlation modeling reinforcement learning correlation priors" Correlation-based Prior Generation (“CPG”; *Editor’s term*) denotes a family of methods in which prior distributions are constructed from explicit correlation structure rather than from independent components or fixed low-capacity covariance families. In the cited literature, the generated prior may be a prior over nonnegative latent loading vectors, multinomial policies or transition rows, jointly Gaussian inverse-problem parameters with prescribed marginals, graph-conditioned latent variables, sparse regression supports and coefficients, sparse Gaussian-process precision factors, implicit conditional distributions over correlation matrices, or calibrated predictive distributions over Pearson correlation values [1905.03826] [1909.05106] [2605.00332] [1905.05335] [1810.00739] [2112.14591] [2107.10606] [2506.03444]. ## 1. Conceptual basis A common starting point across these methods is the inadequacy of independence assumptions when the objects being regularized are semantically or structurally related. In latent-feature models, the coordinates of a loading vector may exhibit co-occurrence, mutual reinforcement, context-specific combinations, and higher-order dependence; in such settings, a simple Gaussian or exponential-family prior can encode some covariance, but modeling all moments up to high order becomes combinatorially intractable [1905.03826]. In finite MDPs, independent Dirichlet priors over policies or transition rows prevent information transfer across related states or contexts, so learning reverts to uninformative defaults wherever counts have not been directly observed [1909.05106]. In joint Bayesian inversion, treating parameters as independent is not statistically neutral: it asserts that the correlation is known a priori to be zero [2605.00332]. In VAEs, an i.i.d. latent prior is inappropriate when one knows a priori that certain data points are correlated through an undirected graph [1905.05335]. This shared critique leads to a more general design principle: prior structure should follow dependence structure. The dependence source varies by domain. It may be induced by shared latent-function evaluations, by a user-defined distance kernel over covariates, by a cross-correlation operator in whitened coordinates, by singleton and pairwise marginals on a graph, by the observed predictor Gram matrix, by pairwise correlations in a covariance model, by empirical regime labels for correlation matrices, or by natural-language metadata describing a variable pair [1905.03826] [1909.05106] [2605.00332] [1810.00739] [2112.14591] [2107.10606] [2506.03444]. A plausible implication is that CPG is best understood not as one model class but as a constructional pattern: first specify what should be statistically coupled, then map that coupling into a prior object whose support and inductive bias match the downstream task. ## 2. Formal constructions One of the most explicit prior generators is the population random measure embedding (PRME), introduced for correlated latent loadings. Starting from a latent-feature model with nonnegative \(Z_n=[Z_{n1},\dots,Z_{nK}]\), PRME embeds the array as a discrete random measure \[ \xi = \sum_{n,k} Z_{nk}\,\delta_{T_n,\theta_k}, \] assumes separate exchangeability, and, after excluding a less useful component of Kallenberg’s representation, arrives at the operative prior \[ Z_{nk}=f_n(\vartheta_k). \] Here a shared Poisson process generates feature-specific latent locations \(\vartheta_k\), while each object \(n\) has its own random measurable function \(f_n(\cdot)\). Correlation arises because all coordinates of \(Z_n\) are evaluations of the same random function at shared feature locations, which induces pairwise, higher-order, and nonlinear dependence by construction [1905.03826]. For discrete reinforcement learning and related multinomial models, a different construction couples simplex-valued parameters through latent Gaussian logits and logistic stick-breaking: \[ \mathbf p_c=\boldsymbol \Pi_{\mathrm{SB}(\boldsymbol \psi_{c\cdot}), \qquad \boldsymbol \psi_{\cdot k} \sim \mathcal N(\boldsymbol\mu_k,\boldsymbol\Sigma),\; k=1,\dots,K-1. \] The correlation prior is therefore encoded in \(\boldsymbol\Sigma\), typically instantiated through \[ (\boldsymbol \Sigma_{\boldsymbol \theta})_{cc'} = \theta \exp\!\left(-\frac{d(c,c')^2}{l^2}\right), \] so that nearby covariates under a chosen distance \(d(c,c')\) inherit similar multinomial structure. This prior can be placed over policies across states, transition distributions across states for a fixed action, or subgoal distributions across states [1909.05106]. In joint Bayesian inversion, the prior object is a block Gaussian covariance with prescribed marginals: \[ \Gamma= \begin{pmatrix} \Gamma_p & L_p^{-1} C L_m^{-T}\\ L_m^{-1} C^T L_p^{-T} & \Gamma_m \end{pmatrix}, \] or, under principal square roots, \[ \Gamma= \begin{pmatrix} \Gamma_p & \Gamma_p^{1/2} C \Gamma_m^{1/2}\\ \Gamma_m^{1/2} C^* \Gamma_p^{1/2} & \Gamma_m \end{pmatrix}. \] The marginals remain exactly \(\mathcal N(p_\ast,\Gamma_p)\) and \(\mathcal N(m_\ast,\Gamma_m)\); correlation is introduced only through \(C\), with validity guaranteed by the strict contraction condition \(\|C\|_2<1\) [2605.00332]. Graph-structured latent-variable models implement CPG through consistent singleton and pairwise marginals. On an acyclic undirected graph \(G=(V,E)\), \[ p_0^{\textrm{corr}}(\bm z) = \prod_{i=1}^n p_0(\bm z_i) \prod_{(v_i,v_j)\in E} \frac{p_0(\bm z_i,\bm z_j)}{p_0(\bm z_i)p_0(\bm z_j)}. \] For cyclic graphs, the paper replaces the invalid direct construction by a uniform mixture over maximal acyclic subgraphs, \[ p_0^{\textrm{corr}_g}(\bm z) = \frac{1}{|\mathcal A_G|} \sum_{G'=(V,E')\in\mathcal A_G} p_0^{G'}(\bm z), \] which yields a tractable lower bound and interpretable edge weights given by maximal-acyclic-subgraph inclusion probabilities [1905.05335]. These constructions differ in technical detail, but they share a common property: correlation is not added as an afterthought to a fixed prior family. It is the mechanism from which the prior is generated. ## 3. Representative prior objects The literature spans several distinct prior objects. The following summary organizes recurring instances. | Paper | Prior object | Correlation mechanism | |---|---|---| | [1905.03826] | Nonnegative latent loadings \(Z_n\) | Shared random functions at Poisson-process feature locations | | [1909.05106] | Multinomial policies, transitions, subgoals | Gaussian covariance \(\boldsymbol\Sigma\) over stick-breaking logits | | [2605.00332] | Joint Gaussian inverse-problem parameters | Cross-correlation operator \(C\) preserving fixed marginals | | [1905.05335] | VAE latent variables across data points | Singleton and pairwise marginals on an undirected graph | | [1810.00739] | Sparse regression support and coefficients | Design-dependent prior via \(X_S^\top X_S\) and \(\lambda\) | | [2112.14591] | Approximate GP prior precision factor | Correlation-based ordering and neighbor selection | | [2107.10606] | Conditional distribution over correlation matrices | cGAN generator with projection to the elliptope | | [2506.03444] | Prior over Pearson correlation coefficients | LLM logits calibrated into a continuous predictive density | In high-dimensional sparse regression, the empirical correlation-adaptive prior (ECAP) uses the Gram determinant \[ D(S)=|X_S^\top X_S| \] to define a support prior \[ \pi_\lambda(S\mid |S|=s)= \frac{ D(S)^{-\lambda/(2s)}\,1\{\kappa(S)