---
title: Joint Parameter Selection (JPS) Overview
url: https://www.emergentmind.com/topics/joint-parameter-selection-jps
type: topic
---

# Joint Parameter Selection (JPS) Overview

Joint Parameter Selection (JPS) denotes a family of optimization and inference problems in which multiple interdependent parameter blocks are selected or estimated simultaneously under a shared criterion, rather than optimized one at a time. Across the literature considered here, this includes mixed discrete–continuous design in integrated sensing and communication, joint sparsity selection in multivariate regression and precision modeling, coherent estimation and model selection under missing covariates, joint state-path and parameter MAP estimation in stochastic differential equations, sparse fine-tuning in domain generalization, and coordinated parameter–data selection in large language model fine-tuning [2410.13446, 2201.05653, 1805.04602, 1704.01670, 2508.16976, 2605.06166]. A common premise is that the performance of one parameter class depends on the choice of another, so decoupled optimization induces bias, suboptimality, or unnecessary overhead.

## 1. Conceptual scope

In the sources surveyed here, JPS is best understood as a methodological pattern rather than a single canonical model. In one explicit formulation for domain generalization, Joint Parameter Selection restricts updates to a small, sparse subset of parameters chosen jointly across source domains so as to preserve the generalization strength of pre-trained models [2508.16976]. In another explicit formulation for LLM fine-tuning, parameter and data selection are cast as two bilevel selection problems under a common validation objective, and parameter importance and data utility emerge as column-wise and row-wise aggregations of a single gradient interaction matrix [2605.06166]. In ISAC, the transmitter jointly chooses which antennas are active and what transmit covariance matrix to use because both achievable rate and sensing beampattern quality depend on these variables jointly [2410.13446].

Several related works do not use the term “Joint Parameter Selection” literally, but instantiate the same logic. High-dimensional multivariate regression requires simultaneous sparse inference over a regression coefficient matrix \(B\) and an error precision matrix \(\Omega\), since misspecification of one block can distort inference on the other [2201.05653]. Logistic regression with missing covariates is treated through joint modeling of \((Y,X)\), with \((\beta,\mu,\Sigma)\) estimated together and model comparison performed on the observed-data likelihood rather than on completed datasets [1805.04602]. In nonlinear SDEs, joint optimization over a latent state path and static parameters is necessary because state uncertainty and parameter uncertainty are coupled, and because MAP estimation over paths requires the Onsager–Machlup functional rather than an energy-only surrogate [1704.01670].

The term itself is not universal. In collaborative imperfect-information games, JPS denotes “Joint Policy Search,” not Joint Parameter Selection [2008.06495]. In FDD cell-free massive MIMO, JPS denotes “Joint Port Selection,” a combinatorial resource-selection problem over ports, users, and base stations [2307.10730]. This terminological ambiguity is substantive: some papers use JPS for parameter masks, some for structural subsets such as antennas or ports, and some for policies rather than parameters.

## 2. Canonical mathematical structures

A useful summary suggested by these works is that JPS problems usually involve two or more coupled variable classes, often with different geometries: binary masks, subset indicators, covariance matrices, latent variables, regression structures, or trajectories. The coupling may enter through a shared objective, a shared likelihood, or a shared validation criterion.

| Setting | Joint objects | Selection/optimization form |
|---|---|---|
| ISAC [2410.13446] | binary antenna vector \(\mathbf p\), covariance \(\mathbf R\), scaling \(\alpha\) | mixed-integer scalarized objective \(F(\mathbf p,\alpha,\mathbf R)-\mu C(\mathbf\Delta(\mathbf p),\mathbf R)\) |
| Multivariate regression [2201.05653] | sparsity in \(B\) and \(\Omega\) | generalized posterior \(\pi_{g,\mathrm{joint}}(B,\Omega\mid Y,X)\) |
| Missing-covariate logistic regression [1805.04602] | \((\beta,\mu,\Sigma)\) plus regression structure \(\mathcal M\) | SAEM estimation and BIC on the observed-data likelihood |
| SDE joint MAP estimation [1704.01670] | state path \(x(\cdot)\), \(z_0\), \(\theta\) | maximize posterior fictitious density with Onsager–Machlup functional |
| Domain generalization [2508.16976] | sparse parameter mask \(M^{\mathrm{step2}}\) | one-shot joint parameter mask from cross-domain gradient importance and variance |
| LLM fine-tuning [2605.06166] | parameter mask \(\mathbf m_\theta\), data mask \(\mathbf m_{\mathcal D}\) | two bilevel selection problems under a shared validation objective |
| Discrete Mumford–Shah [2109.13651] | hyperparameters \((\beta,\lambda)\) | continuous joint optimization of SURE/SUGAR-estimated risk |
| Joint anomaly-feature selection [2606.13244] | sample indicators \(s_i\), feature indicators \(f_j\) | exact-budget bipartite selection with bilinear coupling \(\sum_{i,j}W_{ij}s_if_j\) |

Representative formulations make the coupling explicit. In ISAC, the core design is
\[
\underset{\mathbf{p}, \alpha, \mathbf{R}}{\text{min.}} \quad F(\mathbf{p},\alpha,\mathbf{R})-\mu C(\mathbf{\Delta}(\mathbf{p}),\mathbf{R})
\]
subject to
\[
\mathbf{R}\succeq 0,\qquad \operatorname{tr}(\mathbf{R})\le P_{\text{Tx}},\qquad \mathbf{p}\in\mathbb{B}^N,\qquad \sum_{n=1}^N p_n=K.
\]
The point is not merely that \(\mathbf p\) is discrete and \(\mathbf R\) is continuous, but that both communication and sensing metrics depend on them jointly [2410.13446].

In LLM fine-tuning, the parameter-selection and data-selection problems are written as parallel bilevel programs:
\[
\min_{\|\mathbf{m}_{\theta}\|_0\le k}\  \mathcal{L}_{\text{val}\!\left(\boldsymbol{\theta}_{\text{old}}+\mathbf{m}_{\theta}\odot\boldsymbol{\delta}^*(\mathbf{m}_{\theta})\right)
\]
and
\[
\min_{\|\mathbf{m}_{\mathcal D}\|_0\le b}\  \mathcal{L}_{\text{val}\!\left(\boldsymbol{\theta}_{\text{old}}+\boldsymbol{\delta}^*(\mathbf{m}_{\mathcal D})\right),
\]
with a common outer objective
\[
\mathcal L_{\text{val}}(\boldsymbol{\theta})=\mathcal L_{\text{new}}(\boldsymbol{\theta})+\lambda \mathcal L_{\text{prior}}(\boldsymbol{\theta}).
\]
Under first- and second-order local approximations, both axes are scored from the same interaction matrix \(\mathbf M\), with parameter scores given by column sums and data scores by row sums [2605.06166].

In joint anomaly-feature selection, the exact-budget bipartite objective is
\[
H(s, f) = -\sum_{i=1}^{N} a_i s_i -\sum_{j=1}^{D} b_j f_j -\lambda \sum_{i=1}^{N} \sum_{j=1}^{D} W_{ij} s_i f_j
\]
subject to
\[
\sum_{i=1}^N s_i = k,\qquad \sum_{j=1}^D f_j = m.
\]
Here the bilinear term \(W_{ij}s_if_j\) is the formal reason that sequential feature-first selection can fail: feature utility depends on which samples are selected, and sample utility depends on which features are selected [2606.13244].

## 3. Methodological patterns

A recurring pattern is conditional tractability. When one parameter block is fixed, another becomes convex, Gaussian, or otherwise manageable. In ISAC, fixing the binary antenna vector \(\mathbf p\) makes the problem convex in \((\mathbf R,\alpha)\), which is then solved with convex optimization tools, while a DP-style predecessor-recursion heuristic explores the combinatorial antenna-selection structure [2410.13446]. The paper is explicit that the resulting algorithm is sub-optimal because the principle of optimality does not strictly hold.

In missing-covariate logistic regression, the full observed-data likelihood is analytically intractable because
\[
p(y_i,x_{i,\mathrm{obs}};\theta)=\int p(y_i\mid x_i;\beta)p(x_i;\mu,\Sigma)\,dx_{i,\mathrm{mis}}
\]
has no closed form in general. The proposed solution replaces the EM E-step by stochastic approximation, with missing covariates treated as latent variables and simulated from \(p(x_{i,\mathrm{mis}}\mid x_{i,\mathrm{obs}},y_i;\theta)\) using an independence Metropolis–Hastings kernel whose acceptance ratio depends only on the logistic likelihood contribution [1805.04602]. This is a joint-modeling strategy rather than a preprocessing strategy.

In high-dimensional multivariate regression, JRNS uses a generalized bi-convex likelihood together with spike-and-slab priors, and performs Metropolis-within-Gibbs updates over entries of \(B\), off-diagonal entries of \(\Omega\), and diagonal entries of \(\Omega\) [2201.05653]. The crucial algorithmic move is to remove the \(\log\det\Omega\) term from the exact Gaussian likelihood and replace it with a generalized likelihood that remains bi-convex, enabling entrywise closed-form point-mass-plus-normal updates for large parts of the posterior.

In SDEs, the joint problem is posed as an optimal-control-like nonlinear program over an absolutely continuous state path, initial conditions, and parameters, then transcribed by third-order Legendre–Gauss–Lobatto direct collocation, stated to be equivalent to the Hermite–Simpson method, and solved with IPOPT [1704.01670]. In the discrete Mumford–Shah problem, the hyperparameters \((\beta,\lambda)\) are selected by differentiating through the SL-PAM alternating minimization scheme and optimizing SURE/SUGAR estimates with a low-memory BFGS quasi-Newton method with box constraints [2109.13651].

The modern fine-tuning papers add a different pattern: one-shot joint scoring before optimization. In domain generalization, the final mask \(M^{\mathrm{step2}}\) is generated before training and remains unchanged; it is obtained by intersecting per-domain top-\([m_{\mathcal L_0}\rho]\) gradient supports and then filtering by across-domain gradient variance [2508.16976]. In DualSFT, a warmup checkpoint \(\bar{\boldsymbol\theta}\), a shared projection vector \(\mathbf u\), and per-sample ghost-dot products are sufficient to co-extract a parameter mask and a data subset from the same gradient statistics [2605.06166]. This suggests a shift from iterative alternating selection toward shared local-response surrogates.

## 4. Representative domains and problem classes

Communications and signal processing supply several of the clearest JPS formulations. In ISAC, JPS couples binary antenna activation with covariance design and a beampattern scaling parameter under a scalarized communication–sensing objective [2410.13446]. In wireless federated learning, device participation and uplink power control are optimized jointly, with aggregation weights
\[
\rho_k(t)=\frac{a_k(t)p_k(t)}{\sum_j a_j(t)p_j(t)}
\]
induced by the same power variables that govern communication quality; selection and power control therefore co-determine the statistical aggregation rule [2205.09306]. In FDD cell-free massive MIMO, joint port selection chooses which ports from which cooperating BSs should be assigned to each user, while an eigenvalue-decomposition-based transformation compresses feedback by exploiting correlation among selected port coefficients [2307.10730].

Statistics and biostatistics present JPS as coherent model specification under latent or missing structure. Logistic regression with missing covariates jointly estimates the regression coefficients \(\beta\) and the Gaussian covariate parameters \((\mu,\Sigma)\), then performs model selection through
\[
\mathrm{BIC}(\mathcal M)=-2\,\ell(\hat\theta_{\mathcal M};x_{\mathrm{obs}},y)+\log(n)\,d(\mathcal M),
\]
where selection is based on the observed-data likelihood, not on complete cases or singly imputed data [1805.04602]. A two-stage Bayesian method for multiple longitudinal markers and competing risks first fits one-marker joint models to reduce bias from informative dropout and then performs spike-and-slab variable selection over marker-current-value effects \(\alpha_{lk}\) and baseline covariate effects \(\gamma_{lj}\) in a cause-specific hazard model [2412.03797]. By contrast, the latent-class shared-parameter joint model is mainly a structure-selection method for the number of latent classes \(G\), estimated through an overfitted mixture and non-empty class counting rather than coefficient shrinkage [1802.10015].

Dynamical systems and control use JPS in a state–parameter sense. The joint MAP estimator for nonlinear SDEs maximizes a posterior fictitious density over \(x(\cdot)\), \(z_0\), and \(\theta\), with objective
\[
\ell(x,z,\theta,y) = \ln\psi(y|x,z,\theta) +\ln\pi(x_0, z_0, \theta)
-\frac12 \int_0^T \operatorname{div}_x f\, dt
-\frac12 \int_0^T \left\| G^{-1}\big[\dot x_t - f(t, x_t, z_t, \theta)\big] \right\|^2 dt.
\]
The paper’s central point is that omitting the drift-divergence term yields MAP estimation of the noise path rather than the state path, and can bias parameters such as damping [1704.01670].

Machine learning contributes two newer interpretations. In domain generalization, JPS is explicitly a sparse fine-tuning rule for pre-trained vision models, based on parameters whose gradients are strong in every source domain and sufficiently consistent across domains [2508.16976]. In LLM fine-tuning, the parameter mask and data subset are selected jointly under matched budgets, with one-shot dual scoring giving a more favorable joint-constrained trade-off than sequential hybrid baselines [2605.06166]. A related but more combinatorial formulation appears in joint anomaly-feature selection, where exact-cardinality budgets over samples and features define a coupled bipartite selection problem, and the paper argues that feature-first ranking is structurally mismatched when \(W\) is non-separable [2606.13244].

## 5. Performance criteria, bounds, and guarantees

The literature does not use a single universal optimality criterion. Instead, JPS inherits whatever performance measure is native to the underlying domain. In ISAC, the criterion is a scalarized trade-off between beampattern MSE and communication rate, with \(\mu=0\) corresponding to sensing-only design, \(\mu=1\) to communication-only design, and intermediate \(\mu\) tracing approximate Pareto trade-offs [2410.13446]. The numerical results explicitly show that when \(\mu=0\), achievable rate is reduced by more than 50% relative to \(\mu=1\), while when \(\mu=1\), beampattern MSE is about four times that at \(\mu=0\).

In high-dimensional Bayesian regression, the guarantees are partly asymptotic and partly algorithmic. For the stepwise generalized Bayesian method, the paper proves selection consistency for \(B\) and \(\Omega\) under growing \(p,q\), and posterior contraction for \(B\) at Frobenius rate
\[
\sqrt{\frac{\delta_n\log(pq)}{n}},
\]
but it is explicit that these guarantees do not yet cover the full JRNS joint sampler [2201.05653]. In missing-covariate logistic regression, the emphasis is on unbiasedness, coverage, and coherent model comparison. The paper reports that SAEM confidence interval coverage stayed close to nominal 95% in one large-sample setting, whereas multiple imputation by chained equations undercovered for some coefficients, and model selection based on the observed-data likelihood outperformed complete-case BIC in correct-model recovery [1805.04602].

Several papers derive bounds whose form directly motivates joint selection. In domain generalization, the target-risk bound contains a stability term improved by sparser updates, together with domain discrepancy terms:
\[
R(A(\theta,\hat S),T)\leq R(A(\theta,\hat S),\hat S) +\sqrt{\frac{C^2+12Cn\mathcal{K}_2}{2n\sigma}+\frac{1}{2}\mathcal{H}\nabla \mathcal{H}(S,T)}.
\]
This motivates the two operators of JPS: importance across all source domains and variance filtering for cross-domain consistency [2508.16976]. In DualSFT, the shared local-utility approximation yields a row-column correspondence between parameter and data scores, and the appendix states first-order truncation error \(O(\eta^2)\) and second-order truncation error \(O(\eta^3)\) for the local surrogate [2605.06166]. In joint anomaly-feature selection, the calibration-perturbation theory is especially sharp: if \(\|\widehat W-W^\star\|_\infty\le \eta\), joint recovery loses at most \(2\eta\) in margin, whereas the analyzed feature-first rule can lose \(2N\eta\), because each column score aggregates perturbations over all \(N\) samples [2606.13244].

Post-selection estimation theory provides a complementary perspective. Estimation after parameter selection introduces post-selection mean squared error (PSMSE), a corresponding \(\Psi\)-Cramér–Rao-type bound for \(\Psi\)-unbiased estimators, and the post-selection maximum-likelihood (PSML) estimator. The framework is primarily for single-index selection, but the paper also contains an explicit extension to subset selection, making it partially transferable to genuine JPS settings [1503.02045]. This line of work clarifies that even when selection is predetermined by a rule \(\Psi\), the conditional density
\[
f(\mathbf{x}\mid \Psi=m;\boldsymbol{\theta}) = \frac{f(\mathbf{x};\boldsymbol{\theta})}{\Pr(\Psi=m;\boldsymbol{\theta})}
\]
changes the estimation problem itself.

## 6. Terminological ambiguity, limitations, and boundary cases

One persistent misconception is that “joint” automatically implies globally optimal simultaneous optimization. The sources do not support that reading. The ISAC dynamic-programming scheme is explicitly sub-optimal because the antenna-selection problem cannot be split into independent subproblems [2410.13446]. JRNS is a true joint generalized posterior method in practice, but the available high-dimensional consistency theorem applies only to a stepwise approximation, not to the full joint sampler [2201.05653]. The two-stage joint-model variable-selection method for competing risks is computationally attractive, yet it neglects between-marker correlation in stage 1 and does not fully propagate stage-1 uncertainty into stage 2 [2412.03797].

A second misconception is that JPS always means sparse variable selection. Several papers instead treat joint estimation or structure selection. The SDE paper concerns joint MAP estimation of a path and parameters, not discrete selection [1704.01670]. The latent-class shared-parameter joint model focuses on selecting the number of latent classes rather than selecting coefficients [1802.10015]. Conversely, in domain generalization and LLM fine-tuning, JPS is explicitly budgeted sparsification over parameter coordinates, but both methods are static and one-shot: masks are generated before training and then held fixed [2508.16976, 2605.06166].

A third misconception is that joint methods necessarily dominate sequential procedures in every regime. The anomaly-feature paper makes the opposite point precise: JPS is most useful when the coupling matrix \(W\) is non-separable. If \(W=\mathbf u\mathbf v^\top\) with nonnegative factors, optimal selection reduces to independent top-\(k\) and top-\(m\) ranking, so the joint problem collapses [2606.13244]. This suggests that the real discriminator is not the word “joint,” but the presence of irreducible cross-block interaction.

Finally, the acronym itself is unstable. In one major paper JPS means Joint Policy Search for collaborative agents in imperfect-information games [2008.06495]. In another it means Joint Port Selection in cell-free massive MIMO [2307.10730]. Within the broader research landscape assembled here, “Joint Parameter Selection” is therefore best treated as an umbrella description for coupled multi-block selection and estimation problems, not as a uniquely standardized term. The unifying idea is that parameter classes, structural choices, or resource allocations are selected together because their effects are not additively separable. Where that coupling is weak, sequential or decoupled methods may suffice; where it is strong, the literature repeatedly shows that conditional convexity, shared likelihoods, shared validation surrogates, or constraint-preserving combinatorial search become the central design principles.

Source: https://www.emergentmind.com/topics/joint-parameter-selection-jps