Parameter Placement Problem: Optimization Insights

• The Parameter Placement Problem is a framework for optimizing the allocation of limited degrees of freedom under structural constraints across diverse applications.
• It encompasses formulations such as free-knot placement in B-spline regression, trainable-entry selection in LoRA, sensor positioning for inverse problems, and allocation in caching and chip design.
• Practical insights include algorithmic strategies like the Iterative Local Placement (ILP) for adaptive knot selection and gradient-informed methods that enhance approximation accuracy and system performance.

In the literature assembled here, the Parameter Placement Problem does not denote a single canonical optimization problem. It instead refers to a family of placement questions in which performance depends on where limited degrees of freedom are assigned: free knots in B-spline bases, trainable entries in a LoRA adapter, graph nodes onto constrained resources, sensors in a physical domain, file fragments in coded caching, or products across display locations. The common theme is the optimization of a placement variable under structural constraints and a finite budget, with objectives ranging from approximation accuracy and regression quality to runtime, average load, estimator covariance, and expected revenue (Shi et al., 2024, Sehanobish et al., 12 May 2026, Goldie et al., 2020, Neitzel et al., 2019, Jin et al., 2017, Housni et al., 2023).

1. Cross-domain formulation and recurring mathematical structure

Across these works, placement is expressed through domain-specific decision variables. In functional approximation, the decision variable is an open knot vector for a degree-$p$ B-spline basis, chosen so that one or many functions satisfy a prescribed global error tolerance with as few interior knots as possible (Shi et al., 2024). In LoRA, the decision variable is a subset $S$ of trainable entries within the $B$ matrix, with $|S|=k$, while $A$ is frozen and gradients are masked outside $S$ (Sehanobish et al., 12 May 2026). In systems and chip design, placement is a mapping $\pi:V\to R$ or a geometric assignment $x=\{x_v\}$ of graph nodes to limited resources or physical locations, subject to capacity, legality, timing, and non-overlap constraints (Goldie et al., 2020).

Budgeted sensor placement is formulated either as a binary vector $x\in\{0,1\}^n$ with $\sum_i x_i=B$ in black-box optimization (Attia, 2024), as a positive Borel measure $S$0 over an observation set in PDE-constrained design (Neitzel et al., 2019), or as a continuous location vector $S$1 in online filtering and diffusion models (Sharrock et al., 2020). In coded caching, placement is the file partition parameter $S$2 or its symmetry-reduced form $S$3, constrained by per-file partition and cache-capacity constraints (Jin et al., 2017). In retail placement, the variable is a mapping $S$4 assigning products to unit-capacity display locations under a browsing distribution and a discrete choice model (Housni et al., 2023).

Taken together, these formulations suggest a shared abstraction: a constrained optimization over a placement variable, with feasibility imposed by cardinality, geometry, support, cache, or legality constraints, and with performance determined by how placement modulates approximation error, gradient accumulation, information, communication, or choice behavior. A plausible implication is that “placement” is best understood as a structural allocation problem rather than as a domain-specific heuristic.

2. Free-knot placement in nonlinear functional regression

A particularly explicit use of the term appears in nonlinear functional data analysis, where the Parameter Placement Problem is the task of determining an adaptive, data-driven knot distribution for B-spline bases so that one or many functions can be approximated to a prescribed accuracy with as few knots as possible. The proposed solution is the Iterative Local Placement (ILP) algorithm, embedded in a Mapping-to-Parameter (M2P) functional regression framework (Shi et al., 2024).

The approximation problem is posed in two forms. For a single function $S$5 on $S$6, one chooses a degree-$S$7 B-spline basis and knot vector so that the approximant $S$8 satisfies a global error tolerance $S$9. For multifunction approximation, given $B$0, the goal is to determine a common knot set and shared basis such that each $B$1 admits coefficients $B$2 with $B$3 for all $B$4. The placement criterion is governed by the worst local complexity across functions, quantified by the magnitude of their $B$5-th derivatives. Coefficients are then obtained by least squares on the fixed shared basis. The paper supplies Lemma 1 and Theorem 1, which provide local-to-global sup-norm control and extend the guarantee to unified multifunction approximation.

ILP is deterministic and constructive. It estimates $B$6-th derivatives by finite differences, forms the worst-case local complexity profile $B$7, starts with no interior knots, and proceeds left to right. On each span it extends the right boundary as far as possible while preserving the local error criterion; it then places an interior knot and repeats until the domain is partitioned. No iterative gradient updates are needed. Its stated complexity is $B$8 for derivative computation, $B$9 for the placement pass, and $|S|=k$0 total for coefficient solves with $|S|=k$1 basis functions.

The resulting basis acts as a dimension-reduction map from infinite-dimensional function space to finite-dimensional parameter space. With shared bases for inputs and outputs, each function is mapped to a coefficient vector $|S|=k$2 or $|S|=k$3, so scalar-on-function and function-on-function problems become ordinary supervised learning in $|S|=k$4 or $|S|=k$5. The framework is explicitly agnostic to the downstream learner and permits MLP, CNN, LSTM, and related models.

Empirically, ILP outperformed equidistant knot placement in both single-function and unified multifunction approximation. For four representative solar irradiance days, ILP achieved consistently lower MaxAE and RMSE than equidistant knots under equal knot counts, and under matched GCV error required fewer knots; for the summer day 06-21, equidistant achieved a minimal GCV of $|S|=k$6 using 136 knots, whereas ILP achieved a better GCV of $|S|=k$7 with only 85 knots. In regression, M2P-MLP attained the lowest MSPE on Tecator, Gasoline, Bike Sharing, and DTI, and lower MSPEM and MSPEF than FLR-FOF and FBNN on Gait, Daily, Electricity, and Traffic (Shi et al., 2024).

The main limitations are equally explicit. Because ILP uses the worst-case derivative profile, it may allocate more knots than smoother functions need. Finite-difference derivatives are unstable under noisy or sparse observations, so denoising or robust derivative estimation is recommended. Very small $|S|=k$8 values inflate knot counts and can induce overfitting, while discontinuities may require alternative bases such as wavelets.

3. Trainable-entry selection in low-rank adaptation

In low-rank adaptation, the Parameter Placement Problem is posed in a sharply discrete form: given a fixed budget of $|S|=k$9 trainable entries within the $A$0 matrix of a LoRA adapter, with $A$1 frozen, does the choice of which $A$2 entries matter? The answer depends strongly on training regime (Sehanobish et al., 12 May 2026).

The setup freezes all $A$3, initializes all $A$4, and selects a subset $A$5 of entries across the collection of $A$6 matrices, with $A$7. A binary mask $A$8 then enforces updates of the form $A$9. Two training regimes are analyzed: supervised fine-tuning (SFT) and GRPO. The central empirical observation is regime dependence. Under SFT, random and informed subsets achieve comparable performance. Under GRPO on base models, random placement fails to improve over the base model, while gradient-informed placement recovers standard LoRA accuracy.

The explanation is given in terms of gradient structure. SFT gradients are reported as low-rank and directionally stable, with low effective rank, high cosine similarity across steps, and high accumulation efficiency. GRPO gradients on base models are high-rank and near-orthogonal across steps, so arbitrary subsets experience cancellation. For Qwen2.5-1.5B on MATH, the reported initialization statistics are: SFT $S$0, cosine $S$1, $S$2; GRPO $S$3, cosine $S$4, $S$5. This is the basis for the paper’s claim that under SFT “how many” dominates, whereas under GRPO “which ones” becomes decisive.

The scoring procedure is lightweight. At initialization, over $S$6 forward-backward passes, one computes either the gradient-magnitude score $S$7 or the diagonal Fisher score $S$8, selects the global top-$S$9 entries, and keeps that mask fixed throughout training. The reported discovery cost is under 10 seconds and less than 0.5% of training cost, with top-10K circuits matching $\pi:V\to R$0 references at at least 97–99% overlap already at $\pi:V\to R$1.

The numerical contrast between SFT and GRPO is large. On the 7-benchmark SFT average, Qwen2.5-1.5B with $\pi:V\to R$2K scores Base 53.0, Random 61.6, $\pi:V\to R$3 61.9, $\pi:V\to R$4 62.1, and Full LoRA 62.4. On GSM8K under GRPO, Qwen2.5-1.5B with $\pi:V\to R$5K scores Base 9.5, Random 9.8, $\pi:V\to R$6 64.2, $\pi:V\to R$7 63.2, and Full LoRA 62.8. On MATH-500, the same model with $\pi:V\to R$8K scores Base 17.4, Random 18.8, $\pi:V\to R$9 43.6, $x=\{x_v\}$0 42.2, and Full LoRA 41.4. The paper also reports that selected parameters concentrate on residual-stream-writing projections—especially V, O, and Down—and in early layers, with 70–98% of selected entries falling in layers 0–4.

A common misconception addressed by these results is that sparsity budget alone determines performance. The paper’s data reject that claim for GRPO on base models: random placement remains near the base model across budgets and LoRA $x=\{x_v\}$1, whereas gradient-informed selection matches or exceeds Full LoRA at the same tiny budgets. By contrast, under SFT the gap between random and informed placement is usually modest, which prevents the result from being universalized across regimes.

4. Placement as constrained graph-to-resource optimization

In systems and chip design, placement is formalized as a constrained optimization over mappings from graph nodes to limited resources, and deep reinforcement learning is presented as a general end-to-end solution strategy (Goldie et al., 2020). Here the placement variable may be discrete, $x=\{x_v\}$2, or geometric, $x=\{x_v\}$3, and the objective aggregates domain-specific costs such as wirelength, congestion, timing slack, communication, load balancing, and latency.

The paper casts sequential placement as an episodic Markov decision process. The state encodes graph features, learned GNN embeddings, partial assignments, utilization vectors, and feasibility masks. Actions assign the next node to a resource or location. Rewards may be purely terminal, as when runtime is defined only for full placements, or incrementally shaped by partial costs such as wirelength or density updates. Constraints are handled through action masking for hard feasibility and graded penalties for soft constraints.

The policy is parameterized by a deep network operating on graph-structured state representations. Graph neural networks are emphasized because they support domain adaptation across unseen graphs and capture multi-hop structure. The optimization target is the expected reward under the stochastic policy, with policy-gradient methods such as REINFORCE, PPO, and TRPO serving as the algorithmic backbone. The discussion stresses that fast, correlated proxy objectives are often necessary because exact evaluation—routing, static timing analysis, or full runtime measurement—can be prohibitively expensive.

Within this framework, the paper explicitly maps a parameter placement problem for distributed machine learning into the same formalism. The set $x=\{x_v\}$4 becomes parameters, tensor shards, tasks, or blocks; $x=\{x_v\}$5 becomes devices, servers, bins, or CLBs; edges encode communication or netlist connectivity; and the cost includes communication, update latency, memory and compute balance, and affinity or anti-affinity rules. A placement policy then assigns parameter shards to devices with capacity masks and reward terms derived from communication, latency, and balance.

The paper does not report head-to-head quantitative benchmarks, but it states several practical lessons. DRL is strongest when proxy rewards are fast and strongly correlated with the true objective, and when transfer across many instances amortizes training cost. It underperforms when feasible actions are sparse without effective masking or when rewards are too slow to compute. Classical analytical placers therefore remain highly competitive, especially in large ASIC global placement with differentiable surrogates.

5. Sensor placement for parameter estimation and inverse problems

Optimal sensor placement is one of the most developed forms of the Parameter Placement Problem in the supplied literature. The shared objective is to choose locations, weights, or binary selections of sensors so as to maximize information about unknown parameters or, equivalently, reduce estimator uncertainty.

A continuous-time bilevel formulation appears for partially observed diffusion processes, where sensor locations $x=\{x_v\}$6 affect the observation operator $x=\{x_v\}$7 and possibly the measurement-noise covariance $x=\{x_v\}$8. The lower-level objective is the asymptotic log-likelihood used for recursive maximum-likelihood estimation of $x=\{x_v\}$9, and the upper-level objective is an asymptotic A-optimal criterion based on the time-averaged trace of the conditional covariance. The paper proposes a projected two-timescale continuous-time stochastic gradient scheme, with a slow timescale for $x\in\{0,1\}^n$0 and a fast timescale for $x\in\{0,1\}^n$1, and establishes almost sure convergence to stationary points under additive-noise and Markovian-noise assumptions (Sharrock et al., 2020).

A complementary PDE-constrained approach encodes a measurement design as a positive measure $x\in\{0,1\}^n$2 on an observation set $x\in\{0,1\}^n$3, with Fisher information

$x\in\{0,1\}^n$4

where $x\in\{0,1\}^n$5 is the sensitivity vector of the parameter-to-state map. The optimization problems are either penalized, $x\in\{0,1\}^n$6, or budget-constrained, $x\in\{0,1\}^n$7 subject to $x\in\{0,1\}^n$8. Existence, duality, and sparsity are proved, including the finite-atomic bound

$x\in\{0,1\}^n$9

Algorithmically, the paper derives accelerated conditional-gradient methods in measure space, together with sparsification and semismooth Newton subsolvers. In the reported convection-diffusion example with $\sum_i x_i=B$0, the optimal design yields approximately $\sum_i x_i=B$1, $\sum_i x_i=B$2, $\sum_i x_i=B$3, and $\sum_i x_i=B$4, compared with $\sum_i x_i=B$5 for a naive 3-sensor reference design of the same mass (Neitzel et al., 2019).

When the objective is black-box and binary, the same theme is addressed probabilistically. The decision variable is $\sum_i x_i=B$6 with a hard budget $\sum_i x_i=B$7 or $\sum_i x_i=B$8, and the optimizer is a Conditional Bernoulli or generalized Conditional Bernoulli policy over feasible placements. The original objective is replaced by $\sum_i x_i=B$9, and score-function gradients are computed with exact feasible sampling, dynamic-programming recurrences of complexity $S$00, and variance-reduction baselines. The stated advantages are that sampling is restricted to the feasible region, computational cost is reduced relative to soft constraints, and no regularization parameter is needed to promote sparsity (Attia, 2024).

A PINN-based formulation uses automatic differentiation to compute parameter sensitivities and then selects sensors through a D-optimality surrogate. Parameters are appended to the PINN inputs, sensitivities are obtained by differentiating the network output with respect to those inputs, and the placement objective maximizes $S$01 for transient problems or $S$02 in the steady-state rank-1 case. In the reported 1D reaction–advection–diffusion example, the trace profile peaks at $S$03 for the prior $S$04; using data from that location, the inferred value is $S$05 for a ground truth of $S$06, whereas using the intuitive outlet location $S$07 gives $S$08. In the 2D transient case with ground truth $S$09 and $S$10, optimal placements consistently outperform intuitive ones, and for the 3-sensor noisy setting the reported means are $S$11 versus $S$12, and $S$13 versus $S$14 (Venianakis et al., 19 Nov 2025).

A recurrent misconception in inverse problems is that the number of sensors is the dominant issue. These works instead show that placement quality can dominate count, whether the metric is asymptotic covariance, a Fisher-information surrogate, or final parameter error. The stronger claim that more sensors always help is explicitly contradicted in the PINN study, where intuitive placements do not necessarily improve with more sensors.

6. Placement in coded caching and product display

Two further strands show that parameter placement can also mean allocating informational or commercial exposure under combinatorial constraints.

In coded caching with uncoded placement and coded delivery, the design variable is the file partition parameter $S$15, the normalized size of subfile $S$16 for file $S$17 and user subset $S$18. The objective is to minimize the average load over i.i.d. demand vectors under cache-capacity constraints. A direct convex formulation uses $S$19 variables, but under type symmetry and popularity monotonicity it reduces to a linear program with $S$20 variables for arbitrary popularity, and under uniform popularity to a $S$21-variable LP. For integer $S$22, the optimal uniform-popularity solution is

$S$23

which recovers Yu et al.’s optimal centralized coded caching scheme. When low subpacketization is required, the paper adds per-file $S$24 constraints and reformulates the problem as a Difference-of-Convex program (Jin et al., 2017).

In the placement of substitutable products, the variable is a mapping $S$25 from unit-capacity locations to products, under a browsing distribution $S$26 over subsets of locations and a weakly substitutable choice model $S$27. Expected revenue is

$S$28

with assortment revenue $S$29. The main algorithmic result is a randomized best-of-many construction: for each cardinality $S$30, compute an $S$31-approximate assortment $S$32, place products by independent uniform sampling from $S$33, estimate the resulting revenue, and return the best candidate. This yields an expected guarantee of

$S$34

and the logarithmic dependence is stated to be tight in the worst case. The same paper gives a deterministic $S$35 approximation for the Markov choice model and a tight $S$36 approximation when prices are identical (Housni et al., 2023).

These formulations show that placement need not be geometric or continuous. In both coded caching and product display, the decisive issue is how a constrained combinatorial allocation interacts with downstream demand structure: common-request exploitation in the former, and browsing-plus-choice behavior in the latter. This suggests that the unifying content of the Parameter Placement Problem lies less in the object being placed than in the structural role played by placement in shaping achievable performance.