Scaling Laws for Parameter Identifiability

• Scaling laws for parameter identifiability are quantitative frameworks that delineate how data size, noise, and model structure determine the resolvability of parameters.
• They employ tools like the Fisher Information Matrix and perturbed Hessians to establish hierarchical identifiability orders, enhancing model inference and uncertainty quantification.
• Applications in biological modeling and bilinear inverse problems validate these laws, guiding experimental design and improving the practical interpretability of complex models.

Parameter identifiability scaling laws delineate the explicit quantitative relationships between data size, model structure, perturbation/noise, and the resolvability of parameters in high-dimensional models. These laws govern both deterministic and probabilistic settings, formulating precise thresholds and hierarchies for when linear combinations of parameters can be robustly inferred from noisy finite data. Central frameworks utilize the Fisher Information Matrix (FIM), perturbed Hessians, and, in bilinear inverse problems, rank-constrained matrix liftings. Scaling laws for parameter identifiability thus underpin uncertainty quantification, experimental design, and the practical interpretability of complex mechanistic or data-driven models (Wang et al., 24 Feb 2026, Choudhary et al., 2014).

1. Theoretical Foundations of Practical Identifiability

Practical identifiability assesses whether small perturbations in input data induce only small variations in estimated model parameters. Formally, for a least-squares inference

$$l(\theta) = \sum_{i=1}^{N} \|h(t_i; \theta) - \hat h_i\|_2^2,$$

a parameter $\theta$ is practically identifiable if, for any $\epsilon>0$, some $C>0$ exists such that $$||\delta|| < \epsilon \implies ||\theta_\delta-\theta^*|| < C\epsilon$$, where $\theta_\delta$ is the minimizer with perturbed data $\hat h - \delta$.

Coordinate identifiability is determined by profiling the likelihood for each parameter and verifying if a unique local minimum exists along each axis. In bilinear inverse problems (BIPs), identifiability links to the uniqueness of rank-one solutions in a rank-constrained linear system. For

$S(x, y) = z,$

with $x\in\mathbb{R}^m$, $y\in\mathbb{R}^n$, identifiability of $(x, y)$ up to scaling is equivalent to uniqueness of the rank-one matrix $W = x y^T$ subject to the measurement operator $\mathscr S(W) = z$ (Choudhary et al., 2014).

2. Asymptotic Hierarchies: Fisher Information, Perturbed Hessian, and Decomposition

In the small-residual regime, the Hessian of the loss function expands as

$$\nabla_\theta^2 l(\theta^*; \hat h - \delta) \approx F + \epsilon H, \quad \epsilon = O(\|\delta\|),$$

where $F$ is the FIM and $H$ collects higher-order second derivatives. Spectral analysis proceeds via eigenvalue decomposition and Schur complements, yielding:

$$F + \epsilon H \simeq \tilde{U} \begin{pmatrix} \Sigma_{r_0} + B_0 & 0 & \dots \ 0 & \epsilon(\Sigma_{r_1} + B_1) & 0 \ \vdots & 0 & \epsilon^2(\Sigma_{r_2} + B_2) \ \end{pmatrix} \tilde{U}^T,$$

with $r_0 = \operatorname{rank}(F)$, $r_1 = \operatorname{rank}(H|_{\ker(F)})$, etc. This yields a hierarchy:

• Zero-order identifiable subspace: span of $r_0$ FIM eigenvectors.
• First-order identifiable: those in $\ker(F)$ but lifted by $H$.
• Higher-order: so forth for unresolved directions.

Coordinate-level identifiability metrics arise from the expansion:

$$\mathcal{L}_i(\theta_i; \epsilon) = (\mathcal{H}_i^{(0)} + \epsilon \mathcal{H}_i^{(1)} + \epsilon^2 \mathcal{H}_i^{(2)}) \theta_i^2 + \ldots$$

where nonvanishing $\mathcal{H}_i^{(0)}$ denotes zero-order identifiability, etc. (Wang et al., 24 Feb 2026).

3. Explicit Scaling Laws

The scaling behavior depends explicitly on data size ($N$) and perturbation magnitude ($\epsilon$). Scaling laws manifest at two main levels:

• Spectrum-level scaling: If $N \mapsto \alpha N$, then $F \mapsto \alpha F$ and nonzero eigenvalues scale as $O(N)$. Flat (nullspace) directions of $F$ become identifiable at scale $O(\epsilon)$ if $H$ is non-singular there.
• Coordinate-level thresholds: For a parameter $i$,
  • $\mathcal{H}_i^{(0)} > 0$ yields $O(N)$ curvature (fast identifiability),
  • $\mathcal{H}_i^{(0)}=0$, $\mathcal{H}_i^{(1)}>0$ gives $O(N)\epsilon$ curvature,
  • Thus, first-order identifiability requires $\epsilon \gtrsim O(1/N)$.

Corresponding uncertainty quantification reflects this hierarchy. Propagated variance from zero- and first-order non-identifiable subspaces contracts from $O(\epsilon)$ to $O(\epsilon^2)$.

In BIPs, probabilistic scaling laws relate the failure probability for robust identifiability to nullspace complexity $p$ (or $f$), data size, and distributional assumptions:

• For dependent uncorrelated vectors, $P_{\mathrm{fail}} \le 4f/(mn(1-\delta))$.
• For i.i.d. Bernoulli, $$P_{\mathrm{fail}} \le 16 \exp\left( p \ln(1/\epsilon) - (m+n)(1-\delta')/4 \right)$$.
• For i.i.d. Gaussian, $$P_{\mathrm{fail}} \le C \exp\left( p \ln(1/\epsilon) - (m+n)\ln(1/\sqrt{\delta}) \right)$$. Thus, generically $m+n = O(p + \ln(1/\Delta))$ suffices for failure probability $\le \Delta$ (Choudhary et al., 2014).

4. Case Studies and Numerical Validation

Scaling law formulations have been validated in both mechanistic biological modeling and bilinear inverse problem settings.

• HIV–Host Dynamics: With six parameters, the FIM has rank three, corresponding to three zero-order identifiable directions. Hessian $H$ restricted to $\ker(F)$ identifies two first-order directions, and one remains non-identifiable. Uncertainty quantification bands reflect nested (zero- and first-order) contributions.
• Amyloid-β Spatiotemporal Propagation: Of 136 parameters, $\sim$34 are zero-order identifiable, three first-order, and the rest remain unresolved even at first order. Uncertainty bands vanish only at late time points or outbreak windows (Wang et al., 24 Feb 2026).
• Blind Deconvolution and BIPs: Experimental results confirm that failure probability transitions occur at $m+n$ scaling linearly with nullspace complexity. Simulations with small, large, and infinite nullspace complexity show empirical failure probabilities in agreement with predicted scaling exponents (Choudhary et al., 2014).

Table: Scaling Law Manifestations Across Domains

Domain	Identifiability Hierarchy	Failure Probability/Uncertainty Scaling
HIV–Host dynamics	$r_0=3$ (zero), $r_1=2$ (first)	Nested UQ bands per order
Aβ propagation	$r_0 \sim 34$, $r_1=3$	Vanishing UQ at specific times
Blind deconvolution (BIP)	Rank-one via lifting	$P_{\mathrm{fail}}$ exponential in $m+n-p$

5. Impact on Experimental Design and Data-Driven Modeling

Scaling laws inform data collection strategies and model selection in high-dimensional settings:

• The number of truly $O(1)$ identifiable combinations is upper-bounded by the FIM rank, which grows linearly with $N$ at best.
• Flat directions can possess biological significance; first-order analysis via $H$ prevents discarding structurally informative but weakly identifiable directions.
• Scaling laws prescribe that increasing $N$ (data) or decreasing $\epsilon$ (noise) moves flat directions over identifiability thresholds: zero-order at $N\to\infty$, first-order at $\epsilon\gtrsim 1/N$, higher-order at yet smaller $\epsilon$.
• For parameter-wise or coordinate-wise uncertainty quantification, nested confidence bands (per identifiability order) partition predictive uncertainty by its geometric origin.
• In digital twin applications, computing all $\mathcal{H}_i^{(0,1,2)}$ enables targeted experimental interventions to boost identifiability along otherwise ambiguous directions (Wang et al., 24 Feb 2026).

6. Limitations and Generalizations

The standard theory assumes:

• Small-residual regime: Hessian approximated by $F + \epsilon H$.
• $F$ and $H$ are positive semi-definite at the optimal parameter.
• In higher dimensions, computation of $H$ can be intensive but may be mitigated by finite-difference or Gauss–Newton approximations.

Extensions include substituting the loss Hessian for $F$ in alternative loss functions (e.g., cross-entropy). For BIPs, the approach adapts to conic-constraint settings, and identifiability scaling laws continue to be governed by nullspace complexity and sample size interplay (Choudhary et al., 2014, Wang et al., 24 Feb 2026).

7. Broader Context and Connections

Parameter identifiability scaling laws unify perspectives originating from statistical inference (FIM-based analysis), mechanistic modeling, and modern signal processing (low-rank matrix recovery in BIPs). They clarify the achievable resolution of parameter inference given dataset size, noise, and model structure, and motivate uncertainty quantification stratified by identifiability order. A plausible implication is that future high-dimensional modeling frameworks—including mechanistic digital twins—will increasingly rely on such scaling hierarchies to guide both model development and experimental design. This suggests that identifiability diagnostics based on these laws will become an essential component of robust data-driven inference and interpretable modeling practice in diverse domains (Wang et al., 24 Feb 2026, Choudhary et al., 2014).