--- title: Collocation-Point Resampling Methods url: https://www.emergentmind.com/topics/collocation-point-resampling type: topic --- # Collocation-Point Resampling Methods Searching arXiv for recent papers on collocation-point resampling and closely related methods. Collocation-point resampling denotes a family of techniques that modify where a collocation method enforces its governing equations. In one line of work, resampling is an interpolation operation: values known on a source collocation grid are transferred to delayed, shifted, or otherwise transformed target points through a barycentric resampling matrix, yielding global spectral discretizations for functional and delay differential equations [2402.12952]. In another line of work, especially for physics-informed neural networks (PINNs), resampling means replacing or moving the collocation set itself during training so that residual evaluations are concentrated in regions judged most informative by residual magnitude, curvature, Hessian structure, Neural Tangent Kernel criteria, reduced-order residual dynamics, or direct gradient ascent on the squared residual [2404.12282], [2504.00910], [2411.19632], [2404.07662], [2501.07700]. Related uses also appear in rectangular collocation for the Kohn–Sham equation, where machine learning reduces the collocation set size while retaining mHartree accuracy, and in boundary element collocation, where oversampling produces rectangular systems that improve robustness and sometimes convergence [1904.07122], [2103.17212]. ## 1. Definitions and major variants The unifying idea is to treat the collocation set as an object that can be transformed rather than fixed. For classical spectral collocation, this transformation is algebraic. Let $\{x_j\}_{j=0}^N$ be the Chebyshev–Gauss–Lobatto nodes on $[-1,1]$, $$ x_j=-\cos\Bigl(\frac{j\pi}{N}\Bigr),\qquad j=0,\dots,N. $$ Given nodal data $y_j=y(x_j)$, the degree-$N$ interpolant can be written in the second barycentric form, $$ p(x)=\frac{\displaystyle\sum_{j=0}^N \frac{w_j}{x-x_j}\,y_j}{\displaystyle\sum_{j=0}^N \frac{w_j}{x-x_j}}, $$ with barycentric weights $$ w_0=w_N=\frac12(-1)^j,\qquad w_j=(-1)^j,\quad j=1,\dots,N-1. $$ Resampling consists of evaluating this interpolant at target points $\xi_i$ through a matrix $R$ whose entries are $R_{i,j}=\ell_j(\xi_i)$, where $\ell_j$ are the Lagrange basis functions [2402.12952]. For PINNs, the term usually refers to adaptive replacement of the set $\mathcal T$ of collocation points used in the residual loss $$ \mathcal L(\theta)=\frac{1}{|\mathcal T|}\sum_{(x,t)\in\mathcal T}|f(x,t;\theta)|^2. $$ The objective is to relocate a fixed number of points toward regions where the current network estimate is least accurate, thereby improving the objective used in subsequent optimization [2404.12282]. This can be implemented as probability-based resampling from a candidate pool, deterministic low-discrepancy initialization followed by adaptive replacement, or direct continuous movement of points by gradient-based optimizers [2404.12282], [2411.19632]. A broader collocation literature uses the same principle in other ways. Rectangular collocation for the Kohn–Sham equation begins with a very large spatial grid and then filters and optimizes a much smaller collocation set via Gaussian process regression and a genetic algorithm [1904.07122]. Oversampled boundary element collocation uses $M\ge N$ collocation points for an $N$-dimensional trial space and solves the resulting $M\times N$ system in weighted least squares form [2103.17212]. This suggests that “collocation-point resampling” is best understood as a methodological umbrella rather than a single algorithm. ## 2. Spectral collocation resampling via barycentric interpolation In the Chebyshev-spectral setting, the resampling operator is explicit. If values are known on the source grid $x_j$ and interpolation is required at target points $\xi_i$, the barycentric resampling matrix is $$ R_{i,j} =\ell_j(\xi_i) =\frac{w_j(\xi_i-x_j)^{-1}}{\sum_{k=0}^N w_k(\xi_i-x_k)^{-1}}. $$ The paper gives a short MATLAB routine, ```matlab function P = barymat(tau,t,w) P = w.'./(tau - t.'); P = P./sum(P,2); P(isnan(P)) = 1; end ``` which forms the matrix entry-by-entry and handles the diagonal case $\xi_i=x_j$ by setting $P_{i,j}=1$ [2402.12952]. This resampling matrix is integrated with the pseudospectral differentiation matrix $D$, defined by differentiating the Lagrange basis at the collocation points, $$ D_{i,j}=\ell_j'(x_i),\qquad i,j=0,\dots,N. $$ For the delay differential equation $$ y'(t)=-y(t)-y(t/2)+e^{-t/2},\qquad y(0)=1, $$ one maps Chebyshev points to $[0,1]$, forms $D$, the identity $I$, and the resampling matrix $$ R=\texttt{barymat}(x/2,x,w). $$ The collocation-point discretization enforces $$ (Dy)_i=-y_i-(Ry)_i+e^{-x_i/2}, $$ or in matrix form, $$ (D+I+R)y=f,\qquad f_i=e^{-x_i/2}. $$ The boundary condition is imposed by row-replacement: the first row of $A:=D+I+R$ is replaced by $[1,0,\dots,0]$ and $f_0$ by $1$, after which one solves the $(N+1)\times(N+1)$ system $Ay=f$ [2402.12952]. The same construction extends to more general functional terms $y(\tau(t))$ by building $R=\texttt{barymat}(\tau(x),x,w)$, and it also extends to state-dependent delays by including $R$ and its Jacobian inside a Newton iteration [2402.12952]. Because the method is a natural extension of standard spectral collocation based on polynomial interpolants, it can be incorporated into existing spectral discretizations such as Chebfun/Chebop [2402.12952]. The principal significance of this version of resampling is that it does not seek new collocation points; it transports an existing polynomial representation to functionally transformed arguments. A common misconception is therefore to equate all collocation-point resampling with adaptive point selection. In the spectral DDE/FDE literature, resampling is instead an interpolation operator embedded in the discretization itself. ## 3. Multidomain treatment, discontinuities, and convergence in spectral schemes When the history function or the delay induces jumps in derivatives at known times $01$ about $[-1,1]$, then the Chebyshev interpolant satisfies $$ \|y-p_N\|_\infty=O(\rho^{-N}), $$ with analogous behavior for derivatives [2402.12952]. This carries over to smooth spectral-collocation solutions of ODEs and DDEs built from such interpolants. If the solution is piecewise analytic with jumps in its $k$-th derivative and subinterval boundaries are introduced at those jumps, the same $\rho^{-N_k}$ geometric convergence is recovered on each subinterval [2402.12952]. By contrast, if no domain splitting is used in the presence of a derivative jump, the observed convergence is only algebraic, $O(1/N^k)$, because of Gibbs-type effects [2402.12952]. These statements place resampling within the standard spectral narrative: the resampling matrix itself is not a cure for nonsmoothness, but it preserves the global high-order structure once discontinuities are isolated. A plausible implication is that in this setting the decisive design variable is not merely the interpolation formula, but the compatibility between resampling and domain decomposition. ## 4. Adaptive resampling in PINNs: probability densities and guiding metrics In PINNs, adaptive collocation-point resampling is usually framed as sampling from a candidate set $\mathcal X$ of size $M\gg N$ according to a normalized probability density function derived from a guiding metric $Y(x)$. A generic form is $$ P(x)=\frac{Y(x)^k}{\overline{Y^k}+c},\qquad \hat P(x)=\frac{P(x)}{\sum_{x'\in\mathcal X}P(x')}, $$ with hyper-parameters $k,c>0$ [2404.12282]. The strategies summarized in the literature include fixed uniform random sampling, fixed Hammersley sampling, residual-based sampling with either random or Hammersley initialization, solution geometric curvature with $Y(x)=|\partial_x\partial_t u(x;\theta)|$, and residual curvature with $Y(x)=|\partial_x\partial_t f(x;\theta)|$ [2404.12282]. For Burgers’ equation, $$ u\,u_x+u_t=\nu u_{xx}, $$ the residual is $$ f(x,t)=u\,u_x+u_t-\nu u_{xx}. $$ The relevant guiding metrics are then the residual magnitude $|f|$, the mixed derivative of the solution $|\partial_x\partial_t u|$, and the mixed derivative of the residual $|\partial_x\partial_t f|$, all computed by automatic differentiation [2404.12282]. A typical cycle samples $M$ candidate points uniformly in the domain, evaluates the guiding metric, constructs the normalized PDF, draws $N$ new collocation points, replaces the previous set, and continues training for another $R$ steps [2404.12282]. The empirical results in the comparative study are problem-dependent but structured. For Burgers’ with $\nu=\pi/100$, $N=2000$, and 30 resamples, the averaged $L^2$ errors over 20 runs were reported as $4.5\times10^{-3}$ for fixed random, $2.1\times10^{-3}$ for fixed Hammersley, $1.3\times10^{-3}$ for PDE,R, $1.0\times10^{-3}$ for PDE,H, $8.5\times10^{-4}$ for $U_{xt}$, and $4.8\times10^{-4}$ for $PDE_{xt}$ [2404.12282]. For Allen–Cahn with $D=0.001$, $N=500$, and 50 resamples, the corresponding values were $2.0\times10^{-2}$, $1.2\times10^{-2}$, $8.5\times10^{-3}$, $6.7\times10^{-3}$, $9.2\times10^{-3}$, and $5.4\times10^{-3}$ [2404.12282]. The same source states that all adaptive methods outperform fixed for small $N$, that $PDE_{xt}$ often yields $2\times$–$5\times$ lower error than residual-only sampling at fixed $N$, and that there exists a problem-dependent threshold $N^*$ above which fixed Hammersley sampling suffices [2404.12282]. A central caution in this literature is explicit: no single approach is likely to be “optimal” [2404.12282]. This is not merely a rhetorical qualification. Residual-based sampling directly targets the largest local contributions to the PINN loss, curvature-based metrics focus on shocks or interfaces, and fixed low-discrepancy sets preserve global coverage [2404.12282]. The trade-off is therefore intrinsic to the choice of guiding information. ## 5. Hessian-, gradient-, NTK-, and DEIM-based variants Several later methods refine the choice of guiding information. A Hessian-based method defines the interior residual $r(x)$, the pointwise loss integrand $f(x)=[r(x)]^2$, and the Hessian matrix $H_f(x)=\nabla^2 f(x)$. It then uses a local indicator such as $\gamma(x)=\|H_f(x)\|_F$ or $\|H_f(x)\|_2$ and samples from a density $$ p(x)\propto \gamma(x)^\tau, $$ optionally shifted by a small constant $c$ for exploration [2504.00910]. The motivation comes from a refined quadrature argument. In one dimension, if $[a,b]$ is partitioned into cells $I_j$ of length $\ell=(b-a)/k$, with $$ M_j=\max_{x\in I_j}|f''(x)|,\qquad n_j=\left\lceil N\cdot\frac{\sqrt{M_j}}{\sum_{p=1}^k\sqrt{M_p}}\right\rceil, $$ then Theorem 3.1 gives $$ E_{\text{refined}}\le \sum_{j=1}^k \frac{\ell^3|f''(\xi_j)|}{12\,n_j^2} \le E_{\text{uniform}}, $$ and Theorem 3.2 states that the refined Hessian-based allocation yields a tighter worst-case error bound when $f''$ is nonuniform [2504.00910]. Comparative experiments summarized in the paper report, for example, that Hessian-RAD converges in $\sim 3000$ epochs to a ten-times smaller error on 1D Newton’s law of cooling, whereas the uniform PINN converges in $\sim 12000$ epochs to $L_2$-error $\approx 10^{-3}$ [2504.00910]. PACMANN formulates adaptive placement differently. It defines the pointwise residual $r(x,t)$ and squared residual $R(x,t)=r(x,t)^2$, then moves the collocation points by gradient-based optimization guided by $\nabla R$ [2411.19632]. Training is cast as a saddle-point problem in which the network parameters are minimized while the collocation set is chosen to maximize the residual term over the domain [2411.19632]. The method updates points every $P$ training iterations, performs $T$ ascent steps using gradient ascent or Adam, and reinitializes points that leave the domain by uniform random draws [2411.19632]. The paper states that the method matches the performance of state-of-the-art methods in low-dimensional problems and outperforms available approaches for high-dimensional problems, with the best performance observed for Adam [2411.19632]. It also states a limitation: there is no formal guarantee of global optimality of the collocation-set placement, and the process may settle in local residual maxima [2411.19632]. PINNACLE replaces pointwise residual heuristics by an NTK-based score. For a candidate point $z$, the “convergence-degree” score is $$ s(z)=\nabla_\theta F[u_t](z)^\top\,\Theta_t\,\nabla_\theta F[u_t](z), $$ where $\Theta_t$ is the empirical NTK [2404.07662]. The method forms a candidate pool over PDE and boundary point types, computes the eNTK via Nystrom approximation, selects collocation points either by sampling proportional to $s(z)$ or by a K-means++ procedure in an NTK-derived embedding, and updates the fractions of PDE, boundary, and experimental points automatically [2404.07662]. The theoretical result given in the summary relates the convergence degree of a training set to the PINN generalization error through the bound $$ E_{x\sim\Omega}[|u_\infty(x)-u(x)|] \le C_1\|r(S)\|_1 + C_2\,\alpha(S)^{-1/2}+C_3, $$ where $\alpha(S)$ is defined from NTK eigenpairs and residual components [2404.07662]. QR-DEIM-based resampling uses residual dynamics over time rather than an instantaneous metric. Over a period $P$, residual snapshots on a fixed set $\mathcal S=\{s_j\}_{j=1}^{N_s}\}$ are assembled into $$ Y=[r_1,\dots,r_P]\in\mathbb R^{N_s\times P}, $$ with thin SVD $$ Y=V\Sigma W^T. $$ A reduced basis $U=V(:,1\!:\!k)$ is selected by thresholding the singular values, and a column-pivoted QR factorization of $U^T$ identifies the DEIM interpolation points [2501.07700]. The resulting approximation $$ r\approx U(P_{\rm ind}^TU)^{-1}P_{\rm ind}^T r $$ selects the most informative residual locations in the reduced basis sense [2501.07700]. The paper reports benchmark improvements on the wave, Allen–Cahn, and Burgers’ equations, including relative $L^2$ errors of $7.04\times10^{-3}\pm5.31\times10^{-3}$ for QR-DEIM on the wave equation, compared with $1.85\times10^{-2}\pm3.64\times10^{-2}$ for uniform random sampling, and $3.75\times10^{-2}\pm2.22\times10^{-2}$ for Burgers’, compared with $2.69\times10^{-1}\pm1.42\times10^{-1}$ for uniform random sampling [2501.07700]. The same source states that no PINN-specific convergence proof is given [2501.07700]. ## 6. Other collocation settings: rectangular collocation and oversampling Outside the PINN literature, collocation-point resampling also appears as point-set compression or oversampling. In rectangular collocation for the Kohn–Sham equation, $$ \left[-\frac12\Delta+V_{\rm eff}(x)\right]\psi(x)=E\psi(x), $$ one expands the wavefunction in a basis and enforces the equation at $M$ collocation points, typically with $M\gg N$ [1904.07122]. The matrix form is $$ (D+VF)c=EFc, $$ and after left-multiplication by $F^T$ one obtains the square generalized eigenproblem $$ F^T(D+VF)c=E(F^TF)c. $$ The collocation set is first filtered from a large cube grid using a potential-based probability $$ P(x_i)=\frac{V_{\max}-V_{\rm eff}(x_i)+\delta}{V_{\max}-V_{\min}+\delta}, $$ then optimized by a Gaussian-process-regression fitness based on the RMSE of the predicted effective potential on a fixed test grid, with a genetic algorithm performing selection, uniform crossover, and pointwise mutation at probability $p_{\rm mut}=4\%$ [1904.07122]. The reported outcome is a reduction of the collocation point set size by more than an order of magnitude, from about 51,000 points to 2,000 points, while retaining mHartree accuracy [1904.07122]. Oversampled boundary element collocation uses a different resampling logic. One chooses $$ \Delta_M=\{x_1,\dots,x_M\}\subset\Gamma,\qquad M=JN,\ J\ge 1, $$ forms the $M\times N$ collocation matrix $$ \mathcal G_{MN}=\bigl(V\phi_n(x_m)\bigr), $$ and solves the overdetermined system through the normal equations $$ (\mathcal G_{MN}^*\mathcal W_M\mathcal G_{MN})\mathbf a =\mathcal G_{MN}^*\mathcal W_M\mathbf f. $$ The summary states that oversampling at least lowers the error by a constant factor in most cases of practical interest, that oversampled collocation is much less affected by a poor choice of collocation points, and that in some cases convergence rates can be higher than the corresponding Galerkin method, although this requires oversampling at a faster than linear rate [2103.17212]. For equispaced points and Fourier-multiplier operators, Theorem 3.8 gives $$ \|u_N^{(M)}-u\|_{H^{4\alpha-(d+1)}} \le C\Bigl(M^{-(d+1)+2\alpha}+N^{-2(d+1)+4\alpha}\Bigr)\|u\|_{H^{d+1}}, $$ from which the source concludes that for any fixed $N$ and $J>1$, the error constant decays like $J^{-(d+1)+2\alpha}$ [2103.17212]. These examples show that resampling is not restricted to online adaptation during optimization. It can also mean selecting a smaller but better collocation set before solving the discretized problem, or enforcing the same trial space at a larger set of collocation points to stabilize the method. ## 7. Practical issues, misconceptions, and open directions Several practical recommendations recur across the literature. For adaptive PINN resampling, candidate-set sizes on the order of $5{,}000$–$20{,}000$ or $10{,}000$–$50{,}000$ per cycle are reported, with resampling every $1{,}000$–$2{,}000$ optimizer steps or every fixed number of epochs, and defaults such as $k=1$, $c\sim 10^{-6}$, or comparable small exploration constants [2404.12282], [2504.00910]. Hammersley initialization is repeatedly used as a strong fixed baseline [2404.12282]. PACMANN states that the extra cost remains a small constant factor when $T\ll P$, and its scaling is linear in both the number of residual points $N_r$ and the ambient dimension $d$ [2411.19632]. QR-DEIM reports one SVD-plus-QR update every $P$ iterations and describes the overall overhead as approximately $1$–$5\%$ of training time if $P\gtrsim 10^3$ [2501.07700]. Several misconceptions are explicitly contradicted by the cited work. First, adaptive resampling is not universally superior: fixed Hammersley sampling can suffice once the collocation budget exceeds a problem-dependent threshold $N^*$ [2404.12282]. Second, residual magnitude is not the only meaningful guide. Curvature-based and Hessian-based indicators may better resolve shocks, interfaces, or steep regions, while NTK-based methods target training dynamics across point types rather than only instantaneous residuals [2404.12282], [2504.00910], [2404.07662]. Third, adaptive methods do not all scale the same way. The PACMANN abstract states that previous adaptive collocation point sampling methods have been reported to scale poorly to higher dimensions, which motivates its gradient-based point-movement formulation [2411.19632]. Fourth, stronger empirical performance does not necessarily come with a formal PDE-specific guarantee: QR-DEIM explicitly lacks a PINN-specific convergence proof, and PACMANN explicitly lacks a formal guarantee of global optimality for the point placement [2501.07700], [2411.19632]. A plausible implication of the collected literature is that collocation-point resampling is increasingly being treated as a numerical design problem in its own right. In spectral DDE/FDE solvers, the design variable is the evaluation operator that maps global polynomial data to transformed arguments [2402.12952]. In PINNs, it is the online distribution of residual evaluations under a fixed computational budget [2404.12282], [2504.00910], [2411.19632], [2404.07662], [2501.07700]. In rectangular collocation and boundary element methods, it is the geometry and cardinality of the collocation set relative to the trial space [1904.07122], [2103.17212]. What remains common across these settings is the premise that the accuracy, robustness, and efficiency of collocation methods depend not only on the approximation space and optimizer, but also on how the collocation points are transformed, selected, or oversampled. Source: https://www.emergentmind.com/topics/collocation-point-resampling