Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for Implicit Neural Representations

Abstract: The Minkowski functionals of a field's excursion sets -- area, boundary measure, and Euler characteristic -- describe its level-set morphology; the Euler characteristic is the cheapest handle on topology. We derive smooth Monte-Carlo estimators for all three of a continuous neural field, evaluated at scattered points via the co-area formula and Gauss-Bonnet, using only autodiff: no grid, no complex, no persistence. The estimator is accurate to 1-3% against exact topology in 2D and 3D, and costs about 3 ms per iteration where a persistent-homology (PH) loss on a cubical grid costs 650-1000 ms -- a 250x gap. We establish four design rules without which these losses silently fail: a dense level ladder (invariants are flat in the parameters away from transitions), a $C<sup>2$ backbone (ReLU nets hide curvature in kinks), the full Minkowski vector (Euler characteristic alone is an alternating sum, gamed by debris-hole cancellation; pricing perimeter closes the channel), and sampling-scale coverage. In 2D the vector-valued cap is the only method in a controlled comparison that both repairs topology (3/3 seeds) and preserves fidelity -- uniform smoothing repairs at 11-17x the fidelity cost, and the Euler term alone repairs nothing. In 3D neural-SDF fitting, however, a failure mode we believe general to any sampled soft topology objective appears: gradient descent adversarially hides topological noise below the sampling density, where the estimator is blind -- spurious-feature counts are invariant to 4x more samples, and closing the window needs cubically many points, erasing the cost advantage. A grid-based PH baseline, whose complex is the evaluation resolution, solves the same benchmark ($4/9$ exact; median $b_1$ error 1 vs. ours above $10^4$). The 250x cost of persistence is, at present, the price of having no null space. We release estimators, receipts, and benchmarks.

• The paper presents a novel auxiliary loss that leverages Rice level-crossing density to capture high-frequency content in INRs.
• It employs a Monte-Carlo estimator and co-area formula to compute a differentiable loss that is robust to irregular sampling and noisy gradients, improving PSNR by up to 3 dB in sparse regimes.
• Experimental results show that the proposed Kac–Rice loss matches or exceeds traditional frequency-domain losses in high-frequency fidelity, particularly in texture-rich and underdetermined settings.

Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for INRs

Overview and Motivation

Implicit Neural Representations (INRs), where a neural network $f_\theta:\mathbb{R}^d\to\mathbb{R}^m$ parameterizes a signal as a function from coordinates to values, have demonstrated wide applicability for tasks such as image representation, signed distance fields, and radiance fields. However, these models are prone to spectral bias: when trained via standard reconstruction losses, INRs learn low-frequency content rapidly, while high-frequency details are captured slowly or even omitted. Conventional high-frequency loss remedies—frequency-domain objectives (e.g., Focal Frequency Loss, FFL), architectural modifications (e.g., SIREN, Fourier features), and frequency progression curricula—typically assume regular sampling grids, which limits their effectiveness in settings with irregularly sampled data.

This paper introduces a mesh-free, grid-agnostic auxiliary loss for INRs based on the Rice level-crossing density: a classic statistic quantifying the number of times a signal crosses a specified value per unit domain, and which is directly tied, via spectral moments, to the signal's high-frequency content. The proposed loss can be evaluated and differentiated at arbitrary input locations, without requiring grids, finite-difference approximations, or frequency transforms. It is inherently robust to noise in gradient-based targets and is applicable in challenging settings where supervision arrives as scattered or nonuniform samples—a common occurrence in neural SDFs, nonuniform sensor data, or adaptive measurement scenarios.

Methodology

The central technical contribution is the formulation of a Monte-Carlo estimator for the $\varepsilon$-smoothed level-crossing density based on the Kac–Rice integrand:

$$\hat c_\varepsilon(u) = \frac{1}{N} \sum_{i=1}^N \delta_\varepsilon(f_\theta(x_i) - u) \, \lVert \nabla_x f_\theta(x_i) \rVert$$

where $\delta_\varepsilon$ is a Gaussian kernel of standard deviation $\varepsilon$ centered at level $u$, and the $x_i$ are arbitrary (possibly irregular) sample locations. The magnitude of the spatial derivative $\nabla_x f_\theta$ is computed via automatic differentiation, leveraging the neural field's differentiability. The auxiliary Kac–Rice loss compares the predicted crossing density at each of $L$ levels (placed at quantiles of the target signal) to the empirical crossing density estimated from the supervision signal at those locations.

Notably, this approach is rooted in the co-area formula, which connects statistics over differential structure (e.g., gradient magnitude) and integrals over level sets, and generalizes to higher dimensions and arbitrary domain geometries without any assumption of stationarity or Gaussianity.

Figure 1: Left is a multisine signal with a marked level and its crossings; right shows collapse of crossing-density profiles when high-frequency content is missing.

Estimator Validation

The paper empirically validates the Monte-Carlo crossing density estimator against both the exact crossing counts (obtained via direct sign changes on a dense grid) and the classical Rice formula for Gaussian fields. Across synthetic and random signals, the estimator tracks exact counts to within a few percent Monte-Carlo error and matches the Rice theoretical prediction for stationary random fields up to $10\%$ error, even capturing non-Gaussian deviations that the idealized formula omits.

Figure 2: The Monte-Carlo estimator accurately tracks ground-truth crossing densities and the Rice formula for a random field.

Experimental Results

Regular Grids

With dense, regular supervision, all auxiliary losses—including Kac–Rice, FFL, and normalized Sobolev—converge to parity on 1D and 2D tasks. The accuracy is set by the expressivity of the architecture: MSE-only training of a positional encoding MLP (PE-MLP) achieves maximal PSNR, and adding auxiliary high-frequency losses yields no substantial improvement. In fact, with perfect supervision, gradient-based auxiliary losses can even degrade PSNR due to inconsistencies when the target gradients are estimated via finite differences rather than computed exactly.

Figure 3: Auxiliary losses do not improve grid-supervised INR reconstruction and may slightly hurt fidelity when gradient targets are estimated via finite differences.

Scattered Nonuniform Domains

In scenarios where training data is scattered and supervision is sparse, the impact of the proposed auxiliary loss is significant. All auxiliary spectral losses—Kac–Rice, FFL (via resampling and interpolation), and normalized Sobolev—add $\varepsilon$0 to $\varepsilon$1 dB over MSE-only training on a PE-MLP (and $\varepsilon$2 to $\varepsilon$3 dB on a SIREN backbone). The Kac–Rice loss matches FFL and Sobolev in PSNR (to within $\varepsilon$4–$\varepsilon$5 dB) and exceeds FFL in SSIM and relative spectral error in the highest frequency bands. However, it does not decisively outperform FFL in aggregate metrics on edge-dominated images.

Figure 4: All auxiliary losses substantially improve convergence and high-frequency fidelity versus MSE-only in scattered sampling regimes.

Per-frequency analysis demonstrates that Kac–Rice yields the lowest spectral error among the spatial-domain losses in the highest frequency bands, highlighting its effectiveness as a high-frequency drive.

Figure 5: Spectral band analysis shows Kac–Rice achieves the lowest relative error in high-frequency bands among spatial-domain losses.

Visual inspection of reconstructed images confirms that all auxiliary losses are effective in restoring spatial coherence and fine detail lost by MSE-only training under sparse supervision.

Figure 6: Reconstructions under scattered supervision: auxiliary losses restore coherence and high-frequency detail absent from the MSE-only baseline.

Content Dependence and Theoretical Regimes

The unique strength of the crossing-density loss is revealed on statistically homogeneous, texture-rich signals—settings closest to the random-field theory underpinning the Rice formula. In these cases, Kac–Rice overtakes FFL by $\varepsilon$6 dB. For edge-dominated, non-stationary content (natural images), it achieves parity rather than dominance, confirming the theory's limitations and scope.

Robustness to Gradient-Target Quality and Mesh-Free Operation

The method's robustness is supported by oracle diagnostics: replacing estimated with true image gradients improves Sobolev losses, but the Kac–Rice loss remains insensitive. This evidences its distributional character: rather than enforcing pointwise accuracy, it matches batch-level statistics and is robust to noise or imprecise target gradients—a distinct advantage in unstructured, noisy, or mesh-free settings.

Composability with Architectural Remedies

The Kac–Rice loss composes with architectural motifs such as SIREN activations, providing additive benefits. While the composability is also exhibited by FFL and Sobolev, Kac–Rice edges out Sobolev when combined with SIREN on certain scattered tasks.

Ablation Studies

Ablation on loss bandwidth $\varepsilon$7, number of levels $\varepsilon$8, loss weight $\varepsilon$9, and sampling budget $$\hat c_\varepsilon(u) = \frac{1}{N} \sum_{i=1}^N \delta_\varepsilon(f_\theta(x_i) - u) \, \lVert \nabla_x f_\theta(x_i) \rVert$$0 show that the crossing-density loss is robust to hyperparameter choices: loss quality plateaus for $$\hat c_\varepsilon(u) = \frac{1}{N} \sum_{i=1}^N \delta_\varepsilon(f_\theta(x_i) - u) \, \lVert \nabla_x f_\theta(x_i) \rVert$$1, saturates by $$\hat c_\varepsilon(u) = \frac{1}{N} \sum_{i=1}^N \delta_\varepsilon(f_\theta(x_i) - u) \, \lVert \nabla_x f_\theta(x_i) \rVert$$2, and is insensitive to $$\hat c_\varepsilon(u) = \frac{1}{N} \sum_{i=1}^N \delta_\varepsilon(f_\theta(x_i) - u) \, \lVert \nabla_x f_\theta(x_i) \rVert$$3 across a wide range. The advantage of auxiliary losses over MSE declines as sampling density increases, consistent with their role in underdetermined regimes.

Figure 7: Ablation confirms robustness to most hyperparameters and stability of auxiliary advantage across varying sample budgets.

Implications and Theoretical Insights

The crossing-density auxiliary loss provides a mathematically justified, differentiable, mesh-free surrogate for high-frequency supervision that is effective in sparse, irregular, or noisy environments where grid-based frequency methods or reliable gradient data are unavailable. Its main advantages are structural: it avoids the need for regular grids (unlike FFL), and is robust to gradient noise (unlike Sobolev). The method's reach is limited: it enforces distributional constraints on the geometry of level sets, not phase, and thus cannot recover localization in ambiguous or highly undersampled regimes.

The failure to decisively surpass FFL in all settings, especially edge-dominated images, is attributed to the inherently weaker constraint provided by the distributional statistics (at most $$\hat c_\varepsilon(u) = \frac{1}{N} \sum_{i=1}^N \delta_\varepsilon(f_\theta(x_i) - u) \, \lVert \nabla_x f_\theta(x_i) \rVert$$4 moments per batch, rather than full spectra or $$\hat c_\varepsilon(u) = \frac{1}{N} \sum_{i=1}^N \delta_\varepsilon(f_\theta(x_i) - u) \, \lVert \nabla_x f_\theta(x_i) \rVert$$5 pointwise constraints). The observed parity suggests that existing auxiliary losses are already near-optimal when abundant information is preserved by the sampling and interpolation process, and any further improvement may depend on moving beyond the current information ceiling, for example, by incorporating explicit priors or richer statistic targets.

Prospects for Future Research

The spatial-domain spectral surrogate framework defined by this work could be extended to alternative functionals of the signal, such as higher-order critical-point counts or Euler characteristics of excursion sets. Integration into more complex INR tasks—such as signed distance fitting or few-shot NeRF, where both grid-free estimation and noise-robustness are vital—remains an open direction. The composability with activation engineering and the potential for curriculum-driven adaptation of crossing-density targets offer additional paths to improved performance.

Conclusion

The paper demonstrates that Rice level-crossing density can be instantiated as a mesh-, grid-, and FFT-free auxiliary loss for INR models. This approach recovers the high-frequency structure in neural fields trained with sparse, nonuniform samples, matching the strongest frequency-domain and Sobolev auxiliary losses on natural images, and outperforming FFL on homogeneous texture. Its primary contribution is a robust, distributional, and theoretically principled spatial-domain surrogate for spectral content, particularly suited to regimes lacking regular structure or reliable gradients.