---
title: 'FM-SIREN: Nyquist-Informed Implicit Representation'
url: https://www.emergentmind.com/topics/fm-siren
type: topic
---

# FM-SIREN: Nyquist-Informed Implicit Representation

Searching arXiv for FM-SIREN and foundational INR papers to ground the article in current literature.
FM-SIREN is an implicit neural representation architecture that modifies SIREN-style periodic multilayer perceptrons by replacing the layerwise fixed frequency multiplier with Nyquist-informed, neuron-specific frequency multipliers. Introduced together with the related FM-FINER variant, FM-SIREN is motivated by the claim that conventional periodic-activation INRs such as SIREN and FINER exhibit hidden feature redundancy because neurons in the same layer are driven by an identical frequency scaling, causing overlapping frequency capture and limiting expressivity [2509.23438]. The proposed remedy is to distribute neuron frequencies across a Nyquist-constrained grid, in analogy with classical harmonic constructions such as the Discrete Sine Transform, thereby increasing frequency diversity without adding parameters, additional depth, or frequency hyperparameter tuning [2509.23438].

## 1. Concept and scope

FM-SIREN belongs to the family of coordinate-based implicit neural representations, in which a network maps continuous coordinates to signal values. Within that family, the immediate precursor is SIREN, which uses sinusoidal activations to represent signals with substantial high-frequency content and to support derivative-based supervision [2006.09661]. FM-SIREN retains the periodic-activation MLP structure but changes how activation frequencies are assigned: instead of a single shared multiplier per layer, each neuron receives its own multiplier $\omega_k$ determined from the signal resolution and the number of neurons [2509.23438].

The central problem addressed by FM-SIREN is described as hidden feature redundancy. In standard SIREN and FINER, neurons within a layer use the same frequency scaling, which yields highly correlated activations and overlapping frequency sensitivity [2509.23438]. This suggests that representational inefficiency in periodic INRs is not only a matter of insufficient depth or width, but also of how frequency support is allocated across neurons.

The proposal is therefore architectural rather than optimization-driven. FM-SIREN does not introduce an auxiliary regularizer, basis orthogonality constraint, or additional branch. Instead, it changes the activation parameterization so that neurons span distinct frequency bands up to the Nyquist limit [2509.23438]. A plausible implication is that FM-SIREN can be viewed as a structural bias toward basis diversification inside a learned INR, bridging classical spectral design and neural coordinate networks.

## 2. Relation to SIREN and FINER

SIREN uses the activation $\sin(\omega x)$, where $\omega$ is a fixed multiplier, typically chosen as a hyperparameter per layer [2509.23438]. Its importance in the INR literature derives from the observation that periodic activations improve the representation of complex natural signals and make the network well suited to tasks involving gradients, Laplacians, and other differential operators [2006.09661]. However, as summarized in the FM-SIREN paper, the use of a single frequency multiplier for all neurons within a layer creates overlapping frequency components in hidden features [2509.23438].

FINER is described as employing a chirp-based activation,
\[
\sin(\omega (|x|+1)x),
\]
again with a fixed frequency multiplier per layer [2509.23438]. Its purpose is to provide more flexible spectral-bias tuning than SIREN, yet it is said to suffer from the same redundancy mechanism when all neurons share the same $\omega$ [2509.23438].

FM-SIREN and FM-FINER preserve the baseline network width, depth, and overall MLP topology while altering only the activation frequency assignment [2509.23438]. In that sense, they are drop-in architectural variants rather than entirely new INR families. This point matters methodologically: the reported gains are attributed to frequency diversity rather than scaling the model or extending training.

A concise comparison is given below.

| Model | Activation pattern | Frequency assignment |
|---|---|---|
| SIREN | $\sin(\omega x)$ | Fixed, identical within a layer |
| FINER | $\sin(\omega (|x|+1)x)$ | Fixed, identical within a layer |
| FM-SIREN | $\sin(\omega_k x)$ | Unique, Nyquist-informed per neuron |
| FM-FINER | $\sin(\omega_k (|x|+1)x)$ | Unique, Nyquist-informed per neuron |

## 3. Nyquist-informed frequency assignment

The defining mechanism of FM-SIREN is the assignment of a distinct multiplier $\omega_k$ to each neuron in a layer, motivated by the Nyquist sampling theorem and by classical transforms such as the DST and DFT [2509.23438]. The Nyquist frequency is given as
\[
f_{\text{Nyquist}} = \frac{f_s}{2},
\]
where $f_s$ is the sampling frequency of the signal [2509.23438]. The neuron frequencies are then distributed across the admissible spectral range up to the Nyquist limit.

For a layer with $K$ neurons, FM-SIREN defines
\[
\phi_k(x) = \sin(\omega_k x),
\qquad
\omega_k = \frac{k\,f_{\text{Nyquist}}}{K},
\quad k=0,1,\ldots,K-1
\]
[2509.23438]. The paper characterizes this as uniformly sampling the frequency range up to the Nyquist frequency, with the design intended to avoid aliasing while promoting complementary frequency specialization [2509.23438].

At the layer level, the hidden representation is written as
\[
y(x) = \sum_{k=0}^{K-1} w_k^{\text{out}} \cdot \sin\big(\omega_k \cdot (w_k^{\text{in}}x + b_k)\big)
\]
[2509.23438]. The assignment can be repeated in deeper networks, implying that each layer inherits the same principle of neuron-wise frequency diversity.

The connection to the Discrete Sine Transform is conceptual rather than literal. In the DST, each basis element corresponds to a distinct sine frequency, yielding an orthonormal basis. FM-SIREN is said to mimic this logic within a learned neural architecture by spreading neuron frequencies over a Nyquist-constrained grid instead of using a single shared multiplier [2509.23438]. This suggests an “implicit basis diversification” effect: the network remains fully learned, yet its hidden units are encouraged to behave more like a bank of differentiated harmonic atoms than a set of redundant replicas.

For multidimensional inputs, the paper states that the same policy is applied per dimension [2509.23438]. For FM-FINER, the frequency multipliers are additionally scaled, with an empirical factor of $2/3$ reported to accommodate the wider bandwidth of the chirp-based activation and avoid aliasing [2509.23438].

## 4. Redundancy reduction and representational consequences

The primary empirical justification for FM-SIREN is reduced hidden feature redundancy. The paper reports that covariance maps in standard SIREN and FINER layers exhibit high values, interpreted as correlated and redundant embeddings, whereas FM-SIREN and FM-FINER yield much lower covariance norms [2509.23438]. The reported Frobenius norm reductions are 49.92% for FM-SIREN and 50.43% for FM-FINER [2509.23438].

These measurements are presented as evidence that neuron-specific frequency assignment produces nearly orthogonal feature vectors per neuron within a layer [2509.23438]. Strict orthogonality is not claimed in the sense of a formally constrained basis, but the comparison to classical harmonic decompositions is explicit. The intended effect is that each neuron specializes in a unique and complementary frequency band rather than duplicating the spectral role of its neighbors.

This architectural claim differs from the original SIREN framing. SIREN emphasized the capacity of sinusoidal networks to represent fine detail and to capture derivatives accurately [2006.09661]. FM-SIREN narrows attention to the internal organization of that capacity, arguing that periodic activations alone are insufficient if all neurons in a layer are locked to the same multiplier [2509.23438]. In other words, the issue is not whether sinusoidal activations can represent frequency-rich signals, but whether a given hidden layer allocates its spectral degrees of freedom efficiently.

The paper further states that performance saturates as width increases past the Nyquist limit because all salient frequencies are already covered [2509.23438]. This is a notable design implication: simply widening a conventional periodic INR may be less efficient than ensuring that existing neurons are assigned non-overlapping frequency roles. It also reports that excessive depth can degrade performance because of higher-frequency noise propagation, while shallow FM-SIREN and FM-FINER can outperform much deeper classic SIREN models [2509.23438]. This suggests that FM-SIREN is partly a critique of the default width-and-depth scaling strategy for INRs.

## 5. Empirical performance across INR tasks

FM-SIREN is evaluated on several canonical INR workloads: fitting 1D audio, 2D images, 3D shapes, and neural radiance fields [2509.23438]. Across these tasks, the paper reports consistent improvement over baseline SIREN and, where relevant, over FINER and other INR baselines.

For 1D audio fitting on Spoken English Wikipedia, the evaluation metric is MSE. The reported average MSE values are $4.74 \times 10^{-5}$ for FM-SIREN, $4.06 \times 10^{-5}$ for FM-FINER, $5.14 \times 10^{-4}$ for SIREN, and $2.63 \times 10^{-4}$ for FINER [2509.23438]. FM-SIREN and FM-FINER are said to achieve the lowest average MSE, with qualitative reconstructions visually near indistinguishable from ground truth [2509.23438].

For 2D image fitting on Kodak, BSDS500, and Philips Circle Pattern, the reported metrics are MSE, PSNR, and SSIM [2509.23438]. The paper states that FM-SIREN and FM-FINER set new PSNR benchmarks, including 32.29 dB and 32.48 dB on Kodak and 38.12 dB and 37.47 dB on BSDS500, respectively [2509.23438]. Reconstructions are described as sharper and less noisy, supported by zoom-ins and error distributions [2509.23438].

For 3D shape fitting on the Stanford 3D Scanning Repository, the metric is Intersection over Union. FM-SIREN and FM-FINER are reported to deliver higher or equal IoU than all baselines, with values up to 0.996 and 0.997 for objects such as Armadillo and Dragon, and to do so without increasing parameter count or training time [2509.23438].

For NeRF synthesis on Blender object scenes, the metric is PSNR. The paper reports that FM-SIREN and FM-FINER outperform all baselines in most test scenes and reduce compute time by 14–17% [2509.23438]. The gains are described as modest in absolute terms but effectively obtained “for free,” since they come from better feature assignment rather than a larger model [2509.23438].

The task coverage is significant because it spans one-, two-, and three-dimensional signals as well as view synthesis. This suggests that the claimed benefit of frequency-diverse hidden units is not tied to a single modality, but to a generic property of periodic INRs.

## 6. Efficiency, tuning, and implementation considerations

A central practical claim is that FM-SIREN introduces no increase in parameters or computational cost and requires no additional hyperparameter search for the activation frequency itself [2509.23438]. The neuron-specific multipliers are explicitly computed from the signal resolution and network width, rather than selected by layerwise manual tuning [2509.23438].

The implementation procedure described in the paper is minimal: choose the number of neurons $K$, compute $f_{\text{Nyquist}}$ from the signal’s resolution, and assign
\[
\omega_k = \frac{k\,f_{\text{Nyquist}}}{K}
\]
for $k=0,\dots,K-1$ [2509.23438]. The same principle extends to multidimensional inputs [2509.23438]. Because the baseline architecture is otherwise unchanged, FM-SIREN is positioned as a low-friction modification to existing SIREN pipelines.

At the same time, the design is not entirely free of sensitivity. The paper notes that performance saturates once width passes the Nyquist limit and that excessive depth may degrade performance through higher-frequency noise propagation [2509.23438]. For FM-FINER, explicit care is required to avoid aliasing because the chirp-based activation has broader inherent bandwidth, motivating the empirical $2/3$ scaling factor [2509.23438]. These caveats indicate that Nyquist-informed allocation reduces one form of tuning, but does not eliminate architectural judgment altogether.

Relative to the original SIREN, this is a shift in engineering philosophy. SIREN relied heavily on periodic activations and a special initialization to make training stable and to preserve the ability to represent derivatives [2006.09661]. FM-SIREN instead emphasizes analytically informed activation diversity as a way to improve utilization of hidden units [2509.23438]. A plausible implication is that future INR design may increasingly treat hidden neurons as frequency-indexed resources rather than homogeneous nonlinear features.

## 7. Position within the INR literature

FM-SIREN occupies a specific position in INR research: it is neither a new representation paradigm nor a task-specific adaptation, but a refinement of periodic-activation network design. Its contribution is to reinterpret the hidden layer of a periodic MLP through the lens of sampling theory and classical harmonic analysis [2509.23438].

This connects naturally to the broader trajectory established by SIREN. The original SIREN paper argued that sinusoidal activations are especially effective for representing natural signals and their derivatives, enabling applications in image fitting, audio reconstruction, video, shape representation, and PDE supervision [2006.09661]. FM-SIREN does not challenge that premise; instead, it argues that fixed per-layer multipliers leave part of that potential unrealized because many neurons learn overlapping frequency components [2509.23438].

A possible misconception is to treat FM-SIREN as merely a frequency-sweep variant of SIREN. The paper’s stronger claim is about hidden feature geometry: lower covariance norms are interpreted as a sign of more complementary embeddings and hence greater effective expressivity at fixed model size [2509.23438]. Another possible misconception is that the method derives its gains from network scaling or heavier optimization; the paper explicitly states that width, depth, and parameter count remain unchanged, and that improvements stem from neuron-wise frequency diversity [2509.23438].

Within the taxonomy of INR modifications, FM-SIREN is most closely aligned with approaches that alter the spectral inductive bias of coordinate MLPs rather than their supervision, encoding, or downstream loss. Its distinctive feature is that the spectral prior is tied to the Nyquist limit of the represented signal [2509.23438]. This makes the method unusually explicit in relating network architecture to signal sampling theory.

The broader significance is methodological. FM-SIREN suggests that periodic INRs can benefit from design principles drawn from explicit bases without sacrificing the flexibility of learned implicit functions. If that interpretation holds beyond the reported experiments, then neuron-wise spectral allocation may become a general design principle for periodic activation networks, including variants beyond SIREN and FINER [2509.23438].

Source: https://www.emergentmind.com/topics/fm-siren