SIREN-Based Architecture

• SIREN-based architecture is a class of MLPs using sine activations that capture high-frequency signal details and enable mesh-free function approximation.
• It is applied in image fitting, audio compression, 3D reconstruction, and scientific computing by parameterizing continuous mappings with controlled spectral bias.
• Innovations like FM-SIREN, H-SIREN, and Siamese SIREN enhance performance through improved spectral coverage, reduced redundancy, and tailored initialization strategies.

A SIREN-based architecture denotes a class of multilayer perceptrons (MLPs) that use periodic activation functions—specifically the sine nonlinearity—in every hidden layer. Originating with the Sinusoidal Representation Networks (SIREN) model, these architectures provide an implicit neural representation (INR) capable of efficiently encoding signals rich in high-frequency detail and enabling mesh-free function approximation. SIREN and its derivatives have become standard benchmarks for INR tasks, including image fitting, audio compression, 3D reconstruction, scientific computing, and data sonification.

1. Core Principles of SIREN-based Architectures

The canonical SIREN model parameterizes a continuous mapping $f_\theta: \mathbb{R}^d \to \mathbb{R}^n$ using a stacked composition of sine-activated layers. For a network of depth $L$:

• The $0$-th layer (input):

$h_0(x) = \sin(\omega_0 W_0 x + b_0)$

• Hidden layers $i=1,\dots,L$:

$h_i = \sin(\omega_i W_i h_{i-1} + b_i)$

• Output layer (typically linear):

$f_\theta(x) = W_{L+1} h_L + b_{L+1}$

Here, $\omega_i$ are frequency scaling (bandlimit) hyperparameters, and weights/biases are initialized specifically to ensure propagating gradients and rich Fourier feature bases.

Distinctive features of SIREN-based architectures include:

• Direct encoding of high-frequency features by exploiting the sinusoidal basis.
• Architectural bandlimit and implicit spectral filtering controlled via frequency scaling of input and hidden layers.
• Mesh-free, coordinate-based representation, making them agnostic to underlying discretization in spatial, temporal, or spatio-temporal domains (Miotto et al., 29 Jan 2025).

2. Bandlimit, Spectral Bias, and Architectural Design

Control of frequency content is central to the SIREN paradigm:

• The input layer's frequency $\omega_0$ is usually set proportional to the smallest physical wavenumber in the signal (for pressure fields, $k_{\min}=2\pi/\max(L_x, L_y)$; empirically $L$0 with $L$1).
• Hidden layer frequency scales, $L$2 ($L$3), are typically in the range $L$4–$L$5 for scientific data, but much higher in audio ($L$6).
• Width (neurons per layer) increases the capacity for diverse Fourier components.
• Greater depth enables the composition of higher harmonics, but can lead to overfitting finescale noise if not counteracted by lower $L$7.

Network initialization is tailored:

• Input weights $L$8
• Subsequent weights $L$9 where $0$0 is the input dimension to layer $0$1; biases typically set to zero.

This parametrization ensures that the initial effective spectral transfer function is quasi-flat up to the network bandlimit $0$2 then decays, yielding strong low-pass filtering and mitigating overfitting in noisy tasks (Miotto et al., 29 Jan 2025).

3. Innovations and Architectures Extending SIREN

a) FM-SIREN: Nyquist-informed Frequency Multiplier

FM-SIREN addresses hidden feature redundancy in SIREN, where fixed-frequency multipliers generate overlapping frequency coverage within a layer. By allocating distinct, Nyquist-informed multipliers $0$3 to each neuron (for $0$4), FM-SIREN achieves orthogonality analogous to the basis of a discrete sine transform. This almost halves hidden-feature redundancy and improves representational capacity across audio, image, 3D, and NeRF tasks—yielding metric improvements without increasing depth or parameter count (Alsakabi et al., 27 Sep 2025).

b) H-SIREN: Hyperbolic Periodic Input Activation

H-SIREN replaces the $0$5-layer activation with $0$6, vastly broadening initial frequency support via an infinite-order Taylor expansion but reverting to $0$7 for hidden layers. This strategy injects high-frequency features at the network input, yet subsequent layers restore SIREN's smoothing bias, preventing overfitting. Empirically, H-SIREN yields substantial gains in PSNR/SSIM for image fitting, video, NeRF, and fluid simulation benchmarks (Gao et al., 2024).

c) Siamese SIREN: Twin-Headed Network for Audio Compression

The Siamese SIREN instantiates two parallel SIREN "heads" atop a shared backbone, enabling internal noise estimation via differencing. This structure, combined with positional embedding and quantization, outperforms standard SIREN in audio compression tasks on perceptual metrics with a 30% reduction in parameter size (Lanzendörfer et al., 2023).

4. Applications and Comparative Performance

SIREN-based architectures are widely applied across domains:

• Physics-informed regression and scientific computing: For mesh-free field reconstruction (e.g., pressure from image velocimetry), SIREN circumvents mesh connectivity, handling arbitrarily distributed, noisy data points. It outperforms traditional OS-MODI and GFI approaches, especially for unstructured or ill-conditioned cells (Miotto et al., 29 Jan 2025).
• Audio signal compression and denoising: The Siamese SIREN network, combined with coordinate embeddings and quantization, outperforms conventional SIREN and PE+SIREN variants in ViSQOL, CDPAM, PESQ, and STOI, with better data efficiency and perceptual quality (Lanzendörfer et al., 2023).
• Image, 3D surface, and radiance field fitting: H-SIREN and FM-SIREN yield marked improvements in PSNR (up to +10dB), SSIM, and IoU across image and 3D benchmarks relative to standard SIREN, with demonstrably lower redundancy and accelerated convergence (Alsakabi et al., 27 Sep 2025, Gao et al., 2024).

Model	PSNR (Image Fitting)	SSIM	Description	Reference
SIREN	26.87 ± 3.18	0.808	Original sine-activated MLP	(Gao et al., 2024)
FM-SIREN	32.29 (Kodak)	–	Per-neuron Nyquist scaling	(Alsakabi et al., 27 Sep 2025)
H-SIREN	37.80 ± 3.10	0.973	Hyperbolic input sine layer	(Gao et al., 2024)

5. SIREN in Sonification and Signal Mapping

Beyond scientific INRs, a SIREN-based architecture supports web-based sonification platforms, facilitating data-driven mapping to auditory parameters. The system architecture typically decouples data loading, parameter mapping, event scheduling, and audio synthesis. Each SIREN synthesizer module is an ES module exporting parameter schemas, optional audio worklets, and a factory for Web Audio graphs. Mathematical mappings for data-to-sound are implemented in normalized, linear, or exponential forms (Peng et al., 2024).

6. Limitations, Sensitivities, and Practical Guidance

While SIREN-based architectures offer mesh-free parameterization, efficient spectral encoding, and versatility across domains, several practical considerations arise:

• Training per-instance is computationally intensive (e.g., 25s/10s clip for Siamese SIREN) (Lanzendörfer et al., 2023).
• Overfitting risk increases with depth and width; frequency scaling ($0$8, $0$9) mitigates this.
• Fixed frequency multipliers (in vanilla SIREN) limit diversity, addressed by FM-SIREN (Alsakabi et al., 27 Sep 2025).
• For H-SIREN, the scale $h_0(x) = \sin(\omega_0 W_0 x + b_0)$0 in $h_0(x) = \sin(\omega_0 W_0 x + b_0)$1 must be tuned offline; higher $h_0(x) = \sin(\omega_0 W_0 x + b_0)$2 recovers finer detail at some risk of overfitting (Gao et al., 2024).

It is recommended to:

• Set $h_0(x) = \sin(\omega_0 W_0 x + b_0)$3 based on physical wavenumber for scientific domains; higher for audio.
• Use FM-SIREN or H-SIREN for tasks demanding richer frequency representation.
• Select narrower or shallower architectures, or reduce $h_0(x) = \sin(\omega_0 W_0 x + b_0)$4, for noisy tasks to enforce stronger low-pass bias (Miotto et al., 29 Jan 2025).
• Retain SIREN initialization strategies for stability.

7. Future Directions and Extensions

Research directions include:

• Quantization-aware training for further storage/bitrate reduction in INR-based compression (Lanzendörfer et al., 2023).
• Hierarchical/multi-scale frequency backbones to further generalize SIREN's expressivity.
• Domain-adapted periodic activation functions (e.g., hyperbolic, chirp, or Gabor-based) to overcome the limitations of simple sinusoids (Gao et al., 2024).
• Integration with WebAssembly and modular plugin APIs for extensible real-time sonification environments (Peng et al., 2024).
• Empirical and theoretical study of the trade-offs introduced by frequency scaling and hyperbolic activations in noise-prone or ill-conditioned domains.

SIREN-based architectures represent a fundamental model class for implicit neural representations, with expanding impact across signal processing, scientific computing, compression, and human-computer interaction.