Neural Electromagnetic Fields for High-Resolution Material Parameter Reconstruction

Abstract: Creating functional Digital Twins, simulatable 3D replicas of the real world, is a central challenge in computer vision. Current methods like NeRF produce visually rich but functionally incomplete twins. The key barrier is the lack of underlying material properties (e.g., permittivity, conductivity). Acquiring this information for every point in a scene via non-contact, non-invasive sensing is a primary goal, but it demands solving a notoriously ill-posed physical inversion problem. Standard remote signals, like images and radio frequencies (RF), deeply entangle the unknown geometry, ambient field, and target materials. We introduce NEMF, a novel framework for dense, non-invasive physical inversion designed to build functional digital twins. Our key insight is a systematic disentanglement strategy. NEMF leverages high-fidelity geometry from images as a powerful anchor, which first enables the resolution of the ambient field. By constraining both geometry and field using only non-invasive data, the original ill-posed problem transforms into a well-posed, physics-supervised learning task. This transformation unlocks our core inversion module: a decoder. Guided by ambient RF signals and a differentiable layer incorporating physical reflection models, it learns to explicitly output a continuous, spatially-varying field of the scene's underlying material parameters. We validate our framework on high-fidelity synthetic datasets. Experiments show our non-invasive inversion reconstructs these material maps with high accuracy, and the resulting functional twin enables high-fidelity physical simulation. This advance moves beyond passive visual replicas, enabling the creation of truly functional and simulatable models of the physical world.

• The paper introduces NEMF, a three-stage framework that combines image-derived geometry, ambient-field estimation, and Fresnel-based physics supervision to reconstruct spatially varying permittivity and conductivity.
• NEMF reduces average permittivity error from 0.078 to 0.011 and average conductivity error from 0.639 to 0.317 across three synthetic indoor scenes, while producing sharper material boundaries than a black-box MLP.
• The paper shows that hash-grid capacity and full-network LBFGS fine-tuning substantially improve accuracy, but performance remains sensitive to geometric normal quality and lacks real-world CSI validation.

Overview

This paper introduces NEMF (Neural Electromagnetic Field), a multi-modal framework for dense, non-invasive reconstruction of electromagnetic material parameters—relative permittivity $\epsilon_r$ and conductivity $\sigma$—with the goal of producing functional digital twins: 3D scene replicas that are not merely visually faithful but simulatable under physical laws. The authors identify a fundamental limitation of NeRF-style representations (Tsinovoy et al., 2022): their learned parameters encode optical appearance implicitly and cannot be queried by a physics simulator to predict, for example, how a wall reflects or attenuates RF energy. NEMF addresses this by fusing two sensing modalities—multi-view images and sparse Channel State Information (CSI) measurements—and by decomposing the notoriously ill-posed electromagnetic inverse problem into three sequential stages that progressively constrain the unknowns.

The disentanglement strategy

The core methodological insight is that the inverse problem becomes tractable only when its coupled unknowns are resolved sequentially. The forward model expresses each CSI measurement as a chain of operations: an incident field $\vec{E}_{inc}$ interacts with a surface via a Jones matrix $J$ determined by the material parameters; the reflected field propagates through free space with path loss and phase shift; and the receiver projects onto a known polarization vector. Each measurement therefore entangles geometry $G$, materials, and the ambient field.

NEMF resolves this in three stages:

• Stage 1 — Geometric prior: an instant-ngp-style hash-grid SDF is trained from ~100 multi-view images per scene, yielding a frozen surface $\mathcal{S}$ with normals $\mathbf{n}$.
• Stage 2 — Ambient field reconstruction: a NeWRF-based MLP $f_\theta(\mathbf{x}, \mathbf{d})$ is trained with full supervision against ground-truth incident fields on synthetic data, plus a gradient-norm regularizer for spatial continuity, then frozen.
• Stage 3 — Physics-supervised inversion: with $G$ and $f_\theta$ fixed, target Jones matrices $\sigma$0 are computed analytically by back-propagating measurements through the inverse propagation operator and solving the linear reflection system. A decoder $\sigma$1—combining sinusoidal positional encoding with a multi-resolution hash grid feeding a residual MLP—maps coordinates to a four-dimensional latent $\sigma$2 parameterizing power-law frequency dispersion, $\sigma$3 and $\sigma$4. A non-trainable differentiable Fresnel layer assembles the predicted Jones matrix, including cross-polarization terms, and training minimizes an NMSE loss over frequencies with L2/L1 regularization on hash embeddings and level gates.

A notable design choice is that material parameters are learned in a frequency-independent latent form, with dispersion handled explicitly by the physics layer—an architectural commitment that separates spatial learning from spectral behavior.

Experimental results

Validation uses three synthetic indoor scenes (Office, Bedroom, Conference Room, roughly $\sigma$5) generated via MATLAB ray tracing with multiple reflections and diffraction, sampled at only 500 receiver positions from one transmitter across 8 frequencies (2.4–5.8 GHz). Evaluation reports Mean Relative Error (MRE) on decoded $\sigma$6 and $\sigma$7, chosen because these quantities span orders of magnitude across concrete, wood, and glass.

The headline comparison against the strongest "black box" MLP baseline (8 layers, 8 frequencies) is decisive:

Scene	Baseline Eps MRE	NEMF Eps MRE	Baseline Sigma MRE	NEMF Sigma MRE
Conference Room	0.071	0.021	0.497	0.164
Bedroom	0.102	0.004	0.774	0.114
Office	0.062	0.007	0.647	0.672

Average permittivity error drops from 0.078 to 0.011—a sevenfold reduction—while conductivity error halves from 0.639 to 0.317. Qualitatively, the baseline produces noisy, geometrically incoherent maps, whereas NEMF yields spatially coherent maps with sharp material boundaries. Residual error concentrates at geometric corners and boundaries, which the authors attribute to instability in locally derived normal vectors rather than failures of the inversion logic itself.

Ablation findings

Two ablation families dissect the architecture. On static components, hash capacity dominates: increasing hash layers and features (Arch-6) improves Conference Room Eps MRE from 0.013 to 0.006 and Bedroom from 0.029 to 0.011, while disabling gating or skip connections consistently degrades performance. Positional encoding bandwidth has mixed effects, indicating the hash grid is the primary feature extractor. Notably, capacity gains correlate with geometric-prior quality, since finer encodings improve normal extraction.

On optimization strategies, the most striking result is LBFGS fine-tuning over the full network (Opt-B): it reduces Eps MRE by approximately 42% in the Conference Room, 83% in the Bedroom, and 59% in the Office relative to Adam-only training. Fine-tuning the MLP alone (Opt-F) yields essentially no benefit, demonstrating that second-order optimization must reach the hash grid embeddings. Two strategies fail to deliver expected gains: extreme capacity with progressive unfreezing underperforms Opt-B in the Bedroom, and high-frequency loss weighting (Opt-E) performs worse than the baseline, indicating a sensitive trade-off the paper does not fully resolve.

Limitations and open questions

The paper concedes two substantive limitations. First, Stage 3 accuracy is tightly coupled to Stage 1 geometric fidelity: errors in normals or incidence angles propagate non-linearly through the Fresnel equations, causing localized prediction failure—the dominant residual error source. Second, and more consequentially, all validation is synthetic. Real-world CSI introduces hardware calibration errors, complex noise profiles, and multipath effects beyond the simulator's model, none of which are addressed here. This dependence on ground-truth incident fields for supervising $\sigma$8 also raises the question of whether Stage 2 can be trained without privileged field access in practice. Open questions include robustness of the Fresnel-space inversion under realistic noise, scalability beyond single-transmitter sparse setups, and integration with emerging 6G sensing protocols.

Conclusion

NEMF demonstrates that anchoring electromagnetic inversion with image-derived geometry and a frozen ambient-field representation converts an ill-posed physical problem into a well-posed, physics-supervised learning task, yielding order-of-magnitude reductions in material reconstruction error over entangled black-box baselines on synthetic scenes. The framework's practical significance, however, remains contingent on validation against real measurements, which the authors defer to future work.