MINER: Multiscale Implicit Neural Representations (2202.03532v2)

Abstract: We introduce a new neural signal model designed for efficient high-resolution representation of large-scale signals. The key innovation in our multiscale implicit neural representation (MINER) is an internal representation via a Laplacian pyramid, which provides a sparse multiscale decomposition of the signal that captures orthogonal parts of the signal across scales. We leverage the advantages of the Laplacian pyramid by representing small disjoint patches of the pyramid at each scale with a small MLP. This enables the capacity of the network to adaptively increase from coarse to fine scales, and only represent parts of the signal with strong signal energy. The parameters of each MLP are optimized from coarse-to-fine scale which results in faster approximations at coarser scales, thereby ultimately an extremely fast training process. We apply MINER to a range of large-scale signal representation tasks, including gigapixel images and very large point clouds, and demonstrate that it requires fewer than 25% of the parameters, 33% of the memory footprint, and 10% of the computation time of competing techniques such as ACORN to reach the same representation accuracy.

• The paper introduces MINER, a multiscale implicit neural framework that achieves equivalent accuracy with less than 25% of parameters, 33% of memory, and 10% of computation time.
• MINER leverages a Laplacian pyramid and independent MLPs for disjoint signal patches, adaptively increasing capacity from coarse to fine scales.
• The approach delivers state-of-the-art performance, achieving a 0.999 IoU for a Lucy 3D mesh in under 30 minutes and over 38dB accuracy for gigapixel images.

Introduction to MINER

The paper by Saragadam et al. presents a novel neural signal model, Multiscale Implicit Neural Representation (MINER), which is an advancement in the field of large-scale signal representation. MINER addresses some significant limitations of existing implicit neural representations, specifically their high computational cost which has rendered them impractical for handling extremely high-dimensional signals like gigapixel images or 3D point clouds.

Design and Implementation

MINER employs a multiscale approach, leveraging the self-similarity of visual signals and represents them using a Laplacian pyramid decomposition. This decomposition not only captures the multiscale frequency content of the signal effectively but also does so sparsely. MINER's architecture deviates from traditional methods by representing small disjoint patches of these scales with separate small MLPs. Crucially, the network's capacity increases adaptively from coarse to fine scales, focusing solely on the necessary parts of the signal. This methodology significantly boosts the representation’s sparsity and results in training efficiencies. But perhaps most compelling is the performance of MINER compared to other state-of-the-art techniques: it achieves the same representation accuracy with less than 25% of the parameters, 33% of the memory footprint, and 10% of the computation time.

Performance Benchmarks and Results

The impressive performance of MINER is supported by robust numerical results. In image and 3D volume representation tasks, MINER dramatically outperforms ACORN, its closest contemporary. For instance, in representing a Lucy 3D mesh, MINER achieved a high IoU value of 0.999 at the finest scale in under 30 minutes, which is a significant acceleration compared to the baseline methods. When representing gigapixel images, it reached a greater than 38dB accuracy in less than three hours—an endeavor that would take more than a day using ACORN. These results clearly demonstrate the efficiency and efficacy of the proposed MINER framework.

Contributions and Future Applications

The paper asserts that the design of MINER, which employs a sequential, coarse-to-fine scale training process, and a multipatch decomposition practice, enables a multi-resolution analysis that is both fast and flexible. Notably, sparse signals can benefit from this approach as it prunes unnecessary representation, further reducing computational load. The MINER framework not only provides a more efficient training but also an equally resourceful inference procedure, fit for streaming reconstruction and use in rendering similar to JPEG2000 or with octrees. This opens up the possibility for practical neural representations of exceptionally large-scale visual signals.

MINER's contributions both challenge and progress the capabilities of implicit neural representations. By offering a fast and memory-efficient approach to the rendering of high-dimensional signals, MINER not only provides a pragmatic solution to current challenges but also pushes the envelope on what can be accomplished in terms of signal representation and reconstruction fidelity within reasonable computational confines.