The Surprising Effectiveness of Noise Pretraining for Implicit Neural Representations

A team from Rice University led by Kushal Vyas and Guha Balakrishnan has published a CVPR 2026 paper revealing a surprisingly simple yet powerful technique for training Implicit Neural Representations (INRs). INRs are neural networks that learn to represent complex signals like images or 3D scenes as continuous functions, but their performance is notoriously sensitive to initialization. The researchers found that pretraining these networks on completely unstructured noise—such as uniform or Gaussian noise—before exposing them to real data dramatically improves their ability to fit new signals, outperforming more complex data-driven initialization methods.

The key insight is that different types of 'noise pretraining' impart different priors. While simple Gaussian noise yields excellent signal fitting, it creates a poor 'deep image prior' for tasks like denoising. Conversely, pretraining on 'structured' noise that mimics the 1/f^α spectral characteristics of natural images achieves the best balance, enabling high-quality signal reconstruction and effective performance on inverse imaging tasks. This breakthrough means practitioners can now train high-performing INRs more efficiently, even without large datasets specific to their target domain, unlocking better 3D reconstruction, video compression, and novel view synthesis.