Sparse Network Inference under Imperfect Detection and its Application to Ecological Networks

A research team from Rice University and other institutions has introduced a novel machine learning framework for inferring sparse networks from imperfect observational data, with direct applications in ecology. The paper, "Sparse Network Inference under Imperfect Detection and its Application to Ecological Networks," tackles a critical problem: existing models for analyzing bipartite networks (like predator-prey or plant-pollinator interactions) often fail to properly control sparsity and scale, leading to oversparse or poorly rescaled estimates that degrade structural recovery. The authors' key innovation is a structured sparse nonnegative low-rank factorization model that simultaneously estimates detection probabilities.

To solve this, the team imposed nonconvex ℓ₁/₂ regularization on latent similarity and connectivity structures, which promotes sparsity within-group similarity and cross-group connectivity while maintaining better relative scale. They developed an ADMM-based algorithm with adaptive penalization and scale-aware initialization to solve the resulting nonconvex, nonsmooth optimization problem. The researchers established theoretical guarantees for their method, proving asymptotic feasibility and KKT stationarity of cluster points under mild regularity conditions. Experiments on both synthetic and real-world ecological datasets demonstrated that their framework achieves improved recovery of latent factors and similarity/connectivity structure compared to existing baselines, making it particularly valuable for ecologists working with inherently noisy and incomplete field observation data.