Physics-Informed Neural Networks with Learnable Loss Balancing and Transfer Learning

Traditional physics-informed neural networks (PINNs) often struggle with data scarcity because they rely on fixed or heuristic weighting of physics residuals and data loss—requiring extensive manual tuning and still suffering from poor generalization. A new arXiv paper by Reza Pirayeshshirazinezhad proposes a self-supervised PINN framework that overcomes this by introducing a learnable blending neuron. This neuron dynamically adjusts the relative contribution of physics-based and data-driven supervision based on their uncertainties, enabling stable training and better performance without any manual adjustment.

To further boost efficiency, the framework incorporates a transfer learning strategy that reuses learned representations from related physical domains, then fine-tunes them on the target system with very few data points. The method was validated on a challenging engineering problem: predicting heat transfer in liquid-metal miniature heat sinks using only 87 CFD datapoints. The adaptive PINN achieved an error below 8%, outperforming shallow neural networks, kernel methods, and physics-only baselines. This work provides a general recipe for embedding physics adaptively into neural networks, offering a robust approach for data-scarce problems in fluid dynamics, material modeling, and beyond.