Recursive transformers slash parameters for reliable chip thermal simulation

Transformer-based surrogate models are replacing costly finite element analysis (FEA) in engineering, but conventional transformers are over-parameterized for small, low-dimensional datasets typical of chip design. Kart-leong Lim's new arXiv paper (2607.27251) tackles this by shifting focus from adding parameters to adding compute. He introduces and evaluates three recursive weight-sharing transformers—Tiny Recursive, Depth Recursive, and a simple recursive variant—that reuse weights across layers to drastically reduce parameter count while maintaining predictive capacity. The proposed Depth Recursive transformer emerged as the best trade-off across predictive performance (Recall, Mean Reciprocal Rank), parameter efficiency, and FLOPs.

Validation used two demanding low-data tasks: thermo-mechanical reliability analysis of advanced semiconductor packages, where stress and warpage from thermal cycling require repeated FEA sweeps, and an iterative Laplace PDE solver for capacitance fields. Recursive transformers matched or beat over-parameterized baselines while using a fraction of the memory and compute. The design guidelines show when to choose each recursive paradigm under resource-constrained scenarios, offering a practical path to accurate, efficient surrogate modeling for engineering design spaces where simulation data is scarce and expensive.