GRASP slashes multi-source training memory to constant, boosts accuracy 21.8%

Multi-source transfer learning has long faced a scalability wall: fusing knowledge from K pre-trained models typically requires loading all K models into memory simultaneously (O(K) memory) or deploying them all at inference, making it impractical for production. A new paper on arXiv introduces GRASP (Gradient-Aligned Sequential Parameter Transfer), which elegantly sidesteps this by processing sources one at a time into an evolving target model. Three innovations drive it: (1) sequential merging that integrates one source before moving to the next, (2) parameter-wise gradient alignment that selectively transfers only parameters whose optimization directions agree with the target domain, preventing negative transfer, and (3) iterative fine-tuning to adapt knowledge before integrating the next source.

Extensive tests across three continual learning benchmarks (Yearbook, CLEAR-10, CLEAR-100) spanning 10 to 108 years of temporal distribution shifts, and four architectures from 1.3M to 25.6M parameters, show GRASP hitting 93.5% mean accuracy versus 71.7% for standard ensemble methods — all while requiring constant memory regardless of source count. This O(1) memory footprint means GRASP can scale to arbitrarily many sources without memory growth, making it uniquely suited for edge devices, low-resource environments, and continuously evolving source domains. The work opens the door to practical multi-source learning without expensive hardware.