Researchers unveil training-free AI model fusion method

A team of researchers from institutions including the University of Science and Technology of China has developed a novel approach to transfer knowledge between AI models of different scales without additional training. Their paper, titled 'Training-Free Knowledge Transfer Across Model Scales through Activation-Guided Pruning,' introduces the Activation-Prune-Merge (APM) framework which creates task-conditioned activation maps to identify and extract the most useful components from a larger donor model.

The method works by pruning the donor model to match the smaller recipient's architecture while preserving critical capabilities, then injecting this 'donor slice' into the recipient with a small interpolation coefficient. In experiments on a 3B-parameter recipient model, APM achieved significant accuracy improvements across 16 benchmarks spanning reasoning, mathematics, code generation, and classification tasks. Notable gains included RTE accuracy jumping from 64.3% to 82.3%, QNLI from 52.3% to 65.7%, and BoolQ from 70.8% to 79.2%. The approach maintains the small-ratio fusion regime while demonstrating that cross-scale heterogeneous fusion can succeed without explicit semantic alignment when the donor contribution is carefully selected.