MTFM: A Scalable and Alignment-free Foundation Model for Industrial Recommendation in Meituan

Meituan researchers have published MTFM, a new transformer-based foundation model for industrial recommendation systems. It tackles a major industry pain point by eliminating the need for pre-aligned input data across different scenarios, which is traditionally resource-intensive. The model uses novel techniques like heterogeneous tokenization and Grouped-Query Attention to significantly boost training throughput and reduce memory usage. Offline and online experiments confirm performance gains by scaling both model size and multi-scenario training data.