GALLM's graph-aware LLM boosts recommendations by 9.76% in HR@5
Fusing item co-occurrence into LLM attention delivers big gains without extra encoders
Large language models are increasingly used for recommendation, but their language-centric pretraining often ignores collaborative signals from user-item interactions. Existing fixes either inject vectors from external recommenders or only capture within-sequence dependencies, missing global patterns. In a new arXiv paper, Fenglin Yan and colleagues from Zhejiang University introduce GALLM, a graph-aware LLM framework that directly encodes three types of relations into the attention mechanism: Text–Text relations preserve semantic links between item descriptions, Item–Text relations align items with their textual features, and Item–Item relations derive from global item co-occurrence.
Instead of adding a separate graph encoder, GALLM transforms these relations into lightweight learnable attention biases that modify how tokens attend to each other inside the LLM. This lets the model tap into collaborative signals while keeping training efficient. Evaluated on four real-world benchmarks, GALLM beat all baselines, improving HR@5 by 9.76% on average over the strongest competitor. The work shows a clean way to inject collaborative knowledge into LLMs without architectural bloat, making it practical for large-scale sequential recommendation systems.
- GALLM models three relations—Text–Text, Item–Text, and Item–Item—as learnable attention biases inside the LLM.
- No extra graph encoder needed, preserving efficiency while capturing global collaborative patterns.
- Achieves 9.76% average improvement in HR@5 over the strongest baseline across four real-world benchmarks.
Why It Matters
Enables LLM-based recommenders to harness collaborative signals directly, improving personalization without costly external graph components.