New curriculum TAC boosts multi-domain RLVR reasoning by 10%
TAC uses gradient geometry to prioritize training domains that help others, achieving 2.8 point gains.
Training AI models across multiple reasoning domains (math, programming, science) typically uses fixed or hand-tuned curricula that ignore how skills transfer between domains. A new paper by Yongjin Yang and colleagues (including Bernhard Schölkopf) introduces Transfer-Aware Curriculum (TAC), an online bandit approach that dynamically selects which domain to train on based on both local learnability and cross-domain transferability. TAC repurposes signals already computed during GRPO (Group Relative Policy Optimization): per-domain advantages indicate where the policy is improving, while projected gradients measure alignment between gradient updates across domains—at less than 1% wall-clock overhead.
Evaluated on a six-domain RLVR reasoning suite with Qwen3-1.7B and Llama3.2-3B, TAC achieved the best macro-averaged accuracy across all comparisons—beating proportional random sampling, a hand-designed schedule, and a learnability-only bandit by up to 2.8 points (10% relative). Ablations revealed that removing the transferability term sharply degraded performance, and TAC remained robust on imbalanced mixtures where learnability-only curricula over-commit to dominant domains. The findings establish cross-domain transferability as a critical signal for curriculum design in multi-domain RL training.
- TAC uses gradient geometry alignment to estimate cross-domain transferability with <1% overhead
- Outperformed proportional random sampling and learnability-only bandits by up to 2.8 points (10% relative) on six-domain reasoning suite
- Robust on imbalanced training mixtures where learnability-only curricula over-commit to dominant domains
Why It Matters
TAC makes multi-domain reasoning training more efficient by automatically focusing on domains that benefit others, improving model generalization.