RLVR fine-tuning lifts small AI agents from 0.35 to perfect scores on Atlassian APIs

Large language models excel at next-token prediction but fail silently when asked to call enterprise APIs correctly—dropping required fields, hallucinating endpoints, or stopping after one read. A new arXiv proof-of-concept (2607.01465) tackles this with Reinforcement Learning from Verifiable Rewards (RLVR), training small models to act accurately inside Atlassian workflows. The team built five synthetic environments mimicking Jira REST v3 and Confluence v2 APIs at schema fidelity, then applied GRPO-style training to Qwen3-1.7B and Qwen3.5-4B using rewards computed purely from tool-call traces.

Results are striking: on four scenarios with non-degenerate reward shapes, the RL-trained policy lifted average reward from a 4B baseline range of 0.35–0.92 to 0.95–1.00. The largest jump was on Confluence page creation (0.35 → 1.00). The authors position this as a step toward outcome-optimised small models for niche enterprise APIs. However, they flag two limitations: hand-crafting verifiable rewards does not scale beyond a handful of endpoints, and one scenario (ticket-transition) was already saturated by the prompted 4B model. The work suggests a viable path for deploying smaller, cheaper models on specific SaaS workflows without human labeling.