Research & Papers

[R] Fine-tuning services report

New benchmark reveals Nebius's edge for iterative function-calling tasks, while costs and speeds vary widely.

Deep Dive

A comprehensive new benchmark provides a crucial snapshot of the fast-moving fine-tuning-as-a-service (FTaaS) landscape. The analysis, conducted by Vintage Data, evaluated multiple providers on critical metrics like cost, training speed, and overall developer experience. The goal was to help developers and companies who have proprietary data but lack the expensive GPU hardware required for training. The report found that the "best" service is highly use-case dependent, with new competitors entering the market even during the testing period.

One standout finding was the performance of Nebius, which demonstrated particularly useful capabilities for developers working on function-calling models. The service's features enabled more efficient iteration cycles, a key advantage for refining AI agents that need to execute specific tasks or APIs. For larger models, some providers also offer the option to run inference on their custom-tuned models in the cloud, though the primary value is in outsourcing the intensive training phase before potentially running the final model locally.

Key Points
  • The fine-tuning-as-a-service market is expanding rapidly, with new providers emerging constantly.
  • Nebius showed distinct advantages for iterative development on function-calling models, improving developer workflow.
  • The 'best' provider is highly situational, depending on specific needs for cost, speed, and model size.

Why It Matters

Democratizes custom AI model creation for teams without massive GPU budgets, accelerating specialized AI agent development.