MLflow 3.14.0 launches agent setup, review queues, and LLM playground
Single command to instrument Claude Code, OpenAI Codex, or OpenCode with MLflow tracing.
MLflow 3.14.0 marks a significant leap for GenAI development workflows, introducing several major features aimed at improving agent integration, observability, and evaluation. The headline feature is one-command agent onboarding: `mlflow agent setup` installs MLflow, configures tracing, and equips popular coding agents (Claude Code, OpenAI Codex, or OpenCode) with MLflow skills to automatically instrument applications. This dramatically lowers the barrier for teams adopting AI agents in production.
Durable tracing for Claude Code now uses a write-ahead-log, preventing slowdowns, server overload, or trace loss during network blips or crashes—critical for team-scale rollouts. Review Queues allow assigning traces to reviewers or agents, capturing structured feedback and ground-truth annotations that are written back onto traces for immediate evaluation use. The revamped evaluation dataset UI lets users browse, edit, and bulk-manage records with direct links to source traces. Additionally, the LLM Playground supports prompt iteration against AI Gateway endpoints and Prompt Registry versions with settings, tools, structured output, and template variables. Pytest integration adds the `@mlflow.test` marker for GenAI regression testing in CI, with test history and per-assertion judge results in the UI. Breaking changes include default serialization format updates for sklearn (to skops), PyTorch (to pt2), and LightGBM (to skops).
- Single `mlflow agent setup` command enables tracing for Claude Code, OpenAI Codex, and OpenCode
- Durable tracing with write-ahead log prevents trace loss and performance degradation at team scale
- New Review Queues and LLM Playground with AI Gateway integration accelerate prompt iteration and feedback
Why It Matters
MLflow 3.14.0 streamlines AI agent observability and evaluation, making production GenAI workflows more reliable and collaborative.