ServiceNow Built an AI That Writes Its Own Training Exercises

Companies are rushing to put AI assistants (sometimes called agents — AI that can actually take actions, not just chat) to work doing real jobs: resetting passwords, filing support tickets, updating customer records. The problem is that an AI which looks brilliant in general can still stumble badly inside one specific company. It might misuse a tool, ignore a company policy, or botch a workflow nobody else uses.

That's the gap ServiceNow's research team is trying to close with AutoSynthData. The idea is simple: instead of paying people to hand-write thousands of training examples, let AI build its own practice problems. The system watches where a weaker AI fails, checks how a stronger "teacher" AI handles the same situation, then generates new, realistic tasks that target exactly those weak spots. As the assistant improves, the system shifts toward harder challenges — like a tutor who keeps raising the difficulty.

The clever part is quality control. Every generated task must pass three tests: it has to be possible to complete, it has to sound like something a real employee would actually ask, and it has to be hard enough to teach the AI something new. A separate "verifier" — an automatic grader — decides whether the AI succeeded. That grader must be fair, strict, and flexible enough to accept any valid solution, not just one predetermined answer.

ServiceNow demonstrated the approach using a simulated business environment. The goal is AI that quietly gets better at your job simply by practicing it, without a team of humans writing endless test cases. If it works at scale, that means faster setup, fewer embarrassing errors, and AI tools that actually fit the way your company operates.