FactorSmith: Agentic Simulation Generation via Markov Decision Process Decomposition with Planner-Designer-Critic Refinement
New agentic system combines factored POMDPs with a three-agent workflow to create executable simulations from text.
Researchers Ali Shamsaddinlou and Morteza NourelahiAlamdari have introduced FactorSmith, a novel AI framework designed to tackle the complex challenge of generating executable game simulations directly from natural language specifications. The system addresses the core limitation of large language models (LLMs) struggling with large, interconnected codebases by implementing a two-pronged approach. First, it employs a factored partially observable Markov decision process (POMDP) decomposition, inspired by prior work like FactorSim, to break down a simulation specification into modular steps. Each step operates on only a minimal subset of relevant state variables, drastically reducing the context any single LLM call must process and enabling more focused, accurate code generation.
Second, within each of these factored steps, FactorSmith embeds a hierarchical agentic workflow inspired by architectures like SceneSmith. This workflow features a trio of specialized AI agents: a planner that orchestrates the overall process, a designer that proposes specific code artifacts, and a critic that evaluates quality through structured scoring. This setup allows for iterative refinement at every generation step, with the ability to roll back to checkpoints if quality thresholds aren't met. The combined methodology is formalized with a mathematical framework for context selection and agentic refinement.
The team's experiments, conducted on the PyGame Learning Environment benchmark, demonstrate the framework's practical efficacy. FactorSmith outperformed non-agentic factored baselines by generating simulations with significantly improved alignment to the original text prompt, a substantial reduction in runtime errors, and overall higher code quality. The researchers have also provided an open-source implementation of the system, making the advanced techniques accessible for further development and application in AI-assisted software creation and simulation design.
- Uses factored POMDP decomposition to limit context, allowing LLMs to focus on minimal state variables per step.
- Embeds a planner-designer-critic agentic trio within each step for iterative quality refinement and checkpoint rollback.
- Outperforms baselines on PyGame benchmarks, generating code with better prompt alignment and fewer runtime errors.
Why It Matters
This represents a significant step towards reliable, AI-driven creation of complex software and interactive simulations from simple descriptions.