Hugging Face hosts Doom running purely on an LLM
A 21B-parameter Phi-3 model renders Doom frames without training—just pure inference.
Researcher physicsrob ported Doom's rendering algorithm directly into Phi-3's transformer weights using a custom compiler (torchwright), eliminating traditional training. The 80x50 model weighs 34 GB and runs on 64-80 GB GPUs, while the 320x200 variant hits 21B parameters and 85.87 GB. Each frame is generated from a 3,614-token prompt encoding level geometry, player position, and view direction, followed by 53,747 tokens of drawing commands.
On a B200 GPU, rendering a single frame takes ~40 minutes at fp32 precision. The checkpoints are available on Hugging Face, with the 80x50 version recommended for practical testing. The project demonstrates how LLMs can execute procedural tasks beyond text generation, pushing the boundaries of what transformer architectures can do with pure inference.
- Phi-3ForCausalLM renders Doom frames without training, using 21B params (320x200) or 34 GB (80x50)
- Each frame is generated from 3,614-token prompts + 53,747 tokens in ~40 minutes on a B200 GPU
- Checkpoints hosted on Hugging Face; requires 64-80 GB GPU memory for the 80x50 model
Why It Matters
Proves LLMs can execute complex, non-textual tasks purely via inference—expanding AI's practical capabilities beyond traditional generative use cases.