HobbyLM: Solo developer trains 500M LLM and 330M image gen from scratch for $800
A single developer used Claude Code as an agentic harness to orchestrate the entire training.
Deep Dive
A solo hobbyist (Altruistic-Tea-5612) built HobbyLM, an LLM pretrained on 40B tokens from FineWeb and post-trained for extended context, plus an image generator inspired by ByteDance's DreamLite. Using a mix of distilled datasets from Midjourney, Flux, and Google's CCW3, the entire project ran on 8xH200 GPUs via Modal.com at a total cost of $800. The model weights, GGUF files, playground, and full code are open-sourced on Hugging Face and GitHub.
Key Points
- Solo developer trained a 500M parameter LLM and 330M parameter image generator from scratch
- Total compute cost: $800 using 8xH200 GPUs from Modal.com
- Used Claude Code as an agentic harness for automated orchestration, ablation studies, and training pipeline
Why It Matters
Shows that one person can build capable multi-modal AI for under $1k using agentic orchestration and cloud GPUs.