Dev builds NPC engine with Gemma 4 and RAG for instant RPG responses

A developer known as u/goodive123 has built a game-agnostic NPC engine designed to work entirely with local AI models. The architecture borrows heavily from SillyTavern but is optimized for real-time RPG interactions. It uses NVIDIA Parakeet 0.6 for speech recognition, Gemma 4 26B A4B (a 26B-parameter model with 4B active parameters) as the core language model, and Qwen3-TTS for text-to-speech output. The combination yields response times below one second while maintaining “decent quality” dialogue and voice.

The key innovation is the use of RAG (retrieval-augmented generation) to keep prompts lean. The engine maintains a library of hundreds of possible NPC actions, but only the most contextually relevant ones are injected into each prompt based on the player’s current message and game state. This prevents the model from being overloaded with a giant action list every turn, reducing token usage and latency. The developer believes this approach could become the standard for future RPGs, especially as smaller local models continue to improve in capability and speed.