AnovaX: open-source local multi-agent voice assistant with adaptive recovery

AnovaX is a proof-of-concept desktop voice assistant that runs completely offline, challenging the dominance of cloud-based pipelines like Siri and Alexa. Built as a single Python process, it chains together a wake-word gate, local speech recognition, and an LLM planner (Gemini) that outputs structured JSON plans of tool calls. A whitelist-and-denylist safety layer ensures only approved actions execute. The multi-agent orchestrator spawns typed child agents—AppAgent, TypingAgent, BrowserAgent, and six others—each with its own timeout, retry policy, and shared-resource locks. A recursive MetaAgent allows the planner to delegate sub-goals to itself (capped at two nesting levels).

What sets AnovaX apart is its adaptive recovery loop: when a core step fails, a compact ReAct-style prompt takes over, and the system hides Gemini's latency behind speculative execution of read-only tools. A companion Flask server exposes a phone-friendly remote over local WiFi, mirrors every agent lifecycle event in real time, and streams the laptop's screen via MJPEG. The project shows that a legible, few-thousand-line assistant can open apps, type, run searches, coordinate concurrent actions, recover from single-step failures, and be driven entirely from a phone—without the LLM ever directly touching the keyboard.