Hugging Face's Voice Demo with Gemma 4 31B Replaces OpenAI Realtime API

Andi from Hugging Face showcased a groundbreaking voice AI demo that integrates Nvidia's Parakeet for speech recognition, Google's Gemma 4 31B model (served via Cerebras for ultra-fast inference), and a custom inference pipeline for Qwen3TTS for text-to-speech. The entire pipeline is fully open-source and designed as a drop-in replacement for OpenAI's Realtime API, offering comparable functionality without vendor lock-in. The demo, which runs in the cloud, also includes web search and vision features, reacting faster than a blink.

What makes this announcement significant is the local viability: Andi reported achieving similar latencies on a MacBook Pro M3 with 36GB RAM using the smaller Gemma 4 E4B variant. This means developers can deploy voice AI on consumer hardware without sacrificing performance. The open-source nature allows for modification and self-hosting, challenging proprietary solutions from Big Tech. For those following the reachy minis project, this pipeline is already powering their robots.