X-Translator: real-time multilingual speech translation with speaker voice preservation

Real-time speech-to-speech translation (S2ST) faces a trio of challenges: maintaining translation quality, keeping latency low, and preserving each speaker's unique voice. Most public systems either lag on one of these fronts or rely on closed APIs. The X-Translator team addresses all three with a modular cascaded approach that combines streaming automatic speech recognition (ASR), machine translation (MT), and prompt-conditioned text-to-speech (TTS). A session-level runtime controller orchestrates the pipeline, using incremental segment commitment to convert partial ASR hypotheses into stable translation units — a crucial trick for handling ambiguous turn boundaries in multi-speaker conversations. An online speaker prompt manager binds each source speech span to a specific voice prompt, ensuring that speaker identity carries over into the synthesized target speech.

X-Translator is evaluated on OpenSTBench against commercial speech translation APIs as behavioral baselines. It demonstrates competitive translation quality, speech naturalness, and latency, while excelling in long-form voice stability and multi-speaker speaker preservation. The system is designed to be low-cost and open, providing a practical platform for researchers and developers to explore deployment-oriented S2ST trade-offs. By releasing code and demo, the team aims to bridge the gap between proprietary products and open research, making real-time speaker-aware translation accessible for applications like live multilingual meetings, interviews, and international calls.