CSSEL-P2P boosts simultaneous speech translation without LLM redesign

Simultaneous speech translation (SimulST) remains challenging for decoder-only LLMs due to limited context and cross-lingual reordering under strict latency constraints. Existing solutions often require architectural modifications or explicit read/write policies that struggle with ambiguous conversational speech boundaries. In a new paper on arXiv, Junkun Chen and colleagues from a research team (affiliations not specified, likely industry) present CSSEL-P2P, a purely data-driven alternative: they chunk audio into fixed-length segments, apply cumulative streaming decoding with a rewind-based committed prefix, and fine-tune on teacher-labeled prefix-to-prefix (P2P) targets with bounded waiting. This approach enables the LLM to output translations incrementally without any model architecture changes.

Evaluated on in-house conversational speech, CSSEL-P2P achieves a +1.54 COMETKiwi improvement in streaming quality over the CSSEL streaming baseline at comparable latency (just +0.15s Average Lagging). The method demonstrates that effective SimulST can be achieved through supervised P2P targets rather than brittle policy rules, potentially simplifying deployment for real-time translation systems. The work aligns with the trend of minimizing model modifications and leveraging data to handle streaming constraints, which could have practical implications for live captioning, real-time interpretation, and voice assistants that need to translate as they listen.