Flow of Spans: Generalizing Language Models to Dynamic Span-Vocabulary via GFlowNets

Researchers introduced Flow of SpanS (FoSS), a new AI framework that generates text using flexible text spans instead of fixed tokens. By employing Generative Flow Networks (GFlowNets) to navigate a directed acyclic graph structure, FoSS achieves up to 12.5% higher MAUVE scores than standard Transformers in text generation and shows 3.5% gains on knowledge tasks. The model benefits from scaling with larger models and more data, outperforming state-of-the-art methods.