Swiss AI's Apertus 1.5 delivers fully open multimodal AI with reasoning
Fully open 70B model handles audio, images, and 262k tokens natively.
Swiss AI released Apertus 1.5, a family of fully open language models in 8B and 70B parameter sizes. These decoder-only transformers use the xIELU activation and AdEMAMix optimizer, continuing pretraining from Apertus 1.0 with an additional 4 trillion tokens for the 8B model and 2 trillion for the 70B model. The models support a massive context window of 262,144 tokens and native multimodal inputs including audio, images, and text. A new thinking mode allows the models to reason before generating responses, boosting performance on complex reasoning tasks. The post-training recipe also improves instruction-following and tool-use capabilities.
Uniquely, Apertus 1.5 is fully open: weights, training data, and all training recipes are released under permissive licenses. The development emphasizes responsible AI by respecting opt-out consent of data owners and using methods to prevent memorization of training data. With multilingual support and a focus on transparency, Apertus 1.5 aims to democratize advanced AI capabilities for developers worldwide. The technical report with benchmarks and intermediate checkpoints is expected in the coming weeks.
- Fully open model: weights, training data, and recipes released for both 8B and 70B versions.
- Native multimodal support (audio, images, text) with 262,144 token context and a thinking mode for reasoning.
- Continued pretraining added 2-4 trillion tokens; performance comparable to other state-of-the-art models of similar size.
Why It Matters
Sets a new standard for transparent, multilingual, multimodal AI that developers can fully control and customize.