Subquadratic's SubQ Model Breaks LLM Bottleneck — Independent Tests Confirm the 12x Context Window Is Real
Miami startup claims SubQ is faster, cheaper, and matches top models.
Subquadratic, a Miami-based startup, claims to have broken the mathematical bottleneck holding back large language models for nearly a decade: the quadratic computational cost of dense attention in transformers. Their new model, SubQ, uses a fundamentally different architecture that avoids the O(n²) scaling that makes processing long texts so expensive. According to the company, SubQ can handle up to 12 times as much text at once as existing models, enabling it to analyze hundreds of documents or entire codebases in a single pass—all while matching the performance of top models from Google DeepMind, OpenAI, and Anthropic on key tasks like coding.
Initial skepticism greeted the claims, with some comparing SubQ to Theranos. But the company has now released results from independent tests by Appen, a third-party AI evaluation firm. Appen's director of generative AI research called the results "game changing," validating Subquadratic's speed and efficiency claims. While SubQ won't replace top models across the board, it offers huge speed gains at a fraction of the cost for data-heavy tasks. Co-founder CEO Justin Dangel believes the breakthrough could kick off "a new age of efficiency" and predicts nobody will be building on transformers in a few years.
- Subquadratic's SubQ model uses a new architecture that avoids the quadratic scaling of transformers, enabling 12x longer context processing.
- Independent tests by Appen confirmed SubQ matches top models on coding tasks while being significantly faster and more energy-efficient.
- Subquadratic aims to replace transformers entirely, claiming SubQ could reduce LLM costs by orders of magnitude for document analysis and codebase processing.
Why It Matters
If validated, SubQ could slash LLM costs by orders of magnitude, enabling real-time analysis of massive documents and codebases.