The Million-Label NER: Breaking Scale Barriers with GLiNER bi-encoder
New architecture decouples label encoding to handle thousands of entity types simultaneously with minimal overhead.
A research team has introduced GLiNER-bi-Encoder, a breakthrough architecture that fundamentally rethinks how Named Entity Recognition (NER) systems handle massive label sets. The paper addresses a critical limitation in existing NER models: as the number of entity types grows, traditional joint-encoding approaches suffer from quadratic complexity, making them impractical for industrial-scale applications with thousands or millions of labels.
The technical innovation lies in decoupling the encoding process into two specialized components: a dedicated label encoder and a separate context encoder. This bi-encoder design eliminates the context-window bottleneck that plagued previous approaches. By pre-computing label embeddings independently of the input text, the system achieves remarkable efficiency gains—up to 130 times faster throughput when processing 1,024 entity labels compared to uni-encoder predecessors.
Beyond raw speed, the architecture maintains strong zero-shot performance, achieving state-of-the-art 61.5% Micro-F1 on the challenging CrossNER benchmark. The researchers also introduced GLiNKER, a modular framework built on this architecture that enables high-performance entity linking across massive knowledge bases like Wikidata. This represents a significant step toward practical, large-scale information extraction systems that can operate across diverse domains without retraining for each new entity type.
- Bi-encoder architecture separates label and context encoding, eliminating quadratic complexity bottlenecks
- Achieves 130x throughput improvement at 1024 labels while maintaining 61.5% Micro-F1 on CrossNER
- Enables practical NER systems that can handle thousands to millions of entity types simultaneously
Why It Matters
Enables industrial-scale information extraction from documents without the computational bottlenecks that previously limited real-world deployment.