[Project] PentaNet: Pushing beyond BitNet with Native Pentanary {-2, -1, 0, 1, 2} Quantization (124M, zero-multiplier inference)

Independent AI researcher kyworn has open-sourced PentaNet, a novel language model architecture that advances beyond the popular BitNet 1.58b approach. While BitNet uses ternary quantization {-1, 0, 1} to replace matrix multiplications with additions, PentaNet expands to pentanary weights {-2, -1, 0, 1, 2}. Crucially, multiplying by ±2 requires no hardware multiplier—it's implemented via efficient left bit-shift operations (x << 1). This preserves the "zero-multiplier" inference benefit while giving the network 47% more information capacity per weight (log₂(5) ≈ 2.32 bits vs 1.58 bits for ternary).

In head-to-head testing, kyworn trained two 124M parameter GPT-2 architecture models on WikiText-103 with identical compute budgets. PentaNet achieved approximately 6.4% lower perplexity than the ternary BitNet baseline across three independent training seeds. The weight distribution remained stable without collapsing back to ternary values, and the Straight-Through Estimator (STE) training remained stable throughout. While both models are small, PentaNet generated noticeably more coherent English with proper grammar and avoided the <unk> token collapse seen in BitNet outputs.

The researcher has released complete training code, a PyTorch PentaLinear layer implementation, and model weights on HuggingFace (Kyworn/pentanet-). The current implementation simulates quantization during training; the next major step involves writing custom Triton/CUDA kernels to fully leverage bit-shift operations for real-world inference speedups. This work demonstrates a practical path toward more capable yet computationally efficient models for edge deployment.