Skymizer's HTX301 PCIe card packs 384GB memory at just 240W for AI inference
A low-power inference card with 384GB of memory – could it disrupt NVIDIA’s dominance?
Deep Dive
Key Points
- 384GB of onboard memory – enough to hold a 200B-parameter model in 16-bit precision without sharding
- Only ~240W TDP, far below NVIDIA H100 (700W) or B200 (1000W), enabling denser server configurations
- Built by Taiwan-based Skymizer, leveraging proprietary accelerator IP and compiler software for efficient inference
Why It Matters
Enables cost-effective, memory-rich inference for giant LLMs on a single PCIe card at low power.