New 5MB CUDA inference engine runs NVIDIA's DVLT 3D transformer
No Python, no PyTorch, just raw CUDA and a 5MB binary – 117M parameters.
A developer known as yassa9 has released dvlt.cu, a stunningly lightweight inference engine for NVIDIA's DVLT 3D transformer model. Written from scratch in CUDA/C++, the entire executable fits in just 5MB and has virtually no dependencies: only cuBLASLt (shipped with libcuda) and the header-only cuTLASS library. There is no Python, PyTorch, TensorFlow, ONNX, vLLM, or Hugging Face runtime. The weights (117 million parameters, bfloat16) are NVIDIA's original non-commercial release, fetched separately and memory-mapped for efficient loading. The engine uses static dimensions and a one-shot arena allocator, ensuring deterministic execution with a single bulk GPU upload.
This approach makes 3D reconstruction accessible on any NVIDIA GPU without the overhead of modern ML frameworks. Users simply download the binary, build it, and point it at images or video. The output is a point cloud with camera poses, viewable in a single-file HTML viewer—no additional software required. While the weights are limited to non-commercial use, dvlt.cu demonstrates how lean, high-performance inference can be achieved for specialized transformer models. It's a powerful example of pushing the boundaries of minimalism in AI deployment, potentially enabling edge applications where traditional frameworks are impractical.
- Single 5MB binary written in CUDA/C++ with no Python or deep learning framework dependencies
- Uses mmap'd bfloat16 weights (117M parameters) from NVIDIA's DVLT model, loaded in one bulk GPU transfer
- Outputs 3D point clouds with camera poses, viewable in a standalone HTML file — no install needed
Why It Matters
Proves high-performance 3D AI inference can run without heavyweight frameworks, ideal for edge and embedded deployment.