New GPU engine slashes memory use for massive matrices
A CUDA engine cuts GPU memory needs 4-8x for trillion-edge graphs
Researchers Francesco Tosoni and Gabriele Mencagli have developed a memory-bounded GPU engine that performs right-multiplication on grammar-compressed matrices using a streaming approach. This new system, detailed in their arXiv paper (arXiv:2607.24971), represents matrices as RePair straight-line programs (SLPs) stored in a directed acyclic graph (DAG) with out-degree 2. The key innovation is a layered grammar structure that enables streaming evaluation where each level only needs to read the level below, keeping live memory confined to two alternating buffers rather than the entire grammar.
The CUDA implementation demonstrates significant advantages over traditional methods like cuSPARSE CSR. For genotype matrices, it achieves a device footprint 4-8 times smaller while maintaining compute times within a small factor of the baseline. The same engine can evaluate any monoid homomorphism by swapping combine operations, and scales to massive graphs like the Software Heritage graph (261TB dense), where it maintains its memory advantage over CSR representations.
- 4-8x smaller GPU memory footprint vs cuSPARSE CSR for genotype matrices
- Streaming evaluation uses only two alternating buffers instead of full grammar storage
- Same engine supports monoid homomorphisms and scales to billion-edge graphs like Software Heritage's 261TB dataset
Why It Matters
Enables practical GPU processing of matrices too large for conventional memory, unlocking new applications in genomics and large-scale graph analytics.