Witchcraft brings SQLite-powered semantic search to your device with 20ms latency

Witchcraft is a from-scratch Rust reimplementation of Stanford's XTR-WARP semantic search engine, built at Dropbox by Jacob Gorm. It uses a single-file SQLite database as backing storage, eliminating the need for external vector databases, API keys, or complex chunking strategies. This makes it ideal for client-side deployment, running entirely offline on your device with no cloud dependencies.

Performance is impressive: on an Apple MacBook Pro M2 Max, Witchcraft achieves 20ms p.95 end-to-end search latency on the NFCorpus dataset with 33% NDCG@10 accuracy. That's more than twice as fast as the original XTR-WARP on server-class hardware, while maintaining similar accuracy. The entire system is self-contained, with no fancy re-rankers or additional services required.

The project also includes Pickbrain, a CLI that indexes Claude Code and OpenAI Codex session transcripts, memory files, and authored documents into a Witchcraft database. Users can query 'What was that conversation where I fixed the auth middleware?' and Pickbrain finds it, allowing direct session resumption. A /pickbrain skill for both Claude and Codex equips those tools with global memory across all sessions, enabling agents to pull previous context automatically—for example, 'use /pickbrain to read up on our previous efforts on training with XTR token masking' to populate a new session with relevant history.