MemX: A Local-First Long-Term Memory System for AI Assistants

Researcher Lizheng Sun has published a paper detailing MemX, a novel long-term memory system designed specifically for AI assistants. Unlike cloud-centric alternatives, MemX adopts a 'local-first' architecture, meaning user data and memory stores reside on the user's own device. It's built with performance and stability in mind, using Rust for the core system and libSQL for database operations, coupled with a standard OpenAI-compatible embedding API. The system's key innovation is its sophisticated, multi-stage retrieval pipeline. When an AI assistant needs to recall information, MemX doesn't rely on a single method. Instead, it performs vector recall (for semantic similarity), keyword recall (for exact matches), fuses the results using Reciprocal Rank Fusion (RRF), and then applies a four-factor re-ranking algorithm. Crucially, it includes a low-confidence rejection rule that prevents the system from returning spurious or incorrect information when the answer simply isn't in its memory.

Performance benchmarks are promising. On two custom Chinese-language test suites with over 1,000 records, MemX achieved a 91.3% Hit@1 rate (the correct answer ranked first) in default scenarios and a perfect 100% under high-confusion conditions, thanks to its conservative query suppression. On the larger LongMemEval benchmark with up to 220,349 records, it showed strong performance for factual recall (51.6% Hit@5) but highlighted that temporal and multi-session reasoning remain challenging areas. A major technical achievement is its search speed: by leveraging SQLite's FTS5 full-text indexing, MemX reduces keyword search latency by a staggering 1,100x at the 100,000-record scale, ensuring end-to-end search times remain under 90 milliseconds. The author positions MemX not as a complete agent framework, but as a reproducible, structurally simple, and explainable baseline for developers who prioritize user privacy, local control, and retrieval stability over chasing end-to-end benchmark scores.