A Small AI on Your Own Laptop Can Now Read Huge Codebases

Most AI coding helpers today, like Claude Code or Codex, are huge programs that live on company servers. You type a question, your code goes to the cloud, and the answer comes back. That works well, but it costs money and it means sending your private code somewhere you don't control. For hospitals, banks, government labs, or anyone working on secret software, that's a dealbreaker.

This team tried a different approach. Instead of making the AI bigger, they moved the hard work earlier. Before you ask anything, the system reads the entire codebase once and builds a searchable map of it — like writing a detailed index at the back of a very long textbook. Then, when you ask a question, it just looks up the relevant pages and hands them to a small AI to explain. (This lookup approach is called RAG, or retrieval-augmented generation — letting AI look things up instead of memorising everything.)

The results were surprising. Bigger isn't better here. A 9-billion-parameter model — small enough to run on a decent laptop or a single office computer — scored best of all seven models tested, with 0.795 out of 1.0 on a 100-question quiz covering eleven kinds of coding questions about a large C++ scientific program. It even beat the same models when they were slotted into Claude Code's retrieval system. What mattered was how good the codebase map was, and which family of model you picked, not raw size.

The bigger picture: front-loading the understanding means small, private AI can give grounded, project-specific answers rather than vague guesses. For in-house scientific and industrial code — the kind that never touches the public internet — this points to AI help without the privacy trade-off.