New AI Trick Lets You Update Facts Without Retraining

Large language models (LLMs) like ChatGPT store facts in their billions of parameters. When a fact changes—say, a country gets a new president—you typically have to retrain the whole model, which is slow and expensive. Now, researchers have found a way to update facts without touching the rest of the model, using something called 'conditional memory.'

Their method, EngramEdit, works by giving the model a separate memory bank that looks up facts based on short phrases (n-grams). When you want to update a fact, you only change the relevant entries in this memory, leaving the rest of the model untouched. This is like having a fact database that the AI consults, rather than baking facts into its brain.

The challenge is that the same fact can be expressed in many ways, and updating one expression might accidentally change others. EngramEdit solves this by computing target representations for the new fact across multiple expressions and carefully updating shared memory entries, penalizing changes to frequently used ones. In tests, it achieved near-perfect editing success, and the updated knowledge was usable in new sentences and multi-step reasoning, with three times the accuracy of the best previous method.

This means AI models could stay up-to-date without costly retraining, making them more reliable and adaptable. It also opens the door to personalized AI that can be corrected on the fly, like fixing a mistake in a document. While still a research prototype, it points to a future where AI knowledge is as easy to edit as a Wikipedia page.