Karpathy: LLMs still 10 years away from continual learning

Andrej Karpathy, a prominent AI researcher known for his work at OpenAI and Tesla, has reiterated that large language models (LLMs) are still approximately ten years away from achieving continual learning. This capability would allow models to continuously absorb new information and experiences without suffering from 'catastrophic forgetting,' a phenomenon where learning new data degrades or erases previously stored knowledge. Karpathy's timeline estimate underscores how far the field remains from building AI systems that can adapt in real time like humans do.

To overcome this hurdle, researchers are actively pursuing two main directions: external memory systems that let models store and retrieve information outside their parameters, and self-modifying models that can update their own weights during inference. While current LLMs rely on static training datasets and periodic fine-tuning, true continual learning would enable models to update themselves on the fly—opening the door to personalized assistants that genuinely remember user preferences, AI agents that learn from each interaction, and systems that stay current without costly retraining cycles. However, Karpathy's decade-long estimate suggests that despite rapid progress in model scale and reasoning, the fundamental architecture needed for lifelong learning remains an open research problem.