Open Source

New AI Trick: Deleting Tiny Bits of Code Boosts Learning

New AI Trick: Deleting Tiny Bits of Code Boosts Learning

⚡This could make AI cheaper to update and less forgetful—saving time and money.

Deep Dive

Imagine you have a huge AI model that's already good at many things. To teach it a new skill, you might think you need to retrain the entire model, which is expensive and time-consuming. But researchers discovered that simply turning off a tiny fraction—less than 0.05%—of the model's internal connections can achieve the same result as full retraining. They call this method Deletion Tuning. It's like fixing a car by removing a few unnecessary parts instead of rebuilding the engine.

The big advantage is that the change file is hundreds to thousands of times smaller than a full retrained model. That means it's cheaper to store and share, and updates can be delivered faster. Also, the AI forgets less of what it already knew. In tests, when learning a new task, Deletion Tuning kept its accuracy on old tasks nearly as high as before, while other methods lost up to 5.4 percentage points. This is a big deal for continual learning, where AI needs to keep learning without losing past knowledge.

Why does this work? The researchers analyzed how the model's internal numbers change during learning. They found that Deletion Tuning makes changes that are very similar to full retraining, unlike other efficient methods. This suggests that the key to learning new skills might be more about removing certain connections than adding new ones. The team has released their code and models for others to use.

For everyday people, this means AI apps could get better updates more quickly and cheaply. It also means AI assistants might remember your preferences better over time, even as they learn new things. While still experimental, this approach could lead to more efficient and reliable AI in the future.

Key Points
  • Deleting just 0.05% of an AI's internal settings can match full retraining for new tasks.
  • The update file is up to 1000x smaller, making storage and sharing cheaper and faster.
  • AI forgets less old knowledge, staying near base accuracy while other methods drop up to 5.4 points.

Why It Matters

Cheaper, faster AI updates mean better apps and assistants that remember your preferences without forgetting old skills.

🔒 FREE VIP ACCESS

Read the full breakdown & get daily AI executive intelligence

Join like-minded builders, researchers, and AI professionals who read The Neural Feed daily. 100% free. No spam.

🔔 BREAKING RELEASE RADAR INSTANT ALERTS • 1-CLICK PAUSE

Get instant breaking AI release alerts by topic

Choose the AI labs, models, or chip tracks you follow. Receive fast alerts only when something major ships — plus our free daily morning briefing.

Includes full article unlock • Only major releases • Unsubscribe anytime
🎁 FREE EXECUTIVE DOWNLOAD ISSUE #21

Download This Week's 8-Page Executive AI Briefing (PDF)

Our curated executive intelligence packet: in-depth breakdowns of frontier models, market movements, plus full-color AI comics.

  • ✓ 8-page formatted executive PDF briefing
  • ✓ Frontier AI models & technical breakthroughs
  • ✓ Full-color satirical AI comic strips
Instant 1-click download • 100% free • Unsubscribe anytime
📬 Get the top 10 AI stories daily