Developer Tools

Why Your Company's AI Keeps Dying After the Demo

Most AI pilots never make it to real work — this paper blames messy company data.

Deep Dive

Here's a pattern many businesses know too well. A team buys an AI assistant, runs a flashy demo, everyone claps — and six months later it's quietly switched off. A new position paper from two researchers argues this isn't because the AI is dumb. It's because the AI is reading company information built for humans: spreadsheets, forms, and databases designed for people clicking buttons, not for software that reads.

The authors call this a 'substrate' problem. Think of the substrate as the desk the AI works on. Right now, that desk is covered in forms with little boxes for each fact, so the AI sees 'Customer: Jane' and 'Order: 42' but misses that Jane placed that order. Reading connected prose — paragraphs that hold relationships together — lets the AI grasp far more in one pass. Their proposed fix: rebuild the AI's workspace around formats it reads naturally, such as Markdown (plain text with simple symbols for headings and lists), and keep the awkward translating to the edges, where the AI actually takes action.

Their framework has four layers: Data (the raw facts), Knowledge (what those facts mean), Intelligence (the AI making decisions), and Governance (the rules and audit trails). A 'Sync Agent' sits at the boundary, translating between the AI's world and your existing systems, while a per-skill trust level decides how much freedom each AI task gets. That way, oversight isn't bolted on afterwards — it's built into the structure.

The honest catch: the authors admit Markdown is just what's available today, not proven as the best answer. They also flag a real risk called indirect prompt injection, where hidden instructions planted in company documents could hijack the AI. And this is a position paper — an argument with a research plan, not a finished product with test results.

Key Points
  • Most company AI projects fail after the demo, not because the AI is weak but because it's reading data formatted for humans
  • The authors suggest rebuilding the AI's workspace with plain text formats like Markdown, keeping translations to a single boundary
  • A four-layer plan (Data, Knowledge, Intelligence, Governance) builds oversight and audit trails into the structure instead of adding them later

Why It Matters

If this holds up, companies could stop wasting money on AI pilots that never survive contact with real work.

📬 Get the top 10 AI stories daily