Light-Powered AI: Faster Responses Without New AI Chips

When you ask a chatbot something, it often delays before answering. That's because AI has to read everything you wrote — your entire prompt, any past messages, and support files — before it can begin replying. This step is called 'prefill,' and it gets slower when you give the AI more context, like a long article or a huge spreadsheet. A new paper suggests that the internal wiring between AI chips is partly to blame. The current standard is copper cables, and they struggle when thousands of chips need to share data at the same time.

The lightbulb idea is to replace those copper connections with light. Researchers simulated what happens if AI systems inside a server use photonic interconnects — tiny pipes that send information using lasers rather than electrons. Using light lets every chip talk to every other chip much faster and with less interference. In their tests, the light-based setup delivered 2.1 to 5.8 times better performance than copper, depending on the job. The biggest gains came in 'communication-limited' situations — when the AI spends most of its time waiting for data from other chips rather than actually thinking.

The paper also looked at a landmark milestone: a system people often call 'time to first token.' That's the wait before the AI even begins producing its first word. For advanced AI models, especially ones like reasoning bots that need huge context windows (up to a million tokens, roughly a book series), this wait can be painful. With light connections, the AI can spread its work across many more chips and finish this prefill stage dramatically sooner. Tests showed 2.2 to 4.5 times faster responses across entire production systems.

There is a catch. This is a research simulation, not a shipping product. Building these photonic systems requires new hardware and manufacturing processes, which won't appear overnight. But it points to a clear next step: the biggest speedups in AI often come from improving the network between chips, not just the chips themselves. The next time an AI feels snappier, light may be why.