Media & Culture

Does the AI industry know AI?

Senior engineers at top AI firms lack basics like LSTM or ODE networks

Deep Dive

A viral Reddit post by user RockyCreamNHotSauce highlights a troubling knowledge gap in the AI industry, based on firsthand conversations with a senior engineer from a Mag7 company (like Google or Microsoft) who runs an LLM-wrapper startup. Despite expertise in search and knowledge graphs, the engineer showed no familiarity with LSTM (long short-term memory) networks, ODE (ordinary differential equation) networks, or spiking neural networks. When the user mentioned using these for a physical AI project, the engineer dismissed it, insisting generative models suffice, and began explaining basic concepts like inference vs. training. The user notes the same experience at Nvidia's conference last October, where hundreds of booths and trillions in valuations yielded no interest in AI model design beyond scaling and benchmarks. This suggests many professionals focus on wrapping and deploying existing models rather than understanding the science.

The post raises critical questions about the industry's depth of knowledge, especially as AI expands into robotics and physical systems requiring diverse architectures. The engineer's ignorance of LSTM—a foundational recurrent network—and ODE networks indicates a gap between practical application and theoretical understanding. With AI's rapid growth, this could lead to over-reliance on generative models, missing innovations in energy-efficient spiking networks or continuous-time ODE models for real-world tasks. The user's experience reflects a broader trend: the industry rewards scaling and hype over foundational science, potentially stifling long-term progress.

Key Points
  • Mag7 engineer with LLM startup lacked knowledge of LSTM, ODE networks, and spiking neural networks
  • Nvidia conference had hundreds of booths but zero interest in AI model design beyond scaling
  • User's physical AI project using non-generative models dismissed by industry professionals

Why It Matters

Highlights a critical expertise gap that could limit AI innovation beyond generative models.