NightVision attack infers LLM architecture from restrictive APIs

Most commercial LLM providers have tightened API access to prevent reverse engineering—offering only the top-1 log probability per token and removing logit bias functionality. A team of researchers from Carnegie Mellon University (Christopher Ellis, Shreyas Chaudhari, Mei-Yu Wang, Leighton Barnes, Giulia Fanti, José M. F. Moura) now demonstrates that even these restricted APIs are leaky. Their attack, NightVision, uses a novel "common set prompting" technique: prompting the model with diverse inputs that all force the same set of output tokens, then spectrally analyzing the resulting log probabilities to recover the hidden dimension of the feed-forward network. Additionally, NightVision exploits end-to-end time-to-first-token (TTFT) measurements—combined with the estimated hidden dimension—to infer depth and overall parameter count.

The empirical results are striking. On 32 open-source LLMs, NightVision recovers hidden dimension with 23% average relative error (9% on mixture-of-experts models), and for models exceeding 3 billion parameters, depth and parameter count are estimated within 53% error. The paper includes extensive ablations showing how accuracy scales with token budget and model properties. These findings suggest that current single-logit, no-bias APIs are not enough to fully obfuscate architectural details—potentially forcing LLM providers to rethink API design. The work also raises security and privacy concerns, as architectural information could aid adversarial attacks or competitive intelligence gathering on proprietary models.