ChaosProbe fingerprints LLMs with chaotic math, sorting GPT-2 from BERT

ChaosProbe, developed by Kunal Kumar Pant and Nithin Nagaraj, offers a fundamentally different way to analyze frozen transformer models: instead of judging them by benchmark performance, it examines their input-embedding spaces through controlled deterministic probes. The method applies a chaotic trajectory-based transformation to each prompt-level embedding matrix, then summarizes Firing Rate and Entropy channel responses into a fixed-length signature. This response-based fingerprint captures structural properties of the embedding space before any contextual computation or task adaptation occurs.

In a bounded proof-of-concept study, the team tested ChaosProbe on 80 neutral prompts across four pretrained models: GPT-2, DistilGPT2, BERT-base-uncased, and RoBERTa-base. Pearson correlation, Spearman correlation, and cosine similarity each recovered all four same-family nearest-neighbor assignments (e.g., GPT-2 with DistilGPT2, BERT with RoBERTa) plus both expected mutual family pairs. Euclidean distance recovered three of four assignments and one mutual pair. Paired bootstrap resampling confirmed the stability of Pearson and Spearman pairings, while signature-validity checks ruled out constant or collapsed responses. This cohort-dependent proof of concept suggests deterministic neurochaotic signatures can expose broad structural relationships among transformer embedding spaces—offering a novel, training-free tool for model comparison and analysis.