Gemma 3 27B replicates Anthropic's introspective awareness — thinking vs. not thinking shifts internal representations

This research, conducted by Julius Kamp as a capstone project at ARBOx4, replicates Anthropic's seminal 'Intentional Control of Internal States' experiment on Gemma 3 27B Instruct. The original paper found that Claude Opus 4.1 exhibits stronger internal representations of a concept when prompted to think about it while writing an unrelated sentence, compared to being prompted not to think about it. Kamp reproduced the exact methodology — 50 concept words, 50 neutral sentences, 99 baseline words — and generated 5,050 responses (temperature 0). He found the same directional effect in Gemma 3, though at smaller strength. Notably, in 712 of 2,500 'think' prompts, the model integrated the concept into the sentence, requiring exclusion of those data points (likely stronger effect cases).

Kamp extended the experiment by measuring internal representations via SAE latents (Gemma Scope 2) and a Natural Language Autoencoder (NLA) decoding method. Both additional measurements showed the effect much more clearly than the original method of concept vector extraction. This suggests that introspective awareness — the ability to consciously modulate internal concept activation — is not unique to Claude models but may be a general property of large language models. The 'no mention' baseline (prompt without concept word) also provided a cleaner control. Overall, the work strengthens evidence that LLMs have a measurable degree of introspective control over their internal states, with potential implications for interpretability and AI safety.