Seven of eight binary features are linearly encoded (AUC >0.995) in the 64-dim hidden layer?

Seven of eight binary features are linearly encoded (AUC >0.995) in the 64-dim hidden layer.

Country feature collapses to 0.487 AUC with linear probe, but MLP recovers to 0.987—a 0.500 gap?

Country feature collapses to 0.487 AUC with linear probe, but MLP recovers to 0.987—a 0.500 gap.

The non-linear encoding uses projection magnitude (distance), not sign, to store the country label.

AI Safety

LessWrong AI July 03, 2026

⚡Linear probes hit 0.487 AUC on country detection—MLP recovers to 0.987.

Deep Dive

Key Points

Seven of eight binary features are linearly encoded (AUC >0.995) in the 64-dim hidden layer.
Country feature collapses to 0.487 AUC with linear probe, but MLP recovers to 0.987—a 0.500 gap.
The non-linear encoding uses projection magnitude (distance), not sign, to store the country label.

Linear probes, a standard interpretability tool, can completely miss non-linear feature encodings, leading to blind spots in safety analysis.