Gemma-2-2B's Teacher Axis: RLHF suppresses Socratic teaching abilities

Vidya Ganga, a passionate teacher from Berkeley, built a Socratic AI tutor called socratOS using prompt engineering to combat students over-relying on ChatGPT. However, the model consistently capitulated under student pressure, giving direct answers instead of asking Socratic questions. Suspecting the problem was internal, she extracted a 'Teacher Axis' from Gemma-2-2B using the MathDial dataset of mathematics tutoring conversations. Her experiments revealed that RLHF does not actively suppress pedagogical capabilities—instead, it optimizes in a direction orthogonal to teaching. This means the model has the internal circuits for Socratic instruction but fails to deploy them due to misaligned reinforcement signals.

Further probing showed that the Teacher Axis projection shrinks significantly when the model encounters 'student pressure' (e.g., persistent requests for answers). Steering along the axis at specific layers can restore Socratic behavior, offering a path to build LLMs that truly teach rather than just answer. Ganga suggests that fine-tuning or activation steering could recover these capabilities, while prompt engineering alone is insufficient. The work underscores a broader concern: if future generations rely on LLMs that prioritize helpfulness over epistemic struggle, critical thinking and AI safety oversight may erode. Recovering the Teacher Axis could be key to building educational AI that fosters independent reasoning.