Steering Frozen LLMs: Adaptive Social Alignment via Online Prompt Routing

A team of researchers has published a paper introducing CCLUB (Consensus Clustering LinUCB Bandit), a novel framework designed to solve a critical flaw in current large language models (LLMs). Models like GPT-4 and Claude are typically aligned using static, one-time methods like Reinforcement Learning from Human Feedback (RLHF), which creates a fixed policy. This leaves them vulnerable to evolving jailbreak attacks and unable to adapt to changing societal safety norms without expensive, full model retraining. CCLUB proposes a solution at inference time, treating the problem as an online learning task where the system dynamically selects the safest 'system prompt' from a pool to steer the model's responses.

The core innovation is a 'conservative consensus clustering' mechanism. It analyzes incoming user prompts and routes them by finding consensus within graphs of both utility (helpfulness) and safety similarity. This prevents the system from generalizing safety rules incorrectly across contexts that seem semantically similar but carry different risks. Theoretically, the approach guarantees sublinear regret, meaning it learns efficiently. In extensive experiments, CCLUB demonstrated significant practical gains, outperforming strong baselines with a 10.98% boost in cumulative reward and reducing the average suboptimality gap by 14.42%. This shows a clear path toward LLMs that can maintain safety and usefulness adaptively, in real-time.