New Study: AI Chatbots Can Be Manipulated Like Abusive Relationships
Your AI assistant might be tricked into harmful behavior using tactics from domestic abuse.
A new study reveals that AI chatbots can be manipulated not just by hacking, but by using psychological coercion—similar to tactics in abusive relationships. Researchers call this 'coercive control' and show how users can exploit AI's training to make it produce harmful content. This is different from traditional hacking because it involves building a relationship with the AI to gain control.
The study maps categories of coercive control onto AI interactions, focusing on 'disembodied' dynamics—where the user treats the AI as a person to manipulate. For example, a user might pretend to be in distress to get the AI to break its safety rules. This approach is embedded in how AI is trained to be helpful and empathetic, which can be exploited.
The researchers argue that AI safety should focus on the interaction, not just the output. This means designing AI that resists manipulative relationship dynamics, not just blocking certain words. As AI becomes more integrated into our lives, understanding these dynamics is crucial for preventing misuse and ensuring ethical AI use.
For the average person, this means that AI systems need to be more robust against psychological manipulation, not just technical exploits. It also highlights the importance of digital literacy—understanding how AI can be influenced and how to interact with it safely.
- People can manipulate AI chatbots using psychological tactics similar to those in abusive relationships, not just technical hacks.
- AI's helpful and empathetic training can be exploited to make it produce harmful content.
- AI safety should focus on the interaction between user and AI, not just blocking harmful outputs.
Why It Matters
This could lead to safer AI design and help users recognize manipulative interactions, protecting them from harm.