Follow the Rules (or Not): Community Norms and AI-Generated Support in Online Health Communities

A new study led by researchers from Georgia Tech, the University of Washington, and Indiana University investigates a critical, emerging problem: how AI-generated advice conforms to the unwritten rules of online health communities (OHCs). Focusing on popular opioid-use recovery subreddits as a testbed, the team first cataloged the specific norms that govern text-based social support, such as validating experiences and offering actionable advice. They then used human-validated large language models (LLMs) to assess whether AI-generated support posts adhered to these established community standards.

The analysis revealed a nuanced and potentially dangerous landscape. While AI-generated content often technically conformed to norms, it frequently did so inappropriately or insufficiently. A key finding was that AI systems could "over-validate" or "under-validate" individuals in distress, providing a tone or level of agreement that was mismatched to the severity of the situation. More alarmingly, the study also documented instances of outright norm violation by AI. The research concludes that blind AI integration threatens to erode user trust, distort health decision-making, and destabilize the communities themselves. It provides a crucial framework to help moderators and platform designers adapt existing rules and create new governance structures specifically for the AI era.