LLUMI uses Reddit upvotes to train open-source LLMs for mental health

A team of researchers from multiple institutions has developed LLUMI, a novel system designed to improve LLM-based writing assistance for mental health support while addressing privacy concerns. LLUMI consists of two components: a generation model that drafts supportive responses and an improvement model that revises human-crafted responses. Instead of relying on expensive proprietary cloud models like GPT-4, the system uses smaller open-source models trained with preference signals derived from Reddit mental health communities. By using upvotes and downvotes to construct chosen-rejected response pairs, the team applied Supervised Fine Tuning (SFT) and Direct Preference Optimization (DPO) to align the models with community-valued traits.

The results are striking: despite using smaller models, LLUMI achieved comparable performance to GPT-based systems across five evaluation dimensions including readability, empathy, connection, actionability, and safety. This demonstrates that open-source models, when trained with community-derived preference signals, can provide high-quality mental health support assistance. Crucially, because LLUMI can be hosted entirely in-house within protected environments, it addresses the significant privacy and data-governance concerns that arise when using cloud-based models for sensitive mental health interactions. This approach paves the way for more accessible, secure, and effective AI-assisted mental health tools.