TheraJudge & TheraAgent: Multi-agent AI framework boosts therapeutic response quality by +0.43

A team of researchers from multiple institutions has introduced a novel framework for aligning large language models with human therapeutic standards. The system, called TheraJudge, is an open-source evaluator trained via preference-based optimization on human-annotated data. It assesses mental health responses across seven psychological dimensions—including safety, relevance, and empathy—and achieves strong agreement with clinician ratings, with intraclass correlation coefficients ranging from 0.87 to 0.95. This surpasses both supervised baselines and proprietary closed-source judges, especially on critical safety and empathy metrics.

In the second stage, the framework introduces TheraAgent, which operationalizes TheraJudge's evaluations through a coordinated multi-agent refinement process. Three specialized agents—Critic, Coach, and Therapist—work together to translate evaluative signals into targeted response revisions. The results are striking: under blind evaluation, TheraAgent improves overall therapeutic quality by +0.43 on a 5-point scale, with 96% clinician inter-rater reliability. For low-quality responses (≤3 points), the improvement jumps to +2.45 points with a 94% recovery rate, demonstrating the framework's ability to correct unsafe or ineffective outputs. The team has released the code publicly, making this a significant step toward human-aligned, scalable mental health support using LLMs.