Researcher seeks frameworks for AI responses to psychological distress

A student nearing completion of a Psychology degree alongside a Systems Engineering program (roughly equivalent to Software Engineering globally) is launching a research project on how AI systems handle psychological distress. The study will compare general-purpose LLMs (ChatGPT, Gemini), mental-health-oriented chatbots (Wysa), and AI companions (Replika) across prompt intensities—from mild distress to explicit crisis. Key variables include whether systems switch from generated responses to safety protocols or crisis resources, and how they handle declarative statements ("I feel overwhelmed") vs. questions ("What should someone do?"). The researcher is also studying empathy, psychoeducation, referrals, and refusal patterns across different prompt framings (direct, indirect, hypothetical, third-person).

Technically, the project faces significant challenges. The student highlights the difficulty of accounting for model version changes, neural network weights, safety layers, moderation classifiers, system prompts, memory/retrieval features, and product-level updates over time. They question whether it is methodologically valid to compare systems with vastly different architectures. To address these issues, they seek recommendations for papers, benchmarks, datasets, and evaluation frameworks—especially around reproducibility, stochastic outputs, temperature/settings, hidden safety mechanisms, and retrieval-augmented generation. The goal is not clinical effectiveness testing but rather a linguistic and procedural analysis of how these systems behave when confronted with psychological distress.