NLP Psychometrics: LLM personas predict anxiety from text with 76% variance

A new arXiv paper (2608.07316) from researchers Edoardo Sebastiano De Duro, Emma Franchino, and Massimo Stella tackles a longstanding problem in NLP-based mental health prediction: models rarely specify whether they're measuring emotional content, syntactic structure, or contextual knowledge. Their solution, called NLP Psychometrics, treats psychological prediction from text as a formal psychometric problem, linking predicted scores to interpretable linguistic evidence and testing beyond the training format. The team conditioned nine LLMs on controlled personas—dubbed 'cognitive digital shadows'—to complete psychometric questionnaires with textual explanations per item, then extracted emotional and syntactic-semantic features via textual forma mentis networks.

Using ablated random forest regressors with SHAP feature attribution, the full models explained up to 70.8% of variance in life satisfaction (SWLS), 55.7% in depression (PHQ-9), and for DASS-21, 68.5% depression, 76.0% anxiety, and 72.4% stress. Critically, sociodemographics alone explained no meaningful variance in depression, anxiety, or stress but did for life satisfaction. Without retraining, the models separated diaries from low- and high-score personas (correlation up to 0.91) and, using only network/emotion features, classified clinical from control participants in real transcripts with up to 68% accuracy. The authors emphasize both promise and limits: LLM personas can expose model biases and recover patterns consistent with clinical rumination, but they cannot substitute for human validation.