LLMs for Causal Extraction in Disaster Social Media: New Validation Framework
Can LLMs reliably parse fragmented disaster tweets to uncover causal chains?
Deep Dive
A study proposed an expert-grounded evaluation framework comparing LLM-generated causal graphs from disaster-related social media posts with reference graphs from disaster-specific reports, and assessed whether extracted relations are supported by post-event evidence or instead reflect model priors. The findings highlight both the potential and risks of using LLMs for causal relation extraction in disaster decision-support systems. The paper was submitted to EMNLP.
Key Points
- Proposes an expert-grounded evaluation framework comparing LLM causal graphs to reference graphs from disaster reports
- Assesses whether extracted causal relations are supported by post-event evidence or reflect model priors
- Submitted to EMNLP and addresses both potential and risks for using LLMs in disaster decision-support systems
Why It Matters
Validating LLM causal extraction from social media could revolutionize real-time disaster response—or amplify dangerous misinformation.