RAGthoven ties Gemini 2.5 Flash, agentic scaffolding fails to improve
Multi-stage humor pipeline ties frontier model but extra agents add no value.
Researchers from Slovakia presented RAGthoven at SemEval-2026 Task 1 (MWAHAHA) for multilingual constrained humor generation in English, Spanish, and Chinese. The system decomposes creative text generation into a multi-stage pipeline: a Planner with retrieval-augmented generation (RAG) from a curated joke corpus, a Best-of-N Writer, a Reflector for self-critique, and an LLM-as-a-judge Judge—all grounded in computational humor theory (Benign Violation Theory, Script-based Semantic Theory of Humor). The team ran ten experiments, including two agentic variants: ReAct-style sequential tool-calling (Exp09) and autonomous multi-branch orchestration (Exp10) with a deterministic ConstraintAudit checker. Across four frontier models on a 12-instance English sample, neither agentic variant produced outputs judged superior to the non-agentic pipeline, despite substantially higher tool-call budgets.
RAGthoven shared Rank 1 with the Gemini 2.5 Flash baseline in all three languages, with overlapping confidence intervals. In Spanish, it led by 42 raw Elo points (1182 vs. 1140), while in English (1045 vs. 1081) and Chinese (1045 vs. 1053) the baseline held a higher raw rating within the same statistical tie. These results strongly suggest language-dependent diminishing returns from multi-stage prompt engineering and agentic scaffolding once a strong frontier model is in the loop. The paper (arXiv:2607.13189) provides a detailed analysis of the system architecture and experimental findings.
- RAGthoven uses a four-stage pipeline (Planner, Writer, Reflector, Judge) with RAG from a curated joke corpus.
- Agentic variants (ReAct and multi-branch) required higher tool-call budgets but failed to outperform the simpler pipeline.
- System tied Gemini 2.5 Flash; Spanish showed a 42 Elo point lead, while English and Chinese were statistical ties.
Why It Matters
Suggests complex agentic scaffolding may not improve humor generation quality over strong base models.