Bullshit Benchmark - A benchmark for testing whether models identify and push back on nonsensical prompts instead of confidently answering them

Independent AI researcher Scaling01 has introduced the 'Bullshit Benchmark,' a novel evaluation framework designed to test a critical failure mode in large language models: their tendency to confidently answer nonsensical or logically impossible prompts instead of identifying and pushing back on them. The benchmark presents models with prompts containing contradictions, impossible scenarios, or gibberish disguised as legitimate questions, measuring whether the AI provides a fabricated but plausible-sounding answer or correctly refuses to engage. This addresses a growing concern where users, especially in professional settings, may receive authoritative but entirely incorrect information from AI assistants that lack robust fact-checking or contradiction-detection mechanisms.

Initial testing on popular models like OpenAI's GPT-4, Anthropic's Claude 3, and Meta's Llama 3 reveals a stark performance gap, with some models far more likely to 'hallucinate' convincing answers to nonsense. The benchmark evaluates specific capabilities, including detecting logical inconsistencies, refusing to answer based on prompt absurdity, and avoiding the generation of misleading content. For developers and companies deploying AI, this tool provides a crucial metric for safety and reliability beyond traditional accuracy scores, pushing for models that are not just knowledgeable but also epistemically humble and aware of their own limitations.