New LBA attack tricks LLMs using 95% fewer queries
Researchers discover way to fool AI models with tiny query budgets.
Adversarial attacks on text classifiers typically require either high query budgets (white-box) or produce poor-quality examples. The hard-label scenario—where only final predictions are accessible—is especially challenging. Existing greedy approaches explore one position at a time, often missing globally optimal substitutions and burning through queries. An ideal solution would consider all possible position combinations, but exhaustive search is infeasible even for short texts.
LBA solves this with a sampling framework that builds an approximate distribution of high-quality adversarial examples. It starts from prior knowledge (e.g., word similarity) and updates the distribution using posterior knowledge (results from prior samples). This feedback loop lets LBA focus on promising regions of the search space, achieving stronger attacks with far fewer queries. Experiments across BERT, RoBERTa, and large language models like Llama-3 show LBA reduces query counts by 60–95% compared to baseline methods while generating more semantically coherent texts, as validated by LLM-based assessments.
- LBA uses sampling with prior+posterior knowledge to approximate optimal adversarial examples, bypassing greedy search limitations.
- Achieves state-of-the-art attack success rates on six LLMs with 60–95% fewer queries than prior hard-label methods.
- LLM evaluation confirms LBA's adversarial texts retain better semantic similarity and readability than baselines.
Why It Matters
Reducing query budgets for adversarial attacks could accelerate red-teaming of production LLMs and improve model robustness testing.