New research reveals learning with adversaries is harder than thought
Adding correct examples can make machine learning models 2x slower, study finds.
In a new paper titled 'Optimal Rates for Learning with Monotone Adversaries' submitted to arXiv, researcher Anay Mehrotra challenges conventional wisdom about adversarial learning in machine learning. The study examines scenarios where an adversary appends correctly labeled examples to a dataset, revealing that this can unexpectedly degrade model performance.
The paper demonstrates that for learning problems with VC dimension d, the minimax expected error scales as Θ((d/n)log(n/d))—a logarithmic slowdown compared to the standard Θ(d/n) rate in PAC learning. Mehrotra shows this holds even when the adversary only adds a finite number of examples, with the additional cost being inherent beyond VC dimension one. The findings extend to Littlestone dimension settings, proving that online-to-batch conversion rates of O(d_L/n) are unattainable in adversarial settings. The research introduces a simple improper learner achieving the dimension-one upper bound and provides elementary lower bounds via a construction where two differing hypotheses produce identical samples under adversarial insertions.
- Error rates in adversarial learning scale as Θ((d/n)log(n/d)) for VC dimension d, not Θ(d/n) as previously assumed
- Researcher Anay Mehrotra shows the logarithmic slowdown is inherent beyond VC dimension one
- Even correctly labeled adversarial examples can make learning harder, with worst-case error rates Θ(1/n) at d=1
Why It Matters
This research impacts adversarial robustness and data poisoning defenses, revealing fundamental limits in training with adversarial inputs.