LDPKiT enables privacy-preserving AI distillation with strong accuracy
New superimposition technique cuts privacy loss while keeping accuracy above 98%.
LDPKiT, introduced by Kexin Li, Aastha Mehta, and David Lie, tackles a critical tension in regulated AI: how to let users distill knowledge from remote models without exposing sensitive input data. Traditional distillation requires querying the teacher model with private samples, risking leakage. LDPKiT replaces naive queries with a superimposition technique that blends the user's private data with generated, approximately in-distribution samples. This allows effective knowledge transfer while bounding privacy leakage via local differential privacy (LDP) noise injection.
Tested on Fashion-MNIST, SVHN, and PathMNIST, LDPKiT consistently outperforms baselines. On SVHN, at epsilon=1.25 (strong privacy), accuracy stays within 2% of the non-private epsilon=2.0 case. The framework includes sensitivity analyses on dataset size and latent space visualizations explaining the gains. Published at IWAPS/ARES 2026, LDPKiT offers a practical path for healthcare, finance, and other regulated domains where model owners restrict access to API-only.
- LDPKiT uses a novel superimposition of private and generated samples to enable privacy-preserving distillation under local differential privacy.
- On SVHN, accuracy at epsilon=1.25 is within 2% of epsilon=2.0, demonstrating strong privacy without major utility loss.
- Tested across Fashion-MNIST, SVHN, and PathMNIST; benefits increase at higher noise levels.
Why It Matters
Enables secure AI distillation in healthcare and finance where input data must stay private.