New Study: No Gaussian Release Achieves Both Privacy and Utility in AI

A new paper by Alexander Okezue Bell, submitted to arXiv, tackles a fundamental problem in AI privacy: protecting hidden states (intermediate representations) in neural networks. The study empirically tests 1,536 Gaussian release covariances for single-layer hidden-state privacy and finds that zero achieve both moderate utility and moderate privacy against an adaptive retrieval attacker. This "empty middle" is proven via a Fisher-ball lower bound: every full-rank Gaussian release at O(1) Fisher utility allows a direction where Mahalanobis signal grows linearly with hidden width, ruling out uniform Gaussian safety. The diagonal inverse-Fisher release, Σᵢₐ₉ = (2𝒦/d) diag(1/Fᵢᵢ), emerges as the unique minimax-optimal diagonal mechanism at first-order KL budget 𝒦, but it sits on a privacy/utility edge rather than filling the middle.

Beyond the negative result, the paper explores alternative architectures. A generalized-eigen mechanism achieving 13× Pareto reduction under Euclidean retrieval collapses to 100% top-1 under an adaptive Mahalanobis attacker. A full-trajectory sequence inverter recovers 94% of clean GPT-2 prefixes but 0% under the diagonal mechanism. Crucially, a split-memory transformer trained from scratch reaches a Mahalanobis gain G_Mah ∈ [20, 33] at 90M parameters and maintains a 6–24× advantage over same-budget GPT baselines from 30M to 1B parameters, albeit with a fixed-token language-modeling loss penalty. Pretrained models top out at G_Mah = 9.3. These results reframe hidden-state release from mechanism design within the Gaussian class to architecture or release co-design.