FedPAIE lets AI learn your photo tastes without uploading images

A team of researchers led by Chuanzhi Xu (University of Sydney and collaborators) has released FedPAIE, a federated aesthetic preference learning framework that lets smartphones and edge devices learn a user's color grading taste without ever sending their photos or ratings to a central server. The system tackles a common problem in personalized image enhancement: learning individual preferences typically requires aggregating private user data, which raises major privacy concerns. FedPAIE solves this by keeping all training data on-device and sharing only model updates.

The architecture is elegantly lightweight. A dual-cue aesthetic scorer is trained with at most 0.787 million parameters, then calibrated into a personalized scorer using a small local support set. That scorer guides the adaptation of a lightweight CLUT (color lookup table) enhancer, which updates just 0.265 million parameters. At inference, the entire personalized enhancer is only 0.293 million parameters—small enough to run on resource-constrained smartphones. To prevent the common problem of "proxy-score over-optimization" (where the AI overfits to the metric rather than looking good), FedPAIE applies fidelity constraints and an excess-gap penalty, keeping output natural while matching user preference.

In experiments on the MIT-Adobe FiveK and Flickr-AES benchmarks, FedPAIE demonstrated strong open-world personalization, meaning it generalizes to images outside the training set. It achieves a favorable trade-off between what users want (brighter, warmer, more contrast, etc.) and preserving the original image's realism, all without needing paired user retouches. This makes FedPAIE a practical step toward truly private, on-device photo enhancement that adapts to each individual's aesthetic.