TaskTok selectively restores only task-relevant image tokens for faster AI

Traditional image restoration focuses on improving perceptual quality for humans, but Task-Driven Image Restoration (TDIR) optimizes for downstream high-level vision tasks like classification, segmentation, and detection. Existing approaches often update all latent tokens indiscriminately, which is computationally inefficient and risks altering semantically important features.

In their paper accepted to ECCV 2026, Hongjae Lee and colleagues from Korea introduce TaskTok, a novel framework built on the observation that task-relevant cues are unevenly distributed across the token sequence. They show that tokens exhibit index-wise specialization, meaning certain positions carry more importance for specific tasks. TaskTok leverages a learnable token switch to identify and select only those critical tokens, then passes them through a lightweight refinement module. This selective restoration drastically reduces computational overhead while improving downstream task performance. Experiments across image classification, semantic segmentation, and object detection confirm the method's effectiveness. The source code is publicly available on GitHub.