CARPRT improves zero-shot VLM classification by 20%
New method CARPRT boosts black-box vision-language models' accuracy by adjusting prompt weights per class.
Deep Dive
Researchers introduce CARPRT, a training-free method that adjusts prompt weights for each class based on class-specific relevance. It outperforms existing class-independent reweighting strategies on standard benchmarks. Code is available.
Key Points
- CARPRT improves zero-shot VLM classification accuracy by 20% by dynamically adjusting prompt weights per class.
- Method is training-free and works with black-box VLMs like CLIP, BLIP, or proprietary models.
- Code and demos available on Hugging Face with plug-and-play integration.
Why It Matters
CARPRT unlocks more accurate zero-shot classification in VLMs, reducing reliance on labeled data and manual prompt engineering.