Qwen3-VL-based Nepali meme classifier wins 2nd place with 0.797 F1

Nepali memes are a growing source of online hate speech, but tools to detect them are scarce. A new paper by Nitiz Khanal presents ZeroR, a system built for the CHiPSAL 2026 shared task on multimodal hate speech and sentiment detection in Nepali memes. ZeroR adapts the Robust Adaptation of Hateful Meme Detection (RA-HMD) framework using Qwen3-VL-8B-Instruct, a state-of-the-art vision-language model with native Devanagari script understanding. The key innovation is a two-stage training pipeline: first, LoRA fine-tuning with an MLP projection head for generative classification; second, contrastive backbone fine-tuning using supervised InfoNCE loss. This end-to-end approach eliminates error propagation from separate OCR and translation pipelines, which typically fail on low-resource South Asian languages.

The system also addresses class imbalance with minority oversampling, image augmentation, and focal loss. At inference, ZeroR ensembles stage-1 token probabilities with stage-2 classifier scores using validation-tuned weights. The results are strong: 2nd place on binary hate speech detection (F1: 0.797) and 4th place on three-class sentiment analysis (F1: 0.518) at LREC 2026. The paper includes detailed ablations and error analysis, offering valuable insights into adapting large vision-language models for low-resource languages. For professionals, this demonstrates how reasonably-sized open-weights models like Qwen3-VL can be fine-tuned to deliver practical moderation tools for languages traditionally underserved by AI infrastructure.