Digital Skin, Digital Bias: Uncovering Tone-Based Biases in LLMs and Emoji Embeddings

A team of eight researchers has published a groundbreaking study titled 'Digital Skin, Digital Bias: Uncovering Tone-Based Biases in LLMs and Emoji Embeddings,' accepted at the prestigious WWW'26 conference. This represents the first large-scale comparative analysis of how AI models handle skin-toned emojis, examining both specialized emoji embedding models (emoji2vec and emoji-sw2v) and four modern large language models: Llama, Gemma, Qwen, and Mistral. The research reveals a critical performance gap where widely-used specialized emoji models show severe deficiencies in supporting skin tone modifiers, while LLMs demonstrate more robust capabilities.

The study employed a multi-faceted investigation methodology examining semantic consistency, representational similarity, sentiment polarity, and core biases across these models. The findings uncover systemic disparities where emojis with different skin tones receive significantly different sentiment scores and inconsistent semantic meanings within the same model architecture. This indicates that foundational AI systems mediating online interactions may be perpetuating and amplifying societal biases through their representation of these crucial digital identity symbols.

These biases manifest as skewed sentiment associations and inconsistent contextual meanings assigned to identical emoji gestures across different skin tone variations. The researchers emphasize that as AI models increasingly mediate web platform interactions, these representational harms could significantly impact social inclusion and personal identity expression in digital spaces. The paper underscores the urgent need for developers and platform operators to implement systematic auditing and mitigation strategies to ensure AI promotes genuine equity rather than reinforcing existing societal biases through seemingly neutral technical systems.