Amazon-Style Shopping AI Gets Confused by Other Languages, Study Finds
Same product, different language — and the AI files it somewhere else.
Researchers from a new study asked a simple question: if the same product listing is written in English, Spanish and Japanese, does the AI file it under the same label each time? The label in question is a "semantic ID" — think of it as a short code, like a library shelf number, that AI uses to find and recommend products. The team used real Amazon listings translated into all three languages. The answer was no. A Japanese translation kept the very first code of its English twin only 7.7% of the time. An English rewording of the same listing kept it 89% of the time. The AI apparently treats a translation as a noticeably different item, even when it isn't.
The researchers then checked whether the tool that assigns these codes was simply overreacting to language differences. They compared translations against unrelated products that sat the same distance away in the AI's mind. The mismatch was nearly identical — so no, the tool isn't specifically picking on foreign languages. The underlying problem is baked in earlier, when the AI turns text into numbers.
Here's the twist that will frustrate anyone hoping for an easy fix. When the researchers balanced their training across languages so the codes were used more evenly, cross-language agreement got worse, not better. Spanish translations matching their English code fell from 28.3% to 6.6%. Fitting the system on English only actually preserved that match 67.6% of the time. More languages, it turns out, is not automatically better.
So what does this mean in practice? If you shop in more than one language, you may be shown different products, different prices and different deals for the same item — without either listing being wrong. Sellers listing in a second language can end up invisible to buyers searching in the first. And anyone building multilingual search now knows that simply feeding the AI more languages does not make it agree with itself. Fixing this will take new methods, not just more data.
- AI product search assigns different ID codes to the same item when the description is in a different language — it treats translations as separate products.
- In tests on Amazon listings, a Japanese translation matched its English version's code just 7.7% of the time, while an English rewording matched 89% of the time.
- Trying to fix it by training evenly across languages made agreement worse, meaning shoppers in Spanish or Japanese may see different results, prices and deals for identical items.
Why It Matters
Multilingual shoppers and sellers may miss listings for the exact same product, because the AI files translations separately.