Microsoft Studied 40,000 AI Photo Chats — Here's What People Really Ask

Microsoft researchers wanted to answer a simple question: when people hand a picture to an AI chatbot, what are they actually trying to get done? To find out, they analyzed more than 40,000 de-identified conversations from Microsoft Copilot — chats where someone uploaded an image and asked for something. They then sorted every request into ten categories of ability, grouped into three buckets: seeing (perception), thinking (cognition), and making (generation).

Their biggest finding is that image requests are rarely one thing. Most of the time, people are asking the AI to do several jobs at once — read a receipt, then add up the numbers, then write an email about it. The researchers also found that when you add pictures to the mix, people attempt a wider and stranger variety of tasks than they do with text alone. Some jobs only work if the AI can connect what's in the picture to the words in your question, like matching a photo of a broken part to instructions for fixing it.

Then came the uncomfortable part. The team compared what people actually do against 253 existing AI tests — the standard exams used to grade these tools. Those tests mostly check whether the AI can look at an image and give one correct answer. But the everyday tasks people bring — writing text, producing code, organizing data — are barely tested at all. They confirmed the same pattern in a separate set of ChatGPT conversations.

The takeaway for you: companies tend to build and improve what they can measure. If the official tests don't resemble how you actually use an AI's eyes, the tool may look brilliant in a demo but stumble on your real work. As these assistants spread into phones, cameras, and offices, closing that gap matters — and it's a reminder that when you upload a photo, the request itself becomes research data.