AI Now Reads Movies to Understand Brain Development
This could speed up brain research and help us understand how kids' brains grow.
Watching movies is a great way to study how the brain works because movies are full of social and emotional content. But to link what's happening on screen to brain activity, researchers usually need humans to watch and label every scene, which is slow and expensive. Now, scientists have tested whether AI can do some of that labeling. They used a type of AI that can understand both images and text to rate movie scenes for things like what characters are thinking (theory of mind) and how positive or negative the scene feels (valence).
They compared the AI's ratings to ratings from humans and to brain scans from 66 people, from young children to young adults, who watched three animated movies while their brains were scanned. The AI's ratings partly matched human judgments, but how well they matched brain activity depended on what was being rated and which AI model was used. For example, theory-of-mind ratings matched human and brain data most consistently in visual areas, while valence ratings varied by movie.
The researchers then built a model to predict brain activity from three types of information: visual features (like colors and shapes), scene descriptions (what's happening), and socio-affective ratings (emotions and thoughts). Visual features were the best at predicting brain activity across many areas. Scene descriptions were more specific, predicting activity in attention and visual areas. The model also showed that different types of information were more important at different ages: scene descriptions mattered more for the oldest group in attention areas, while visual features mattered more for the middle group in areas related to salience and emotion.
Overall, the study shows that AI can provide useful labels for brain research, but it's not a perfect substitute for humans. The researchers say AI should be used alongside human annotators, not instead of them. This could make future brain studies faster and cheaper, helping us learn more about how the brain develops.
- AI can rate movie scenes for emotions and thoughts, but its ratings only partly match human judgments.
- Brain scans showed that visual features are the strongest predictor of brain activity, while scene descriptions are more specific.
- Different types of information are more important at different ages, suggesting AI can help study brain development.
Why It Matters
Faster, cheaper brain research could lead to better understanding of child development and mental health.