New AI Tool Makes Video Streaming Quality Comparable Across Apps
Finally, a way to know if your video app is really better than the rest.
Have you ever wondered why video quality seems different on Netflix versus YouTube, even with the same internet speed? Until now, comparing streaming quality across apps was nearly impossible because each study used different tests and viewers. But a new AI tool changes that by acting as a universal referee. It reads the basic playback data—how long the video takes to start, how often it buffers, and how the picture quality changes—and gives a score from 0 to 100. This score is consistent no matter which app or study it comes from.
The AI, built from an open-source language model, doesn't need any special training. It just follows a fixed set of rules to judge the playback events. In tests on over 3,500 sessions from four different studies, the AI's scores matched human ratings with high accuracy. It even outperformed a standard industry method. When the same session was scored multiple times, the AI gave almost identical results, showing it's reliable.
After calibrating the scores, the researchers could place all four studies on the same 0-100 scale. The error was small—between 7.8 and 13.3 points—meaning the scores are trustworthy. They also found that a typical quality metric (VMAF) of 60 corresponded to scores that differed by 17 points between two studies, highlighting how misleading raw metrics can be without a common scale.
So what does this mean for you? Soon, you might see independent quality scores for streaming services, helping you choose the one that actually performs best. For companies, it provides a fair way to benchmark their service against competitors. And for researchers, it's a tool to compare findings across studies. The catch? The AI only looks at playback events, not the actual picture quality, so it's not a perfect measure of visual experience. But it's a big step toward making video quality comparable for everyone.
- A new AI tool scores video streaming quality consistently across different studies, using only basic playback data like startup time and buffering.
- In tests, it matched human ratings closely and even beat a standard industry method, with high reliability.
- This could lead to independent quality scores for streaming apps, helping you pick the best service.
Why It Matters
You'll finally be able to compare streaming services fairly and choose the one that gives you the best viewing experience.