OASIS: New AI Grades Videos, Audio and Essays Like a Teacher

Grading one video with AI is easy. Grading thousands of them — fairly, consistently, with a paper trail — is the hard part. OASIS is the assembly line that makes it possible. It keeps track of which recording is which, which version of the grading checklist was used, and what the AI said, so results can be checked later. It was built for medical schools, but the researchers say it works anywhere a recorded performance needs judging.

Here's how it works in plain terms. A human writes a rubric — a simple list of what "good" looks like, like "makes eye contact" or "explains the diagnosis clearly." The AI then watches or reads the recording and gives a separate score for each item, along with the exact moment or quote it based on and a short reason why. It can run on big commercial AI services, or on free open-source models on your own computers if privacy matters. That choice matters a lot for hospitals and companies handling sensitive recordings.

The platform has quietly been in production at UT Southwestern Medical Center since fall 2023, handling more than 7,000 evaluations. That's notable: most AI grading research never leaves the lab. The upside is real — human graders are slow, expensive and can be inconsistent from person to person. An AI that never gets tired could give every student, job candidate or call-center worker the same fair starting point, with feedback in seconds instead of weeks. The architecture is domain-agnostic, meaning the same tool could score sales calls, customer service chats or teaching demos.

The catch is that this is a 27-page technical report, not a product. The authors released no application code, no installer, no sample data and no official software release, so you can't try it today and outside experts can't verify how well it really works. And automated scoring of people raises obvious questions about bias and how you appeal a score you think is wrong. That's why the human review step matters — and why it should stay.