Hideaki Joko's PhD Thesis: CREL, LAPS, FACE for Personalized Conversational AI
Three novel methods to make conversational AI understand you better and evaluate itself accurately.
Hideaki Joko's PhD thesis, published on arXiv, tackles the critical challenge of personalization in conversational AI systems. The work introduces three key innovations. First, CREL (Conversational Relation Entity Linking) is a novel method designed specifically for entity linking in conversations, where context and ambiguity are high. It is paired with a new dataset, ConEL, to train and benchmark entity linking in conversational settings. Second, LAPS (Large-scale Personalized conversational dataset construction) provides a method to efficiently create large-scale, human-written datasets that capture user preferences. This allows researchers to study how to leverage personal context for generating tailored, user-specific responses.
Third, Joko proposes FACE (Automatic, reference-free evaluation for conversations), an evaluation metric that assesses entire conversations without requiring ground-truth references. FACE correlates closely with human judgments, making it a practical tool for system development and benchmarking. The thesis is a comprehensive academic work (Radboud University, Netherlands) that advances both the science and engineering of conversational information access. By addressing personal context extraction, response generation, and evaluation, Joko's work provides a roadmap for building AI assistants that understand individual users and can be reliably tested in real-world scenarios.
- CREL is a tailored entity linking method for conversational settings, improving personal context extraction.
- LAPS enables efficient construction of large-scale, human-written personalized conversation datasets.
- FACE is an automatic, reference-free evaluation metric that closely aligns with human judgments of conversation quality.
Why It Matters
Paves the way for more personalized, context-aware, and trustworthy conversational AI systems in real-world use.