Report on the 8th Workshop on Search-Oriented Conversational Artificial Intelligence (SCAI 2024) at CHIIR 2024

Conversational Agents are increasingly integrated into our daily routines, assisting us with various tasks, from simple commands such as scheduling events to more complex conversational search interactions. Such conversational search systems are traditionally evaluated with word-overlap metrics such...

Full description

Saved in:
Bibliographic Details
Published in:SIGIR forum 2024-08, Vol.58 (1), p.1-12
Main Authors: Frummet, Alexander, Papenmeier, Andrea, Fröbe, Maik, Kiesel, Johannes, Adlakha, Vaibhav, Braunschweiler, Norbert, Dubiel, Mateusz, Ghosh, Satanu, Gohsen, Marcel, Kreutz, Christin, Momeni, Milad, Nilles, Markus, Cherumanal, Sachin Pathiyan, Pirmoradi, Abbas, Thomas, Paul, Trippas, Johanne R., Zelch, Ines, Zendel, Oleg
Format: Article
Language:English
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Conversational Agents are increasingly integrated into our daily routines, assisting us with various tasks, from simple commands such as scheduling events to more complex conversational search interactions. Such conversational search systems are traditionally evaluated with word-overlap metrics such as F1 score and accuracy. The full-day workshop on Search-Oriented Conversational Artificial Intelligence (SCAI) at CHIIR 2024 explored the evaluation of conversational search systems from the user's perspective. This interactive workshop included multiple panel discussions and working groups focused on developing and discussing innovative, user-centered evaluation methods for these systems. This paper, co-authored by both organizers and participants of the workshop, presents a summary of the insights gathered from the panel discussions and working groups. Date: 14 March 2024. Website: https://scai.info/scai-2024/.
ISSN:0163-5840
DOI:10.1145/3687273.3687282