SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations
Satwik Kottur, Seungwhan Moon, Alborz Geramifard, Babak Damavandi
2021Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing56 citationsDOIOpen Access PDF
Abstract
Next generation task-oriented dialog systems need to understand conversational contexts with their perceived surroundings, to effectively help users in the real-world multimodal environment. Existing task-oriented dialog datasets aimed towards virtual assistance fall short and do not situate the dialog in the user's multimodal context. To overcome, we present a new dataset for Situated and Interactive Multimodal Conversations, SIMMC 2.0, which includes 11K taskoriented user$assistant dialogs (117K utterances) in the shopping domain, grounded in immersive and photo-realistic scenes.
Topics & Concepts
Dialog boxComputer scienceTask (project management)Human–computer interactionMultimodal interactionBenchmark (surveying)Context (archaeology)Dialog systemNatural language processingSituatedDomain (mathematical analysis)Baseline (sea)Pipeline (software)Artificial intelligenceMultimediaWorld Wide WebProgramming languageMathematicsGeodesyGeographyOceanographyPaleontologyEconomicsMathematical analysisGeologyBiologyManagementTopic ModelingMultimodal Machine Learning ApplicationsSpeech and dialogue systems