BART for knowledge grounded conversations
Maxime De Bruyn, Ehsan Lotfi, Jeska Buhmann, Walter Daelemans
Abstract
Transformers have shown incredible capabilities for conversation modeling, however, they store factual knowledge in their learned parameters, which is costly to update with new knowledge since it requires retraining Models trained before the Coronavirus pandemic do not know about COVID-19 In this paper, we investigate how a BART model can be adapted to a knowledge grounded conversational setup We introduce the notion of key and query tokens to retrieve knowledge stored in an external database, that can easily be updated with new knowledge As factual knowledge can hardly be reduced to a single sentence or vector, we allow the model to retrieve multiple sentences from the memory Our analysis shows perplexity decreases with the number of passages retrieved from memory Second, our analysis shows a shared encoder for knowledge retrieval, and conversation understanding reduces the model size and perform as well as a specialized module © 2020 Copyright held by the owner/author(s)