Abstractive Text Summarization Using Multimodal Information
Shaik Rafi, Ranjita Das
Abstract
Much text generates over the internet through news articles, story writing and blogs. Reading and understanding such an enormous amount of data to the user is problematic, including time and effort. Automatic abstractive text summarization has gained more importance to increase the user's understanding and reduce time. It shortens the given input by preserving the meaning and identifying the context of the whole document to generate meaningful sentences. The research community has proposed different methods for text reduction and generating abstractive summaries. However, problems like semantics and contextual relationship in the summary generation process must be still need to improve. The multimodal abstractive text summarization is a technique that combines text and image information which helps in addressing the semantics and contextual relationship by proposing Multimodality Image Text (MIT) layer that fuses the text-extracted global features by glove embedding and preserves the semantics of the vocabulary and text-related images are used to identify the contextual relationship features from inception v3, which cope in the MIT layer to generate efficient multimodal abstractive text summaries by training and testing with seq-to-seq model. Experiments with the MSMO dataset achieve superior performance on other state-of-art results.