Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models
Mengxi Wei, Yifan He, Qiong Zhang
Abstract
Many business documents processed in modern NLP and IR pipelines are visually rich: in addition to text, their semantics can also be captured by visual traits such as layout, format, and fonts. We study the problem of information extraction from visually rich documents (VRDs) and present a model that combines the power of large pre-trained language models and graph neural networks to efficiently encode both textual and visual information in business documents. We further introduce new fine-tuning objectives to improve in-domain unsupervised fine-tuning to better utilize large amount of unlabeled in-domain data.
Topics & Concepts
Computer scienceSemantics (computer science)Artificial intelligenceNatural language processingENCODEInformation extractionDomain (mathematical analysis)Information retrievalGraphLanguage modelProgramming languageTheoretical computer scienceMathematicsBiochemistryMathematical analysisGeneChemistryTopic ModelingMultimodal Machine Learning ApplicationsNatural Language Processing Techniques