X-InstructBLIP: A Framework for Aligning Image, 3D, Audio, Video to LLMs and its Emergent Cross-Modal Reasoning
Artemis Panagopoulou, Le Xue, Ning Yu, Junnan Li, Dongxu Li, Shafiq Joty, Ran Xu, Silvio Savarese, Caiming Xiong, Juan Carlos Niebles
Topics & Concepts
Audio visualComputer scienceModalMultimediaImage (mathematics)Computer visionComputer graphics (images)Artificial intelligenceHuman–computer interactionChemistryPolymer chemistryMultimodal Machine Learning ApplicationsVideo Analysis and SummarizationMusic and Audio Processing