LifeInsight2.0: An Enhanced Approach for Automated Lifelog Retrieval in LSC'24
Gia-Huy Vuong, Van-Son Ho, Tien-Thanh Nguyen-Dang, Xuan-Dang Thai, Thang-Long Nguyen-Ho, Minh-Khoi Pham, Tu-Khiem Le, Van-Tu Ninh, Graham Healy, Cathal Gurrin, Minh–Triet Tran
Abstract
We introduce the LifeInsight 2.0 system - an enhanced version of LifeInsight, built specifically for the sixth annual Lifelog Search Challenge (LSC'23). LifeInsight 2.0 leverages the core functionalities of LifeInsight while incorporating significant improvements to address performance bottlenecks. This refined architecture aims to deliver superior search capabilities within the LSC'24. LifeInsight 2.0 employs an ensemble approach combining two powerful foundation models: CLIP (Contrastive Language-Image Pretraining) and BLIP2 (Bootstrapping Language-Image Pretraining) model. In addition, the system incorporates a temporal query mechanism and an automatic query parser. The former enables LifeInsight 2.0 to interpret queries that include time-based information, while the latter specifically handles tasks involving question answering.