Binary Embedding-based Retrieval at Tencent
Yukang Gan, Yixiao Ge, Chang Zhou, Shupeng Su, Zhouchuan Xu, Xuyuan Xu, Quanchao Hui, Xiang Chen, Yexin Wang, Ying Shan
Abstract
Large-scale embedding-based retrieval (EBR) is the cornerstone of search-related industrial applications. Given a user query, the system of EBR aims to identify relevant information from a large corpus of documents that may be tens or hundreds of billions in size. The storage and computation turn out to be expensive and inefficient with massive documents and high concurrent queries, making it difficult to further scale up.
Topics & Concepts
Computer scienceEmbeddingCornerstoneInformation retrievalScale (ratio)ComputationBinary numberArtificial intelligenceProgramming languagePhysicsQuantum mechanicsArithmeticArtVisual artsMathematicsDomain Adaptation and Few-Shot LearningAdvanced Image and Video Retrieval TechniquesMachine Learning and Algorithms