Chuweb21D: A Deduped English Document Collection for Web Search Tasks
Zhumin Chu, Tetsuya Sakai, Qingyao Ai, Yiqun Liu
Abstract
As a traditional information retrieval task, ad hoc web search has long been an important part of IR research and evaluation tracks (e.g. TREC, NTCIR and CLEF). A crawled, large-scale web document collection is a central component to offline web search evaluation. Although there already exist several English document collections, such as the ClueWeb series, GOV2 and c4, a collection that satisfies properties of both strong timeliness and raw HTML formatting is still relatively scarce.
Topics & Concepts
Computer scienceInformation retrievalClefDisk formattingWorld Wide WebTask (project management)Web pageEconomicsManagementOperating systemWeb Data Mining and AnalysisInformation Retrieval and Search BehaviorImage Retrieval and Classification Techniques