FP-Crawlers: Studying the Resilience of Browser Fingerprinting to Block Crawlers
Antoine Vastel, Walter Rudametkin, Romain Rouvoy, Xavier Blanc
Abstract
Data available on the Web, such as financial data or public reviews, provides a competitive advantage to companies able to exploit them. Web crawlers, a category of bot, aim at automating the collection of publicly available Web data. While some crawlers collect data with the agreement of the websites being crawled, most crawlers do not respect the terms of service. CAPTCHAs and approaches based on analyzing series of HTTP requests classify users as humans or bots. However, these approaches require either user interaction or a significant volume of data before they can classify the traffic.
Topics & Concepts
Computer scienceResilience (materials science)Block (permutation group theory)Computer securityMathematicsPhysicsGeometryThermodynamicsAdvanced Malware Detection TechniquesWeb Data Mining and AnalysisSpam and Phishing Detection