A scalable framework for harmonizing, standardization, and correcting crowd-sourced low-cost sensor PM2.5 data across Europe
Amirhossein Hassani, Vasileios Salamalikis, Philipp Schneider, Kerstin Stebel, Núria Castell
Abstract
Citizen-operated low-cost air quality sensors (LCSs) have expanded air quality monitoring through community engagement. However, still challenges related to lack of semantic standards, data quality, and interoperability hinder their integration into official air quality assessments, management, and research. Here, we introduce FILTER, a geospatially scalable framework designed to unify, correct, and enhance the reliability of crowd-sourced PM 2.5 data across various LCS networks. FILTER assesses data quality through five steps: range check, constant value detection, outlier detection, spatial correlation, and spatial similarity. Using official data, we modeled PM 2.5 spatial correlation and similarity (Euclidean distance) as functions of geographic distance as benchmarks for evaluating whether LCS measurements are sufficiently correlated/consistent with neighbors. Our study suggests a −10 to 10 Median Absolute Deviation threshold for outlier flagging (360 h). We find higher PM 2.5 spatial correlation in DJF compared to JJA across Europe while lower PM 2.5 similarity in DJF compared to JJA. We observe seasonal variability in the maximum possible distance between sensors and reference stations for in-situ (remote) PM 2.5 data correction, with optimal thresholds of ∼11.5 km (DJF), ∼12.7 km (MAM), ∼20 km (JJA), and ∼17 km (SON). The values implicitly reflect the spatial representativeness of stations. ±15 km relaxation for each season remains feasible when data loss is a concern. We demonstrate and validate FILTER's effectiveness using European-scale data originating from the two community-based monitoring networks, sensor.community and PurpleAir with QC-ed/corrected output including 37,085 locations and 521,115,762 hourly timestamps. Results facilitate uptake and adoption of crowd-sourced LCS data in regulatory applications. • A framework to unify and flag crowed-sourced PM 2.5 data is designed. • Includes two Processing levels with five Quality Controls steps within each level. • Labels data as “high quality”, “good quality”, or “other quality”. • Reduces median RMSE by ∼50.3 % (from 7.6 μg m −3 ; 95 % CI = 7.2–8) at raw level. • Reduces median RMSE by ∼49.5 % at corrected level.