Benchmarking Progressive Sampling for Data Quality Profiling
A recent study introduces a new standard that assesses nine innovative sampling techniques aimed at profiling data quality within data-centric AI pipelines. It evaluates both blind methods—such as random uniform, geometric, Yamane, and cluster—and proxy-guided techniques, including Metropolis-Hastings, DAG, stratified, and importance-weighted sampling. The datasets examined encompass NYC 311, NYPD arrests, UCI Adult (up to 500K rows), an IoT sensor stream (2.3M rows), Ultra-Marathon Running (up to 7.4M rows), and synthetic data (5x10^6 rows). Surprisingly, the findings reveal that blind representative samplers consistently surpass proxy-guided methods, regardless of the dataset size.
Key facts
- Data quality profiling computes missing-value rates, duplicate fractions, outlier densities, and functional-dependency violations.
- Exhaustive scans over millions of rows are too slow for near-real-time monitoring.
- Progressive sampling is the standard alternative for data quality profiling.
- Nine sampling strategies were benchmarked: blind (random uniform, geometric, Yamane, cluster) and proxy-guided (Metropolis-Hastings, DAG, stratified by column type or quality score, importance-weighted).
- Real-world datasets include NYC 311, NYPD arrests, and UCI Adult (up to 500K rows).
- An IoT sensor stream dataset contains 2.3 million rows.
- Ultra-Marathon Running dataset has up to 7.4 million rows.
- Synthetic data was scaled to 5 million rows.
- Blind representative samplers dominate uniformly over proxy-guided methods.
Entities
—