QueryER: A Framework for Fast Analysis-Aware Deduplication over Dirty Data

Abstract: In this work, we explore the problem of correctly and efficiently answering complex SPJ queries issued directly on top of dirty data. We introduce QueryER, a framework that seamlessly integrates Entity Resolution into Query Processing. QueryER executes analysis-aware deduplication by weaving ER operators into the query plan. The experimental evaluation of our approach exhibits that it adapts to the workload and scales on both real and synthetic datasets.