CPClean: Certainty-Driven Data Cleaning

• CPClean is a data cleaning methodology that applies certain prediction semantics to improve classification on incomplete datasets.
• It leverages polynomial-time Q1 and Q2 computations for K-NN, efficiently identifying ambiguous examples for targeted intervention.
• Empirical evaluations show CPClean can close up to 66% of the accuracy gap with a limited cleaning budget compared to standard methods.

CPClean is a data cleaning methodology for machine learning grounded in the theory of Certain Predictions (CP), which extends database “certain answer” semantics to supervised classification on incomplete datasets. It aims to identify and prioritize ambiguous training examples whose disambiguation, typically via human intervention, will most efficiently confer stability and accuracy to downstream predictions. The approach is tractable for k-nearest-neighbor (K-NN) classifiers: although the possible completions of an incomplete table are combinatorially large, CPClean offers polynomial-time (and often linear-time) algorithms for determining prediction certainties and supporting cleaning decisions, thereby enabling targeted, cost-effective manual cleaning for robust model performance (Karlaš et al., 2020).

1. Formal Basis: Certain Prediction (CP) Framework

CPClean operates on an incomplete training set, modeled as a Codd table

$D = \{ (C_i, y_i) \mid i = 1,\ldots,N \},$

where each $C_i \subseteq \mathcal{X}$ is a finite candidate set of possible feature-vectors for the $i$th example and $y_i \in \mathcal{Y}$ is the (known) label. The set of possible worlds represented is

$$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$

with $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$. For a fixed learning algorithm $\mathcal{L}$, each $D'$ yields a classifier $h_{D'} = \mathcal{L}(D')$.

A test point $t$ is certainly predicted as label $C_i \subseteq \mathcal{X}$0 if every possible $C_i \subseteq \mathcal{X}$1 yields $C_i \subseteq \mathcal{X}$2. The CP “checking” query (Q1) asks whether this is true, whereas the “counting” query (Q2) computes the number of worlds in which $C_i \subseteq \mathcal{X}$3 receives label $C_i \subseteq \mathcal{X}$4: $C_i \subseteq \mathcal{X}$5

$C_i \subseteq \mathcal{X}$6

These primitives reveal prediction ambiguity stemming from data incompleteness.

2. Efficient Certain Prediction Computation for K-NN

For K-NN classifiers, both Q1 and Q2 admit polynomial-time solutions despite the size of $C_i \subseteq \mathcal{X}$7.

• Compute similarities $C_i \subseteq \mathcal{X}$8 for all candidates $C_i \subseteq \mathcal{X}$9.
• Define for each $i$0 an $i$1-vector $i$2, whose $i$3th entry counts the number of candidates in $i$4 no more similar to $i$5 than $i$6.
• Boundary sets $i$7 characterize the set of worlds in which $i$8 is a boundary $i$9th neighbor of $y_i \in \mathcal{Y}$0; their size is computable via

$y_i \in \mathcal{Y}$1

• Label-tally configurations $y_i \in \mathcal{Y}$2 and the Support$y_i \in \mathcal{Y}$3 quantity (via dynamic programming) yield the contributions per $y_i \in \mathcal{Y}$4-neighbor set.
• For Q2, these components are aggregated to count the total number of worlds in which the label tally favors $y_i \in \mathcal{Y}$5. The complexity is $y_i \in \mathcal{Y}$6 for small $y_i \in \mathcal{Y}$7.

For binary classification, Q1 can be solved even more rapidly: for label $y_i \in \mathcal{Y}$8, construct the $y_i \in \mathcal{Y}$9‐extreme world $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$0 by maximally or minimally favoring that label via candidate selection; Q1 is true iff $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$1 predicts $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$2 for $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$3.

3. CPClean Algorithmic Workflow

The CPClean process consists of iterative, information-theoretic selection of training points to clean:

1. Initialization: Begin with an incomplete training set $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$4 and a clean validation set $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$5.
2. Cleaning model: At each step, select an index $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$6 whose $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$7 will be replaced (by an oracle) with the true $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$8 (singleton), at unit cost. The goal is to minimize the number of such interventions $$\mathcal{I}_D = \{ D' = \{(x_i, y_i)\}_{i=1}^N : x_i \in C_i \text{ for all } i \},$$9 until all $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$0 are certainly predicted (Q1 is true for all).
3. Greedy selection rule: At each round, choose $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$1 to minimize the expected conditional entropy,

$|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$2

where $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$3.

1. Oracle cleaning: For each $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$4, the expected new entropy is averaged over possible $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$5.
2. Termination: When all validation points are certainly predicted, the process halts. Any world completion agrees with the true world on $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$6.
3. Guarantee: The greedy process obtains a $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$7 approximation (up to $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$8 factor) to the maximal possible information gain by any cleaning policy.

4. Illustrative Example

For $|\mathcal{I}_D| = \prod_{i=1}^N |C_i|$9, $\mathcal{L}$0, binary labels:

• $\mathcal{L}$1, $\mathcal{L}$2; $\mathcal{L}$3, $\mathcal{L}$4.
• Four possible worlds: $\mathcal{L}$5, $\mathcal{L}$6, $\mathcal{L}$7, $\mathcal{L}$8.
• Q2 counts the number of worlds where test point’s nearest neighbor has $\mathcal{L}$9 (using $D'$0-products), and similarly for $D'$1.
• At each cleaning step, CPClean evaluates—via Q2—the expected entropy decrease by selecting $D'$2 vs $D'$3 (averaging over possible $D'$4 replacements), and picks the tuple whose cleaning most reduces ambiguity on validation.

This exemplifies CPClean’s capacity for targeted, uncertainty-driven tuple disambiguation.

5. Empirical Evaluation and Comparative Performance

CPClean was evaluated on datasets including BabyProduct ($D'$5, 11.8% real missing), Supreme, Bank, and Puma (each $D'$6–$D'$7, with synthetic 20% missing-not-at-random injection), using $D'$8 and Euclidean distance in K-NN, with 1,000 validation and test examples per dataset.

Cleaning baselines included:

• Ground truth (full manual cleaning),
• Default imputation (mean/mode),
• HoloClean (probabilistic cleaning),
• BoostClean (best heuristic by validation),
• RandomClean.

CPClean consistently outperformed baseline methods. On average, with a fixed 20% cleaning budget, CPClean closed 66% of the accuracy gap (from default-imputed to fully-cleaned) compared to $D'$930% for RandomClean. In many cases, CPClean achieved 100% of the possible gap closure with cleaning performed on only 15–65% of tuples. HoloClean sometimes degraded classifier accuracy (gap closed negative). Beyond 1,000 validation examples, additional validation size had negligible effect on CPClean’s stability. CPClean’s information-theoretic surrogate performed robustly across datasets (Karlaš et al., 2020).

6. Practical Considerations and Broader Impact

CPClean supports both incomplete-data scenarios arising from natural attribute-missingness and those with multiple candidate repair values per tuple. The algorithm steers manual effort to “high influence” cases—tuples whose certainty would most impact the predictive stability on validation or test sets. Its framework is model-agnostic at the conceptual level, but the current tractable implementation assumes K-NN with modest $h_{D'} = \mathcal{L}(D')$0 and $h_{D'} = \mathcal{L}(D')$1.

A plausible implication is that, for other learning algorithms where Q1/Q2-like primitives can be made tractable, analogous versions of CPClean could be developed. The approach is theoretically guaranteed to be near-optimal in its class and empirically demonstrates significant efficiency gains in data cleaning for machine learning pipelines.