DQI-Kit: Data Quality Tool & Quantum Framework

• DQI-Kit is a dual-use toolkit offering both a Data Quality Toolkit for AI and a framework for encoding quantum optimization problems into Max-LINSAT.
• The ML-oriented DQI-Kit automatically assesses data readiness by diagnosing issues like noisy labels, class overlap, and class imbalance while providing actionable remediation recommendations.
• The quantum-oriented DQI-Kit transforms complex constrained optimization problems into Max-LINSAT instances and estimates performance using methods such as belief propagation and information set decoding.

DQI-Kit is an overloaded designation in the arXiv literature. It refers, first, to the Data Quality Toolkit for AI, an ML-oriented system for automatic assessment of training-data quality, issue explanation, remediation, and change tracking for structured datasets (Gupta et al., 2021). It also refers, separately, to DQI-Kit -- A Software Framework for Decoded Quantum Interferometry, a Python framework for translating constrained optimisation problems into Max-LINSAT instances suitable for Decoded Quantum Interferometry (DQI) (Thelen et al., 16 May 2026). These two uses are unrelated in purpose, architecture, and application domain. A further point of clarification is that the survey of open-source data quality tools in "A survey of open-source data quality tools: shedding light on the materialization of data quality dimensions in practice" does not cover DQI-Kit at all (Papastergios et al., 2024).

1. Naming and scope

The term DQI-Kit is used in two distinct technical contexts. In machine learning data preparation, it denotes the Data Quality Toolkit, also described as a Data Quality Toolkit for AI or DQI-Kit, whose role is to assess whether structured labeled data are fit for downstream ML, identify problems such as noisy labels or class overlap, recommend remediations, and generate a Data Readiness Report (Gupta et al., 2021). In quantum optimisation, it denotes a software framework for Decoded Quantum Interferometry that automatically encodes constrained optimisation problems into Max-LINSAT and estimates expected DQI performance (Thelen et al., 16 May 2026).

The distinction is substantive. The ML-oriented DQI-Kit is concerned with data readiness before model training. The quantum-oriented DQI-Kit is concerned with problem transformation for a quantum optimisation method whose native domain is Max-LINSAT. A plausible implication is that any reference to DQI-Kit requires disambiguation by domain rather than by acronym alone.

2. DQI-Kit as the Data Quality Toolkit for AI

In the ML literature, DQI-Kit is presented as a system for automatic assessment of training data quality, explanation of detected issues, remediation of those issues, and tracking/reporting of all dataset changes (Gupta et al., 2021). Its motivation is that conventional data cleaning and profiling tools do not adequately capture problems that matter specifically for machine learning, including noisy labels, existence of overlapping classes, class imbalance in the presence of difficult minority samples, feature irrelevance, fairness bias, correlated features, and outliers.

The toolkit is positioned explicitly at “step 0” of the ML workflow, before downstream modeling. Its functional pipeline is described as: ingest structured labeled data; measure multiple ML-relevant quality dimensions; produce metric-wise scores, explanations, and recommendations; optionally remediate issues; write updated data; and generate a Data Readiness Report that records baseline quality and lineage of operations (Gupta et al., 2021). The architecture has three main components: Data Quality Measurement, Data Remediation, and Data Readiness Report.

The measurement component returns output in human-readable JSON. For each metric, the output contains a quality score, a textual explanation, recommended actions, and additional analysis details. The paper states that all quality scores are normalized to [0, 1], where 1 means no observed issue for that quality dimension (Gupta et al., 2021). The remediation component can be automatic or human-in-the-loop. The report component records baseline quality, detected issues, transformations, and lineage/history of operations, and is described as dynamic, not static.

The paper describes the toolkit as a library of quality metrics and remediation techniques exposed through IBM API Hub as IBM Data Quality for AI APIs, with tutorials on IBM Learning Path (Gupta et al., 2021). It is focused on structured/tabular datasets, primarily in supervised classification settings.

3. Metrics, outputs, and empirical profile of the ML-oriented toolkit

The Data Quality Toolkit paper presents a metric suite for structured ML data. The metrics explicitly described are Class Overlap, Label Purity, Class Parity, Feature Relevance, Data Homogeneity, Data Fairness, Correlation Detection, Data Completeness, Outlier Detection, and Data Duplicates (Gupta et al., 2021). The paper repeatedly emphasizes that, for most of these metrics, it does not provide explicit mathematical formulas or full algorithmic detail, so the system paper is primarily architectural and workflow-oriented rather than derivation-heavy.

Several outputs are concrete. Class Overlap identifies class-wise overlapping regions, an aggregated overlap score, and feature ranges contributing to overlap. Label Purity reports noise ratio, number of noisy samples, indices of noisy samples, original labels, and suggested corrected labels. Correlation Detection identifies correlated numerical columns and recommends dropping the highest correlated columns with respect to all columns from each identified correlated pair. Data Completeness identifies the location of missing values and uses a constraints and association based approach for imputation. Data Duplicates uses pandas .duplicated for duplicate detection (Gupta et al., 2021).

The empirical sections are strongest for overlap, noisy labels, and class imbalance. For Class Overlap, the paper reports evaluation on 25 datasets with 20% overlap points added, finding that for 18/25 datasets classifier performance drops by more than 1%, for 12 datasets the drop is greater than 4%, and the maximum observed drop is 15%. For overlap detection on 19 datasets, average precision is reported as above 0.80 and average recall above 0.90. For Label Purity, on 35 datasets with 10% random noise per class introduced in training only, precision improves by 5–15% over CleanLab, with recall described as similar (Gupta et al., 2021).

The paper also states that a report was generated after running 8 quality metrics and 3 remediations on the UCI breast-cancer dataset. This suggests that the toolkit’s intended unit of use is not a single metric invocation but a multi-metric readiness assessment tied to downstream ML suitability.

4. DQI-Kit as a framework for Decoded Quantum Interferometry

In quantum optimisation, DQI-Kit is described as a software framework that provides a unified, extensible interface for automatically encoding constrained optimisation problems into Max-LINSAT and estimating expected performance of Decoded Quantum Interferometry on the resulting instances (Thelen et al., 16 May 2026). Its motivation is the mismatch between industrial optimisation formulations—such as SAT, MILP, integer constraints, Boolean logic, and QUBO/PUBO-like objectives—and DQI’s native problem class, which is formulated as systems of linear equations over finite fields whose objective is to maximise the number of satisfied equations.

The paper defines the native target problem as Max-LINSAT and also uses Weighted Set-Max-LINSAT. The latter is given as

$$f(x) = \sum_{i=1}^{m} f_i(x), \qquad f_i(x)= \begin{cases} w_i & \text{if } x \cdot b_i \in F_i,\ 0 & \text{otherwise.} \end{cases}$$

The paper explicitly notes that Set-Max-LINSAT is equivalent in expressive power to ordinary Max-LINSAT, although set-valued constraints can reduce constraint count and improve DQI behaviour (Thelen et al., 16 May 2026).

The framework is an open-source Python library with two principal modelling layers. MaxLinSat is the low-level representation for Max-LINSAT and set-valued weighted Max-LINSAT instances over a user-chosen field via Sage. MaxConstraintSat is the higher-level representation for integer and Boolean modelling, with bounded integer variables, polynomial term dictionaries, arithmetic and Boolean operator overloading, and a conversion method to_max_lin_sat() (Thelen et al., 16 May 2026). This split allows users either to work directly with field-linear instances or to remain at a higher abstraction level and delegate transformations to the framework.

The supported modelling scope includes linear objectives over binary variables, linear objectives over bounded integer variables, pseudo-Boolean polynomial objectives, Boolean expressions treated as polynomial expressions, and linear and modular constraints. The paper is explicit that support for general polynomial constraints and objectives over non-binary variables is currently lacking in a satisfactory exact Max-LINSAT-compatible form (Thelen et al., 16 May 2026).

5. Transformation pipeline, gadgets, and performance estimation in the quantum framework

The transformation pipeline begins by reducing polynomial objectives and constraints toward linear form. High-degree binary polynomial terms are split using auxiliary variables, with equality enforced through a penalty term

$-(e_1(x)-e_2(x))^2,$

which is maximised when $e_1(x)=e_2(x)$ (Thelen et al., 16 May 2026). The paper states that DQI-Kit by default splits terms of degree at least four, though the threshold is user-adjustable.

The next stage lifts rational or integer linear constraints into a prime field large enough to preserve intended equalities and inequalities. For ordering constraints, the paper defines

$$L_i = \min_{x\in X} (b_i\cdot x - v_i), \qquad U_i = \max_{x\in X} (b_i\cdot x - v_i),$$

and requires

$p > U_i - L_i.$

Then the ordering constraint is encoded as a set-membership condition over the field, while bounded integer variables receive range restrictions with sufficiently large weight (Thelen et al., 16 May 2026). Linear objectives are then translated into weighted constraints. For a binary variable $x_i\in\{0,1\}$, a term $w_i x_i$ becomes the weighted constraint $x_i=1$ with weight $w_i$. For non-binary bounded integers, the paper gives a bit-decomposition-style construction requiring

$\lceil \log_2(u_i-l_i+1)\rceil$

weighted constraints per term (Thelen et al., 16 May 2026).

At the low level, MaxLinSat supports gadgets, including reusable Boolean gadgets and a truth-table-based gadget generator, though the paper states that exhaustive search makes the generator feasible only for small gadgets and field orders (Thelen et al., 16 May 2026). A notable implementation detail is automatic constraint merging: constraints with the same left-hand side are combined into a single right-hand-side map, which serves as the internal representation of set-valued weighted constraints.

DQI-Kit also estimates expected DQI performance on generated instances. The framework computes the expected number of satisfied constraints by analysing the linear code

$-(e_1(x)-e_2(x))^2,$0

If the polynomial degree $-(e_1(x)-e_2(x))^2,$1 satisfies $-(e_1(x)-e_2(x))^2,$2, expected solution quality can be computed exactly using formulas from the cited DQI theory; otherwise exact computation becomes exponential or is approximated by sampling error vectors (Thelen et al., 16 May 2026). Implemented classical decoders are belief propagation, information set decoding, lookup table syndrome decoding, and nearest-neighbour decoding. If the user does not specify $-(e_1(x)-e_2(x))^2,$3 or a decoder, DQI-Kit auto-selects them based on field order and an approximate minimum distance of the dual code.

The paper places strong emphasis on inefficiencies that can destroy DQI’s potential advantage: increased number of constraints, auxiliary variables, duplicated constraints, small linear dependencies in $-(e_1(x)-e_2(x))^2,$4, and unsupported weighted or heterogeneous set-size structures in standard DQI (Thelen et al., 16 May 2026). This yields a framework whose role is not only translation but also diagnosis of poor encoding paths.

6. Position relative to adjacent literature

The two DQI-Kit usages occupy different research neighborhoods. The ML-oriented toolkit sits near work on ML data readiness, profiling, and remediation rather than generic ISO-style data quality tooling. The paper explicitly argues that generic cleaning and profiling checks are insufficient for ML-specific problems such as noisy labels and class overlap (Gupta et al., 2021). The quantum-oriented toolkit sits near work on Max-LINSAT encodings, finite-field optimisation, and Decoded Quantum Interferometry, where the central problem is transformation overhead and its effect on code structure and expected DQI performance (Thelen et al., 16 May 2026).

A third point is negative but important: the survey of six open-source data quality tools—Deequ, dbt Core, MobyDQ, Great Expectations (GX), Soda Core, and Apache Griffin—contains no statements about DQI-Kit, no mapping to ISO/IEC 25012 dimensions, and no comparison tables involving it (Papastergios et al., 2024). The paper states that DQI-Kit appears nowhere in the survey, including Section 2, Table 1, the appendix table, and the references. Accordingly, that survey cannot be used as evidence for DQI-Kit’s functionality, architecture, interfaces, or ISO/IEC 25012 coverage (Papastergios et al., 2024).

This absence is especially relevant because it prevents the ML-oriented DQI-Kit from being situated, on the basis of that survey alone, within the six-tool open-source DQ landscape. A plausible implication is that references to DQI-Kit in the data-quality literature should be grounded in the toolkit paper itself rather than inferred from adjacent surveys.

7. Significance and interpretation

Taken together, the literature uses DQI-Kit for two unrelated but structurally similar ambitions: a framework that makes an otherwise hard evaluation or transformation problem more systematic. In the ML case, the problem is deciding whether structured labeled data are fit for learning, how to explain deficiencies, and how to remediate them while preserving lineage (Gupta et al., 2021). In the DQI quantum case, the problem is deciding how to transform constrained optimisation formulations into Max-LINSAT without introducing overheads that undermine DQI performance (Thelen et al., 16 May 2026).

The shared label therefore reflects methodological packaging rather than substantive commonality. One DQI-Kit is a data-quality and remediation library for machine learning datasets. The other is a problem-encoding and performance-analysis framework for Decoded Quantum Interferometry. The literature does not support treating them as variants of a single platform, nor does it support importing claims from one domain into the other.