Operational Integrity Score (OIS)

• Operational Integrity Score (OIS) is a domain-specific metric that aggregates diverse integrity dimensions beyond conventional performance measures.
• In tabular models, OIS combines Explainability, Fairness, Robustness, Privacy, and Sustainability to aid effective, high-stakes decision-making.
• In quantum circuits, OIS is defined as one minus the Jensen–Shannon distance, capturing behavioral deviations through direct output distribution comparisons.

Searching arXiv for the specified OIS-related papers to ground the article in current literature. Operational Integrity Score (OIS) denotes a domain-specific integrity metric designed to evaluate whether a system remains acceptable under criteria that are not captured by a single conventional performance measure. In recent arXiv literature, the term appears in two distinct but conceptually related senses. In high-stakes tabular machine learning, OIS is described as directly mirroring the Model Integrity and Responsibility Assessment Index (MIRAI), a unified score that aggregates Explainability, Fairness, Robustness, Privacy, and Sustainability for binary-classification models under a controlled comparison setting (Nguyen et al., 14 May 2026). In quantum-circuit validation, OIS is an execution-level behavioral score defined as one minus the Jensen–Shannon distance between the output distributions of a reference circuit and a circuit under test (Ahmed et al., 29 Apr 2026). In both settings, the central premise is that integrity cannot be inferred from a single familiar proxy such as predictive performance or structural similarity alone.

1. Conceptual scope and domain-specific meanings

The term Operational Integrity Score is not used as a single universal formula across the two cited works. Instead, it names two operationalizations of integrity that share a common motivation: evaluation against failures that are invisible to narrow metrics.

In the tabular-model setting, OIS is a decision-oriented scalar index for regulated or high-stakes deployments. It is intended to complement predictive metrics such as Accuracy and F1 rather than replace them, and it is computed from five integrity dimensions evaluated on the same data-splits and hyperparameter budget across candidate models such as DT, XGB, SVM, MLP, TabResNet, and FTT (Nguyen et al., 14 May 2026).

In the quantum-circuit setting, OIS is a behavioral divergence score within a three-layer framework composed of the Structural Integrity Score (SIS), OIS, and the Interaction Graph Semantic-Logical Score (IGS). Here, OIS is explicitly tied to observed output behavior: identical output distributions yield maximal integrity, while increasing Jensen–Shannon distance implies decreasing integrity (Ahmed et al., 29 Apr 2026).

Domain	OIS formulation	Primary purpose
Tabular binary classification	Weighted sum of five dimension scores mirroring MIRAI	Responsible model selection
Quantum circuits	$1-\mathrm{JSD}(p,q)$ between output distributions	Behavioral anomaly detection

A plausible implication is that OIS functions less as a fixed metric family than as an integrity-design pattern: define a controlled reference, align heterogeneous signals onto a common scale, and produce a scalar suitable for operational ranking or alerting.

2. OIS as a multi-dimensional index for tabular models

In the tabular-model formulation, OIS mirrors MIRAI exactly. The score is defined for tabular binary-classification models and extends evaluation beyond accuracy- or F1-centric practice by quantifying five dimensions: Explainability, Fairness, Robustness, Privacy, and Sustainability (Nguyen et al., 14 May 2026).

The framework assumes a controlled comparison setting. A set of candidate models is selected, each model is evaluated on the same data-splits and under the same hyperparameter budget, and a single scalar index is computed to reflect a balanced trade-off across the five dimensions. The purpose is explicitly decision-oriented: OIS complements Accuracy and F1 and provides a ranking for regulated or high-stakes use cases.

The five dimensions are instantiated through raw metrics. Explainability is computed from example-level SHAP attributions and Quantus tests. Fairness is measured through subgroup disparities between privileged (“male”) and unprivileged (“female”) groups. Sustainability combines carbon estimate, parameter count, FLOPs per sample, and MACs per sample. Robustness is evaluated through HopSkipJump Attack gap and prediction-space drift detection. Privacy is quantified through Membership-Inference Privacy and SHAPr Privacy.

This construction encodes an important methodological claim: predictive strength and integrity are not assumed to coincide. The experiments in healthcare, financial, and socioeconomic datasets were specifically reported to show that higher predictive performance does not necessarily imply better overall integrity and responsibility, and that simpler models can in several cases achieve a stronger cross-dimensional balance than more complex deep tabular architectures (Nguyen et al., 14 May 2026).

3. Dimension metrics, normalization, and aggregation

The tabular OIS inherits MIRAI’s dimension-wise metric structure. For each candidate model, the raw metrics are organized as follows (Nguyen et al., 14 May 2026).

Explainability uses example-level SHAP attributions followed by Quantus’s eight quantitative tests: Local Lipschitz Estimate, Consistency, Faithfulness Correlation, Faithfulness Estimate, Model Parameter Randomization Test, Random Logit Test, Sparseness, and Complexity. These produce scores $x_{e,1}\ldots x_{e,8}$ in $[0,1]$ where higher is better.

Fairness measures subgroup disparities across the privileged (“male”) versus unprivileged (“female”) groups in Accuracy difference, Precision difference, Recall difference, True Positive Rate (TPR) difference, False Positive Rate (FPR) difference, Demographic Parity difference, and Equalized Odds difference. Each raw disparity $\Delta$ is in $[0,1]$, but lower is better. These are denoted $x_{f,1}\ldots x_{f,7}$.

Sustainability includes a carbon estimate,

$$carbon_{raw} = p_t \;(\mathrm{kW}) \times emission\_rate_{\mathrm{kgCO2e/kWh}} \div per\_capita\_daily\_reference$$

together with parameter count, FLOPs per sample, and MACs per sample, each normalized by a reference model. These four metrics are mapped into $x_{s,1}\ldots x_{s,4}$ in $[0,1]$ where higher indicates greater sustainability.

Robustness uses HopSkipJump Attack gap and prediction-space drift detection, yielding raw scores $x_{r,1},x_{r,2}\in[0,1]$ where higher is better.

Privacy uses Membership-Inference Privacy and SHAPr Privacy, with raw scores $x_{e,1}\ldots x_{e,8}$0 where higher indicates better privacy protection.

Because not all raw metrics share the same direction, the framework applies direction alignment:

$x_{e,1}\ldots x_{e,8}$1

where $x_{e,1}\ldots x_{e,8}$2 and $x_{e,1}\ldots x_{e,8}$3 indexes the metrics in each dimension.

The normalized metrics are averaged within each dimension:

$x_{e,1}\ldots x_{e,8}$4

The final OIS is then the weighted sum of the five dimension scores:

$x_{e,1}\ldots x_{e,8}$5

subject to $x_{e,1}\ldots x_{e,8}$6.

Default weights are equal, with $x_{e,1}\ldots x_{e,8}$7 for all five dimensions. The framework also allows custom weights, for example heavier emphasis on Fairness in lending or Robustness in medical devices. This weighting mechanism makes the scalar index controllable while preserving transparency at the dimension level.

4. Empirical trade-offs in healthcare, financial, and socioeconomic datasets

The tabular experiments reported condensed examples from three datasets: Diabetes Hospitals, German Credit, and Census Income. These results were used to illustrate the divergence between predictive metrics and OIS (Nguyen et al., 14 May 2026).

Dataset	Model and F1	OIS
Diabetes Hospitals (101 763 records, 22 features)	XGB, $x_{e,1}\ldots x_{e,8}$8	$x_{e,1}\ldots x_{e,8}$9
Diabetes Hospitals (101 763 records, 22 features)	MLP, $[0,1]$0	$[0,1]$1
Diabetes Hospitals (101 763 records, 22 features)	FTT, $[0,1]$2	$[0,1]$3
German Credit (1 000 records, 22 features)	TRN, $[0,1]$4	$[0,1]$5
German Credit (1 000 records, 22 features)	MLP, $[0,1]$6	$[0,1]$7
Census Income (32 561 records, 14 features)	SVM, $[0,1]$8	$[0,1]$9
Census Income (32 561 records, 14 features)	FTT, $\Delta$0	$\Delta$1

Several dataset-level observations are explicit. In Diabetes, FTT ties XGB for F1 but falls sharply in the overall integrity score because of weaker Sustainability and Privacy. In German Credit, TRN attains the best F1, yet MLP achieves the stronger cross-dimensional balance. In Census Income, SVM trails slightly in predictive performance while obtaining the highest OIS by virtue of stronger Privacy and Sustainability.

Across all three domains, the model with the absolute best F1 or Accuracy is reported to be rarely the same model with the best OIS. Complex deep-tabular architectures such as TRN and FTT can excel on predictive metrics and even Explainability, but may incur steep costs in Sustainability and Privacy leakage. Simpler architectures such as MLP, XGB, and SVM often deliver a more even trade-off across Privacy, Fairness, and Robustness.

These findings support the stated use case of OIS in regulated environments. In settings framed in the paper as involving the EU AI Act, medical deployment, or financial compliance, an OIS-driven ranking provides a compact mechanism for model selection without manually reconciling many separate metrics.

5. OIS as a behavioral score for quantum circuits

In the quantum-circuit literature, OIS is defined formally and directly from output distributions. Let $\Delta$2 be the reference circuit and $\Delta$3 the circuit under test. Their output probability distributions are

$\Delta$4

The score is built from the Kullback–Leibler divergence,

$\Delta$5

the mixture distribution,

$\Delta$6

the Jensen–Shannon divergence,

$\Delta$7

and the Jensen–Shannon distance,

$\Delta$8

Because the logarithm is base 2 and the distributions are finite and discrete, $\Delta$9. The Operational Integrity Score is then

$[0,1]$0

Accordingly, $[0,1]$1, with $[0,1]$2 indicating identical output distributions and values approaching $[0,1]$3 indicating maximal divergence (Ahmed et al., 29 Apr 2026).

The required inputs are two circuits, either executed or simulated in the computational basis. The paper uses $[0,1]$4 shots. For each circuit, counts are collected for each bit string $[0,1]$5 and normalized into empirical distributions:

$[0,1]$6

Under real NISQ conditions, the two circuits are submitted back-to-back under identical conditions, including the same calibration and shot count. On a noiseless or approximate simulator such as Qiskit Aer, the histogram is treated identically.

The step-by-step computation is straightforward: choose $[0,1]$7, run both circuits, form $[0,1]$8 and $[0,1]$9, compute $x_{f,1}\ldots x_{f,7}$0, compute the two KL terms, form $x_{f,1}\ldots x_{f,7}$1, take the square root to obtain JSD, and finally compute $x_{f,1}\ldots x_{f,7}$2. The paper does not explicitly prescribe smoothing for cases where $x_{f,1}\ldots x_{f,7}$3 but $x_{f,1}\ldots x_{f,7}$4, although it notes that in practice a very small pseudocount is common.

6. Sensitivity, blind spots, and interpretive significance

The quantum-circuit study places OIS inside a three-layer integrity framework in which SIS performs structural screening, IGS performs pre-execution interaction-level comparison, and OIS performs execution-level behavioral verification (Ahmed et al., 29 Apr 2026). The rationale is that a single metric is insufficient for reliable circuit validation.

A central result concerns structural blind spots. Structural similarity alone does not guarantee behavioral equivalence. In cases where $x_{f,1}\ldots x_{f,7}$5, OIS still detected anomalies in 93.85% of instances, while IGS detected 72.58%. The severity-specific breakdown for OIS detection in the blind-spot subset was 92.96% at severity $x_{f,1}\ldots x_{f,7}$6, 94.04% at severity $x_{f,1}\ldots x_{f,7}$7, and 95.27% at severity $x_{f,1}\ldots x_{f,7}$8.

OIS is reported to decrease monotonically with anomaly severity for all eight perturbation types examined, including structure-preserving anomalies such as gate substitution and gate reordering that SIS can miss. Even when SIS remains above $x_{f,1}\ldots x_{f,7}$9, OIS often drops to $$carbon_{raw} = p_t \;(\mathrm{kW}) \times emission\_rate_{\mathrm{kgCO2e/kWh}} \div per\_capita\_daily\_reference$$0 or lower under reordering or substitution, indicating substantial behavioral deviation. An anomaly is treated as detected whenever OIS falls below $$carbon_{raw} = p_t \;(\mathrm{kW}) \times emission\_rate_{\mathrm{kgCO2e/kWh}} \div per\_capita\_daily\_reference$$1, corresponding to more than 5% Jensen–Shannon distance.

The study also characterizes the cost profile of OIS. Relative to SIS and IGS, OIS has the lowest false-negative rate because it directly verifies functional correctness at the distributional level, but it also has the highest computational and measurement cost. Beyond approximately 14 qubits, full-state noiseless simulation or large-shot hardware runs become expensive, and measurement noise can increase variance in the empirical estimates of $$carbon_{raw} = p_t \;(\mathrm{kW}) \times emission\_rate_{\mathrm{kgCO2e/kWh}} \div per\_capita\_daily\_reference$$2 and $$carbon_{raw} = p_t \;(\mathrm{kW}) \times emission\_rate_{\mathrm{kgCO2e/kWh}} \div per\_capita\_daily\_reference$$3.

Taken together, the tabular and quantum formulations point to a common interpretive lesson. OIS is most useful when integrity is understood as an operational property that only becomes visible after controlled comparison across multiple axes or against a behavioral reference. In tabular learning, this means exposing trade-offs hidden by Accuracy or F1 alone; in quantum validation, it means exposing semantic deviations hidden by structural similarity alone. This suggests that OIS is best treated not as a generic synonym for quality, but as a formal mechanism for detecting discrepancies between nominal performance and operational acceptability.