Pairwise Transitivity Inconsistency

• Pairwise transitivity inconsistency is the failure of pairwise preference judgments to satisfy the transitivity rule, where if A is preferred to B and B to C, A may not be preferred to C.
• It is quantified using various indices such as Saaty’s CI, GCI, and Koczkodaj’s KII that measure deviations in multiplicative and additive frameworks to assess both localized and global inconsistencies.
• Addressing this inconsistency is critical for enhancing the reliability of ranking systems in decision analysis, social choice theory, and AI evaluation frameworks.

Pairwise transitivity inconsistency refers to the phenomenon where a set of pairwise preference or comparison judgments fails to satisfy the transitivity property—a core axiom in classical decision theory and ranking systems. In the strictest sense, if alternative A is preferred to B, and B is preferred to C, transitivity requires that A should also be preferred to C. When transitivity is violated (through either human error, algorithmic limitations, or structural anomalies in the data), the integrity of any ranking or aggregation derived from such comparisons is compromised. Detecting, quantifying, repairing, and theoretically analyzing such inconsistencies is fundamental in fields such as decision analysis, multi-criteria optimization, social choice, and more recently, automated evaluation systems using machine learning models.

1. Formal Characterization of Pairwise Transitivity Inconsistency

In the context of a pairwise comparison matrix $A = [a_{ij}]$ of order %%%%1%%%% (with $a_{ij} > 0$ and $a_{ij} a_{ji} = 1$), full consistency is defined by the multiplicative transitivity condition: $a_{ik} = a_{ij} \cdot a_{jk},\quad \forall i,j,k.$ Departures from this condition—quantified in various ways—define pairwise transitivity inconsistency.

A triad $(i, j, k)$ is said to be inconsistent if $a_{ik} \neq a_{ij} a_{jk}$. Such violations can be localized (a small number of inconsistent triads in an otherwise mostly consistent matrix) or global (widespread cyclic or contradictory preferences).

In ordinal settings (such as tournament or directed comparison graphs), transitivity inconsistency is typically identified by the presence of cyclic triads (e.g., $A > B$, $B > C$, $C > A$) or equivalence contradictions. Combinatorial indices such as the Kendall-Babington Smith coefficient count the proportion of inconsistent triads to all possible triads (Kułakowski, 2017).

2. Axiomatic Foundations and Inconsistency Indices

Axiomatic approaches establish necessary and desirable properties for any measure of inconsistency. Two primary frameworks are noteworthy:

• Five-Axiom System (unique value at consistency, permutation invariance, monotonicity under intensification, monotonicity on single comparison, continuity) (Brunelli et al., 2013):
  • Axiom 1 (Consistency Uniqueness): $I(A) = \nu$ if and only if $A$ is consistent.
  • Axiom 2 (Permutation Invariance): $I(P A P^T) = I(A)$ for any permutation $P$.
  • Axiom 3 (Preference Intensification): If $b > 1$, $I(A(b)) \geq I(A)$ where $A(b) = [a_{ij}^b]$.
  • Axiom 4 (Single Comparison Monotonicity): Perturbing a single non-diagonal entry monotonically increases inconsistency.
  • Axiom 5 (Continuity): $I(A)$ is continuous in each $a_{ij}$.
• Triad-Focused (Localization) Axioms (Koczkodaj et al., 2013, Koczkodaj et al., 2015, Csató, 2018):
  • Zero for perfect consistency.
  • Normalization (e.g., $[0, 1)$).
  • Local monotonicity: tightening any triad's deviation from consistency strictly increases the measure.
  • Scale, permutation, and inversion invariance.

Key inconsistency indices include:

Index	Principle	Satisfies All Five Axioms?
Saaty's CI	Eigenvalue deviation	Yes
Geometric Consistency Index (GCI)	Log-square deviation from weight ratios	Yes
CI* (average 3×3 subdeterminants)	Aggregates local transitivity failures	Yes
Barzilai's RE	Logarithmic discrepancy averages	No (fails monotonicity, continuity)
Harmonic Consistency (HCI)	Harmonic mean of column sums	No (fails monotonicity under intensification)
Koczkodaj's KII	Worst-case triad inconsistency	Strong local sensitivity; unique triad axiomatic ranking (Csató, 2016, Csató, 2018)

The indices differ in their sensitivity to localized versus distributed inconsistency, their response to scale changes, and their mathematical robustness under transformation and optimization.

3. Mathematical Formulations and Computational Schemes

A central insight is that logarithmic transformation linearizes the consistency condition, permitting more tractable convex optimization and analysis: $$x_{ik} = x_{ij} + x_{jk},\quad \text{with } x_{ij} = \ln a_{ij}.$$ Triad and cycle-based indices are computed as: $$\text{Triad index:}\quad I(i,j,k) = 1 - \min\left\{ \frac{a_{ij} a_{jk}}{a_{ik}}, \frac{a_{ik}}{a_{ij} a_{jk}} \right\}$$ or, in distance-based form,

$$I(i,j,k) = 1 - \exp\left( -\left| \ln\left( \frac{a_{ij} a_{jk}}{a_{ik}} \right) \right| \right)$$

(Koczkodaj et al., 2013, Koczkodaj et al., 2015).

For incomplete matrices, the extension replaces triads by all cycles containing observed comparisons, evaluating the deviation from the (cycled) transitivity condition (Kułakowski et al., 2019): $$R_s = \frac{\prod_{l=1}^{m-1} c_{i_l i_{l+1}}}{c_{i_1 i_m}},\quad K_s = \min\left\{ |1 - R_s|, |1 - 1/R_s| \right\}$$ and globalizing via maximum or average across all cycles.

Optimization-based repair frameworks use convex or mixed-integer formulations to identify minimal modification sets required to bring inconsistency below user-defined thresholds (such as CR $<0.1$) (1311.0748). This typically involves optimization in log-space with binary variables indicating entry modifications and convex constraints enforcing the desired inconsistency criterion.

Gradient-based "consistencization" treats the inconsistency index as a potential function, descending along the gradient (or using difference approximations for non-smooth indices) to iteratively reduce inconsistency, with explicit formulas for instant priority direction available for both multiplicative and additive parameterizations (Magnot et al., 2021).

4. Implications Across Application Domains

• Decision Analysis and MCDM: The reliability of priority vectors and risk of poor ranking underlies the need for robust inconsistency assessment. In AHP and MCDM, indices that fail monotonicity or continuity can mask severe local transitivity violations, or conversely react too sensitively to minor perturbations (Brunelli et al., 2013).
• Social Choice and Ordinal Rankings: In ranking by majority or in the presence of ties, transitivity inconsistency is central to resolving Condorcet cycles or counting cyclic/inconsistent triads (Kułakowski, 2017). In such cases, the consistency coefficient serves as a quality measure for aggregation credibility.
• Learning and AI Evaluation: Automated LLM-based judge systems exhibit pairwise transitivity inconsistency (e.g., $A>B$, $B>C$, $C>A$) especially when using coarse discrete rating systems and ambiguous tie protocols. This motivates probabilistic evaluation frameworks such as TrustJudge, which leverage distribution-sensitive scoring and likelihood-aware aggregation to reduce such inconsistencies. The quantification is formalized via non-transitivity ratio (NTR) over $k$-response subsets:

$NTR_k = \frac{V_k}{\binom{n}{k}}$

where $V_k$ counts cyclic or equivalence violations (Wang et al., 25 Sep 2025).

• Statistical Modeling Beyond Stochastic Transitivity: Models that relax the stochastic transitivity assumption (as in BT or Thurstone) and use low-rank skew-symmetric structures for pairwise probabilities can accurately model and infer rankings in highly intransitive domains such as e-sports or multi-criteria tasks, achieving minimax optimal rates (Lee et al., 13 Jan 2025).

5. Comparative Performance and Theoretical Insights

• Indices that satisfy the full axiom set, notably CI, GCI, and CI*, are mathematically well-behaved under preference intensification, permutation, and single comparison perturbation, and are shown to change monotonically in accordance with theoretical expectations. In real-world repair scenarios, minimal corrections identified by these indices align closely with intuitive notions of inconsistency "location" and magnitude (1311.0748, Koczkodaj et al., 2015).
• Extreme-based indices, such as Koczkodaj’s, offer maximal sensitivity to localized transitivity failure and are uniquely characterized by axiomatic systems for triads; they are indispensable when the worst-case local error is the limiting factor (Csató, 2016).
• Mean-based (averaging) indices can dilute localized but significant errors, particularly in large matrices. A principal criticism of the eigenvalue-based CI is that it can fall below threshold even when certain triads exhibit arbitrarily high inconsistency when the matrix dimension grows (Koczkodaj et al., 2013, Brunelli, 2015).

6. Open Problems and Future Directions

Key research avenues include:

• Scalability: Efficient computation of cycle-based indices for large and incomplete matrices, possibly through graph-theoretic or randomized approaches.
• Normalization and Thresholding: Refinement of inconsistency thresholds for various indices, particularly in the incomplete and probabilistic judgment regimes, as well as generalizing Saaty’s rule for non-exact or missing-data settings (Ágoston et al., 2021).
• Integration with ML and Probabilistic Models: Extending these frameworks to LLM-based or statistical learning evaluators, with explicit mechanisms for propagating and measuring the consequences of pairwise transitivity inconsistency at the system level (Wang et al., 25 Sep 2025).
• Extensions Beyond Classic Models: Development of predictive models and estimators that are robust to (and can model) systemic intransitivity, leveraging spectral methods and nuclear norm constraints for skew-symmetric comparison probability matrices (Lee et al., 13 Jan 2025).
• Theoretical Unification: Ongoing refinement of the axiomatic landscape, with attention to triad-to-global aggregation, invariance principles, and metric properties required for interpretability and algorithmic robustness.

7. Conclusion

Pairwise transitivity inconsistency is a foundational concept that underlies the reliability of preference aggregation, ranking, and automated evaluation. The field has advanced from global, often unintentionally obfuscating measures, to localizable, axiomatized, and metrically grounded indicators that enable both diagnosis and algorithmic remediation of inconsistency. As applications expand to incomplete data, AI-driven evaluation, and highly intransitive domains, new models and criteria for inconsistency quantification and management are necessary. The continuous refinement of theoretical foundations and practical tools is crucial for ensuring computational integrity and interpretability in decision-support, ranking, and automated assessment systems.