Pairwise Complexity in Inference and Models

• Pairwise complexity is a concept that quantifies the burden of using binary comparisons to reconstruct global structure, balancing statistical reduction with a quadratic increase in candidate evaluations.
• It underpins diverse applications such as ranking identification, active learning, parameter estimation in regression, and experimental design through efficient pairwise formulations.
• This framework transforms higher-order dependencies into pairwise interactions, offering insights into trade-offs between simpler statistical models and increased computational demands.

Pairwise complexity is a family of complexity notions that arise when inference, learning, or system description is organized around pairwise relations rather than pointwise observations. In the cited literature, the phrase covers at least four distinct quantities: the number of pairwise comparisons needed to identify a ranking or decision boundary; the computational burden induced by the $\Theta(n^2)$ set of candidate pairs; the stochastic or structural complexity of models whose sufficient statistics or interactions are pairwise; and combinatorial complexity measures defined directly from pairwise dependencies (Jamieson et al., 2011, Beretta et al., 2017, Bar-Yam et al., 2012). This breadth is not accidental. Pairwise formulations often replace absolute labels by relative judgments, or high-order structure by binary relations, which can either reduce statistical complexity through strong structural assumptions or increase computational complexity because the ambient pair space is quadratic.

1. Comparison-query complexity and ranking identification

A central use of pairwise complexity is as a query-complexity notion: how many pairwise comparisons are required to reconstruct a latent order. Without structural assumptions, determining an arbitrary total order on $n$ objects requires $\Theta(n \log_2 n)$ comparisons, because there are $n!$ possible rankings and $\log_2(n!)=\Theta(n\log_2 n)$ bits must be acquired. Under the geometric reference-point model, however, objects are embedded at known locations in $\mathbb{R}^d$, rankings are induced by distance to an unknown reference point $r$, and each pairwise comparison is equivalent to asking on which side of a perpendicular bisector hyperplane $r$ lies. The number of realizable rankings is then $M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$, with recursion

$Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$

so the information-theoretic lower bound becomes $n$0 rather than $n$1. An adaptive algorithm that queries only ambiguous comparisons satisfies

$n$2

whereas uniformly random non-adaptive queries identify a unique ranking with probability

$n$3

implying that $n$4 comparisons are required to recover the ranking with reasonable probability (Jamieson et al., 2011).

A different active-ranking formulation considers agnostic tournaments, where pairwise preferences may be non-transitive. There the objective is to output a permutation whose loss, measured by the minimum feedback arc-set in tournaments objective, is close to optimal while querying only a small number of labels. An active algorithm based on an $n$5-good decomposition returns, with constant probability, such a decomposition using at most

$n$6

queries in expectation, and the resulting surrogate optimization yields a permutation $n$7 satisfying

$n$8

By contrast, non-adaptive VC-style uniform sampling can require essentially quadratic sample size for small positional regret (Ailon, 2010).

Pairwise comparisons can also reduce binary classification to threshold learning. With noisy label and comparison oracles, the ADGAC reduction sorts unlabeled points using the comparison oracle and then locates a threshold by groupwise label queries. Under Tsybakov noise, the resulting label complexity scales like threshold learning,

$n$9

while total query complexity is

$\Theta(n \log_2 n)$0

The lower bound

$\Theta(n \log_2 n)$1

shows that the dependence on the accuracy term and the dimensional term is nearly optimal up to logarithmic factors (Xu et al., 2017).

2. Stochastic comparison models, top-$\Theta(n \log_2 n)$2 selection, and regression

In stochastic comparison models, pairwise complexity is usually a sample-complexity notion. For active best-$\Theta(n \log_2 n)$3 selection under Strong Stochastic Transitivity and the Stochastic Triangle Inequality, the PAC problem admits a matching instance-independent characterization: any algorithm must use

$\Theta(n \log_2 n)$4

comparisons in expectation, and the Tournament-$\Theta(n \log_2 n)$5-Selection procedure achieves

$\Theta(n \log_2 n)$6

comparisons in expectation. For exact best-$\Theta(n \log_2 n)$7 identification, hardness becomes instance dependent through the boundary gaps $\Theta(n \log_2 n)$8. The paper gives a worst-instance lower bound

$\Theta(n \log_2 n)$9

and proposes SEEBS for $n!$0 with complexity

$n!$1

and SEEKS for general $n!$2 with complexity

$n!$3

optimal up to log or loglog factors (Ren et al., 2020).

Rank regression from pairwise comparisons places pairwise complexity in a feature-based estimation setting. There are $n!$4 Gaussian feature vectors in $n!$5, pair indices are sampled uniformly at random from the first $n!$6 samples, and comparison outcomes are generated by a generalized linear model

$n!$7

The estimator

$n!$8

is unbiased up to a positive scale factor $n!$9. A sufficient condition for $\log_2(n!)=\Theta(n\log_2 n)$0 with high probability is

$\log_2(n!)=\Theta(n\log_2 n)$1

with probability bound

$\log_2(n!)=\Theta(n\log_2 n)$2

The analysis is explicitly dependence-aware because the same sample can participate in many queried pairs (Kadioglu et al., 2021).

Pairwise comparisons also support personalization through latent user types. In a mixed Bradley–Terry model with $\log_2(n!)=\Theta(n\log_2 n)$3 users, $\log_2(n!)=\Theta(n\log_2 n)$4 items, and $\log_2(n!)=\Theta(n\log_2 n)$5 user types, the algorithmic pipeline computes each user’s net-win vector, clusters users in this projected space, and then performs Bradley–Terry maximum likelihood within each cluster. The paper states that accurate individual preferences for almost all users are achievable if there are

$\log_2(n!)=\Theta(n\log_2 n)$6

pairwise comparisons per type, and describes this scaling as near optimal when $\log_2(n!)=\Theta(n\log_2 n)$7 only grows logarithmically with $\log_2(n!)=\Theta(n\log_2 n)$8 or $\log_2(n!)=\Theta(n\log_2 n)$9 (Wu et al., 2015).

3. Computational complexity induced by the quadratic pair space

When pairwise formulations are used algorithmically, the dominant difficulty is often not sample complexity but the quadratic cardinality of the candidate set. In D-optimal experimental design for pairwise comparisons, the candidate pool is

$\mathbb{R}^d$0

A naive greedy algorithm for selecting $\mathbb{R}^d$1 comparisons evaluates all candidates at every iteration, giving

$\mathbb{R}^d$2

time. By exploiting pairwise difference geometry, Cholesky factorization, and scalar rank-one updates, this can be reduced to

$\mathbb{R}^d$3

with lazy greedy variants providing additional practical speedups (Guo et al., 2019).

The same quadratic growth appears in pairwise empirical risk minimization. In stagewise proximal pairwise learning, the empirical pair set has size

$\mathbb{R}^d$4

so a deterministic full-gradient method would cost $\mathbb{R}^d$5. The proposed proximal DSGD instead combines adaptive sample-size growth with importance sampling over opposite instances and obtains total iterations

$\mathbb{R}^d$6

hence $\mathbb{R}^d$7 time to reach statistical accuracy, while maintaining the pairwise objective and nonsmooth regularization (AlQuabeh et al., 2022).

Online pairwise learning exhibits the same phenomenon over time rather than dataset size. Naive online pairwise OGD computes, at round $\mathbb{R}^d$8, a gradient against all $\mathbb{R}^d$9 past points and therefore costs $r$0 per round and $r$1 overall. In the kernelized setting with random Fourier features of dimension $r$2, the naive complexity is $r$3. The FPOGD method replaces full-history gradients by a limited-memory, variance-reduced estimator built from a stratified buffer of size $r$4, yielding total time

$r$5

and using

$r$6

features for kernel approximation. Its regret bound isolates the cumulative variance term

$r$7

so computational reduction and statistical improvement are directly linked (AlQuabeh et al., 2023).

In PLM-based learning-to-rank, pairwise complexity is dominated by the joint encoding cost of two candidates. Naive pairwise inference over $r$8 candidates costs

$r$9

where $r$0 is the forward-pass cost for sequence length $r$1. The GLIMPSE framework instead combines pointwise scoring with right-to-left refinement over the top-$r$2 items, giving

$r$3

so the pairwise stage is compressed from $r$4 to $r$5 comparisons per pass (Kannen et al., 2024).

A related full-graph versus sparse-graph tradeoff appears in face clustering. ConPaC formulates pairwise adjacency inference as CRF inference with transitivity constraints. On the full graph, inference costs

$r$6

where $r$7 is the maximum active degree; in the worst case this becomes $r$8. On a $r$9-NN graph, the same mechanism becomes

$M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$0

with memory $M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$1, which the paper describes as linear time complexity given a $M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$2-NN graph (Shi et al., 2017).

4. Noise, exactness, and pairwise access models

Noisy pairwise access creates a second layer of complexity: the number of repeated or auxiliary pairwise operations needed to preserve exactness or controllable approximation error. In the geometric active-ranking model, if each comparison is correct with probability at least $M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$3 and $M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$4, then repeating every ambiguous query $M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$5 times and taking a majority vote yields exact recovery with probability exceeding

$M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$6

while the average number of comparisons is $M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$7. To achieve success probability at least $M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$8, it suffices to take

$M=|\Sigma_{n,d}|=Q(n,d)=\Theta(n^{2d})$9

which gives total complexity

$Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$0

Under persistent errors, repeated querying does not help, so the algorithm switches to a triangular voting scheme. Exact full recovery is then impossible in general, but the method exactly ranks at least

$Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$1

objects and achieves expected Kendall–Tau error

$Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$2

with

$Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$3

comparisons on average (Jamieson et al., 2011).

Exact sampling from pairwise comparisons introduces an analogous distinction between a canonical but potentially slow exact method and a faster structure-aware one. With target distribution $Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$4 over a finite support and pair-distribution $Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$5, the canonical Markov chain has transition probabilities

$Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$6

Coupling from the Past on this chain yields perfect samples, but the naive complexity is

$Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$7

and can be exponential in $Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$8 when the spectral gap $Q(n,d)=Q(n-1,d)+(n-1)Q(n-1,d-1),$9 is exponentially small. The accelerated construction first learns an approximation $n$00, then defines a parametric downscaled chain with near-uniform stationary distribution and mixing time

$n$01

leading to total oracle complexity

$n$02

The resulting dependence is spectral-topological, via $n$03, rather than through the detailed structure of $n$04 (Fotakis et al., 2022).

These results suggest that pairwise access is not merely a substitute for labels or direct samples. It creates a distinct exactness regime in which the main question is how much additional pairwise structure is needed to turn locally noisy or conditionally observed binary relations into globally correct objects.

5. Structural and stochastic complexity of pairwise models

In another line of work, pairwise complexity is not a query count but an intrinsic complexity of a model class. For spin models in the MDL/Bayesian framework, the relevant quantity is the stochastic complexity

$n$05

The central conclusion is that complexity is determined not by interaction order but by the mutual arrangement of interactions. Localized dependencies on non-overlapping groups of variables yield low stochastic complexity, whereas fully connected pairwise Ising models are highly complex because of their extended interaction structure. Gauge transformations partition models into equivalence classes with identical $n$06, and loop structure provides the decisive invariants. The paper therefore rejects the simple identification of “pairwise” with “simple” (Beretta et al., 2017).

A different structural notion is the pairwise complexity profile. Starting from a symmetric coupling matrix $n$07, the construction defines

$n$08

then inverts this relation to obtain $n$09, sums over variables,

$n$10

and corrects for multiple counting via

$n$11

This pairwise profile satisfies linear superposition for unrelated systems, the sum rule

$n$12

and monotonic non-increase with scale. Its principal computational advantage is that it reduces the original factorial-time formulation to

$n$13

time (Bar-Yam et al., 2012).

The same concern with redundancy and randomness underlies pairwise set complexity in information theory. The per-pair contribution is

$n$14

and it is explicitly constructed to vanish both at independence and at deterministic dependence. The authors then generalize this pairwise quantity by differential interaction information and use the resulting multivariable measures as hyperedge weights in hypergraphs of system dependence (Galas et al., 2013).

Pairwise adjustment also appears in clustering comparison. Standard Adjusted Mutual Information corrects mutual information by averaging over full label permutations, but the expectation term costs

$n$15

time. Pairwise Adjusted Mutual Information instead averages over permutations that swap only two samples. This yields an explicit adjustment formula computable in

$n$16

time, or $n$17 in sparse format, while empirical ranking agreement with standard AMI remains high across synthetic and real datasets (Lazarenko et al., 2021).

Taken together, these constructions suggest that pairwise complexity is not only about how many pairwise operations are required. It can also mean how much structure a pairwise representation retains, suppresses, or makes computationally accessible.

6. Pairwise complexity in combinatorial, graphical, and genomic models

Combinatorial pairwise systems often exhibit sharp tractability boundaries. In the stable marriage problem with pairwise preferences, complexity depends jointly on the orderedness class of each side and on the stability notion. Weak stability is polynomial for acyclic pairwise preferences, but deciding existence becomes NP-complete if one side has strict lists and the other has asymmetric pairwise preferences. Strong stability remains polynomial when one side has ties and the other asymmetric preferences, yet becomes NP-complete once posets are allowed. Super-stability is polynomial for posets on one side and asymmetric preferences on the other, but NP-complete when both sides have acyclic preferences (Cseh et al., 2018).

Graphical model learning provides a high-dimensional statistical analogue. For continuous pairwise Markov random fields with bounded domains and basis functions, the GRISE local program yields structure recovery once

$n$18

where

$n$19

Accordingly, structure recovery has sample complexity

$n$20

and the total computational complexity scales as $n$21 up to polylog factors. The logarithmic dependence on $n$22 parallels discrete and Gaussian graphical-model results, while the exponential dependence on degree is retained (Shah et al., 2020).

In genome rearrangement, pairwise complexity appears as the counting complexity of optimal transformations between two genomes. In the Single Cut-and-Join model, exact counting of minimum-length scenarios is $n$23-complete. Parameterizing instead by the number $n$24 of nontrivial components in the adjacency graph yields an FPT algorithm with running time

$n$25

where $n$26 is the $n$27th Bell number. Since $n$28, the problem is also fixed-parameter tractable by distance (Bailey et al., 2024).

These task-specific manifestations point to a common pattern. Pairwise formulations often replace one difficult object by another: a total order by binary comparisons, a full system description by pairwise couplings, a global combinatorial transformation by operations on paired extremities, or a higher-order interaction model by pairwise terms plus structural constraints. This suggests that pairwise complexity is best understood not as a single scalar invariant, but as a task-dependent measure of how much combinatorial, statistical, or computational burden remains after a problem has been recast in pairwise form.