Binary Comparison Queries

• Binary comparison queries are defined as evaluations that compare two objects to determine which better meets a specified criterion without relying on absolute values.
• They offer a practical mechanism to reduce query complexity in settings like active learning by using noise-tolerant oracles and delegation strategies.
• Their applications span active learning, fair division, social choice, and binary similarity matching, with strong theoretical guarantees and empirical validation.

A binary comparison query is a query to an oracle or human that, given two objects $x, y$, outputs which of the two is "greater" (or closer to a target, or more likely to possess some property) according to a specified criterion, typically in the absence of access to absolute values or explicit labels. Such queries—also termed pairwise comparison queries—are a fundamental primitive in diverse algorithmic domains including active learning, preference elicitation, fair division, and code or data similarity analysis. The central appeal of binary comparison queries is their cognitive simplicity for annotators and their efficiency in certain restricted computational settings, often enabling exponential or near-optimal reductions in query complexity compared to explicit value or label queries.

1. Formal Definitions and Query Models

Binary comparison queries come in several instantiations, distinguished by the property being compared, the noise model assumed, and the presence or absence of access to ground-truth labels. Characteristic formalizations include:

• Positivity Comparison Oracle ($O_1$): Given two unlabeled data points $x_1, x_2$, return $O_1(x_1, x_2) \in \{+1, -1\}$, where $+1$ denotes " $x_1$ is more likely positive than $x_2$". The oracle is noisy, with

$$\mathbb{E}_{x_1, x_2} \left[ 1 \{ O_1(x_1, x_2) \cdot (\eta(x_1) - \eta(x_2)) < 0 \} \right] = \epsilon_1, \quad 0 \leq \epsilon_1 < \frac12.$$

Here, $\eta(x)$ is the probability $x$ is positive (Cui et al., 2020).

• Uncertainty Comparison Oracle ($O_2$): Given $x_1, x_2$, returns which is closer to the classification threshold (higher uncertainty). For $u(x)=|\eta(x)-\frac12|$, lower $u(x)$ implies higher uncertainty. The associated noise is

$$\mathbb{E}_{x_1, x_2}\left[ 1 \{ O_2(x_1, x_2) \cdot (u(x_2) - u(x_1)) < 0 \} \right] = \epsilon_2, \quad 0 \leq \epsilon_2 < \frac12.$$

(Cui et al., 2020)

• Distance-to-Boundary Comparison: Given a class of halfspaces $c(x)=\operatorname{sign}(\langle w, x \rangle)$, the query "Is $f(x_i) \geq f(x_j)$?" returns which of $(x_i, x_j)$ is closer to the classification boundary (Kane et al., 2017).
• Preference Comparison Query: For fair division or voting, given two bundles (or alternatives) $X, Y$ and an agent $i$, the query $Q(i; X, Y)$ reports which is preferred, revealing orderings without access to cardinal utilities (Bu et al., 2024, Conitzer, 2014).
• Binary Similarity Comparison (code/data similarity): Given two binary artifacts $S_a, S_b$, output a binary value or a graded similarity score estimating the semantic similarity, possibly learning from data-dependent embeddings (Song et al., 2022, Song et al., 2022, Hu et al., 2019).

Across contexts, comparison queries are often subject to noise, adversarial responses, or limited cognitive bandwidth, and their information content depends on structural properties of the domain (e.g., margins, single-peakedness, valuation decomposability).

2. Binary Comparison Queries in Algorithmic Active Learning

Binary comparison queries are pivotal in advancing the efficiency of active learning algorithms, especially in the context of binary classification and halfspace learning.

• Halfspace Learning via Comparison Queries: With a positivity or distance-to-boundary comparison, all labels can be inferred with $O(\log n)$ queries under favorable conditions—specifically, if the sample sits in $[N]^d$ (bounded bit-description) or enjoys margin $\gamma$ (Kane et al., 2017). Formally, the inference dimension $D$ characterizes the minimal sample size at which a label can always be inferred from other labels plus comparisons; query complexity is $O(D \log D \log n)$. For points in high margin, $D = O(d \log d \log(1/\eta))$.
• Noisy Oracles and Adaptive Labeling: With noisy comparators, a combination of positivity ($O_1$) and uncertainty ($O_2$) comparison oracles yields adaptive algorithms that make only $O(n)$ queries to $O_1$ and $O(n\log\log n)$ queries to $O_2$ (details below). Prior approaches using noisy quick-sort have $O(n\log n)$ query complexity and can be unstable; use of a delegation set $D'$—the $t$ most uncertain points under $O_2$—enables label inference for the rest of the dataset using only a majority vote over pairwise comparisons (Cui et al., 2020).
• Lower Bounds: In the absence of margin or bounded description, the inference dimension becomes infinite, and $\Omega(n)$ queries are required (worst-case). This delimits the regimes where exponential improvements from comparison queries are possible (Kane et al., 2017).

3. Efficiency and Theoretical Guarantees

A key motivation for binary comparison queries is information efficiency under structural assumptions:

Setting	Query Complexity	Assumptions	Reference
Halfspace, large margin	$O(\log n)$	High margin, low complexity	(Kane et al., 2017)
General, noisy oracles	$O(n)$ ($O_1$), $O(n\log\log n)$ ($O_2$)	Delegation scheme, $\epsilon_i < 1/2$	(Cui et al., 2020)
Label queries only	$\Theta(n)$	None	(Kane et al., 2017)
Single-peaked voting, known axis	$\Theta(m)$	Known axis	(Conitzer, 2014)
Single-peaked voting, known cardinals	$\Theta(\log m)$	Known cardinal positions	(Conitzer, 2014)
Fair division, constant agents $n$	$O(\log m)$	Additive/identical values	(Bu et al., 2024)

Under bounded inference dimension, comparison queries enable efficient label or preference recovery that would otherwise be intractable given only label or value queries.

4. Methodologies and Algorithms

Active Learning: Delegation Set Algorithm

The adaptive labeling strategy leveraging binary comparison queries proceeds as follows (Cui et al., 2020):

1. Delegation via Uncertainty Oracle ($O_2$): Identify the top $t$ most uncertain points using a tournament-plus-heap selection with m-fold repeated queries for robustness.
2. Label Inference via Positivity Oracle ($O_1$): For each remaining point, use majority vote over $O_1$ queries against the delegation set.
3. Threshold Point Assignment: The most uncertain (delegation set) points, unassignable via pairwise comparisons, are labeled randomly or recursively (to reduce their error further).

In active learning with a limited labeling budget, this approach integrates into disagreement-based batch selection loops, delivering provable generalization error bounds and requiring sublinear total label budget.

Fair Division and Social Choice

For fair allocation among $n$ agents and $m$ indivisible goods, comparison-based query algorithms iteratively partition items, extract EF1 or PROP1 allocations via bundle comparisons, and use augmenting paths in matching graphs to achieve the desired fairness guarantees in $O(\log m)$ queries for constant $n$ (Bu et al., 2024). For voting with single-peaked or single-crossing preferences, peak-finding and ranking recovery can be accomplished with $\Theta(\log m)$ comparison queries when the alternative ordering is known, and $\Theta(m)$ otherwise (Conitzer, 2014).

Similarity and Matching via Binary Comparison

• IoT Binary Similarity Matching: The Inter-BIN architecture employs co-attention across instruction sequences from binaries. The essence is to use learned, multi-feature instruction representations and soft alignment (via attention) to compare code at the function or block level, reporting a binary similarity score (Song et al., 2022). The Multi-Relational Instruction Association Graph constructs a per-pair graph with six relation types and aggregates via R-GCN and Bi-LSTM pooling, again producing binary similarity outcomes (Song et al., 2022).
• Semantics-based Hybrid Comparison: BinMatch leverages semantic signatures (memory writes, comparison operand values, library calls) obtained via dynamic instrumentation and static emulation, comparing function behaviors via Jaccard-indexed LCS or SimHash+HD measures to support binary similarity comparisons robust to obfuscation, optimization, and cross-ISA variation (Hu et al., 2019).

5. Applications and Empirical Evidence

Binary comparison queries are empirically validated in diverse settings:

• Crowdsourcing: Users can often answer uncertainty comparison queries more reliably than explicit labeling in perceptually difficult visual tasks; the O₂ oracle is cognitively simple and can match or outperform explicit label-based supervised benchmarks for character recognition and car-preference tasks (Cui et al., 2020).
• Active Classification: On image datasets (MNIST, FMNIST, KMNIST, CIFAR-10), even with high comparison noise, the O₁+O₂ scheme reaches 80–94% k-NN test accuracy. Label inference requiring explicit labels is replaced entirely with pairwise comparisons (Cui et al., 2020).
• Preference Elicitation: In voting, proper exploitation of domain structure (single-peakedness: known axis or cardinals) reduces the number of queries from $\Theta(m \log m)$ (unconstrained) to $\Theta(m)$ or $\Theta(\log m)$, underpinning scalable collective decision methods (Conitzer, 2014).
• Fair Division: Algorithms relying solely on bundle comparison queries (no explicit utilities) return EF1 and PROP1 allocations with optimal query complexity in terms of $m$, with matching lower bounds (Bu et al., 2024).
• Binary Code Similarity: Large-scale cross-architecture malware and IoT datasets demonstrate that state-of-the-art similarity models based on comparison queries (binarized outputs or retrieval via soft binary relevance) are robust to ISAs, compilers, and obfuscations (Song et al., 2022, Song et al., 2022, Hu et al., 2019).

6. Lower Bounds, Limitations, and Optimality Regimes

Exponential query complexity reductions from binary comparison queries are only available with structural restrictions, such as low inference dimension, large margin, or restricted preference domains. If such constraints are absent, worst-case lower bounds are linear (or worse):

• For general halfspaces with unbounded margin or description length, the inference dimension is infinite and $\Omega(n)$ queries are necessary (Kane et al., 2017).
• In social choice, even under single-peakedness, linear-in-$m$ queries are needed unless the cardinal axis is known; in unknown-ordinal domains, no sublinear methods exist (Conitzer, 2014).
• For fair division with only comparison queries, the lower bound for EF1 and PROP1 allocations is $\Omega(\log m)$ even with two agents, realized by adversarial binary-valued goods and minimax-responses (Bu et al., 2024).
• For binary similarity learning, methods reliant on specific features (e.g., instruction n-grams) degrade under heavy obfuscation, while neural comparison approaches require carefully tailored embeddings to maintain robustness (Song et al., 2022, Hu et al., 2019).

Open questions persist—robustness to persistent or adversarial noise in comparisons, efficient comparison-based recovery in high-complexity domains (e.g., general Boolean thresholds), streaming and memory-limited extensions, and the design of comparison schemes for domains beyond $\mathbb{R}^d$ halfspaces or monotonic preference structures (Kane et al., 2017, Conitzer, 2014).

7. Broader Perspectives and Contextualization

Binary comparison queries occupy a vital position at the interface of computational learning theory, human-computer interaction, and algorithmic social science. They underpin active learning protocols that are more label-efficient and cognitively ergonomic; enable scalable preference aggregation and fair allocation in multiagent systems; and drive robust, cross-architecture matching in binary code analysis—a critical task in software security and maintenance. Theoretical frameworks such as inference dimension, budget-constrained optimization, and soft matching inform the optimality and practical implementation of such queries, while ongoing research continues to refine their capabilities and address the challenges imposed by noise, adversarial query models, and high-dimensional or structurally complex domains (Cui et al., 2020, Kane et al., 2017, Bu et al., 2024, Conitzer, 2014, Song et al., 2022, Song et al., 2022, Hu et al., 2019).