Conditional Sorting in Algorithms & ML

• Conditional sorting is defined as the process of ordering elements under constraints, employing forbidden comparisons and differentiable relaxations in both algorithmic and machine learning contexts.
• It leverages frameworks like ColorSolve and CliqueSolve, using graph parameters such as chromatic and clique numbers to efficiently reconstruct total or partial orders.
• The technique extends to differentiable sorting, where continuous relaxations permit gradient-based optimization in ranking tasks, ensuring computational efficiency and stability.

Conditional sorting refers to the process of ordering elements under additional constraints or conditions, extending classical sorting to contexts with restricted comparisons, differentiable objectives, or partial information. In discrete algorithmic settings, conditional sorting often arises through forbidden comparisons, while in machine learning and neural network optimization, it appears via differentiable relaxations of the canonical comparator-and-swap operation. Conditional sorting unifies themes from theoretical computer science, combinatorial optimization, and modern statistical learning.

1. Fundamental Models of Conditional Sorting

Two principal frameworks define conditional sorting. The first is the forbidden comparisons model, formalized via a forbidden-pairs graph $H=(V,E_f)$, where each edge $(u,v)\in E_f$ represents a pair of elements whose direct comparison is disallowed (cost $\infty$). The set of allowable (probeable) comparisons forms the complement $G=(V,E)$ with $E = \binom{V}{2}\setminus E_f$. The objective is to reconstruct the underlying total (or partial) order efficiently, probing as few allowed pairs as possible; when $H$ is empty, this recovers classical sorting with $\Theta(n\log n)$ probes (Manas, 31 Aug 2025).

The second framework, prominent in learning-to-rank and neural network contexts, involves conditional—specifically, differentiable—sorting. Here, the hard comparator (min/max operation) is replaced with a continuous relaxation, permitting gradient-based optimization even when only an ordering or ranking of samples is observed (Petersen et al., 2022, Petersen et al., 2021).

2. Conditional Sort Algorithms with Forbidden Comparisons

Let $V$ be a set of $n$ elements subject to an unknown total order $\prec$, but only certain pairs can be compared due to the forbidden graph $H$. The challenge is to adaptively probe the allowed pairs in $G$ to reconstruct the order efficiently. Algorithmic performance is elegantly governed by two parameters of $H$:

• The clique number $\omega(H)$ denotes the largest clique in $H$.
• The chromatic number $\chi(H)$ is the minimal number of colors in a proper coloring of $H$.

Two deterministic algorithms attain state-of-the-art probe complexity:

Algorithm	Probes (Big-O)	Parameter
ColorSolve	$O(n\log n + n\chi(H))$	Chromatic number $\chi(H)$
CliqueSolve	$O(n\,\omega(H)\log n)$	Clique number $\omega(H)$

ColorSolve partitions $V$ using a proper coloring of $H$, sorts within color classes fully, and establishes inter-class relations using multi-pointer routines. CliqueSolve processes vertices sequentially: for each new vertex $u$, a recursive pivoting and partial-probe approach exploits the clique bound for efficiency, reducing subproblem sizes geometrically and maintaining $O(k \log n)$ probes per vertex for $k = \omega(H)$ (Manas, 31 Aug 2025). Both methods do not require the existence of a unique total order; when the comparability graph is merely acyclic, the induced partial order is recovered.

3. Partial Order Discovery and Non-Sortable Instances

If the underlying comparability graph $\vec{G}$ is not Hamiltonian, no total order exists. The conditional sorting algorithms above nevertheless reconstruct the full partial order (i.e., the Hasse diagram of the poset), orienting all allowed edges in $G$ in compliance with the order constraints. The probe complexities remain $O(n\chi(H)+n\log n)$ or $O(n\,\omega(H)\log n)$, depending on the parameterization. This generalizes classical sorting to poset discovery in the presence of incomplete information (Manas, 31 Aug 2025).

4. Random Constraints: Erdős–Rényi Analysis and Uniform Algorithms

In random graph models, where $G \sim G(n,p)$ and the forbidden pairs correspond to $H = \overline G$, several structural parameters concentrate sharply:

• The independence number $\alpha(G) = O(p^{-1}\log n)$ (w.h.p.), so $\omega(H)=\alpha(G)$.
• The number of permissible comparisons is $|E| = pn^2/2 \pm O(\sqrt{pn^2\log n})$.

A two-regime strategy attains $O(n^{3/2}\log n)$ probes for all $p$:

1. Sparse regime ($p \leq \frac{\log n}{\sqrt n}$): Probe all edges in expected $O(n^{3/2}\log n)$ time.
2. Dense regime ($p > \frac{\log n}{\sqrt n}$): Use CliqueSolve, achieving $O(p^{-1}n\log^2 n) \le O(n^{3/2}\log n)$ (w.h.p.).

This protocol is robust across all edge densities and relies on sharp concentration of random graph invariants (Manas, 31 Aug 2025).

5. Differentiable Conditional Sorting in Neural Networks

Differentiable sorting replaces the hard, discontinuous min/max comparator with a continuous operator suitable for backpropagation, central to ranking-supervision tasks (Petersen et al., 2021). In classical sorting networks, a hard decision is made based on $\text{sign}(b - a)$, leading to zero gradients almost everywhere. Differentiable relaxations utilize sigmoids, e.g., logistic, reciprocal, or Cauchy CDFs, to interpolate between the two permutations (Petersen et al., 2022, Petersen et al., 2021):

$$\min_f(a, b) = a\,f(b-a) + b\,f(a-b), \quad \max_f(a, b) = a\,f(a-b) + b\,f(b-a)$$

Notably, only specific sigmoids satisfying $f'(x) = \Theta(1/x^2)$ guarantee monotonicity, ensuring nonnegative gradients w.r.t. inputs and stabilizing optimization (Petersen et al., 2022). For example, the reciprocal sigmoid $$R(x) = \frac{1}{2}\left(1 + \frac{\beta x}{1 + 2\beta|x|}\right)$$ satisfies this asymptotic decay property.

Two architectures permit scalable, differentiable permutations:

• Odd–even sorting networks: $n$-layer networks with adjacent pairwise comparators.
• Bitonic sorting networks: $\log^2 n$-layer depth for $n=2^k$, superior scaling for large $n$ (Petersen et al., 2021).

The activation-replacement trick, $\varphi(x) = \text{sign}(x)|x|^\lambda$, helps maintain gradients away from the saturated regime of the sigmoid, further improving training stability (Petersen et al., 2021).

6. Empirical Performance and Comparative Analysis

Empirical benchmarks establish substantial gains of neural-network-based conditional sorting over prior global relaxations, both in element-wise (EW) and exact-match (EM) accuracy:

Task / $n$	NeuralSort (EM/EW)	OT-Sort (EM/EW)	Odd–Even (EM/EW)	Bitonic (EM/EW)
MNIST-4 / 15	12.2% / 73.4%	12.6% / 74.2%	35.4% / 83.7%	34.7% / 82.9%
SVHN / 32	0.0% / 29.9%	0.0% / --	-- / --	0.0% / 42.4%

For monotonic differentiable sorting networks, individual-rank and full-sequence accuracy exceed prior non-monotonic relaxations by 15–20 percentage points for moderate $n$ and by an order of magnitude in full-sequence recovery for larger $n$ (Petersen et al., 2022). In addition, monotonicity enforces correct gradient flow, leading to stable optimization and error-bounded sorting dynamics.

Bitonic networks achieve forward+backward runtimes of 15 ms for $n=1024$ (vs 660 ms for odd-even) and practical GPU memory consumption ($\sim$0.55 GB), scaling to thousands of items and outperforming black-box global approaches such as NeuralSort and OT-Sort (Petersen et al., 2021).

7. Open Questions, Extensions, and Practical Limitations

Key research directions include tightening lower bounds for probe complexity in forbidden-comparisons models—determining if $\Omega(n\omega(H)\log n)$ or $\Omega(n\chi(H)+n\log n)$ is necessary in the worst case—and seeking algorithms that closely mirror the information-theoretic lower bounds (as in $\log(\#\text{linear extensions})$). Extensions to cost models beyond $0$–$\infty$ comparisons, as well as parallel/distributed probing schemes, remain areas of active investigation (Manas, 31 Aug 2025).

For differentiable sorting, constraints include the necessity for bitonic networks to operate with $n=2^k$, the need for tuning of steepness and activation-exponent hyperparameters, and potential memory bottlenecks arising from the full $n \times n$ soft-permutation matrix when only the sorted outputs are required (Petersen et al., 2021).

Conditional sorting thus constitutes a rich subject at the intersection of combinatorial algorithms, optimization, and machine learning, with continually evolving methodologies and broad application scope spanning partial-information sorting, learning-to-rank, and end-to-end differentiable pipelines.