LapSum Framework for Differentiable Ranking

• LapSum is an analytic framework that constructs fully differentiable surrogates for order-based operators like soft ranking, sorting, and top-k selection.
• It employs a closed-form inversion of a Laplace sum function over input elements to achieve low computational and memory complexity.
• The framework enables efficient forward and backward passes in neural and optimization pipelines for tasks such as learning-to-rank and structured output prediction.

LapSum is an analytic framework for constructing fully differentiable surrogates to order-based operators, comprising soft ranking, differentiable sorting, top-$k$ selection, and permutation-based transformations. The core methodology centers on the closed-form inversion of a functional known as the LapSum, defined as the sum of Laplace cumulative distribution functions over the elements of an input vector. This construction yields low computational and memory complexity and facilitates efficient forward and backward passes in neural and optimization pipelines relevant to learning-to-rank, attention, and structured output prediction tasks (Struski et al., 8 Mar 2025).

1. Core Definition and Mathematical Structure

Let $r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$ denote an input vector and $\alpha \neq 0$ a temperature (scale) parameter. Define the standard Laplace CDF as

$$Lap(x) = \begin{cases} \frac{1}{2} \exp(x) & x \le 0 \ 1 - \frac{1}{2} \exp(-x) & x > 0 \end{cases}$$

The thermally scaled Laplace CDF is $Lap_\alpha(x) = Lap(x/\alpha)$. The LapSum function is given by

$$\mathrm{LapSum}_\alpha(x; r) = \sum_{i=0}^{n-1} Lap_\alpha(x - r_i) \in (0, n)$$

Given a target “sum” $w \in (0, n)$, one inverts this function to obtain $b = \mathrm{LapSum}^{-1}_\alpha(w; r)$ satisfying $\sum_i Lap_\alpha(b - r_i) = w$. Evaluating $Lap_\alpha(b-r_i)$ for each coordinate then enables soft selection and ordering tasks.

2. Closed-Form Inversion and Algorithmic Construction

For sorted $r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$0, the piecewise-linear nature of the LapSum is exploited. Precomputing sequences $r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$1 in $r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$2 time (via forward and backward recursions), the function admits the region-wise form for $r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$3: $$\mathrm{LapSum}(x; r) = \frac{1}{2} a_k e^{\frac{x - r_{k+1}}{\alpha}}</p> <ul> <li>\frac{1}{2} b_{k+1} e^{\frac{r_k - x}{\alpha}}</li> <li>c_{k+1}$$r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$$4w_k = c_{k+1} - \frac{1}{2} b_{k+1} + \frac{1}{2} a_k e^{(r_{k+1} - r_k)/\alpha}$$ demarcate the regions.

The closed-form inversion is as follows:

• For $r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$5:

$r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$6

• For $r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$7 $r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$8:

$r = (r_0, \ldots, r_{n-1}) \in \mathbb{R}^n$9

• For $\alpha \neq 0$0:

$\alpha \neq 0$1

Locating the corresponding interval in $\alpha \neq 0$2 time per query enables overall $\alpha \neq 0$3 forward propagation.

3. Differentiable Operators Derived from LapSum

The LapSum inversion ground the construction of several differentiable surrogates for combinatorial operators:

• Soft Top-$\alpha \neq 0$4 (generalizing Top-$\alpha \neq 0$5 selection):

$\alpha \neq 0$6 for $\alpha \neq 0$7, where $\alpha \neq 0$8. The resulting vector $\alpha \neq 0$9 yields a soft indicator, converging to the hard Top-$$Lap(x) = \begin{cases} \frac{1}{2} \exp(x) & x \le 0 \ 1 - \frac{1}{2} \exp(-x) & x > 0 \end{cases}$$0 as $$Lap(x) = \begin{cases} \frac{1}{2} \exp(x) & x \le 0 \ 1 - \frac{1}{2} \exp(-x) & x > 0 \end{cases}$$1.

• Soft Ranking:

$$Lap(x) = \begin{cases} \frac{1}{2} \exp(x) & x \le 0 \ 1 - \frac{1}{2} \exp(-x) & x > 0 \end{cases}$$2 for each $$Lap(x) = \begin{cases} \frac{1}{2} \exp(x) & x \le 0 \ 1 - \frac{1}{2} \exp(-x) & x > 0 \end{cases}$$3. In the zero-temperature limit, this retrieves classical ranking (plus