Divergence-Based Similarity Functions (DSF)

• DSF is a family of similarity functions that derives a score from divergence measures by applying a monotonic decreasing transform, such as the exponential function.
• It encompasses various formulations including asymmetric KL-based measures, symmetric affine divergences, and extropy-based ratios, each capturing distinct structural properties.
• DSFs find applications in neural representation learning, statistical geometry, and structured-domain comparisons by linking divergence properties to similarity metrics.

to=arxiv_search 荣富({"query":"\"divergence-based similarity\" arXiv DSF", "max_results": 10}) to=arxiv_search 天天中彩票怎么买{"query":"divergence-based similarity function", "max_results": 10} The literature surveyed here suggests that a divergence-based similarity function (DSF) is not a single canonical formula but a family of constructions in which similarity is obtained from a divergence, discrepancy, or divergence-pattern comparison between objects. In the most explicit formulation, a DSF has the form $S(p,q)=f(D(p\|q))$, where $D\ge 0$ is a divergence and $f$ is monotonically decreasing, such as $f(t)=e^{-t}$; in other formulations, similarity is produced by normalized ratios built from extropy or by comparing within-group divergence magnitudes rather than paired values directly (Jayashree et al., 2019, Nishiyama, 2018, P. et al., 19 Aug 2025, Hoorn, 19 Oct 2025).

1. General formulation

A recurrent formal pattern is to begin with a divergence $D$ and convert it into similarity by a decreasing transform. Representative forms stated in the literature include

$$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$

with the last expression requiring a natural maximum scale $D_{\max}$ (Nishiyama, 2018). In the word-distribution setting, the energy or similarity score is written explicitly as

$$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$

so that identical distributions attain similarity $1$, while increasing divergence reduces similarity toward $0$ (Jayashree et al., 2019).

The algebraic behavior of a DSF is inherited from the underlying divergence. If $D\ge 0$0 is asymmetric, then the resulting similarity is typically asymmetric as well. This is the case for KL-based constructions, where

$D\ge 0$1

and therefore $D\ge 0$2 in general (Jayashree et al., 2019). If $D\ge 0$3 is symmetric, as with the affine divergence $D\ge 0$4, then the corresponding DSF is symmetric (Nishiyama, 2018). Other constructions are normalized differently: the generalized extropy similarity ratio is

$D\ge 0$5

which is symmetric and lies in $D\ge 0$6 (P. et al., 19 Aug 2025). By contrast, the correlation-of-divergency coefficient

$D\ge 0$7

is nonnegative and unbounded above, and the paper explicitly states that its values range between $D\ge 0$8 and $D\ge 0$9 (Hoorn, 19 Oct 2025).

A useful conceptual distinction in this literature is between DSFs that compare objects directly and DSFs that compare their internal divergence structure. The former include KL-, Jeffreys-, Bhattacharyya-, Jensen–Shannon-, or Wasserstein-derived similarities. The latter include $f$0, which does not correlate paired values $f$1 and $f$2 directly but instead compares the magnitudes

$f$3

thereby measuring whether the way values differ within one group is mirrored in another (Hoorn, 19 Oct 2025).

2. Geometric and statistical foundations

A major information-geometric source of DSFs is the theory of dually flat spaces. In that setting, a statistical manifold carries dual affine coordinates $f$4, dual strictly convex potentials $f$5, and a canonical divergence

$f$6

For exponential families and mixture families, this canonical divergence coincides with the Kullback–Leibler divergence (Nishiyama, 2018). The same framework introduces the affine divergence

$f$7

which is a symmetric Jeffreys-type divergence, and the $f$8- and $f$9-divergences, which are consistent with Bhattacharyya and Jensen–Shannon divergences, respectively (Nishiyama, 2018). These divergences provide a menu of DSFs with different symmetry and overlap properties.

The dually flat formalism also supplies structural identities that are relevant for DSF design. The canonical divergence satisfies the triangular relation

$f(t)=e^{-t}$0

which is presented as a generalization of the Euclidean law of cosines (Nishiyama, 2018). The affine divergence inherits Euclidean-style relations such as a generalized law of cosines, a generalized parallelogram law, and a generalized polarization identity. This suggests that some DSFs do not merely score pairwise closeness but encode the intrinsic dual-affine geometry of the model space.

A second route from divergence to similarity is to induce a Riemannian metric from a convex generator. For separable Bregman divergences generated by

$f(t)=e^{-t}$1

the Hessian metric is

$f(t)=e^{-t}$2

and the associated geodesic distance becomes

$f(t)=e^{-t}$3

Thus the induced metric is Euclidean after the coordinate transform $f(t)=e^{-t}$4 (Gzyl, 2018). This construction turns a nonmetric divergence into a genuine metric-based DSF and changes the associated notions of mean and best predictor.

A third foundational perspective is the unifying framework for directed distances between statistical functionals. There the central object is

$f(t)=e^{-t}$5

which covers density-based $f(t)=e^{-t}$6-divergences, Bregman divergences, scaled Bregman divergences, distribution-function divergences, quantile divergences, depth-based divergences, and cumulative paired divergences (Broniatowski et al., 2022). In that framework, classical Cramér–von Mises, Anderson–Darling, Kantorovich, and Wasserstein-type quantities are all realizations of a common divergence template over statistical functionals.

3. Functional and information-theoretic variants

The functional density power divergence class provides a precise characterization of which scalar transforms preserve divergence structure. For $f(t)=e^{-t}$7,

$f(t)=e^{-t}$8

and the paper proves that, for fixed $f(t)=e^{-t}$9, this is a valid divergence if and only if $D$0 is convex, strictly increasing, and finite on $D$1 (Ray et al., 2021). The density power divergence and logarithmic density power divergence appear as the special cases $D$2 and $D$3, respectively. This result defines a principled design space for DSFs derived from robust divergence families.

A different information-theoretic specialization is the Jensen–Fisher divergence,

$D$4

where

$D$5

is Fisher information (Sánchez-Moreno et al., 2010). Because Fisher information is a local gradient functional, this divergence is explicitly described as very sensitive to fluctuations and particularly informative for oscillatory distributions. The paper compares Jensen–Fisher and Jensen–Shannon divergences on sinusoidal, generalized gamma-like, and Rakhmanov–Hermite densities, showing that Jensen–Fisher reacts much more strongly to node structure and local oscillation (Sánchez-Moreno et al., 2010). A plausible implication is that DSFs built from Fisher-information divergences are suited to settings where local regularity matters more than global spread.

Extropy-based constructions replace logarithmic information measures by quadratic $D$6-type forms. With

$D$7

the generalized extropy divergence ratio is

$D$8

and the generalized extropy similarity ratio is

$D$9

The paper proves that this similarity equals $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$0 in $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$1, where $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$2 is the angle between $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$3 and $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$4 (P. et al., 19 Aug 2025). This establishes an explicit relationship between an extropy-based DSF and cosine similarity.

Not all DSFs are based on direct distribution comparison. The correlation-of-divergency coefficient $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$5 compares the divergence profile of each observation within its own group, and normalizes by the average root mean square divergence of each group (Hoorn, 19 Oct 2025). The paper emphasizes that Pearson and Spearman correlations ask whether high $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$6 goes with high $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$7, whereas $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$8 asks whether items that are more atypical relative to peers in one group are also more atypical relative to peers in the other. This yields a DSF over variability structure rather than over raw values.

4. Learned DSFs in representation learning

In neural representation learning, DSFs appear as task-specific learned divergences. Deep Bregman divergences are defined by parameterizing a convex functional $$S_D(P,Q)=\exp(-\lambda D(P,Q)),\qquad S_D(P,Q)=\frac{1}{1+D(P,Q)},\qquad S_D(P,Q)=1-\frac{D(P,Q)}{D_{\max}},$$9 with neural networks, so that the resulting functional Bregman divergence

$D_{\max}$0

becomes learnable (Cilingir et al., 2020). In the symmetric case, the paper proves that functional Bregman divergences reduce to

$D_{\max}$1

with $D_{\max}$2 symmetric positive semidefinite, and shows that deep metric learning, Mahalanobis metric learning, kernel metric learning, and moment-matching functions for comparing distributions arise as special cases (Cilingir et al., 2020). In practice, the divergence replaces Euclidean distance inside contrastive or triplet losses.

A language-modeling instantiation represents each word $D_{\max}$3 as a Gaussian mixture distribution

$D_{\max}$4

with multiple components encoding polysemy and diagonal covariances encoding uncertainty (Jayashree et al., 2019). Similarity is then

$D_{\max}$5

but because exact KL divergence between Gaussian mixtures has no closed form, the paper uses an approximation obtained by averaging a lower and an upper bound built from component-wise Gaussian KL terms, expected likelihood kernels, and entropies (Jayashree et al., 2019). The max-margin objective encourages low KL for positive word–context pairs and higher KL for negative pairs. The asymmetry of KL is used explicitly to model entailment-friendly ordered relations.

A more recent computer-vision instantiation addresses multi-view contrastive learning. Each set of augmented views is mapped to unit-norm embeddings on the hypersphere, fitted with a von Mises–Fisher distribution $D_{\max}$6, and compared by

$D_{\max}$7

inside an InfoNCE objective (Jeon et al., 9 Jul 2025). The paper states that this DSF explicitly captures the joint structure across all views, establishes a theoretical connection to cosine similarity, and shows that, unlike cosine similarity, the method operates effectively without requiring a temperature hyperparameter (Jeon et al., 9 Jul 2025). In the special case $D_{\max}$8 with equal concentration, DSF-based InfoNCE is exactly equivalent to cosine-similarity InfoNCE after the appropriate scaling relation $D_{\max}$9.

5. Structured-domain instantiations

For positive semidefinite matrices, DSFs are built by combining subspace geometry and divergence on positive definite fibers. If $$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$0 and $$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$1, with support subspaces $$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$2, the geometric distance is

$$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$3

where $$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$4 measures subspace discrepancy and $$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$5 is a Hausdorff-type extension of a PD divergence (Liu et al., 2023). The framework extends KL, Bhattacharyya, Rényi, Stein, Burg, and related divergences to non-equidimensional PD matrices and then to arbitrary PSD matrices. The paper proves that the resulting measurement can be recognized either as the cost of a parallel transport or as the length of a quasi-geodesic curve (Liu et al., 2023). A DSF then arises by applying a decreasing transform such as $$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$6.

In intuitionistic fuzzy theory, a strict divergence-based distance is constructed from Jensen–Shannon divergence. For an intuitionistic fuzzy value $$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$7, the paper defines a normalized IFV distance

$$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$8

and aggregates it over weighted attributes to obtain

$$E_\theta(w,c)=\exp\!\bigl(-\mathrm{KL}(f_w\|f_c)\bigr),$$9

Its dual similarity is

$1$0

(Wu et al., 2022). The paper introduces the notions of strict intuitionistic fuzzy distance measure and strict intuitionistic fuzzy similarity measure, and proves that the new distance/similarity pair satisfies the strict axioms while previously proposed nonlinear distances do not (Wu et al., 2022). In particular, the maximum distance $1$1 is reserved for the endpoint pair $1$2, avoiding the infinitely many non-endpoint maximizers exhibited by earlier constructions.

Extropy-based similarity ratios furnish another structured-domain example. For density, survival, or cumulative-distribution functionals, the generalized extropy similarity ratio remains invariant under common positive scaling, and the paper derives bounds under proportional hazards and proportional reversed hazards models (P. et al., 19 Aug 2025). Nonparametric estimators are given for density-, survival-, and cdf-based similarities, and the paper demonstrates applications in lifetime data analysis and image analysis (P. et al., 19 Aug 2025).

6. Properties, interpretative issues, and limitations

The surveyed literature repeatedly distinguishes divergence from metric. Many DSFs inherit asymmetry and the failure of the triangle inequality from their generating divergences. KL-based word-distribution similarities are asymmetric by design; deep Bregman divergences are generally asymmetric and need not satisfy the triangle inequality; the correlation-of-divergency coefficient is symmetric but unbounded and therefore not interpretable on the same scale as a normalized similarity (Jayashree et al., 2019, Cilingir et al., 2020, Hoorn, 19 Oct 2025). By contrast, some constructions deliberately restore metric structure, either through Hessian-induced Riemannian geometry or through square-root Jensen–Shannon metrics on intuitionistic fuzzy values (Gzyl, 2018, Wu et al., 2022).

A second recurring issue is the choice between global and local sensitivity. Jensen–Shannon- and extropy-based DSFs emphasize different information structures than Jensen–Fisher-based DSFs. The former are tied to entropy-like or $1$3-type overlap, whereas the latter are explicitly controlled by derivatives of densities and are therefore very sensitive to oscillations (Sánchez-Moreno et al., 2010, P. et al., 19 Aug 2025). This suggests that DSF choice is not merely a normalization decision but a modeling commitment about what kind of discrepancy counts as similarity-relevant.

A third issue is approximation and numerical stability. Gaussian-mixture word models require approximate KL bounds because exact mixture KL is intractable (Jayashree et al., 2019). Multi-view contrastive DSF requires stabilized estimation of the vMF concentration parameter $1$4, including dimension-based normalization and rescaling of the mean resultant length, to prevent numerical instability (Jeon et al., 9 Jul 2025). Extropy-based measures require square-integrability, while nonparametric estimation of density-based extropy similarities depends on kernel density estimation (P. et al., 19 Aug 2025). In PSD-matrix geometry, non-generic cases may require an additional maximization over a smaller unitary group, although the generic formulas are spectral and explicit (Liu et al., 2023).

A final interpretative point concerns what is being compared. Some DSFs compare two distributions directly, some compare statistical functionals such as cdfs or quantiles, and some compare the internal divergence structure of two collections. The literature therefore supports a broad but technically precise view: DSF denotes a similarity mechanism grounded in divergence, but the divergence may live in distribution space, manifold geometry, function space, embedding space, or variability-pattern space (Broniatowski et al., 2022, Nishiyama, 2018, Hoorn, 19 Oct 2025). This breadth is a source of power, but it also means that DSFs are best understood through the geometry, invariances, and failure modes of the specific divergence from which they are built.