---
title: Rao’s Quadratic Entropy
url: https://www.emergentmind.com/topics/rao-s-quadratic-entropy
type: topic
---

# Rao’s Quadratic Entropy

Rao’s quadratic entropy is a foundational diversity measure that quantifies not only the relative abundance of types within a population but also their pairwise dissimilarities. Introduced by C. R. Rao in 1982, it has found broad utility in ecology, engineering design, information theory, and machine learning. The definition encompasses and generalizes traditional indices like the Gini–Simpson index, and its structure admits deep connections to information geometry and statistical estimation. The quadratic form structure and accommodation of arbitrary dissimilarity metrics enable modeling of functional, phylogenetic, or semantic distances, making it highly applicable in modern data-driven diversity analysis.

## 1. Mathematical Definition and Fundamental Properties

Rao’s quadratic entropy for a finite population of $n$ types with probability vector $p=(p_1, ..., p_n)$ and symmetric, nonnegative dissimilarity matrix $D=(d_{ij})$ (with $d_{ii}=0$) is formally defined as
$$
Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.
$$
This quantifies the expected dissimilarity between two independently drawn elements from the population under $p$ [2205.09931, 2408.00684, 2412.03835].

Key properties:
- **Agnostic to the choice of dissimilarity:** $d_{ij}$ can encode Euclidean, Manhattan, Jaccard, Hamming, or kernel-based dissimilarities.
- **Symmetry:** $Q(p)$ is invariant under permutations of the types.
- **Continuity:** $Q(p)$ varies continuously in both $p$ and $d_{ij}$.
- **Monotonicity:** Adding a new type with average dissimilarity greater than current $Q$ increases entropy; otherwise, it decreases [2205.09931].
- **Generalization:** If $d_{ij}=1$ for $i\neq j$, $Q(p) = 1 - \sum_i p_i^2$, which is exactly the Gini–Simpson index.

## 2. Statistical Estimation and Unbiased Index

The empirical or “plugin” estimator for Rao's quadratic entropy with $N$ observed types (counts $n_i$ for each type, $\sum n_i = N$), is
$$
\widehat{Q}_{\mathrm{naive}} = \frac{1}{N^2}\sum_{i, j} d_{ij} n_i n_j.
$$
For uniform counts $n_i \equiv 1$, this becomes $\widehat{Q}_{\mathrm{naive}} = (1/N^2)\sum_{i\neq j} d_{ij}$ [2408.00684].

However, sample covariance under finite $N$ introduces negative bias. The unbiased estimator, denoted “RQID” in design applications, is
$$
Q_{\mathrm{unbiased}} = \frac{1}{N(N-1)} \sum_{i=1}^N \sum_{\substack{j=1 \\ j \ne i}}^N d_{ij}.
$$

This estimator is exactly unbiased for the true population $Q$ when using simple random sampling without replacement [2408.00684]. For binary distances, it reduces to the Gini–Simpson Index.

## 3. Connections to Related Diversity Indices

Rao’s quadratic entropy generalizes classical measures such as the Gini–Simpson index and Hill numbers:
- **Gini–Simpson index:** For $d_{ij} = 1$ $(i \neq j)$, $Q(p) = 1 - \sum_i p_i^2$ [2412.03835].
- **Hill numbers:** The Hill number of order $q$ is ${}^q D(p) = (\sum_i p_i^q)^{1/(1-q)}$; for $q=2$, ${}^2 D(p) = (\sum_i p_i^2)^{-1}$.
- **Leinster–Cobbold index:** For similarity matrix $Z$, ${}^qD_Z(p) = (\sum_i p_i (Zp)_i^{q-1})^{1/(1-q)}$. For $q=2$ and $Z = I-D$, $1-Q(p)$ appears as the “similarity-weighted” analogue of $\sum_i p_i^2$.

The relationship is summarized in the following table:

| Metric                    | Parameters                | Formula                                                      |
|---------------------------|---------------------------|--------------------------------------------------------------|
| Rao's quadratic entropy   | $d_{ij}$ arbitrary        | $Q(p) = \sum_{i,j} p_i p_j d_{ij}$                          |
| Gini–Simpson index        | $d_{ij}=1$ $(i\neq j)$    | $1 - \sum_i p_i^2$                                          |
| Leinster–Cobbold index    | similarity matrix $Z$     | ${}^q D_Z(p) = (\sum_i p_i (Zp)_i^{q-1})^{1/(1-q)}$         |

## 4. Methodological Extensions and Application Frameworks

The generality of $d_{ij}$ underlies methodological extensions. In software engineering, Rao’s entropy is instantiated as “fork entropy,” where $d_{ij}$ quantifies kernel-smoothed differences in file changes between forks [2205.09931]. In engineering design, semantic distances between concept descriptions are computed via SBERT embeddings over SAPPHIRE causality model levels, aggregated using weights to produce the pairwise distance matrix [2408.00684]. Unbiased estimation (RQID) is emphasized for robust assessment of variety.

Algorithmic workflow in engineering design includes:
1. Encoding design concepts at multiple abstraction levels.
2. Computing embedding vectors and pairwise cosine dissimilarities.
3. Aggregating level-wise distances with specified weights.
4. Computing the unbiased Rao index by summing off-diagonal distances and normalizing by $N(N-1)$.

Empirical illustrations confirm that the unbiased Rao index corrects the downward bias of the naive estimator, especially in small samples [2408.00684].

## 5. Role in Information Geometry and Maximum Diversity

Rao’s quadratic entropy admits a natural information-geometric interpretation. On the simplex $\Delta_{n-1}$, $Q(p) = p^\top D p$ is a quadratic form; its gradient is $2Dp$ and its Hessian is $2D$ [2412.03835]. The maximizer of $Q(p)$ under affine constraints (e.g., resource allocation $\sum_i a_i p_i = C$) is obtained as
$$
\hat{p}^{(Q)}_\theta = \frac{D^{-1}(\mathbf{1} + \theta a)}{\mathbf{1}^\top D^{-1}(\mathbf{1} + \theta a)}.
$$
In the unconstrained case, this simplifies to
$$
\hat{p}^{(Q)}_0 = \frac{D^{-1} \mathbf{1}}{\mathbf{1}^\top D^{-1} \mathbf{1}},
$$
provided the resulting vector is a valid probability vector.

Connections to the Fisher–Rao metric further integrate Rao’s entropy into the framework of information geometry, where geodesics and dual connections can be studied in relation to the quadratic form structure of $Q$ [2412.03835].

## 6. Empirical Applications and Interpretive Properties

Rao's quadratic entropy and its variants have been deployed in a range of empirical contexts:
- **Open-source software:** Fork entropy quantifies diversity among GitHub forks based on kernel distances of file-modification fingerprint vectors. Observed fork entropy is positively correlated with project productivity, pull-request acceptance rates, and negatively with bug issue rates [2205.09931].
- **Engineering design:** The unbiased Rao index computed on SBERT-embedded SAPPHIRE causal descriptions robustly quantifies conceptual “variety” [2408.00684].
- **Ecology and biodiversity:** Functional and genetic trait dissimilarities are incorporated via $d_{ij}$, enabling sensitivity to phylogenetic or niche differences [2412.03835].

Desirable mathematical axioms—symmetry, continuity, and monotonicity—are empirically validated in these applications, and the index provides a one-number summary that is sensitive both to abundance heterogeneity and to the spectrum of dissimilarity.

## 7. Limitations and Theoretical Considerations

While Rao’s quadratic entropy enjoys broad applicability, its properties depend critically on the definition and empirical fidelity of $d_{ij}$. In computation, choice of dissimilarity metric, embedding model, and abstraction weights can influence results and require context-specific validation [2408.00684]. Statistical estimation is straightforward under simple random sampling but requires bias correction for small samples. There is no requirement for $d_{ij}$ to satisfy the triangle inequality; any symmetric, nonnegative matrix with zeros on the diagonal is admissible. The estimator's variance decreases as $1/N$, and in the large-population limit, plug-in and unbiased estimators coincide.

In information-geometric settings, the invertibility and positivity of $D$ are necessary for the existence of the maximizing $p$, and these restrictions must be considered in ecological modeling.

---

Rao’s quadratic entropy provides a mathematically robust, axiomatically sound, and computationally transparent measure for quantifying diversity in settings where both frequency and dissimilarity among types are non-negligible. Its versatility and foundational character make it central to modern studies of variety, disparity, and informational geometry [2205.09931, 2408.00684, 2412.03835].

Source: https://www.emergentmind.com/topics/rao-s-quadratic-entropy