Rao’s Quadratic Entropy

• Rao’s quadratic entropy is a diversity measure that quantifies the average dissimilarity between types by integrating their relative abundances with pairwise differences.
• Its quadratic formulation generalizes classic indices like the Gini–Simpson index and finds applications in ecology, engineering design, information theory, and machine learning.
• The measure’s robust mathematical properties, including symmetry, continuity, and unbiased statistical estimation, enable reliable analysis in various empirical contexts.

Rao’s quadratic entropy is a foundational diversity measure that quantifies not only the relative abundance of types within a population but also their pairwise dissimilarities. Introduced by C. R. Rao in 1982, it has found broad utility in ecology, engineering design, information theory, and machine learning. The definition encompasses and generalizes traditional indices like the Gini–Simpson index, and its structure admits deep connections to information geometry and statistical estimation. The quadratic form structure and accommodation of arbitrary dissimilarity metrics enable modeling of functional, phylogenetic, or semantic distances, making it highly applicable in modern data-driven diversity analysis.

1. Mathematical Definition and Fundamental Properties

Rao’s quadratic entropy for a finite population of $n$ types with probability vector $p=(p_1, ..., p_n)$ and symmetric, nonnegative dissimilarity matrix $D=(d_{ij})$ (with $d_{ii}=0$) is formally defined as

$$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$

This quantifies the expected dissimilarity between two independently drawn elements from the population under $p$ (Wang et al., 2022, Majumder et al., 2024, Eguchi, 2024).

Key properties:

• Agnostic to the choice of dissimilarity: $d_{ij}$ can encode Euclidean, Manhattan, Jaccard, Hamming, or kernel-based dissimilarities.
• Symmetry: $Q(p)$ is invariant under permutations of the types.
• Continuity: $Q(p)$ varies continuously in both $p$ and $p=(p_1, ..., p_n)$0.
• Monotonicity: Adding a new type with average dissimilarity greater than current $p=(p_1, ..., p_n)$1 increases entropy; otherwise, it decreases (Wang et al., 2022).
• Generalization: If $p=(p_1, ..., p_n)$2 for $p=(p_1, ..., p_n)$3, $p=(p_1, ..., p_n)$4, which is exactly the Gini–Simpson index.

2. Statistical Estimation and Unbiased Index

The empirical or “plugin” estimator for Rao's quadratic entropy with $p=(p_1, ..., p_n)$5 observed types (counts $p=(p_1, ..., p_n)$6 for each type, $p=(p_1, ..., p_n)$7), is

$p=(p_1, ..., p_n)$8

For uniform counts $p=(p_1, ..., p_n)$9, this becomes $D=(d_{ij})$0 (Majumder et al., 2024).

However, sample covariance under finite $D=(d_{ij})$1 introduces negative bias. The unbiased estimator, denoted “RQID” in design applications, is

$D=(d_{ij})$2

This estimator is exactly unbiased for the true population $D=(d_{ij})$3 when using simple random sampling without replacement (Majumder et al., 2024). For binary distances, it reduces to the Gini–Simpson Index.

3. Connections to Related Diversity Indices

Rao’s quadratic entropy generalizes classical measures such as the Gini–Simpson index and Hill numbers:

• Gini–Simpson index: For $D=(d_{ij})$4 $D=(d_{ij})$5, $D=(d_{ij})$6 (Eguchi, 2024).
• Hill numbers: The Hill number of order $D=(d_{ij})$7 is $D=(d_{ij})$8; for $D=(d_{ij})$9, $d_{ii}=0$0.
• Leinster–Cobbold index: For similarity matrix $d_{ii}=0$1, $d_{ii}=0$2. For $d_{ii}=0$3 and $d_{ii}=0$4, $d_{ii}=0$5 appears as the “similarity-weighted” analogue of $d_{ii}=0$6.

The relationship is summarized in the following table:

Metric	Parameters	Formula
Rao's quadratic entropy	$d_{ii}=0$7 arbitrary	$d_{ii}=0$8
Gini–Simpson index	$d_{ii}=0$9 $$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$0	$$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$1
Leinster–Cobbold index	similarity matrix $$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$2	$$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$3

4. Methodological Extensions and Application Frameworks

The generality of $$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$4 underlies methodological extensions. In software engineering, Rao’s entropy is instantiated as “fork entropy,” where $$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$5 quantifies kernel-smoothed differences in file changes between forks (Wang et al., 2022). In engineering design, semantic distances between concept descriptions are computed via SBERT embeddings over SAPPHIRE causality model levels, aggregated using weights to produce the pairwise distance matrix (Majumder et al., 2024). Unbiased estimation (RQID) is emphasized for robust assessment of variety.

Algorithmic workflow in engineering design includes:

1. Encoding design concepts at multiple abstraction levels.
2. Computing embedding vectors and pairwise cosine dissimilarities.
3. Aggregating level-wise distances with specified weights.
4. Computing the unbiased Rao index by summing off-diagonal distances and normalizing by $$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$6.

Empirical illustrations confirm that the unbiased Rao index corrects the downward bias of the naive estimator, especially in small samples (Majumder et al., 2024).

5. Role in Information Geometry and Maximum Diversity

Rao’s quadratic entropy admits a natural information-geometric interpretation. On the simplex $$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$7, $$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$8 is a quadratic form; its gradient is $$Q(p) = \sum_{i=1}^n \sum_{j=1}^n p_i p_j d_{ij} = p^\top D p.$$9 and its Hessian is $p$0 (Eguchi, 2024). The maximizer of $p$1 under affine constraints (e.g., resource allocation $p$2) is obtained as

$p$3

In the unconstrained case, this simplifies to

$p$4

provided the resulting vector is a valid probability vector.

Connections to the Fisher–Rao metric further integrate Rao’s entropy into the framework of information geometry, where geodesics and dual connections can be studied in relation to the quadratic form structure of $p$5 (Eguchi, 2024).

6. Empirical Applications and Interpretive Properties

Rao's quadratic entropy and its variants have been deployed in a range of empirical contexts:

• Open-source software: Fork entropy quantifies diversity among GitHub forks based on kernel distances of file-modification fingerprint vectors. Observed fork entropy is positively correlated with project productivity, pull-request acceptance rates, and negatively with bug issue rates (Wang et al., 2022).
• Engineering design: The unbiased Rao index computed on SBERT-embedded SAPPHIRE causal descriptions robustly quantifies conceptual “variety” (Majumder et al., 2024).
• Ecology and biodiversity: Functional and genetic trait dissimilarities are incorporated via $p$6, enabling sensitivity to phylogenetic or niche differences (Eguchi, 2024).

Desirable mathematical axioms—symmetry, continuity, and monotonicity—are empirically validated in these applications, and the index provides a one-number summary that is sensitive both to abundance heterogeneity and to the spectrum of dissimilarity.

7. Limitations and Theoretical Considerations

While Rao’s quadratic entropy enjoys broad applicability, its properties depend critically on the definition and empirical fidelity of $p$7. In computation, choice of dissimilarity metric, embedding model, and abstraction weights can influence results and require context-specific validation (Majumder et al., 2024). Statistical estimation is straightforward under simple random sampling but requires bias correction for small samples. There is no requirement for $p$8 to satisfy the triangle inequality; any symmetric, nonnegative matrix with zeros on the diagonal is admissible. The estimator's variance decreases as $p$9, and in the large-population limit, plug-in and unbiased estimators coincide.

In information-geometric settings, the invertibility and positivity of $d_{ij}$0 are necessary for the existence of the maximizing $d_{ij}$1, and these restrictions must be considered in ecological modeling.

---

Rao’s quadratic entropy provides a mathematically robust, axiomatically sound, and computationally transparent measure for quantifying diversity in settings where both frequency and dissimilarity among types are non-negligible. Its versatility and foundational character make it central to modern studies of variety, disparity, and informational geometry (Wang et al., 2022, Majumder et al., 2024, Eguchi, 2024).