Boltzmann-Shannon Index: A Geometric-Aware Measure of Clustering Balance

Abstract: We introduce the Boltzmann-Shannon Index (BSI), a normalized measure for clustered continuous data that captures the interaction between frequency-based and geometry-based probability distributions. Building on ideas from geometric coarse-graining and information theory, the BSI quantifies how well a partition reflects both the population of each cluster and its effective geometric extent. We illustrate its behavior on synthetic Gaussian mixtures, the Iris benchmark, and a high-imbalance resource-allocation scenario, showing that the index provides a coherent assessment even when traditional metrics give incomplete or misleading signals. Moreover, in resource-allocation settings, we demonstrate that BSI not only detects severe density-geometry inconsistency with high sensitivity, but also offers a smooth, optimization-ready objective that naturally favors allocations balancing demographic weight with each group's effective spread in the outcome space, while providing a smooth, gradient-friendly regularizer that can be easily embedded in modern policy-making and algorithmic governance optimization frameworks.