Balanced-Information Score (BIS) in Clustering

• Balanced-Information Score (BIS) is a normalized index, based on the Boltzmann–Shannon Index, that measures cluster balance by comparing frequency-based and SVD-derived geometry distributions.
• It computes one minus the Jensen–Shannon divergence between these two discrete distributions, effectively capturing how cluster sizes align with their geometric extents.
• BIS is computationally efficient and gradient-friendly, making it suitable as a regularizer in clustering, fairness, and resource-allocation optimization tasks.

Searching arXiv for the cited papers and topic to ground the article. Searching arXiv for "(Bossi et al., 2 Dec 2025) Boltzmann-Shannon Index" Searching arXiv for "(Shi et al., 2 Aug 2025) Importance Sampling is All You Need BIS" Balanced-Information Score (BIS), as referred to in the query, corresponds to the Boltzmann–Shannon Index (BSI), a normalized measure for clustered continuous data that captures the interaction between frequency-based and geometry-based probability distributions (Bossi et al., 2 Dec 2025). It is defined on a fixed partition of a dataset and evaluates how well cluster populations align with each cluster’s effective geometric extent. In the formulation introduced in "Boltzmann-Shannon Index: A Geometric-Aware Measure of Clustering Balance" (Bossi et al., 2 Dec 2025), the score is obtained as one minus the Jensen–Shannon divergence between a frequency-based distribution over clusters and a geometry-based distribution derived from singular-value-based volume surrogates.

1. Nomenclature and scope

The term Balanced-Information Score (BIS) is not the name used in the source paper. The quantity introduced on arXiv is the Boltzmann–Shannon Index (BSI), and the exposition provided for this topic states that it is “sometimes referred to in your query as the Balanced-Information Score (BIS)” (Bossi et al., 2 Dec 2025). In this sense, BIS is an alternate query label rather than the paper’s formal terminology.

This distinction matters because the acronym “BIS” is also used in an unrelated 2025 code-generation evaluation paper, "Importance Sampling is All You Need: Predict LLM’s performance on new benchmark by reusing existing benchmark" (Shi et al., 2 Aug 2025). That paper does not introduce or use any quantity called “Balanced-Information Score (BIS),” and its core concepts are importance sampling, Importance-Weighted Autoencoders, weight truncation, marginal expectation, theoretical error bounds, pseudocode, and empirical prediction of LLM benchmark performance. A common source of confusion is therefore purely terminological: in the clustering context, BIS refers to the Boltzmann–Shannon Index; in the code-evaluation context, “BIS” denotes a prompt-centric evaluation framework unrelated to clustering balance (Shi et al., 2 Aug 2025).

Within its intended scope, BSI is designed for clustered continuous data and evaluates the consistency between two induced discrete distributions on cluster labels: one from cluster frequencies and one from cluster geometry. This suggests that BIS is best understood not as a generic clustering-quality metric, but as a balance metric targeted at size–volume proportionality.

2. Mathematical definition

Let $X=\{x_1,\dots,x_N\}\subset\mathbb{R}^d$ be a dataset already partitioned into $K$ disjoint groups, with labels $\ell_i\in\{1,\dots,K\}$. The construction associates two discrete probability distributions on $\{1,\dots,K\}$ (Bossi et al., 2 Dec 2025).

The first is the frequency-based distribution

$$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$

The second is the geometry-based distribution $q=(q_1,\dots,q_K)$. For each cluster $i$, let $X^{(i)}$ be the $n_i\times d$ matrix whose rows are the $d$-dimensional points in cluster $K$0, possibly after centering. Its singular value decomposition is

$K$1

The geometric volume surrogate is then

$K$2

With $K$3 and $K$4 defined, the Boltzmann–Shannon Index is

$K$5

where

$K$6

The formal role of the score is therefore precise: it measures the divergence between cluster frequencies and cluster volumes, then converts that divergence into a normalized agreement score by subtracting from $K$7.

3. Normalization, interpretation, and core properties

For any two discrete distributions $K$8 over the same alphabet,

$K$9

with $\ell_i\in\{1,\dots,K\}$0 if and only if $\ell_i\in\{1,\dots,K\}$1, and $\ell_i\in\{1,\dots,K\}$2 when $\ell_i\in\{1,\dots,K\}$3 and $\ell_i\in\{1,\dots,K\}$4 have disjoint support. Hence

$\ell_i\in\{1,\dots,K\}$5

so $\ell_i\in\{1,\dots,K\}$6 (Bossi et al., 2 Dec 2025).

The endpoints admit a direct interpretation. $\ell_i\in\{1,\dots,K\}$7 if and only if $\ell_i\in\{1,\dots,K\}$8, meaning perfect alignment of cluster sizes and cluster volumes. $\ell_i\in\{1,\dots,K\}$9 if and only if $\{1,\dots,K\}$0, corresponding to extremely inverted or disjoint frequency/geometry and hence maximal imbalance. The score is symmetric, since $\{1,\dots,K\}$1, and bounded in $\{1,\dots,K\}$2 (Bossi et al., 2 Dec 2025).

The paper emphasizes sensitivity to size–volume mismatch: any departure of $\{1,\dots,K\}$3 from $\{1,\dots,K\}$4, whether a small cluster occupying large volume or a large cluster occupying small volume, yields a strictly smaller BSI. This distinguishes the measure from purely frequency-based summaries. In the Iris example, Shannon entropy of sizes is close to $\{1,\dots,K\}$5 but is described as insensitive to geometric spread, whereas BSI is near $\{1,\dots,K\}$6 only when both cluster sizes and cluster volumes are nearly equal in $\{1,\dots,K\}$7D (Bossi et al., 2 Dec 2025). A plausible implication is that BIS should be interpreted as a proportionality criterion rather than a dispersion criterion.

A notable special case is the two-cluster reversal lemma. For $\{1,\dots,K\}$8, with

$\{1,\dots,K\}$9

the score becomes

$$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$0

the binary entropy of a Bernoulli$$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$1 (Bossi et al., 2 Dec 2025). This curve peaks at $$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$2 with $$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$3 and falls to $$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$4 as $$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$5 or $$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$6.

4. Computational procedure and complexity

Given $$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$7, the computational procedure is analytic and does not require Monte Carlo sampling (Bossi et al., 2 Dec 2025). The prescribed steps are:

1. Compute cluster sizes $$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$8 and $$p_i \;=\; \frac{1}{N}\,\bigl|\{\,x_j : \ell_j=i\}\bigr|\;\,, \qquad \sum_{i=1}^K p_i=1.$$9.
2. For each $q=(q_1,\dots,q_K)$0: a. Form the $q=(q_1,\dots,q_K)$1 data matrix $q=(q_1,\dots,q_K)$2. b. Optionally center and compute its SVD at cost $q=(q_1,\dots,q_K)$3, or $q=(q_1,\dots,q_K)$4. c. Extract singular values $q=(q_1,\dots,q_K)$5 and form $q=(q_1,\dots,q_K)$6.
3. Normalize $q=(q_1,\dots,q_K)$7.
4. Form $q=(q_1,\dots,q_K)$8 and evaluate

$q=(q_1,\dots,q_K)$9

in $i$0 time.

1. Return

$i$1

The overall time is

$i$2

The exposition states that no Monte Carlo is needed because the SVD-based surrogate is analytic and always defined, even for overlapping or nonconvex clusters (Bossi et al., 2 Dec 2025).

This computational profile is important for interpretation. The score depends on a specific choice of geometry surrogate, namely the product of singular values. It is therefore not an abstract information-theoretic functional alone, but an information-theoretic functional coupled to an SVD-based coarse-graining of cluster extent.

5. Empirical behavior on benchmark and synthetic settings

The paper illustrates the behavior of BSI on synthetic Gaussian mixtures, the Iris benchmark, and a resource-allocation fairness setting (Bossi et al., 2 Dec 2025).

For synthetic $i$3D Gaussian mixtures with $i$4, three isotropic Gaussians produce a progression described as follows: balanced and well-separated yields BSI approximately high and close to $i$5; moderate size imbalance with the same variance yields BSI approximately intermediate; and strong size imbalance with overlap yields BSI approximately low. The stated interpretation is that BSI tracks how population proportions align with cluster spreads in $i$6.

For the Iris benchmark with $i$7, the ground-truth labels have $i$8 points each, so

$i$9

The best K-means run over $X^{(i)}$0 restarts yields BSI (proposed) $X^{(i)}$1 and BSI (true labels) $X^{(i)}$2 (Bossi et al., 2 Dec 2025). The same exposition reports comparison values of Silhouette approximately $X^{(i)}$3, Calinski–Harabasz approximately $X^{(i)}$4, Davies–Bouldin approximately $X^{(i)}$5, and Shannon entropy of sizes approximately $X^{(i)}$6 bits, close to $X^{(i)}$7, while remaining insensitive to geometric spread. The conclusion drawn there is that BSI is approximately $X^{(i)}$8 only when both cluster sizes and cluster volumes are nearly equal in $X^{(i)}$9D.

In the resource-allocation fairness example under extreme imbalance, the population shares are

$n_i\times d$0

A resource-allocation vector $n_i\times d$1 is defined by

$n_i\times d$2

Using $n_i\times d$3 generated $n_i\times d$4D points whose SVD-volume surrogates exactly reflect $n_i\times d$5, the reported values are: $n_i\times d$6 gives BSI approximately $n_i\times d$7, $n_i\times d$8 gives BSI approximately $n_i\times d$9, and $d$0 gives BSI approximately $d$1 (Bossi et al., 2 Dec 2025). The paper characterizes the resulting curve as smooth and monotonic, indicating that BSI can quantify degrees of fairness in a continuous, differentiable way.

6. Optimization use, significance, and limitations of interpretation

Because $d$2 is differentiable as a function of an allocation vector $d$3, the paper proposes its use as an objective or penalty term in constrained optimization (Bossi et al., 2 Dec 2025). If $d$4 parameterizes resource shares or cluster-prior weights, one may maximize BSI subject to budget or other constraints. The gradient is written as

$d$5

together with the chain-rule derivatives of $d$6 and $d$7.

The accompanying pseudo-code recipe for a gradient step is:

1. Compute $d$8.
2. Compute

$d$9

and

$K$00

1. Chain-rule through $K$01 and $K$02 to get $K$03.
2. Take a gradient step:

$K$04

The significance assigned to this construction is that BSI can serve as a smooth, gradient-friendly regularizer that is easily embedded in modern policy-making and algorithmic governance optimization frameworks (Bossi et al., 2 Dec 2025). This suggests a broader role for BIS beyond descriptive cluster analysis: it can function as a differentiable proportionality objective whenever one distribution encodes demographic or frequency information and another encodes geometric or allocation structure.

At the same time, the interpretation of BSI is specific. It rewards partitions whose cluster populations $K$05 and cluster volumes $K$06 are in precise proportion, and its geometry term is explicitly built from an SVD-based volume surrogate (Bossi et al., 2 Dec 2025). A plausible implication is that high BIS should not be read as a universal certificate of clustering quality; rather, it indicates agreement between frequency and effective geometric extent under the paper’s chosen geometric coarse-graining.