Thresholded Entropy: Foundations & Applications

• Thresholded entropy is a method that integrates entropy calculations with threshold operations to optimally segment images and analyze data distributions.
• It includes diverse formulations such as Shannon, Tsallis, and Kaniadakis, each driving specific segmentation approaches and abrupt image transitions.
• Practical applications range from histogram-based threshold selection in remote sensing to entropy-aware token pruning in language models.

Thresholded entropy is not a single standardized entropy functional. In the arXiv literature, it denotes several related constructions in which entropy is coupled to a threshold, a thresholded representation, or a threshold regime. The dominant usage is histogram-based image segmentation, where a gray-level threshold is selected by maximizing or minimizing an entropy criterion computed from the classes induced by the threshold. Closely related usages include low-entropy multiplicative estimation when the entropy itself is near a critical threshold, entropy computed on already thresholded binary signals, entropy behavior at physical transition thresholds, and threshold-free alternatives that reject arbitrary cutoff selection (Katiyar et al., 2013, Sparavigna, 2015, Golia et al., 2022, Janik, 2019, Kang et al., 2021, Felippe et al., 2021).

1. Conceptual scope and terminological ambiguity

In image analysis, thresholded entropy usually means selecting one or more segmentation thresholds from an image histogram by optimizing an information measure. The histogram is interpreted as a probability distribution, and threshold selection is posed as an information-theoretic optimization problem. This usage appears in grayscale thresholding, multilevel segmentation, remote sensing, and entropy-based edge detection (Katiyar et al., 2013, El-Sayed, 2012).

Outside image processing, the same phrase or closely related language is used more loosely. In quantitative information flow, the relevant issue is the low-entropy or “thresholded regime,” where additive error is not meaningful because the true entropy can be very close to zero (Golia et al., 2022). In studies of binary configurations, entropy may be computed after the data have already been thresholded into binary states, so the threshold enters as a preprocessing step rather than as the optimization variable (Janik, 2019). In explosive percolation, entropy is analyzed at the physical transition threshold itself (Kang et al., 2021).

Several papers also delimit what thresholded entropy is not. One work on Pearson correlation matrices argues that thresholding is an arbitrary, information-losing preprocessing step and proposes a threshold-free entropy directly on the full matrix (Felippe et al., 2021). Another survey of discrete entropy under thinning states explicitly that it does not define a quantity named thresholded entropy, even though thinning and truncation are closely related to thresholded or truncated behavior (Johnson, 2015). A polymatroidal study of the entropy region likewise does not use the phrase as a named concept, but introduces a truncation operator

$f_{L,t}^*(I)=\min\{f(I),\, f(L\cup I)-t\},$

with the case $L=N$ explicitly called truncation (Matúš et al., 2013).

2. Histogram-based threshold optimization

The canonical thresholded-entropy construction starts from a grayscale histogram. If gray level $x_i$ occurs $N_i$ times in an image with $N$ pixels, then

$f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$

A threshold $t$ partitions the histogram into two classes, $A=\{x_1,\dots,x_t\}$ and $B=\{x_{t+1},\dots,x_g\}$, with cumulative probabilities

$$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$

In the Shannon case, the class entropies are

$L=N$0

and the optimal threshold is chosen by maximizing $L=N$1 (Sparavigna, 2015).

A remote-sensing comparison treats thresholding as an image segmentation problem in which an image histogram is interpreted as a probability distribution and an optimal threshold is chosen by maximizing an entropy criterion. The implementations were carried out in Matlab, modified to allow entropy variation, and applied to LISS III and LISS IV satellite images of the Bhopal area, with reference information from ERDAS Imagine, Google Earth, and DGPS survey data. Thresholded outputs were evaluated by Average Score and Correlation Coefficient, with higher values preferred. For one dataset, Tsallis gave Average Score $L=N$2 and Correlation $L=N$3, Rényi gave $L=N$4 and $L=N$5, and Shannon gave $L=N$6 and $L=N$7. For another dataset, Tsallis gave $L=N$8 and $L=N$9, Rényi gave $x_i$0 and $x_i$1, and Shannon gave $x_i$2 and $x_i$3. The authors’ conclusion was that Tsallis entropy is comparatively more suitable for thresholding, followed by Rényi, then Shannon (Katiyar et al., 2013).

Entropy thresholding has also been embedded in edge-detection pipelines. One algorithm first computes a global threshold $x_i$4 by Shannon entropy, then computes two local thresholds $x_i$5 and $x_i$6 by Tsallis entropy with $x_i$7, builds a binary image from the combined thresholds, and finally applies a $x_i$8 edge detector based on local homogeneity. If the count of equal-valued pixels in the window satisfies $x_i$9, the center pixel is treated as non-edge; otherwise it is labeled as edge. The paper reports comparisons against Canny, LOG, and Sobel, emphasizing robustness, flexibility, and reduced computation time, while also noting thick edges and sensitivity to the choice of $N_i$0 (El-Sayed, 2012).

3. Generalized entropy families and parameter-driven image transitions

Entropy thresholding in imaging has expanded well beyond Shannon entropy. The main formulations represented in the cited literature differ not only in the entropy functional but also in whether the threshold is chosen by maximization or minimization, whether the entropy is additive or generalized, and whether extra uncertainty components are introduced (Sparavigna, 2015, Patrascu, 2019, Chakraborty et al., 2021).

Formulation	Threshold criterion	Reported feature
Shannon / Rényi / Tsallis	Maximize class entropy from histogram	Remote-sensing comparison favored Tsallis
Tsallis non-extensive entropy	Maximize $N_i$1	$N_i$2 can induce abrupt image transitions
Kaniadakis entropy	Maximize total $N_i$3 from class entropies	$N_i$4 recovers Shannon
Neutrosophic Shannon entropy	Select local minima of $N_i$5	Supports multiple thresholds
t-entropy	Maximize $N_i$6	Bounded arctan-based entropy

Tsallis thresholding is the best-developed example of parameter-sensitive thresholded entropy. For a threshold $N_i$7, the class entropies are

$N_i$8

and the total entropy is

$N_i$9

with

$N$0

The reported distinctive phenomenon is that, for some images and $N$1, the optimal threshold $N$2 changes abruptly rather than smoothly. In the lower blood-cells image, the threshold is around $N$3 below a critical point and jumps to about $N$4 above $N$5, causing the bi-level image to change abruptly; analogous transitions are also reported in three-level thresholding, including a critical value around $N$6 for one blood-cells image and around $N$7 for a lower blood-cell image (Sparavigna, 2015). A companion Kaniadakis-thresholding paper describes the entropic index as determining the threshold and driving an abrupt “image transition” in the appearance of the corresponding bi-level image (Sparavigna, 2015).

Kaniadakis thresholding uses

$N$8

with $N$9 yielding Shannon entropy. For the two thresholded classes, the paper defines $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$0 and $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$1, combines them through a generalized composition rule, and again chooses $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$2 by entropy maximization. On the tested images, Tsallis and Kaniadakis produce the same or very similar thresholds and segmentations. Reported examples include Lena, where the best threshold is $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$3 for both, Cameraman, where Tsallis gives $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$4 and Kaniadakis $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$5, and cell and blood images with thresholds such as $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$6, $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$7, $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$8, and $f_i=\frac{N_i}{N}, \qquad \sum_{i=1}^g f_i = 1.$9 (Sparavigna, 2015).

Neutrosophic Shannon entropy changes the thresholding problem more radically. For each candidate threshold $t$0, every gray level $t$1 is assigned a degree of truth $t$2, neutrality $t$3, and falsity $t$4, together with bifuzzy undefinedness $t$5 and contradiction $t$6. These are converted into escort fuzzy degrees $t$7 and $t$8, which define a Shannon-style entropy

$t$9

The total objective is

$A=\{x_1,\dots,x_t\}$0

and the optimal threshold or thresholds are the local minima of $A=\{x_1,\dots,x_t\}$1, not maxima. The paper reports one clear local minimum for ball and multiple local minima, hence multilevel segmentations, for block, mammography, and spider (Patrascu, 2019).

The t-entropy framework introduces

$A=\{x_1,\dots,x_t\}$2

a bounded entropy whose maximum occurs at the uniform distribution and whose value is at most $A=\{x_1,\dots,x_t\}$3. In multilevel thresholding, it replaces Shannon entropy in the posterior objective

$A=\{x_1,\dots,x_t\}$4

optimized by Differential Evolution. On BSDS500, the paper reports that t-entropy with $A=\{x_1,\dots,x_t\}$5 achieved $A=\{x_1,\dots,x_t\}$6, $A=\{x_1,\dots,x_t\}$7, and $A=\{x_1,\dots,x_t\}$8, compared with Shannon’s $A=\{x_1,\dots,x_t\}$9, $B=\{x_{t+1},\dots,x_g\}$0, and $B=\{x_{t+1},\dots,x_g\}$1 in the same setup (Chakraborty et al., 2021).

4. Threshold-free reformulations and critiques of arbitrary cutoffs

A major criticism of thresholded entropy arises in the analysis of Pearson correlation matrices. A common preprocessing step converts a correlation matrix $B=\{x_{t+1},\dots,x_g\}$2 into an adjacency matrix by thresholding,

$B=\{x_{t+1},\dots,x_g\}$3

after which one computes a graph-based entropy. The critique is fourfold: information loss, arbitrariness of the threshold $B=\{x_{t+1},\dots,x_g\}$4, discontinuity introduced by the Heaviside step function, and indirectness because the entropy is then measured on graph topology rather than on the original correlation matrix (Felippe et al., 2021).

The threshold-free alternative rescales the Pearson matrix by its dimension,

$B=\{x_{t+1},\dots,x_g\}$5

and interprets $B=\{x_{t+1},\dots,x_g\}$6 as an analog of a density matrix. Because $B=\{x_{t+1},\dots,x_g\}$7 is real and symmetric, has unit diagonal, and is positive semidefinite, $B=\{x_{t+1},\dots,x_g\}$8 is Hermitian, unit trace, and positive semidefinite. Its eigenvalues $B=\{x_{t+1},\dots,x_g\}$9 are therefore nonnegative and sum to $$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$0, allowing the definition

$$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$1

If $$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$2, then $$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$3; if all variables are perfectly correlated, $$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$4. The empirical demonstration uses $$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$5 resting-state BOLD time series from nine healthy volunteers before and after ingestion of ayahuasca, and reports that entropy generally increases after ayahuasca (Felippe et al., 2021).

This threshold-free line directly contests a common misconception: entropy associated with correlations need not be defined only after binarizing the correlations. A plausible implication is that some applications of thresholded entropy are best understood as expedient graph surrogates rather than as intrinsic entropy measures on the original data.

5. Threshold regimes, low-entropy estimation, and critical phenomena

In quantitative information flow, the central issue is not threshold selection in the data space but reliable estimation when the entropy itself is below a practically relevant scale. One work studies Shannon entropy estimation in a probability-revealing conditional sampling model and targets multiplicative $$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$6-approximation in the low-entropy regime. Given tolerance $$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$7, confidence $$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$8, and $$P_A=\sum_{i=1}^{t} f_i, \qquad P_B=\sum_{i=t+1}^{g} f_i.$$9, the estimator returns $L=N$00 such that

$L=N$01

using only

$L=N$02

queries, independently of $L=N$03. The paper identifies a structural threshold at entropy $L=N$04: if $L=N$05, then there must exist $L=N$06 such that $L=N$07; if all masses satisfy $L=N$08, then $L=N$09. This regime split enables variance bounds that avoid the $L=N$10 dependence present in earlier multiplicative estimators (Golia et al., 2022).

A different threshold phenomenon appears in explosive percolation. There the entropy is

$L=N$11

where $L=N$12 is the probability that a randomly chosen cluster has size $L=N$13. In Erdős–Rényi percolation, entropy is symmetric around the threshold and reaches its maximum at $L=N$14. In the da Costa–type explosive model, however, the scaling functions below and above the threshold differ:

$L=N$15

with opposite signs in the small-$L=N$16 expansions of $L=N$17 and $L=N$18. The consequence is that entropy is not maximal at the threshold, $L=N$19, and the second derivative diverges to $L=N$20 from the left and $L=N$21 from the right (Kang et al., 2021).

Related but distinct constructions appear in discrete probability and entropy geometry. The thinning operator

$L=N$22

is presented as the closest analogue to thresholded or truncated behavior in one survey, which also proves monotonicity of entropy under thinning for IID ultra-log-concave sums (Johnson, 2015). At the level of entropy regions, the truncation formula

$L=N$23

provides a threshold-like operation on polymatroid rank functions (Matúš et al., 2013). These usages suggest that thresholded entropy can denote either an operational threshold in data processing or a structural threshold in the behavior of entropy itself.

6. Thresholded representations in binary data and token pipelines

In machine-learning-based entropy estimation, the threshold may precede the entropy calculation. For a binary vector $L=N$24, the Shannon entropy

$L=N$25

is factorized through the exact chain rule

$L=N$26

which turns entropy estimation into a sequence of supervised binary classification tasks. Each term is a conditional cross-entropy. The method is explicitly intended for binary configurations or thresholded signals, including Ising spins, spike trains, and other multidimensional binary signals. The paper validates the method on synthetic Boolean data and on a $L=N$27 periodic 2D Ising model using 20,000 Monte Carlo configurations per temperature, 5-fold cross-validation, XGBoost as the main nonlinear classifier, and symmetry-based data augmentation at low temperature (Janik, 2019).

A recent large-language-model preprint uses thresholded entropy in yet another sense: entropy-aware token pruning. Each token receives an information energy

$L=N$28

with default weights $L=N$29, $L=N$30, and $L=N$31, and the system is cooled by

$L=N$32

In deterministic mode, the retained count is

$L=N$33

so the algorithm keeps the top-$L=N$34 tokens by energy; in Boltzmann mode, survival is governed by

$L=N$35

Compression stops when the energy-weighted similarity

$L=N$36

falls below a fidelity threshold $L=N$37. The paper reports roughly $L=N$38 prompt compression across five prompt categories while maintaining $L=N$39, an additional $L=N$40 savings from context deduplication on repeated blocks, and about $L=N$41 reduction from output-side quenching (Agyemang et al., 2 Jun 2026).

This suggests an emerging extension of thresholded entropy from histogram segmentation to representation pruning. In one branch, the threshold acts on gray levels, cluster scales, or candidate cut points; in another, it acts on features, symbols, or tokens after they have been assigned an entropy-derived importance score.