Entropy Thresholding in Image Segmentation

• Entropy thresholding is a technique that determines optimal image thresholds by maximizing or minimizing entropy values computed from gray-level histograms.
• It employs various entropy measures, including Shannon, Tsallis, and Kaniadakis, to enhance contrast sensitivity and robustness against noise.
• Practical implementations use both exhaustive search and metaheuristic algorithms to achieve efficient segmentation in diverse applications such as biomedical imaging and deep learning.

Entropy thresholding is a methodological paradigm in information theory, signal processing, and computer vision in which optimal thresholds are determined by the maximization (or, in some settings, minimization) of an entropy functional defined over a partitioned data representation. The principal application domain is image segmentation, where the gray-level histogram of an image is divided into classes by one or more grayscale thresholds such that the entropy of these classes—categorized by Shannon, Tsallis, Kaniadakis, fuzzy, or related quantities—is extremized, yielding a robust, nonparametric rule for separating foreground and background, object and background, or multiple tissue or texture regions. The entropy measure’s parametric flexibility (notably in the Tsallis and Kaniadakis families) allows fine control over contrast sensitivity and robustness to noise or non-uniform illumination, with wide applicability ranging from edge detection, color segmentation, to collaborative label correction in noisy-label deep neural networks.

1. Entropy Measures for Histogram-Based Thresholding

Entropy thresholding frameworks operate on the discrete probability mass function $p = \{p_0,\ldots,p_g\}$ obtained by normalizing an image’s gray-level histogram. The foundational entropy models are:

• Shannon Entropy: $H_S(p) = -\sum_{i=0}^g p_i \ln p_i$, parameter-free and strictly additive.
• Tsallis Entropy (entropic index $q \neq 1$): $H_q(p) = \frac{1}{q-1}(1-\sum_{i=0}^g p_i^q)$, which recovers Shannon entropy as $q \to 1$. The index $q$ modulates sensitivity to tail probabilities and is non-additive: $S_q(A+B) = S_q(A) + S_q(B) + (1-q)S_q(A)S_q(B)$ (Sparavigna, 2015, Sparavigna, 2015, El-Sayed et al., 2014).
• Kaniadakis Entropy (entropic index $\kappa, |\kappa|<1$): $$H_\kappa(p) = -\frac{1}{2\kappa}\sum_{i=0}^g(p_i^{1+\kappa} - p_i^{1-\kappa})$$, reducing to Shannon’s form as $\kappa \to 0$, with a deformed, symmetric algebra for class composition (Sparavigna, 2015, Sparavigna, 2015).
• Type-II Fuzzy Entropy: Interval-valued membership functions assigned to histogram bins, with entropy defined over the induced “ultra-fuzzy” probability distributions (Nag, 2017).
• Neutrosophic Shannon Entropy: Entropy of neutrosophic classes (truth, falsity, neutrality) for each gray level, integrating degrees of affiliation and uncertainty (Patrascu, 2019).

The entropy functional provides the criterion for evaluating a candidate threshold or set of thresholds, with variations for bi-level, multi-level, and multi-modal optimization.

2. Mathematical Formulation and Threshold Selection

The generic workflow consists of:

1. Histogram Partition: For a candidate threshold $H_S(p) = -\sum_{i=0}^g p_i \ln p_i$0, split gray levels into two classes:
   • Class A: $H_S(p) = -\sum_{i=0}^g p_i \ln p_i$1, with probability $H_S(p) = -\sum_{i=0}^g p_i \ln p_i$2.
   • Class B: $H_S(p) = -\sum_{i=0}^g p_i \ln p_i$3, with probability $H_S(p) = -\sum_{i=0}^g p_i \ln p_i$4.
   • Normalize to class-PMFs: $H_S(p) = -\sum_{i=0}^g p_i \ln p_i$5, $H_S(p) = -\sum_{i=0}^g p_i \ln p_i$6.
2. Class Entropy Calculation: Compute class-wise entropies $H_S(p) = -\sum_{i=0}^g p_i \ln p_i$7 according to the chosen entropy type (see Table 1).

Entropy Type	Class Entropy Formula	Additive/Combinatorial Rule
Shannon	$H_S(p) = -\sum_{i=0}^g p_i \ln p_i$8	$H_S(p) = -\sum_{i=0}^g p_i \ln p_i$9
Tsallis	$q \neq 1$0	$q \neq 1$1
Kaniadakis	$q \neq 1$2	$q \neq 1$3

1. Objective Function Maximization: The optimal threshold $q \neq 1$4 maximizes the combined entropy functional. For multi-level cases, all possible combinations of $q \neq 1$5 thresholds are considered, sometimes with heuristics (e.g., artificial bee colony, APPA).

For fuzzy and neutrosophic generalizations, interval-valued membership or degrees augment the classical PMF, leading to more complex entropy objectives (Nag, 2017, Patrascu, 2019).

3. Algorithmic Implementation and Computational Strategies

The algorithmic core is an exhaustive or stochastic search over candidate thresholds, evaluating the entropy objective at each step:

• Exhaustive Histogram Scan: For bi-level thresholding, the entropy functional is computed for each $q \neq 1$6; the maximizer is selected directly (Sparavigna, 2015, Sparavigna, 2015).
• Multi-Level/Multimodal: To avoid the $q \neq 1$7 scaling, population-based metaheuristics are often employed:
  • Adaptive Plant Propagation Algorithm (APPA): Runner-based memetic exploration for the non-concave, high-dimensional fuzzy entropy surface (Nag, 2017).
  • Multi-Objective Swarm Optimization: Vectorized entropy (and mixed entropy–variance) criteria across channels or image regions, optimized for Pareto efficiency (Boldaji et al., 2021).
• 2D Histogram Extensions: Operating on joint (gray value, local mean) histograms, two-dimensional Tsallis-based thresholding provides higher noise immunity and improved segmentation (El-Sayed et al., 2014).
• Practical Heuristics: For computational efficiency, symmetric thresholds (e.g., $q \neq 1$8 along the diagonal) and quantized search spaces are standard.

Specialized implementations include MATLAB vectorization, pruning zero-count bins, and, where needed, parallelization for real-time processing (El-Sayed, 2012).

4. Applications and Empirical Performance

Entropy thresholding techniques are widely used in:

• Bi-level and Multi-level Image Segmentation: Segmentation of standard images (“Lena”, “Cameraman”), biomedical (microscopic cells, blood smears), and remote sensing data, frequently evaluated by edge count or information preservation post-thresholding (Sparavigna, 2015, Sparavigna, 2015, El-Sayed et al., 2014).
• Edge Detection: Entropy-based threshold maps, coupled with homogeneity-based local operators, yield competitive or superior performance to classical gradient methods (e.g., Canny, LoG), particularly with fine-scale texture (El-Sayed, 2012).
• Noisy Label Correction in Deep Learning: Shannon entropy of predicted posteriors is thresholded (using an empirically chosen entropy cutoff) to identify high-confidence predictions during robust training with noisy supervision. Entropy thresholding retains more training samples and avoids discarding potentially informative data (Wu et al., 2021).
• Color Image Segmentation: Vectorized entropy and hybrid (entropy + variance) objectives embedded in multi-objective optimizers segment color images with reduced threshold storage and increased region homogeneity (Boldaji et al., 2021).
• Correlation Entropy Estimation: In contrast to classical threshold-based adjacency construction from correlation matrices, threshold-free approaches rescale the Pearson matrix, compute its eigenspectrum, and evaluate the von Neumann type entropy—eliminating arbitrariness and discontinuities (Felippe et al., 2021).

Empirical findings indicate that generalized entropies (Tsallis, Kaniadakis) with appropriately chosen parameters ($q \neq 1$9) can yield modest improvements over Shannon, particularly in challenging or noisy settings, with the caveat of requiring parameter tuning (Sparavigna, 2015, Sparavigna, 2015, El-Sayed et al., 2014).

5. Theoretical and Practical Advantages, Limitations, and Parameterization

Advantages:

• Adaptivity: Parametric forms (Tsallis, Kaniadakis) offer control over sensitivity to intensity distribution tails, enabling texture discrimination and contrast enhancement.
• No Distributional Assumptions: Methods require only histogram data; no model of the underlying image-generating process is needed.
• Computational Efficiency: With histogram-based operations and, for bi-level thresholding, per-threshold evaluations in $H_q(p) = \frac{1}{q-1}(1-\sum_{i=0}^g p_i^q)$0, methods are real-time compatible for standard image sizes (El-Sayed, 2012).
• Flexible Generalization: Extensions to fuzzy, neutrosophic, 2D, and color histogram-based entropies admit applications to multilevel segmentation, uncertainty modeling, and complex scenes (Nag, 2017, Patrascu, 2019, Boldaji et al., 2021).

Limitations:

• Parameter Calibration: Generalized entropic indices require empirical tuning (usually by secondary quality criteria or cross-validation). Parameter sweeping in $H_q(p) = \frac{1}{q-1}(1-\sum_{i=0}^g p_i^q)$1 is standard; extreme values may induce unwanted “image transitions” or excessive region uniformity (Sparavigna, 2015).
• Singularities and Stability: Sharp transitions in optimal threshold (as $H_q(p) = \frac{1}{q-1}(1-\sum_{i=0}^g p_i^q)$2 or $H_q(p) = \frac{1}{q-1}(1-\sum_{i=0}^g p_i^q)$3 varies) can induce abrupt changes in segmentation. This “first-order-like” behavior may be desirable or problematic depending on the context (Sparavigna, 2015, Sparavigna, 2015).
• No Spatial Context: Classical entropy thresholding treats pixel intensities as independent; incorporating Markov, spatial, or prior models is necessary for spatially coherent segmentations in highly textured or noisy data (Patrascu, 2019).
• Scalability in Multi-Level Problems: Exhaustive enumeration is infeasible as class number increases (curse of dimensionality), mandating approximate or metaheuristic optimization (Nag, 2017, Boldaji et al., 2021).

6. Extensions and Future Research Directions

Recent work explores several important directions:

• Threshold-Free Entropy Estimation: Direct entropy metrics on correlation matrices (e.g., Pearson, mutual information) using von Neumann formalism eliminate the need for explicit threshold selection, with applications in brain connectivity analysis (Felippe et al., 2021).
• Hybridization with Other Criteria: Combining entropy-based objectives with Otsu’s between-class variance or region homogeneity enables multi-objective optimization frameworks suitable for Pareto filtering and robust segmentation (Boldaji et al., 2021).
• Fuzzy and Neutrosophic Approaches: Modeling uncertainty and transition regions via interval-valued or neutrosophic extensions yields more robust handling of noisy or low-contrast boundaries, especially in multi-phase images (Nag, 2017, Patrascu, 2019).
• Optimization Methodologies: Plant propagation, swarm intelligence, and bio-inspired algorithms are increasingly replacing traditional search—offering scalability, rapid convergence, and suitability for high-dimensional threshold vectors (Nag, 2017, Boldaji et al., 2021).

Validation and parameter selection are increasingly grounded in quantitative secondary measures (edge density, classification accuracy, entropy loss) and, for deep learning, generalization metrics on retained or corrected sample sets (Wu et al., 2021).