Dataset Complexity Indicators Overview

Updated 12 December 2025

Dataset Complexity Indicators are quantitative metrics that assess the inherent difficulty of learning tasks by measuring separability, overlap, and dimensionality across various domains.
They integrate methods from spectral analysis, neighborhood, and topology-based metrics to provide classifier-agnostic insights into data structure and learning challenges.
Researchers can leverage these indicators to inform model selection, parameter tuning, and data preprocessing while benchmarking performance across diverse applications.

Dataset complexity indicators are quantitative metrics designed to assess, compare, and predict the inherent difficulty of learning tasks associated with a given dataset, independent of particular classifiers. These indicators span information theory, geometry, spectral graph theory, topology, feature selection, neighborhood analysis, and more, reflecting the multifaceted nature of what makes data “hard” or “easy” for machine learning models. Their development is grounded in both theoretical principles—such as separability, cover numbers, homogeneity, and dimensionality—and practical correlations with empirical learning performance across domains including classification, regression, clustering, knowledge-graph link prediction, image recognition, biological microarrays, and human trajectory prediction.

1. Taxonomy and Foundational Theories

Dataset complexity indicators can be classified according to the structural or statistical property they measure (Lorena et al., 2018). Foundational families include:

Feature-space discriminatory metrics (e.g., Fisher’s discriminant ratio, overlap volume) quantify separability based on marginal/axis-aligned attributes.
Boundary/linearity metrics (e.g., error rate of linear classifier, linearity nonlinearity) reflect the topological complexity or nonlinearity of class boundaries.
Neighborhood/morphology metrics (e.g., 1-NN error, MST-based fraction of borderline points, local set cardinality) capture local structure and overlap.
Network measures (e.g., graph density, clustering coefficient, hub score) reveal graph-theoretic properties of point clouds induced by k-NN or ε-NN graphs.
Dimensionality and imbalance metrics (e.g., PCA dimension ratio, class entropy, imbalance ratio) assess sparsity and bias.
Spectral/theory-rooted indicators utilize the eigenspectrum of similarity graphs to integrate global structure, class mixing, and manifold connectivity (Branchaud-Charron et al., 2019, Li et al., 2022, Gul et al., 2 Sep 2025).
Distributional/divergence-based indicators (e.g., Distance-based Separability Index) compare intra- and inter-class distances using nonparametric divergence statistics (Guan et al., 2021).
Topology/covering-based measures employ geometric partitioning or covering numbers (e.g., shattering coefficient, number of pure-class balls) for fine-grained quantification of overlap and combinatorial complexity (Pascual-Triana et al., 2020, Mello, 2019).

This taxonomy enables targeted analysis; for example, spectral indicators can capture non-local mixing poorly reflected by axis-aligned projections, while cover-number–derived metrics can expose pathologies in highly imbalanced datasets.

2. Spectral, Graph-Laplacian, and Eigenvalue-Based Metrics

Spectral indicators leverage the Laplacian spectrum or graph-theoretic surrogates to aggregate global structural and overlap properties into continuous, often interpretable scores. The principal representatives include:

Cumulative Spectral Gradient (CSG): Formulates a similarity matrix over classes—aggregated via nearest-neighbor statistics in an embedding space—and constructs a normalized Laplacian. The complexity metric is the cumulative sum of its eigengaps, with higher values reflecting stronger intrinsic entanglement of class distributions. Originally correlated with CNN performance (Branchaud-Charron et al., 2019), recent work demonstrates that its predictive power and “scaling with class count” can collapse under different parameterizations (notably, the number of nearest neighbors K), rendering it unreliable in knowledge-graph link prediction settings (Gul et al., 2 Sep 2025).
Cumulative Maximum Scaled Area Under Laplacian Spectrum (cmsAULS): Defines a normalized area under the cumulative maximum curve of scaled Laplacian eigenvalues of a class similarity graph, post-embedding. Empirically, cmsAULS outperforms CSG and a wide array of classical complexity measures in correlating with deep network test error, with stability across image datasets and embedding types (Li et al., 2022). It exhibits key thresholds—cmsAULS ≲ 0.5 (easy, well-separated); cmsAULS ≳ 2.0 (hard, high-overlap).
Principal Cubic Complexes: A topological-graph approach to structural complexity in general clouds, optimizing the fit of a principal graph or higher-dimensional complex by minimizing both data-approximation and elastic energies. Three natural complexity measures arise: geometric complexity (total harmonic deviation), structural complexity (barcode of graph components), and construction complexity (number of elementary graph operations needed to grow the optimal approximator) (Zinovyev et al., 2012).

Empirical best practices recommend parameter sweeps for all neighborhood-sensitive spectral measures, explicit reporting of their dependence on free parameters (such as K, sample size M), and benchmarking against classifier-agnostic alternatives like entropy, margin, and pairwise divergence measures (Gul et al., 2 Sep 2025, Li et al., 2022).

3. Distance-Based, Neighborhood, and Covering Indicators

Distance and neighborhood metrics provide both localized and global insight into class entanglement and boundary shape. Key formulations:

Distance-based Separability Index (DSI): A nonparametric, classifier-agnostic approach that compares the empirical distributions of intra-class versus between-class Euclidean distances using Kolmogorov–Smirnov statistics. DSI approaches 0 for maximally mixed classes and 1 for perfectly separable ones. Unlike many feature- or boundary-based measures, DSI’s invariance under translation, rotation, and scaling, and its direct “if and only if” connection to class-conditional distribution equality, provide theoretical robustness. Across synthetic and real datasets, “1 – DSI” tracks error monotonicity more reliably than standard indicators like F1, N3, or density (Guan et al., 2021).
Overlap Number of Balls (ONB): Quantifies the number of minimal-radius, class-pure “balls” required to cover all points in a class, with the average-class ratio (ONB_avg) serving as a global overlap score. ONB metrics surpass nearest-neighbor measures (N3, N1) in correlating (|ρ| > 0.9) with classifier AUC/GEOM under both class overlap and imbalance (Pascual-Triana et al., 2020).
Morphology and Neighborhood Indicators: MST-based borderline fractions (N1), intra- versus extra-class neighbor distance ratios (N2), leave-one-out 1-NN error (N3), and non-linearity metrics (N4) are included in standard benchmarking (Lorena et al., 2018, Pascual-Triana et al., 2020). Their combination with ONB allows a two-dimensional dissection of local versus global complexity.

These indicators permit precise thresholding for practical action: e.g., ONB < 0.3 indicates sufficiency of simple models, while ONB > 0.6 denotes the need for ensemble or cost-sensitive classifiers (Pascual-Triana et al., 2020).

4. Information-Theoretic, Entropy, and Dimensionality Measures

Information-theoretic and dimensionality metrics offer interpretable, computationally efficient proxies for dataset “hardness,” especially in large image corpora and high-dimensional spaces:

Shannon pixel entropy, GLCM (texture) entropy, delentropy, and intrinsic dimensionality: Used to benchmark image datasets, with clear rank-ordering of autonomous driving and general recognition corpora (Cityscapes < IDD < BDD < Vistas), matching deep network mIoU rankings. High entropy or ID correlates with lower segmentation performance (Rahane et al., 2020). Shannon entropy tracks early network layer learning difficulty; texture and delentropy reflect deeper feature extraction challenges.
Class-entropy and imbalance ratios (C1, C2), PCA-based dimension estimates (T2–T4): Standardized in complexity survey libraries (e.g., ECoL (Lorena et al., 2018)), yielding key context for interpreting other measures and guiding preprocessing via rebalancing or dimensionality reduction.
Regularity-based indicators (trajectory, context, and interaction entropy): Used in human trajectory and motion prediction, these metrics (e.g., unconditional spatial entropy, GMM-based positional modality, efficiency, angular deviation, minimum distance to collision, density) provide an explicit framework for quantifying scene- and context-level predictive uncertainty, crucial for model and benchmark selection (Amirian et al., 2020, Hug et al., 2020).

5. Complexity Indicators in High-Dimensional and Special-Purpose Data

In domains with extreme feature dimensionality or unknown signal interactions—e.g., biological microarrays—feature-robust and interaction-sensitive metrics have been developed:

Depth (GA-wrapped feature subset size): The “depth” indicator quantifies the number of features required to recover fixed percentiles (90–100%) of peak model performance (AUC) using a genetic algorithm wrapper for feature selection. The depth plot, and derived indices (s_90%, s_95%, etc.), succinctly capture the order of feature coupling and robustness to noise/irrelevant features. Depth differentiates uni-variate, additive, and epistatic (non-linear interaction) regimes, which classical metrics fail to resolve in the presence of high-dimensional noise (Sha et al., 2023).
In microarray and genotype data, depth plots empirically reveal that gene expression datasets are dominated by a few highly predictive features (s_90% = 1–3), whereas genotype datasets require dozens of features, indicating greater complexity (Sha et al., 2023).

6. Unified, Composite, and Adaptive Complexity Metrics

Recent work addresses the limitations of single-facet indicators by proposing composite metrics that integrate multiple dataset properties:

Dataset-adaptive, normalized performance metrics: These integrate sample size, feature dimensionality, class imbalance, and signal-to-noise into an adjusted performance score (M*), where sub-factors penalize small sample-per-feature ratios, high imbalance, or low SNR. The adjusted metric converges to asymptotic performance levels more quickly than the raw metric, providing effective early-stage model assessment and cross-dataset comparison (Ossenov, 2024).
These metrics serve as scalable, task- and data-agnostic meta-indicators, suitable for workflow resource allocation, hyperparameter budgeting, and “learning curve” extrapolation.

7. Operationalization, Software, and Practical Considerations

Several established open-source libraries provide implementations for standard complexity indicators: ECoL (22 measures; R) (Lorena et al., 2018), shattering (SLT-combinatorial measures; R) (Mello, 2019); bespoke code for spectral and depth-based methods accompanies respective research (Li et al., 2022, Sha et al., 2023).

Practical guidance includes:

Always use multiple, complementary indicators per dataset (feature-space, boundary, neighborhood, spectral, and dataset-level).
For spectral measures, always report parameter sweeps and curves rather than single values, and benchmark against entropy/divergence baselines (Gul et al., 2 Sep 2025).
For class imbalance/overlap, rely on ONB_avg_man and N3 for direct performance correlations. For feature-coupling regimes, depth plots provide actionable feature-selection thresholds (Pascual-Triana et al., 2020, Sha et al., 2023).
In trajectory and context prediction, leverage pseudo-entropy or regularity/density scores for curriculum design and scene selection (Amirian et al., 2020).
Remarks on causal artifacting: spectral measures can be misleading when overlap is induced by data embedding or when parameter choices drive the metric, underscoring the need for classifier-agnostic validation.

Table 1: Summary of Notable Dataset Complexity Indicators

Indicator Name	Type/Domain	Key Formula / Principle
CSG (Cumulative Spectral Gradient)	Spectral, image/KG	$\sum_{i=0}^{K-2} \delta_i$ , $\delta_i = \lambda_{i+1} - \lambda_i$ (Branchaud-Charron et al., 2019, Gul et al., 2 Sep 2025)
cmsAULS (Cumulative Max Scaled AULS)	Spectral, image	$\sum_{k=1}^n m_k$ , $m_k = \max_{1\leq i\leq k} \tilde{\lambda}_i$ (Li et al., 2022)
DSI (Distance-based Separability Index)	Distance-based, general	$DSI = \frac{1}{n}\sum_{i=1}^{n} KS(\{d_{C_i}\},\{d_{C_i,\overline{C}_i}\})$ (Guan et al., 2021)
Depth (GA feature thresholds)	Feature selection, microarray	$s_{95\%} = \min\{s: f(s)\geq 0.95 f_{max}\}$ (Sha et al., 2023)
ONB_avg (Overlap Number of Balls)	Morphology, general	$(1/K)\sum_{i=1}^K (b_i/N_i)$ (Pascual-Triana et al., 2020)
MST-border (N1), 1-NN error (N3)	Neighborhood	$N1 = \|border\ edges\|/N$ , $N3 = LOOCV\ error$ (Lorena et al., 2018)

Conclusion

Dataset complexity indicators synthesize theoretical and empirical properties of data—including separability, overlap, dimensionality, and redundancy—into actionable metrics. They are essential tools for anticipating model difficulty, selecting algorithms, guiding resource allocation, and interpreting learning success or failure, provided their domain-specific limitations and parameter sensitivities are robustly controlled (Gul et al., 2 Sep 2025, Li et al., 2022, Lorena et al., 2018, Sha et al., 2023, Guan et al., 2021, Mello, 2019, Pascual-Triana et al., 2020).