Handcrafted Descriptors Overview

• Handcrafted Descriptors are algorithmically defined feature representations constructed using mathematical rules and expert insights, ensuring transparency and data efficiency.
• They encompass methods such as color statistics, texture (LBP, GLCM), gradient (HOG), and shape descriptors, often fused with deep features for superior performance.
• Their applications span medical imaging, bioimage analysis, forensics, and action recognition, with standardized pipelines enhancing cross-study comparability.

Handcrafted descriptors are algorithmically defined feature representations constructed according to mathematically specified rules and expert insight into visual, geometric, or statistical image structures, as opposed to features discovered via end-to-end learning. They remain fundamental in both classical and contemporary computer vision, pattern recognition, forensics, bioimage analysis, and medical imaging, providing transparent, interpretable, and often highly data-efficient alternatives or complements to learned features.

1. Mathematical Forms and Families of Handcrafted Descriptors

Handcrafted descriptors are designed to encode specific statistical, geometric, or domain-specific invariances. Commonly employed types include:

• Color Statistics and Histograms: Mean, variance, skewness, kurtosis, and histograms of intensities are extracted over multiple color spaces (RGB, HSV, CIELAB), either globally or within segmented regions of interest (ROIs). For example, for channel $c$ over $N$ pixels, moments are:

$$\mu_c = \frac{1}{N} \sum_{i=1}^N I_c(i), \quad \sigma_c^2 = \frac{1}{N} \sum_{i=1}^N (I_c(i) - \mu_c)^2$$

Higher-order moments and entropy-based measurements often augment this core set (Hoang et al., 20 Oct 2025).

• Texture Descriptors:
  • Local Binary Patterns (LBP): For $P$ neighbors and radius $R$, LBP outputs for center pixel $g_c$:

  $$\mathrm{LBP}_{P,R}(x) = \sum_{p=0}^{P-1} s(g_p - g_c) \, 2^p, \quad s(u) = \begin{cases} 1 & u \ge 0 \ 0 & u < 0 \end{cases}$$

  Uniform patterns (max two bitwise transitions) are commonly used for dimensionality reduction (Hoang et al., 20 Oct 2025, Hansley et al., 2017, Tschuchnig et al., 26 Jul 2025). - Gray-Level Co-occurrence Matrix (GLCM): Encodes spatial relationships via matrices $P(i,j)$ quantifying the normalized frequency of gray-level pairings at fixed offset and angle. Derived features (Haralick statistics) include contrast, correlation, energy, and homogeneity. - Completed LBP (CLBP), Local Ternary Patterns (LTP), Adaptive Hybrid Pattern (AHP), Multi-threshold Local Phase Quantization (MLPQ): Variations exist to encode magnitude, ternary relations, multiscale or multithreshold phase features (Nanni et al., 2019).

• Gradient- and Frequency-based Descriptors:

  • Histogram of Oriented Gradients (HOG): Computes normalized histograms of gradient orientations within localized grid cells, producing descriptors robust to illumination and local deformation [261.19262, (Hansley et al., 2017)].
  • Discrete Cosine Transform (DCT), Wavelet Features: Decompose images into frequency subbands, capturing periodic structures or compressibility signatures (Nirob et al., 27 Jan 2026).
  • Residual-based descriptors: High-pass filtering (e.g., with $h = [1, -3, 3, -1]$), followed by quantization and higher-order co-occurrence histograms, as in SPAM/SRM, for forensic analysis (Cozzolino et al., 2017).
• Shape and Morphological Descriptors:
  • Metrics such as region area, perimeter, compactness, sphericity, and moment invariants encode geometric properties of objects and ROIs (Salmanpour et al., 20 Nov 2025, Nanni et al., 2019).
• Spatial and Temporal Geometry:
  • In spatiotemporal contexts (e.g., sign language recognition), geometric relations between detected semantic points (e.g., hand and face centroids) are encoded via scaled distances, area, angles, and motion dynamics, e.g., per-frame features for centroid triangles (Carneiro et al., 2024).
• Bag-of-Visual-Words/Statistical Encodings:
  • Bag-of-Words histograms and Fisher Vectors over local descriptors (e.g., SIFT, dense trajectories) aggregate local feature statistics for variable-length data, with BoW as histogram encoding and FV as GMM-based moment statistics (Wang et al., 2019).
• Covariance Embedding:
  • Covariance matrices aggregating local descriptors (radiometric, geometric, structural) for each spatial block, compared via Log–Euclidean distance, form compact multiscale texture encodings (Pham, 2018).

2. Descriptor Engineering: Pipelines and Extraction Protocols

The construction of handcrafted descriptors involves precise algorithmic pipelines:

• Segmentation and ROI selection: Features are often extracted within automatically or semantically defined ROIs to enhance specificity (e.g., fish eye, breast lesion, ear region) (Hoang et al., 20 Oct 2025, Tschuchnig et al., 26 Jul 2025, Hansley et al., 2017).
• Pixelwise or local operator application: Operators such as LBP, HOG, Gabor filters, or phase quantization are evaluated in sliding windows or dense grids.
• Statistical aggregation: Histograms, means, variances, covariances, and spatial statistics transform pixelwise codes into compact fixed-length vectors. Entropy, normalization (L1, L2), and power-law (e.g., RootSIFT) postprocessing further enhance robustness and comparability (Balntas et al., 2017).
• Preprocessing: Quantization (fixed bin size or number), intensity normalization, and morphological filtering precede extraction to improve reproducibility and standardization (Salmanpour et al., 20 Nov 2025).

A representative example is the incremental fusion of feature groups: Rather than aggregating all descriptors at once, features are concatenated in staged steps with performance validation at each fusion, yielding compact, non-redundant final vectors (e.g., a 17-stage pipeline forming a 161-dimensional vector for fish freshness assessment (Hoang et al., 20 Oct 2025)).

3. Comparative Efficacy, Complementarity, and Fusion with Deep Features

Systematic benchmarking demonstrates that fused handcrafted descriptors achieve strong, often state-of-the-art, performance:

• Performance Benchmarks:
  • In bioimage classification, fusion of eleven handcrafted descriptors (e.g., LTP, MLPQ, CLBP, RIC, COL) via sum-rule ensemble achieved average accuracies of ~93.8%, nearly matching or slightly exceeding best CNN-only ensembles (Nanni et al., 2019).
  • For fish freshness classification, LightGBM over fused handcrafted features surpassed deep CNN baselines by >14% absolute accuracy (63.21% → 77.56%), with further gains under augmentation (Hoang et al., 20 Oct 2025).
  • AI-generated image detection yielded PR-AUC ≈ 0.99 and F1 = 0.94 using a mixed handcrafted set with gradient-boosted trees, outperforming basic models (Nirob et al., 27 Jan 2026).
• Complementarity in Hybrid Systems:
  • Fusing handcrafted features (edges, LBP, threshold maps) as early- or late-fusion channels in ResNet-50 and DINOv2 classifiers yielded AUC/F1 improvements (e.g., d₁ edges boosting AUC from 0.781 to 0.796) and superior recall compared to each alone (Tschuchnig et al., 26 Jul 2025).
  • In unconstrained ear and facial recognition, composite score-level fusion of handcrafted matchers (HOG, LBP, POEM, SIFT-style DSIFT, etc.) and CNN embeddings consistently outperforms single-type systems, demonstrating distinct error profiles (Hansley et al., 2017).
• Interpretability and Data Efficiency:
  • Handcrafted descriptors possess explicit statistical or geometrical meaning, facilitating interpretability and forensic audit, and permit strong discrimination with much less data than required by general-purpose deep nets (Nirob et al., 27 Jan 2026, Salmanpour et al., 20 Nov 2025).
  • In resource-constrained or real-time scenarios, low-dimensional handcrafted vectors support lightweight models with negligible computational overheads (e.g., 2 ms inference addition in sign language pipelines) (Carneiro et al., 2024).

4. Modern Standardization and Software Toolchains

Contemporary frameworks prioritize reproducible and standardized computation of handcrafted descriptors:

• PySERA Framework: Implements 557 handcrafted radiomic features (first-order, shape, GLCM, GLRLM, GLSZM, moment invariants, diagnostic) following Image Biomarker Standardization Initiative (IBSI) protocols, with full logging of preprocessing (resampling, discretization), and supports integrations with machine learning ecosystems for end-to-end pipelines (Salmanpour et al., 20 Nov 2025).
• HPatches Benchmark: Introduces strict protocols for descriptor evaluation (verification, matching, retrieval) under geometric and photometric distortion regimes, and demonstrates that proper normalization (ZCA-whitening, power-law) can elevate classic SIFT to near-deep performance on patch tasks (Balntas et al., 2017).

These frameworks ensure cross-study comparability and scalability across multicenter datasets.

5. Representative Descriptors: Comparative Properties

The table summarizes key families, their core principles, and application strengths:

Descriptor Type	Key Operation / Statistic	Application Domains
Color statistics	Moments, histograms in color spaces	Food, histology, general texture
LBP family	Pixelwise sign thresholding	Texture, bioimages, ear/facial biometrics
GLCM/Co-occurrence	Spatial gray-level relationships	Texture, lesion heterogeneity
HOG/Gradient	Gradient orientation histograms	Object detection, biometrics, forgery
Covariance-based	Region statistics, Log-Euclidean dist.	Texture retrieval
Residual/Forensic	High-pass + quantized co-occurrences	Image forensics, manipulation detection
Morphological	Area, perimeter, shape descriptors	Bioimage, radiomics, medical imaging
BoW/Fisher Vector	Cluster/statistical encoding	Video/action recognition

Each family is selected for targeted invariance or sensitivity—e.g., LBP for rotation-invariant microtexture, GLCM for heterogeneity, HOG for object edges, moments for shape invariants.

6. Impact and Use Cases Across Scientific Domains

Handcrafted descriptors are central to:

• Medical Imaging: Radiomics analysis (PySERA) for tumor characterization, risk modeling, outcome prediction, leveraging texture/morphology features for precision medicine (Salmanpour et al., 20 Nov 2025, Tschuchnig et al., 26 Jul 2025).
• Food Industry: Automated classification tasks (e.g., fish freshness estimation) utilizing fused color, histogram, and texture descriptors with interpretable, validated models (Hoang et al., 20 Oct 2025).
• Image Forensics: Residual-based and co-occurrence histograms for manipulation detection and provenance analysis, with architectures now being recast as constrained CNNs for further performance gain (Cozzolino et al., 2017).
• Bioimage and Phenotype Analysis: Ensembles of LBP variants, color stats, shape features enabling robust classification across tissue, cellular, and subcellular microscopy, often exceeding default CNNs without parameter tuning (Nanni et al., 2019).
• Action Recognition: Dense trajectory-based encodings (HOG, HOF, MBH) processed via BoW or Fisher Vectors for human action classification in video, now hybridized with learned hallucination by networks (Wang et al., 2019).
• Unconstrained Biometrics: LBP/HOG/BSIF/LPQ fused with deep CNNs for ear and facial recognition under large pose/illumination variability (Hansley et al., 2017).

These examples underscore continued relevance in scientific and applied settings demanding transparency, sample efficiency, domain integration, and computational frugality.

7. Limitations and Perspectives

Despite substantial advances, handcrafted descriptors have inherent limitations:

• Task Adaptivity: They encode only those invariances and cues anticipated by their formal definitions; extreme geometric or photometric distortions, and complex context-sensitive semantics, remain challenging relative to flexible deep architectures (Balntas et al., 2017).
• Scalability to Semantic Granularity: High-level or relational patterns (e.g., action verbs, anatomical relationships) often require augmentation through hybrid or learned models.
• Dimensionality and Redundancy: Uncontrolled fusion may yield prohibitively high-dimensional vectors (e.g., spatial pyramid GOLD), motivating staged or validation-driven fusion (Hoang et al., 20 Oct 2025, Nanni et al., 2019).

Nevertheless, carefully engineered, incrementally fused, and standard-compliant handcrafted features deliver state-of-the-art or near-state-of-the-art performance when matched to problem structure and fused with contemporary machine learning techniques. This highlights their role as vital, interpretability-preserving, and efficient components in modern vision and medical AI workflows (Hoang et al., 20 Oct 2025, Salmanpour et al., 20 Nov 2025, Nirob et al., 27 Jan 2026).